From rajshorya at gmail.com  Wed Oct  1 09:35:29 2014
From: rajshorya at gmail.com (Shorya Raj)
Date: Wed, 1 Oct 2014 20:35:29 +1300
Subject: [Python-Dev] Sysadmin tasks
Message-ID: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>

Hello
Just curious, is there any sort of tasklist for any sort of sysadmin sort
of work surrounding CPython development? There seem to be plenty of tasks
for the actual coding part, but it would be good to get something up for
the more systems admin side of things. If there is no one managing that
side yet, I would be more than happy to start to do so.



Thanks
SbSpider
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141001/5367c519/attachment.html>

From rdmurray at bitdance.com  Wed Oct  1 19:41:12 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 01 Oct 2014 13:41:12 -0400
Subject: [Python-Dev] Sysadmin tasks
In-Reply-To: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
References: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
Message-ID: <20141001174112.EC883250002@webabinitio.net>

On Wed, 01 Oct 2014 20:35:29 +1300, Shorya Raj <rajshorya at gmail.com> wrote:
> Just curious, is there any sort of tasklist for any sort of sysadmin sort
> of work surrounding CPython development? There seem to be plenty of tasks
> for the actual coding part, but it would be good to get something up for
> the more systems admin side of things. If there is no one managing that
> side yet, I would be more than happy to start to do so.

We have a sysadmin team (python-infrastructure is the mailing list,
there's also a #python-infra channel on freenode).  I'm sure they will
be happy for additional help!  I don't know that they have a formal task
list, but they might.  (Although I also do sysadmin stuff, I've confined
myself to bugs.python.org for my python sysadmin work...and that isn't
currently handled by the python-infrastructure team).

--David

From mark at hotpy.org  Wed Oct  1 19:59:47 2014
From: mark at hotpy.org (Mark Shannon)
Date: Wed, 01 Oct 2014 18:59:47 +0100
Subject: [Python-Dev] Sysadmin tasks
In-Reply-To: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
References: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
Message-ID: <542C4113.70204@hotpy.org>

Hi,

http://speed.python.org/
could do with some love.

Cheers,
Mark.

On 01/10/14 08:35, Shorya Raj wrote:
> Hello
> Just curious, is there any sort of tasklist for any sort of sysadmin
> sort of work surrounding CPython development? There seem to be plenty of
> tasks for the actual coding part, but it would be good to get something
> up for the more systems admin side of things. If there is no one
> managing that side yet, I would be more than happy to start to do so.
>
>
>
> Thanks
> SbSpider
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org
>

From dholth at gmail.com  Wed Oct  1 22:25:23 2014
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 1 Oct 2014 16:25:23 -0400
Subject: [Python-Dev] Sysadmin tasks
In-Reply-To: <542C4113.70204@hotpy.org>
References: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
 <542C4113.70204@hotpy.org>
Message-ID: <CAG8k2+70S9BagagTxNgaddqsrite7SGh5v7r7nd3WK3aFieokQ@mail.gmail.com>

On Wed, Oct 1, 2014 at 1:59 PM, Mark Shannon <mark at hotpy.org> wrote:
> Hi,
>
> http://speed.python.org/
> could do with some love.

+1

From ncoghlan at gmail.com  Thu Oct  2 00:17:44 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 2 Oct 2014 08:17:44 +1000
Subject: [Python-Dev] Sysadmin tasks
In-Reply-To: <CAG8k2+70S9BagagTxNgaddqsrite7SGh5v7r7nd3WK3aFieokQ@mail.gmail.com>
References: <CANk+dkOaw0PkMc_vx2ai4bTKMbPkqwescUCkBYQ4qFiRRqFUNQ@mail.gmail.com>
 <542C4113.70204@hotpy.org>
 <CAG8k2+70S9BagagTxNgaddqsrite7SGh5v7r7nd3WK3aFieokQ@mail.gmail.com>
Message-ID: <CADiSq7cCwBPvpF_bKhXYZPs8DWeYW-CGYyGdehaH-bEbsnSYaw@mail.gmail.com>

On 2 Oct 2014 06:26, "Daniel Holth" <dholth at gmail.com> wrote:
>
> On Wed, Oct 1, 2014 at 1:59 PM, Mark Shannon <mark at hotpy.org> wrote:
> > Hi,
> >
> > http://speed.python.org/
> > could do with some love.
>
> +1

Indeed :)

More generally, see the list Benjamin Peterson put together at
https://wiki.python.org/moin/PSFInfrastructure of the services the infra
team manage.

infrastructure at python.org is the relevant mailing list.

As you've noticed, one of the things currently missing is a "getting
started" guide akin to the developer guide for CPython. The infrastructure
list would be the place to discuss potentially creating one (although that
would also require discussion of what it should say!)

Cheers,
Nick.

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141002/29459354/attachment.html>

From status at bugs.python.org  Fri Oct  3 18:08:12 2014
From: status at bugs.python.org (Python tracker)
Date: Fri,  3 Oct 2014 18:08:12 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20141003160812.D7E3C560C4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-09-26 - 2014-10-03)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4631 (-46)
  closed 29679 (+92)
  total  34310 (+46)

Open issues with patches: 2169 


Issues opened (25)
==================

#20079: Add support for glibc supported locales
http://bugs.python.org/issue20079  reopened by haypo

#22139: python windows 2.7.8 64-bit did not install
http://bugs.python.org/issue22139  reopened by loewis

#22502: <Ctl-C> after continue in pdb stops in signal.py
http://bugs.python.org/issue22502  opened by xdegaye

#22503: Signal stack overflow in faulthandler_user
http://bugs.python.org/issue22503  opened by schwab

#22506: `dir` on Enum subclass doesn't expose parent class attributes
http://bugs.python.org/issue22506  opened by cool-RR

#22507: PyType_IsSubtype doesn't call __subclasscheck__
http://bugs.python.org/issue22507  opened by ionel.mc

#22508: Remove __version__ string from email
http://bugs.python.org/issue22508  opened by r.david.murray

#22515: Implement partial order on Counter
http://bugs.python.org/issue22515  opened by cool-RR

#22518: integer overflow in encoding unicode
http://bugs.python.org/issue22518  opened by pkt

#22519: integer overflow in computing byte's object representation
http://bugs.python.org/issue22519  opened by pkt

#22520: integer overflow in computing unicode's object representation
http://bugs.python.org/issue22520  opened by pkt

#22521: ctypes compilation fails on FreeBSD: Undefined symbol "ffi_cal
http://bugs.python.org/issue22521  opened by haypo

#22522: sys.excepthook doesn't receive the traceback when called from 
http://bugs.python.org/issue22522  opened by Claudiu.Popa

#22524: PEP 471 implementation: os.scandir() directory scanning functi
http://bugs.python.org/issue22524  opened by benhoyt

#22525: ast.literal_eval() doesn't do what the documentation says
http://bugs.python.org/issue22525  opened by Behdad.Esfahbod

#22533: Counter with no keys does not compare equal to Counter with ke
http://bugs.python.org/issue22533  opened by ethan.furman

#22535: headerregistry.Address introduces extra quotes without addr_sp
http://bugs.python.org/issue22535  opened by krother

#22536: subprocess should include filename in FileNotFoundError except
http://bugs.python.org/issue22536  opened by charpov

#22538: turtledemo two_canvases reversion
http://bugs.python.org/issue22538  opened by terry.reedy

#22539: Table formatting errors in pydoc
http://bugs.python.org/issue22539  opened by Artur.de.Sousa.Rocha

#22540: speed up isinstance and issubclass for the usual cases
http://bugs.python.org/issue22540  opened by georg.brandl

#22543: -W option cannot use non-standard categories
http://bugs.python.org/issue22543  opened by remram

#22544: Inconsistent cmath.log behaviour
http://bugs.python.org/issue22544  opened by pitrou

#22546: Wrong default precision in documentation for format
http://bugs.python.org/issue22546  opened by Barium

#22547: Implement an informative `BoundArguments.__repr__`
http://bugs.python.org/issue22547  opened by cool-RR



Most recent 15 issues with no replies (15)
==========================================

#22547: Implement an informative `BoundArguments.__repr__`
http://bugs.python.org/issue22547

#22539: Table formatting errors in pydoc
http://bugs.python.org/issue22539

#22538: turtledemo two_canvases reversion
http://bugs.python.org/issue22538

#22536: subprocess should include filename in FileNotFoundError except
http://bugs.python.org/issue22536

#22524: PEP 471 implementation: os.scandir() directory scanning functi
http://bugs.python.org/issue22524

#22522: sys.excepthook doesn't receive the traceback when called from 
http://bugs.python.org/issue22522

#22507: PyType_IsSubtype doesn't call __subclasscheck__
http://bugs.python.org/issue22507

#22502: <Ctl-C> after continue in pdb stops in signal.py
http://bugs.python.org/issue22502

#22495: merge large parts of test_binop.py and test_fractions.py
http://bugs.python.org/issue22495

#22493: Deprecate the use of flags not at the start of regular express
http://bugs.python.org/issue22493

#22489: .gitignore file
http://bugs.python.org/issue22489

#22475: asyncio task get_stack documentation seems to contradict itsel
http://bugs.python.org/issue22475

#22470: Possible integer overflow in error handlers
http://bugs.python.org/issue22470

#22468: Tarfile using fstat on GZip file object
http://bugs.python.org/issue22468

#22463: Warnings when building on AIX
http://bugs.python.org/issue22463



Most recent 15 issues waiting for review (15)
=============================================

#22540: speed up isinstance and issubclass for the usual cases
http://bugs.python.org/issue22540

#22525: ast.literal_eval() doesn't do what the documentation says
http://bugs.python.org/issue22525

#22522: sys.excepthook doesn't receive the traceback when called from 
http://bugs.python.org/issue22522

#22515: Implement partial order on Counter
http://bugs.python.org/issue22515

#22508: Remove __version__ string from email
http://bugs.python.org/issue22508

#22501: Optimise PyLong division by 1 or -1
http://bugs.python.org/issue22501

#22493: Deprecate the use of flags not at the start of regular express
http://bugs.python.org/issue22493

#22490: Using realpath for __PYVENV_LAUNCHER__ makes Homebrew installs
http://bugs.python.org/issue22490

#22489: .gitignore file
http://bugs.python.org/issue22489

#22486: Add math.gcd()
http://bugs.python.org/issue22486

#22470: Possible integer overflow in error handlers
http://bugs.python.org/issue22470

#22462: Modules/pyexpat.c violates PEP 384
http://bugs.python.org/issue22462

#22457: load_tests not invoked in root __init__.py when start=package 
http://bugs.python.org/issue22457

#22456: __base__ undocumented
http://bugs.python.org/issue22456

#22450: urllib doesn't put Accept: */* in the headers
http://bugs.python.org/issue22450



Top 10 most discussed issues (10)
=================================

#22515: Implement partial order on Counter
http://bugs.python.org/issue22515  41 msgs

#19915: int.bit_at(n) - Accessing a single bit in O(1)
http://bugs.python.org/issue19915  20 msgs

#20079: Add support for glibc supported locales
http://bugs.python.org/issue20079  15 msgs

#12029: Catching virtual subclasses in except clauses
http://bugs.python.org/issue12029  12 msgs

#22486: Add math.gcd()
http://bugs.python.org/issue22486   9 msgs

#22518: integer overflow in encoding unicode
http://bugs.python.org/issue22518   9 msgs

#21963: 2.7.8 backport of Issue1856 (avoid daemon thread problems at s
http://bugs.python.org/issue21963   7 msgs

#22472: OSErrors should use str and not repr on paths
http://bugs.python.org/issue22472   7 msgs

#22525: ast.literal_eval() doesn't do what the documentation says
http://bugs.python.org/issue22525   7 msgs

#17873: _ctypes/libffi missing bits for aarch64 support
http://bugs.python.org/issue17873   6 msgs



Issues closed (85)
==================

#2399: Patches for Tools/msi
http://bugs.python.org/issue2399  closed by benjamin.peterson

#4093: add gc/memory management tests to pybench
http://bugs.python.org/issue4093  closed by lemburg

#5309: distutils doesn't parallelize extension module compilation
http://bugs.python.org/issue5309  closed by pitrou

#5979: strptime() gives inconsistent exceptions
http://bugs.python.org/issue5979  closed by belopolsky

#6320: Standard string encodings should include GSM0.38
http://bugs.python.org/issue6320  closed by haypo

#7336: traceback module not properly printing exceptions on interpret
http://bugs.python.org/issue7336  closed by haypo

#7574: PyUnicode_FromFormat broken and not documented for 2.x
http://bugs.python.org/issue7574  closed by haypo

#8473: doctest fails if you have inconsistent lineendings
http://bugs.python.org/issue8473  closed by r.david.murray

#10007: Visual C++ cannot build _ssl and _hashlib if newer OpenSSL is 
http://bugs.python.org/issue10007  closed by zach.ware

#10349: g++ compilation error of Module/python.c with ./configure --wi
http://bugs.python.org/issue10349  closed by haypo

#10384: SyntaxError should contain exact location of the invalid chara
http://bugs.python.org/issue10384  closed by haypo

#10702: bytes and bytearray methods are not documented
http://bugs.python.org/issue10702  closed by berker.peksag

#10800: libffi build failure on HP-UX 11/PA
http://bugs.python.org/issue10800  closed by haypo

#10937: WinPE 64 bit execution results with errors
http://bugs.python.org/issue10937  closed by tim.golden

#11406: There is no os.listdir() equivalent returning generator instea
http://bugs.python.org/issue11406  closed by ncoghlan

#12780: Clean up tests for pyc/pyo in __file__
http://bugs.python.org/issue12780  closed by r.david.murray

#14341: sporadic (?) test_urllib2 failures
http://bugs.python.org/issue14341  closed by berker.peksag

#16038: ftplib: unlimited readline() from connection
http://bugs.python.org/issue16038  closed by georg.brandl

#16252: bytes and bytearray methods are undocumented
http://bugs.python.org/issue16252  closed by berker.peksag

#16324: MIMEText __init__ does not support Charset instance
http://bugs.python.org/issue16324  closed by berker.peksag

#16537: Python???s setup.py raises a ValueError when self.extensions i
http://bugs.python.org/issue16537  closed by berker.peksag

#16555: Add es_cu to locale aliases
http://bugs.python.org/issue16555  closed by serhiy.storchaka

#16758: IDLE SubprocessStartupError
http://bugs.python.org/issue16758  closed by terry.reedy

#17174: Posix os.path.join should raise TypeError when passed unusable
http://bugs.python.org/issue17174  closed by serhiy.storchaka

#17219: cross add Python's library directory when building python stan
http://bugs.python.org/issue17219  closed by doko

#17442: code.InteractiveInterpreter doesn't display the exception caus
http://bugs.python.org/issue17442  closed by r.david.murray

#17997: ssl.match_hostname(): sub string wildcard should not match IDN
http://bugs.python.org/issue17997  closed by georg.brandl

#18052: IDLE 3.3.2 Windows taskbar icon needs reboot after install
http://bugs.python.org/issue18052  closed by terry.reedy

#18096: bad library order returned by python-config.in
http://bugs.python.org/issue18096  closed by doko

#18554: os.__all__ is incomplete
http://bugs.python.org/issue18554  closed by yselivanov

#18711: Add PyErr_FormatV
http://bugs.python.org/issue18711  closed by pitrou

#18729: In unittest.TestLoader.discover doc select the name of load_te
http://bugs.python.org/issue18729  closed by python-dev

#18747: Re-seed OpenSSL's PRNG after fork
http://bugs.python.org/issue18747  closed by georg.brandl

#18854: is_multipart and walk should document their treatment of 'mess
http://bugs.python.org/issue18854  closed by r.david.murray

#19062: Idle: problem confighandler getting userdir
http://bugs.python.org/issue19062  closed by terry.reedy

#19342: Improve grp module docstrings
http://bugs.python.org/issue19342  closed by python-dev

#19367: IDLE wont open
http://bugs.python.org/issue19367  closed by terry.reedy

#19434: Wrong documentation of MIMENonMultipart class
http://bugs.python.org/issue19434  closed by python-dev

#19529: Fix unicode_aswidechar() with 4byte unicode and 2byte wchar_t,
http://bugs.python.org/issue19529  closed by georg.brandl

#19677: IDLE displaying a CoreAnimation warning
http://bugs.python.org/issue19677  closed by terry.reedy

#20076: Add UTF-8 locale aliases
http://bugs.python.org/issue20076  closed by serhiy.storchaka

#20135: FAQ need list mutation answers
http://bugs.python.org/issue20135  closed by r.david.murray

#20858: Enhancements/fixes to pure-python datetime module
http://bugs.python.org/issue20858  closed by benjamin.peterson

#20974: email module docs say not compatible with current python versi
http://bugs.python.org/issue20974  closed by r.david.murray

#21339: IDLE crash on OS X 10.9 upon shut-down with many windows open
http://bugs.python.org/issue21339  closed by terry.reedy

#21397: tempfile.py: Fix grammar in docstring, comment typos
http://bugs.python.org/issue21397  closed by yselivanov

#21739: Add hint about expression in list comprehensions (https://docs
http://bugs.python.org/issue21739  closed by r.david.murray

#21909: PyLong_FromString drops const
http://bugs.python.org/issue21909  closed by serhiy.storchaka

#21922: PyLong: use GMP
http://bugs.python.org/issue21922  closed by benjamin.peterson

#21971: Index and update turtledemo doc.
http://bugs.python.org/issue21971  closed by terry.reedy

#22103: bdist_wininst does not run install script
http://bugs.python.org/issue22103  closed by ned.deily

#22251: Various markup errors in documentation
http://bugs.python.org/issue22251  closed by berker.peksag

#22379: Empty exception message of str.join
http://bugs.python.org/issue22379  closed by python-dev

#22396: AIX posix_fadvise and posix_fallocate
http://bugs.python.org/issue22396  closed by haypo

#22422: IDLE closes all when in dropdown menu
http://bugs.python.org/issue22422  closed by mandolout

#22437: re module: number of named groups is limited to 100 max
http://bugs.python.org/issue22437  closed by serhiy.storchaka

#22465: Number agreement error in section 3.2 of web docs
http://bugs.python.org/issue22465  closed by terry.reedy

#22466: problem with installing python 2.7.8
http://bugs.python.org/issue22466  closed by steve.dower

#22492: small addition to print() docs: no binary streams.
http://bugs.python.org/issue22492  closed by terry.reedy

#22494: default logging time string is not localized
http://bugs.python.org/issue22494  closed by vinay.sajip

#22504: Add ordering between `Enum` objects
http://bugs.python.org/issue22504  closed by ethan.furman

#22505: Expose an Enum object's serial number
http://bugs.python.org/issue22505  closed by ethan.furman

#22509: Website incorrect link
http://bugs.python.org/issue22509  closed by ned.deily

#22510: Faster bypass re cache when DEBUG is passed
http://bugs.python.org/issue22510  closed by serhiy.storchaka

#22511: Assignment Operators behavior within a user-defined function a
http://bugs.python.org/issue22511  closed by mark.dickinson

#22512: 'test_distutils.test_bdist_rpm' causes creation of directory '
http://bugs.python.org/issue22512  closed by r.david.murray

#22513: grp.struct_group is not hashable
http://bugs.python.org/issue22513  closed by ethan.furman

#22514: incomplete programming example on python.org
http://bugs.python.org/issue22514  closed by benjamin.peterson

#22516: Windows Installer won't - even when using "just for me"option
http://bugs.python.org/issue22516  closed by r.david.murray

#22517: BufferedRWpair doesn't clear weakrefs
http://bugs.python.org/issue22517  closed by python-dev

#22523: [regression] Lib/ssl.py still references _ssl.sslwrap
http://bugs.python.org/issue22523  closed by python-dev

#22526: file iteration SystemError for huge lines (2GiB+)
http://bugs.python.org/issue22526  closed by python-dev

#22527: Documentation warnings
http://bugs.python.org/issue22527  closed by georg.brandl

#22528: Missing hint to source code
http://bugs.python.org/issue22528  closed by python-dev

#22529: Why copyright only 1990-2013 and not 2014
http://bugs.python.org/issue22529  closed by ned.deily

#22530: re rejects index of type long on 2.7
http://bugs.python.org/issue22530  closed by python-dev

#22531: Turn contextlib.{redirect_stdout,suppress} into ContextDecorat
http://bugs.python.org/issue22531  closed by ncoghlan

#22532: A suggested change
http://bugs.python.org/issue22532  closed by eric.smith

#22534: bsddb memory leak with MacPorts bdb 4.6
http://bugs.python.org/issue22534  closed by ned.deily

#22537: Failure building 2.7 docs on Windows
http://bugs.python.org/issue22537  closed by python-dev

#22541: Support both side_effect and return_value in a more human way
http://bugs.python.org/issue22541  closed by michael.foord

#22542: Use syscall (eg. arc4random or getentropy) rather than /dev/ur
http://bugs.python.org/issue22542  closed by alex

#22545: incomplete complex repr() with negative zeroes
http://bugs.python.org/issue22545  closed by pitrou

#1764286: inspect.getsource does not work with decorated functions
http://bugs.python.org/issue1764286  closed by yselivanov

#1176504: locale._build_localename treatment for utf8
http://bugs.python.org/issue1176504  closed by serhiy.storchaka

From alex.gaynor at gmail.com  Fri Oct  3 23:57:42 2014
From: alex.gaynor at gmail.com (Alex Gaynor)
Date: Fri, 3 Oct 2014 21:57:42 +0000 (UTC)
Subject: [Python-Dev] PEP476: Enabling certificate validation by default
References: <loom.20140919T185218-488@post.gmane.org>
 <CAP7+vJLwO2HPWJd2WGXFCtyA-c33D6dK-=QYVxLAEpF1svUP1w@mail.gmail.com>
 <CAFRnB2UJFErffyZTAJPM8f=_DDAbD5aGN-e9G7DacBfYUqb8Og@mail.gmail.com>
 <CADiSq7fLz80aV2g7hVCU4K3w==tU6smid+C292L1N5m-KwsWfw@mail.gmail.com>
 <CAP7+vJKQwrjPJydcys+UFOty0xzzOuTgrYg=WLnAfUNQjY4nJA@mail.gmail.com>
 <CAFRnB2Vs9bPOuR1HRsvbYoVT4=URqsa1xMF+2gH9qyUf4imavA@mail.gmail.com>
 <CAP7+vJKioet=Bm-BvXakG9pBB7eYiF1F=Z7FHf=2=Z8UtvAu8g@mail.gmail.com>
 <CAFRnB2WviuB-=gKgFAKsW6a0vZvXpMWp36knDoSqqM4c42TPQw@mail.gmail.com>
 <CADiSq7cCM-YNZFO_1Vkss+Ja6zmPB-CvT34hOFg1eWKSwipW6g@mail.gmail.com>
 <CAP7+vJ+C897KLH7XNMcQwTYBH2CNeSu8NfH6=r0d9474RCjDBw@mail.gmail.com>
 <CADiSq7dXtDxZM14WvTH-0ZbrRES6Mh=9sqfJrf9O8VpBJtEtWw@mail.gmail.com>
 <CAP7+vJ+1D=b=4r3qL4B2+BBkJPSpoji2R4Ds=R+h73JErDrkqw@mail.gmail.com>
Message-ID: <loom.20141003T235603-112@post.gmane.org>

Guido van Rossum <guido <at> python.org> writes:

> 
> OK, I'll hold off a bit on approving the PEP, but my intention is to approve
> it. Go Alex go!
> 

A patch for the environmental variable overrides on Windows has landed; thanks
Benjamin!

Alex


From donald at stufft.io  Sat Oct  4 00:06:01 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 3 Oct 2014 18:06:01 -0400
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
Message-ID: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>

I'm working on the backport of ensurepip to Python 2.7, and I realized that
I'm not sure which commands to install. Right now by default pip (outside of
the context of ensurepip) will install pip, pip2, and pip2.7 if installed in
Python 2.7. In Python 3's ensurepip we modified it so that it would install
pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4 if it
was an "alt install".

My question is, does this behavior make sense for ensurepip in 2.7? Or should
it also install the "pip" command if it is an "install"?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From guido at python.org  Sat Oct  4 02:31:51 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 3 Oct 2014 17:31:51 -0700
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
Message-ID: <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>

That is copying the (alt)install targets of Python's own Makefile, and I
think those are exactly right.
On Oct 3, 2014 3:07 PM, "Donald Stufft" <donald at stufft.io> wrote:

> I'm working on the backport of ensurepip to Python 2.7, and I realized that
> I'm not sure which commands to install. Right now by default pip (outside
> of
> the context of ensurepip) will install pip, pip2, and pip2.7 if installed
> in
> Python 2.7. In Python 3's ensurepip we modified it so that it would install
> pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4 if
> it
> was an "alt install".
>
> My question is, does this behavior make sense for ensurepip in 2.7? Or
> should
> it also install the "pip" command if it is an "install"?
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141003/b51f3ddb/attachment.html>

From donald at stufft.io  Sat Oct  4 02:33:08 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 3 Oct 2014 20:33:08 -0400
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
 <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
Message-ID: <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>

Ok, so neither Python 2.7 nor Python 3.x?s ensure pip command will install a
``pip`` binary by default without a flag. That's fine with me, just wanted to
make sure it made sense for Python 2.x. Thanks!

> On Oct 3, 2014, at 8:31 PM, Guido van Rossum <guido at python.org> wrote:
> 
> That is copying the (alt)install targets of Python's own Makefile, and I think those are exactly right.
> 
> On Oct 3, 2014 3:07 PM, "Donald Stufft" <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> I'm working on the backport of ensurepip to Python 2.7, and I realized that
> I'm not sure which commands to install. Right now by default pip (outside of
> the context of ensurepip) will install pip, pip2, and pip2.7 if installed in
> Python 2.7. In Python 3's ensurepip we modified it so that it would install
> pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4 if it
> was an "alt install".
> 
> My question is, does this behavior make sense for ensurepip in 2.7? Or should
> it also install the "pip" command if it is an "install"?
> 
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <mailto:Python-Dev at python.org>
> https://mail.python.org/mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org <https://mail.python.org/mailman/options/python-dev/guido%40python.org>

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141003/cc12a701/attachment.html>

From guido at python.org  Sat Oct  4 02:46:01 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 3 Oct 2014 17:46:01 -0700
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
 <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
 <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>
Message-ID: <CAP7+vJ+Dw64piWG680C4tBYRoBwgMSODCDQXjmMQVBQ4=8gZEw@mail.gmail.com>

That's not what I meant. Python 2.7 does install "python" unless you use
altinstall.
On Oct 3, 2014 5:33 PM, "Donald Stufft" <donald at stufft.io> wrote:

> Ok, so neither Python 2.7 nor Python 3.x?s ensure pip command will install
> a
> ``pip`` binary by default without a flag. That's fine with me, just wanted
> to
> make sure it made sense for Python 2.x. Thanks!
>
> On Oct 3, 2014, at 8:31 PM, Guido van Rossum <guido at python.org> wrote:
>
> That is copying the (alt)install targets of Python's own Makefile, and I
> think those are exactly right.
> On Oct 3, 2014 3:07 PM, "Donald Stufft" <donald at stufft.io> wrote:
>
>> I'm working on the backport of ensurepip to Python 2.7, and I realized
>> that
>> I'm not sure which commands to install. Right now by default pip (outside
>> of
>> the context of ensurepip) will install pip, pip2, and pip2.7 if installed
>> in
>> Python 2.7. In Python 3's ensurepip we modified it so that it would
>> install
>> pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4
>> if it
>> was an "alt install".
>>
>> My question is, does this behavior make sense for ensurepip in 2.7? Or
>> should
>> it also install the "pip" command if it is an "install"?
>>
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141003/0ab03763/attachment.html>

From donald at stufft.io  Sat Oct  4 02:51:00 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 3 Oct 2014 20:51:00 -0400
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <CAP7+vJ+Dw64piWG680C4tBYRoBwgMSODCDQXjmMQVBQ4=8gZEw@mail.gmail.com>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
 <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
 <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>
 <CAP7+vJ+Dw64piWG680C4tBYRoBwgMSODCDQXjmMQVBQ4=8gZEw@mail.gmail.com>
Message-ID: <E84DD16E-BE06-4908-9E47-63A83E8806DE@stufft.io>

Whoops, I misred.

So to be clear, you think:

install -> pip, pip2, pip2.7
altinstall -> pip2.7

> On Oct 3, 2014, at 8:46 PM, Guido van Rossum <guido at python.org> wrote:
> 
> That's not what I meant. Python 2.7 does install "python" unless you use altinstall.
> 
> On Oct 3, 2014 5:33 PM, "Donald Stufft" <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> Ok, so neither Python 2.7 nor Python 3.x?s ensure pip command will install a
> ``pip`` binary by default without a flag. That's fine with me, just wanted to
> make sure it made sense for Python 2.x. Thanks!
> 
>> On Oct 3, 2014, at 8:31 PM, Guido van Rossum <guido at python.org <mailto:guido at python.org>> wrote:
>> 
>> That is copying the (alt)install targets of Python's own Makefile, and I think those are exactly right.
>> 
>> On Oct 3, 2014 3:07 PM, "Donald Stufft" <donald at stufft.io <mailto:donald at stufft.io>> wrote:
>> I'm working on the backport of ensurepip to Python 2.7, and I realized that
>> I'm not sure which commands to install. Right now by default pip (outside of
>> the context of ensurepip) will install pip, pip2, and pip2.7 if installed in
>> Python 2.7. In Python 3's ensurepip we modified it so that it would install
>> pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4 if it
>> was an "alt install".
>> 
>> My question is, does this behavior make sense for ensurepip in 2.7? Or should
>> it also install the "pip" command if it is an "install"?
>> 
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>> 
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org <mailto:Python-Dev at python.org>
>> https://mail.python.org/mailman/listinfo/python-dev <https://mail.python.org/mailman/listinfo/python-dev>
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/guido%40python.org <https://mail.python.org/mailman/options/python-dev/guido%40python.org>
> 
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
> 

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141003/97edf703/attachment-0001.html>

From guido at python.org  Sat Oct  4 03:05:08 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 3 Oct 2014 18:05:08 -0700
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <E84DD16E-BE06-4908-9E47-63A83E8806DE@stufft.io>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
 <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
 <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>
 <CAP7+vJ+Dw64piWG680C4tBYRoBwgMSODCDQXjmMQVBQ4=8gZEw@mail.gmail.com>
 <E84DD16E-BE06-4908-9E47-63A83E8806DE@stufft.io>
Message-ID: <CAP7+vJKCVK8gLOAYyfbQ5+UDWWXjqCS7yZn44z82E4DN1ZWtgQ@mail.gmail.com>

Yes.

On Friday, October 3, 2014, Donald Stufft <donald at stufft.io> wrote:

> Whoops, I misred.
>
> So to be clear, you think:
>
> install -> pip, pip2, pip2.7
> altinstall -> pip2.7
>
> On Oct 3, 2014, at 8:46 PM, Guido van Rossum <guido at python.org
> <javascript:_e(%7B%7D,'cvml','guido at python.org');>> wrote:
>
> That's not what I meant. Python 2.7 does install "python" unless you use
> altinstall.
> On Oct 3, 2014 5:33 PM, "Donald Stufft" <donald at stufft.io
> <javascript:_e(%7B%7D,'cvml','donald at stufft.io');>> wrote:
>
>> Ok, so neither Python 2.7 nor Python 3.x?s ensure pip command will
>> install a
>> ``pip`` binary by default without a flag. That's fine with me, just
>> wanted to
>> make sure it made sense for Python 2.x. Thanks!
>>
>> On Oct 3, 2014, at 8:31 PM, Guido van Rossum <guido at python.org
>> <javascript:_e(%7B%7D,'cvml','guido at python.org');>> wrote:
>>
>> That is copying the (alt)install targets of Python's own Makefile, and I
>> think those are exactly right.
>> On Oct 3, 2014 3:07 PM, "Donald Stufft" <donald at stufft.io
>> <javascript:_e(%7B%7D,'cvml','donald at stufft.io');>> wrote:
>>
>>> I'm working on the backport of ensurepip to Python 2.7, and I realized
>>> that
>>> I'm not sure which commands to install. Right now by default pip
>>> (outside of
>>> the context of ensurepip) will install pip, pip2, and pip2.7 if
>>> installed in
>>> Python 2.7. In Python 3's ensurepip we modified it so that it would
>>> install
>>> pip3, and pip3.4, but *not* pip if it was an "install", and only pip3.4
>>> if it
>>> was an "alt install".
>>>
>>> My question is, does this behavior make sense for ensurepip in 2.7? Or
>>> should
>>> it also install the "pip" command if it is an "install"?
>>>
>>> ---
>>> Donald Stufft
>>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> <javascript:_e(%7B%7D,'cvml','Python-Dev at python.org');>
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>
>>
>> ---
>> Donald Stufft
>> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>>
>>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
>

-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141003/eaf86b21/attachment.html>

From ncoghlan at gmail.com  Sat Oct  4 13:28:45 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 4 Oct 2014 21:28:45 +1000
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <E84DD16E-BE06-4908-9E47-63A83E8806DE@stufft.io>
References: <D5112D00-A986-4268-8C4D-968C4C301B8A@stufft.io>
 <CAP7+vJ+dCJMvm6jofMPDEA5Lb-7McFmsLCNKdFpdZ8aciooJYw@mail.gmail.com>
 <B02FB722-ECF6-4065-BCF9-836BB806C920@stufft.io>
 <CAP7+vJ+Dw64piWG680C4tBYRoBwgMSODCDQXjmMQVBQ4=8gZEw@mail.gmail.com>
 <E84DD16E-BE06-4908-9E47-63A83E8806DE@stufft.io>
Message-ID: <CADiSq7f3fumg2743K5o5wtvyFg6eO4T=MXJ_9iXyVBqkCLEDDQ@mail.gmail.com>

On 4 October 2014 10:51, Donald Stufft <donald at stufft.io> wrote:
> Whoops, I misred.
>
> So to be clear, you think:
>
> install -> pip, pip2, pip2.7
> altinstall -> pip2.7

To spell out the assumption I didn't make clear when helping with the
backport PEP, the difference comes from PEP 394, which specifies the
following behaviour when installing Python itself:

Python 2.7: python, python2, python2.7
Python 3.4: python3, python3.4

That maps to ensurepip as:

Python 2.7: pip, pip2, pip2.7
Python 3.4: pip3, pip3.4

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From georg at python.org  Sat Oct  4 15:21:47 2014
From: georg at python.org (Georg Brandl)
Date: Sat, 04 Oct 2014 15:21:47 +0200
Subject: [Python-Dev] [RELEASED] Python 3.2.6rc1, Python 3.3.6rc1
Message-ID: <542FF46B.2040109@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm happy to announce
the release of Python 3.2.6rc1 and 3.3.6rc1.  Both are release candidates
for security-fix releases, which are provide source-only on python.org.

The list of security-related issues fixed in the releases is given in
the changelogs:

    https://hg.python.org/cpython/raw-file/v3.2.6rc1/Misc/NEWS
    https://hg.python.org/cpython/raw-file/v3.3.6rc1/Misc/NEWS

To download the pre-releases visit one of:

    https://www.python.org/downloads/release/python-326rc1/
    https://www.python.org/downloads/release/python-336rc1/

These are pre-releases, please report any bugs to

     http://bugs.python.org/

The final releases are scheduled one week from now.

Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlQv9GsACgkQN9GcIYhpnLC93gCfVm74lhOysPYCO0fy9/l5LUfJ
bUYAn2u1EygfsPw2oa4CSoj5t0TYUJq7
=HnOK
-----END PGP SIGNATURE-----

From g.brandl at gmx.net  Sun Oct  5 12:22:18 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Oct 2014 12:22:18 +0200
Subject: [Python-Dev] Microsoft Visual C++ Compiler for Python 2.7
In-Reply-To: <a08e4b0867994538bc6e6a31fefddf62@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <eabd03f13d314698af86e29e5772fc9f@DM2PR0301MB0734.namprd03.prod.outlook.com>,
 <20140927141226.5ad2979d@fsol>
 <a08e4b0867994538bc6e6a31fefddf62@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <m0r64q$kej$1@ger.gmane.org>

I just tried out the compiler and built wininst and wheel dists. Thanks!

distutils was *almost* fine using it, but for two snags:

* I had to set VS90COMNTOOLS

* distutils expects vcvarsall.bat in VC, while you have it in the parent dir

The first could be set by the installer of your package.  But if it is not
feasible to do so, setting an envvar is something that developers can surely
be expected to do with instruction.

The second is a little more inconvenient.  Wouldn't it be possible to put
the .bat file in the "right" folder, or if it has to be there, put *another*
one in "VC"?  (That is what I did.)

cheers,
Georg

On 09/27/2014 05:16 PM, Steve Dower wrote:
> Plain distutils won't detect it. It would be easy enough to fix 2.7.9, but
> "update Python" is a big/impossible ask for a lot of people, whereas "update
> setuptools" is easy and also covers Python 2.6 and <3.3.
> 
> The compiler installer can't set the keys that distutils looks for without
> losing the per-user installation, and it may also corrupt actual installs of
> Visual Studio. A monkey patch via setuptools was the best way to handle this -
> covers pip and Cython and can be ported to other libraries that care but avoid
> setuptools.
> 
> Now that we have a patch, there's very limited value in fixing 2.7.9, IMO. But
> I'm willing to be convinced - we can always add a version check to the
> setuptools patch.
> 
> Cheers,
> Steve



From Steve.Dower at microsoft.com  Sun Oct  5 17:03:03 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 5 Oct 2014 15:03:03 +0000
Subject: [Python-Dev] Microsoft Visual C++ Compiler for Python 2.7
In-Reply-To: <m0r64q$kej$1@ger.gmane.org>
References: <eabd03f13d314698af86e29e5772fc9f@DM2PR0301MB0734.namprd03.prod.outlook.com>, 
 <20140927141226.5ad2979d@fsol>
 <a08e4b0867994538bc6e6a31fefddf62@DM2PR0301MB0734.namprd03.prod.outlook.com>,
 <m0r64q$kej$1@ger.gmane.org>
Message-ID: <405f3b5b27184033b6e8c7c8e7ed27b5@DM2PR0301MB0734.namprd03.prod.outlook.com>

If you update setuptools to 6.0 or later, it shouldn't have any trouble detecting it. No need to set any environment variables.

FWIW, I put my vcvarsall.bat where I did because it's not the original version. All the files in VC and WinSDK after unmodified from their original release.

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Georg Brandl<mailto:g.brandl at gmx.net>
Sent: ?10/?5/?2014 3:23
To: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Microsoft Visual C++ Compiler for Python 2.7

I just tried out the compiler and built wininst and wheel dists. Thanks!

distutils was *almost* fine using it, but for two snags:

* I had to set VS90COMNTOOLS

* distutils expects vcvarsall.bat in VC, while you have it in the parent dir

The first could be set by the installer of your package.  But if it is not
feasible to do so, setting an envvar is something that developers can surely
be expected to do with instruction.

The second is a little more inconvenient.  Wouldn't it be possible to put
the .bat file in the "right" folder, or if it has to be there, put *another*
one in "VC"?  (That is what I did.)

cheers,
Georg

On 09/27/2014 05:16 PM, Steve Dower wrote:
> Plain distutils won't detect it. It would be easy enough to fix 2.7.9, but
> "update Python" is a big/impossible ask for a lot of people, whereas "update
> setuptools" is easy and also covers Python 2.6 and <3.3.
>
> The compiler installer can't set the keys that distutils looks for without
> losing the per-user installation, and it may also corrupt actual installs of
> Visual Studio. A monkey patch via setuptools was the best way to handle this -
> covers pip and Cython and can be ported to other libraries that care but avoid
> setuptools.
>
> Now that we have a patch, there's very limited value in fixing 2.7.9, IMO. But
> I'm willing to be convinced - we can always add a version check to the
> setuptools patch.
>
> Cheers,
> Steve


_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141005/b8dd1ecf/attachment.html>

From rdmurray at bitdance.com  Sun Oct  5 18:11:58 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 05 Oct 2014 12:11:58 -0400
Subject: [Python-Dev] bytes-like objects
Message-ID: <20141005161158.9A52F250069@webabinitio.net>

Over the past while we've been cleaning up the docs in the area of "how
do we refer to bytes, bytearray, memoryview, etc, etc?" in the APIs that
deal with bytes.  As you may or may not remember, we settled on the term
'bytes-like object', and have changed the docs to (we hope) consistently
use this term, which has a glossary entry most of the references to it
link to.

I just committed (to 3.5) the final changes for issue 16518, the change
to consistently using the term 'bytes-like object' for things that
support the buffer interface.  This means that messages that previously
read:

    'sometype' does not support the buffer interface

now read

    a bytes-like object is required, not 'sometype'

The 'buffer interface' messages were rather confusing, since you have to
dig to find out what the 'buffer interface' is.  It isn't any sort of
python object, nor is there any object in python3 that has a name
related to it.  'bytes-like object', on the other hand, is fairly
intuitive[*], and if you look it up in the glossary it links to the
explanation of the buffer interface.

We felt it was worth explicitly mentioning this patch on python-dev to point
out to everyone that the docs and error messages now use a consistent
terminology, instead of the mis-mash we had before, which should make it easier
to help people with issues where this comes up.

If there are objections to this change to the messages it is easy enough to
back out, but personally I find the new error messages *much* clearer, and I'm
an experienced dev.

(This work was done by Ezio Melotti, but I committed the final patch as
part of my quest to clear the 'commit review' queue in the tracker.)

--David

[*] the more esoteric cases are not often encountered in regular (as opposed to
NumPy) code, and when they are, the extension to "something that can be
interpreted as a sequence of bytes" is pretty straightforward conceptually.

From g.brandl at gmx.net  Sun Oct  5 18:59:56 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Oct 2014 18:59:56 +0200
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <20141005161158.9A52F250069@webabinitio.net>
References: <20141005161158.9A52F250069@webabinitio.net>
Message-ID: <5431790C.1000702@gmx.net>

On 10/05/2014 06:11 PM, R. David Murray wrote:
> Over the past while we've been cleaning up the docs in the area of "how
> do we refer to bytes, bytearray, memoryview, etc, etc?" in the APIs that
> deal with bytes.  As you may or may not remember, we settled on the term
> 'bytes-like object', and have changed the docs to (we hope) consistently
> use this term, which has a glossary entry most of the references to it
> link to.
> 
> I just committed (to 3.5) the final changes for issue 16518, the change
> to consistently using the term 'bytes-like object' for things that
> support the buffer interface.  This means that messages that previously
> read:
> 
>     'sometype' does not support the buffer interface
> 
> now read
> 
>     a bytes-like object is required, not 'sometype'
> 
> The 'buffer interface' messages were rather confusing, since you have to
> dig to find out what the 'buffer interface' is.  It isn't any sort of
> python object, nor is there any object in python3 that has a name
> related to it.  'bytes-like object', on the other hand, is fairly
> intuitive[*], and if you look it up in the glossary it links to the
> explanation of the buffer interface.
> 
> We felt it was worth explicitly mentioning this patch on python-dev to point
> out to everyone that the docs and error messages now use a consistent
> terminology, instead of the mis-mash we had before, which should make it easier
> to help people with issues where this comes up.
> 
> If there are objections to this change to the messages it is easy enough to
> back out, but personally I find the new error messages *much* clearer, and I'm
> an experienced dev.

I agree. Thanks for the effort you two!

Georg


From g.brandl at gmx.net  Sun Oct  5 18:59:56 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 05 Oct 2014 18:59:56 +0200
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <20141005161158.9A52F250069@webabinitio.net>
References: <20141005161158.9A52F250069@webabinitio.net>
Message-ID: <5431790C.1000702@gmx.net>

On 10/05/2014 06:11 PM, R. David Murray wrote:
> Over the past while we've been cleaning up the docs in the area of "how
> do we refer to bytes, bytearray, memoryview, etc, etc?" in the APIs that
> deal with bytes.  As you may or may not remember, we settled on the term
> 'bytes-like object', and have changed the docs to (we hope) consistently
> use this term, which has a glossary entry most of the references to it
> link to.
> 
> I just committed (to 3.5) the final changes for issue 16518, the change
> to consistently using the term 'bytes-like object' for things that
> support the buffer interface.  This means that messages that previously
> read:
> 
>     'sometype' does not support the buffer interface
> 
> now read
> 
>     a bytes-like object is required, not 'sometype'
> 
> The 'buffer interface' messages were rather confusing, since you have to
> dig to find out what the 'buffer interface' is.  It isn't any sort of
> python object, nor is there any object in python3 that has a name
> related to it.  'bytes-like object', on the other hand, is fairly
> intuitive[*], and if you look it up in the glossary it links to the
> explanation of the buffer interface.
> 
> We felt it was worth explicitly mentioning this patch on python-dev to point
> out to everyone that the docs and error messages now use a consistent
> terminology, instead of the mis-mash we had before, which should make it easier
> to help people with issues where this comes up.
> 
> If there are objections to this change to the messages it is easy enough to
> back out, but personally I find the new error messages *much* clearer, and I'm
> an experienced dev.

I agree. Thanks for the effort you two!

Georg

From tx_babee at yahoo.com  Sun Oct  5 19:04:31 2014
From: tx_babee at yahoo.com (Lyndal)
Date: Sun, 5 Oct 2014 12:04:31 -0500
Subject: [Python-Dev]  Weekly Python Patch/Bug Summary
Message-ID: <64C3B3BC-FD30-4E77-AAD8-B3C4AE1C914B@yahoo.com>



Sent from my iPhone

From techtonik at gmail.com  Sun Oct  5 20:35:17 2014
From: techtonik at gmail.com (anatoly techtonik)
Date: Sun, 5 Oct 2014 21:35:17 +0300
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <5431790C.1000702@gmx.net>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
Message-ID: <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>

That's a cool stuff. `bytes-like object` is really a much better name for users.

From tjreedy at udel.edu  Sun Oct  5 21:49:48 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 05 Oct 2014 15:49:48 -0400
Subject: [Python-Dev] 3.4.2rc1 online docs: cannot access tkinter, idle pages
Message-ID: <m0s7ct$2r5$1@ger.gmane.org>

I can access every page I have tried except for the tkinter and idle 
pages, which either give interminable 'Connecting...' or in one try, 404.
https://docs.python.org/3/library/tkinter.html
https://docs.python.org/3/library/idle.html

This is true from the index, previous/next from adjacent pages, and the 
index page
https://docs.python.org/3/library/tk.html

-- 
Terry Jan Reedy


From greg.ewing at canterbury.ac.nz  Sun Oct  5 23:24:35 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Oct 2014 10:24:35 +1300
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
Message-ID: <5431B713.2060708@canterbury.ac.nz>

anatoly techtonik wrote:
> That's a cool stuff. `bytes-like object` is really a much better name for users.

I'm not so sure. Usually when we talk about an "xxx-like object" we
mean one that supports a certain Python interface, e.g. a "file-like
object" is one that has read() and/or write() methods. But you can't
create an object that supports the buffer protocol by implementing
Python methods.

I'm worried that using the term "bytes-like object" will lead
people to ask "What methods do I have to implement to make my
object bytes-like?", to which the answer is "mu".

-- 
Greg

From victor.stinner at gmail.com  Sun Oct  5 23:32:08 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Sun, 5 Oct 2014 23:32:08 +0200
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <20141005161158.9A52F250069@webabinitio.net>
References: <20141005161158.9A52F250069@webabinitio.net>
Message-ID: <CAMpsgwaLt=vWh3haZnWKwTg6OCBwWkZqdtibGpHQUkYFpz-_dw@mail.gmail.com>

Hi,

I prefer "bytes-like" than "buffer protocol". By the way, is there a
documentation in Python doc which explains "bytes-like" and maybe list most
compatible types?

I'm not sure that the term has an unique definition. In some parts of
Python, I saw explicit checks on the type: bytes or bytearray, sometimes
memoryview is accepted. The behaviour is different in C functions using
PyArg API. It probably depends if the function relies on the content (read
bytes) or on methods (ex: call .find).

Victor
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141005/a799ec8d/attachment.html>

From storchaka at gmail.com  Sun Oct  5 23:58:02 2014
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 06 Oct 2014 00:58:02 +0300
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <5431B713.2060708@canterbury.ac.nz>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz>
Message-ID: <m0sett$jht$1@ger.gmane.org>

On 06.10.14 00:24, Greg Ewing wrote:
> anatoly techtonik wrote:
>> That's a cool stuff. `bytes-like object` is really a much better name
>> for users.
>
> I'm not so sure. Usually when we talk about an "xxx-like object" we
> mean one that supports a certain Python interface, e.g. a "file-like
> object" is one that has read() and/or write() methods. But you can't
> create an object that supports the buffer protocol by implementing
> Python methods.
>
> I'm worried that using the term "bytes-like object" will lead
> people to ask "What methods do I have to implement to make my
> object bytes-like?", to which the answer is "mu".

Other (rarely used) alternatives are "buffer-like object" and 
"buffer-compatible object".


From dw+python-dev at hmmz.org  Mon Oct  6 01:06:43 2014
From: dw+python-dev at hmmz.org (David Wilson)
Date: Sun, 5 Oct 2014 23:06:43 +0000
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <CAMpsgwaLt=vWh3haZnWKwTg6OCBwWkZqdtibGpHQUkYFpz-_dw@mail.gmail.com>
References: <20141005161158.9A52F250069@webabinitio.net>
 <CAMpsgwaLt=vWh3haZnWKwTg6OCBwWkZqdtibGpHQUkYFpz-_dw@mail.gmail.com>
Message-ID: <20141005230643.GA30429@k2>

On Sun, Oct 05, 2014 at 11:32:08PM +0200, Victor Stinner wrote:

> I'm not sure that the term has an unique definition. In some parts of
> Python, I saw explicit checks on the type: bytes or bytearray,
> sometimes memoryview is accepted. The behaviour is different in C
> functions using PyArg API. It probably depends if the function relies
> on the content (read bytes) or on methods (ex: call .find).

Buffers aren't "bytes like" in that many of them aren't immutable, even
when the buffer object itself is hashable. An example of this is the
buffer exposed by mmap.mmap() with MAP_SHARED.

This came up during the StringIO optimization changes a few months back,
it seems some oversight in the original design that got carried through
to Python 3. If we're approaching the topic of defining bytes-like
things with improved rigor, it might be worth discussing.


Making bytes-like objects an explicit thing in the language feels like
solidifying what is mostly a CPython implementation detail. For example,
at least until recently, PyPy emulated buffers in ways that often made
them much slower to use than regular bytes.

Another aspect is that the ability to twiddle bits derived from a
buffer mostly implies that the code you are calling is going to always
be written in C, or its future implementation will remain sufficiently
restricted as to always accept buffers (perhaps by first copying them to
bytes -- exactly what PyPy did until recently).


+1 on improving the notion of bytes-like things in Python, but not
necessarily by concretizing the existing interface.


David
> 
> Victor
> 

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/dw%2Bpython-dev%40hmmz.org


From rosuav at gmail.com  Mon Oct  6 05:13:02 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 6 Oct 2014 14:13:02 +1100
Subject: [Python-Dev] 3.4.2rc1 online docs: cannot access tkinter,
	idle pages
In-Reply-To: <m0s7ct$2r5$1@ger.gmane.org>
References: <m0s7ct$2r5$1@ger.gmane.org>
Message-ID: <CAPTjJmqEA-LsZviCx4TziZMeqeTf5ZYFasEbvde6crhRjKu2aQ@mail.gmail.com>

On Mon, Oct 6, 2014 at 6:49 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> I can access every page I have tried except for the tkinter and idle pages,
> which either give interminable 'Connecting...' or in one try, 404.
> https://docs.python.org/3/library/tkinter.html
> https://docs.python.org/3/library/idle.html

Is this still the case? I just tried them and they work for me.

ChrisA

From ncoghlan at gmail.com  Mon Oct  6 05:26:28 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Oct 2014 13:26:28 +1000
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <CAMpsgwaLt=vWh3haZnWKwTg6OCBwWkZqdtibGpHQUkYFpz-_dw@mail.gmail.com>
References: <20141005161158.9A52F250069@webabinitio.net>
 <CAMpsgwaLt=vWh3haZnWKwTg6OCBwWkZqdtibGpHQUkYFpz-_dw@mail.gmail.com>
Message-ID: <CADiSq7faaiWi96_iERwc+4XczziBMPPiFK=Qdz-inWScND3yeQ@mail.gmail.com>

On 6 October 2014 07:32, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> I prefer "bytes-like" than "buffer protocol". By the way, is there a
> documentation in Python doc which explains "bytes-like" and maybe list most
> compatible types?

https://docs.python.org/3/glossary.html#term-bytes-like-object

It only specifically lists bytes, bytearray and memoryview.

It also links to the current formal definition of what constitutes a
bytes like object:
https://docs.python.org/3/c-api/buffer.html#bufferobjects

memoryview.cast() offers a lot of capabilities to write interesting
buffer exporters in pure Python, but it's currently not possible as
there's no standard API defined for doing so:
http://bugs.python.org/issue13797

PyPy created a private Python level protocol to allow bytes-like
objects to be defined in RPython, but their approach hasn't been
proposed for standardisation (it's also not clear whether or not their
internal protocol offers the full range of features offered by PEP
3118).

> I'm not sure that the term has an unique definition. In some parts of
> Python, I saw explicit checks on the type: bytes or bytearray, sometimes
> memoryview is accepted. The behaviour is different in C functions using
> PyArg API. It probably depends if the function relies on the content (read
> bytes) or on methods (ex: call .find).

The core practical definition of a "bytes-like object" is whether or
not "memoryview(obj)" works. An object can be bytes like without
supporting the full bytes/bytearray API.

Some APIs that theoretically *could* accept arbitrary bytes like
objects are currently more limited - expanding those cases to accept
arbitrary bytes-like objects is generally considered a minor RFE
rather than a bug fix.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From greg.ewing at canterbury.ac.nz  Mon Oct  6 02:15:58 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 06 Oct 2014 13:15:58 +1300
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <m0sett$jht$1@ger.gmane.org>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz> <m0sett$jht$1@ger.gmane.org>
Message-ID: <5431DF3E.9000500@canterbury.ac.nz>

I wrote:

>> But you can't
>> create an object that supports the buffer protocol by implementing
>> Python methods.

Another thing is that an object implementing the buffer
interface doesn't have to look anything at all like a
bytes object from Python, so calling it "bytes-like"
could be rather confusing.

-- 
Greg

From ncoghlan at gmail.com  Mon Oct  6 05:36:08 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Oct 2014 13:36:08 +1000
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <5431B713.2060708@canterbury.ac.nz>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz>
Message-ID: <CADiSq7dgU2_VipFFawino0Qo4fpgjRNXf4vx7NWWO99od9HKcA@mail.gmail.com>

On 6 October 2014 07:24, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> anatoly techtonik wrote:
>>
>> That's a cool stuff. `bytes-like object` is really a much better name for
>> users.
>
>
> I'm not so sure. Usually when we talk about an "xxx-like object" we
> mean one that supports a certain Python interface, e.g. a "file-like
> object" is one that has read() and/or write() methods. But you can't
> create an object that supports the buffer protocol by implementing
> Python methods.
>
> I'm worried that using the term "bytes-like object" will lead
> people to ask "What methods do I have to implement to make my
> object bytes-like?", to which the answer is "mu".

That's a defect in the language definition, which requires volunteers
willing and able to work through the task of defining and gathering
consensus around a Python level counterpart to PEP 3118 that works
across arbitrary Python implementations:
http://bugs.python.org/issue13797

Ideally the drive for this would come from the Cython, PyPy,
IronPython and Jython communities, as they would all likely benefit
from a C independent way to express support for the buffer protocol.
Enabling this level of flexibility in defining bytes-like objects is
likely to be a key step in extending deep array oriented programming
support from CPython to other Python implementations.

In the meantime, the answer to "How do I define a bytes-like type?" is:

1. Use the appropriate implementation specific buffer protocols; or
2. Inherit from an existing bytes-like type

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Mon Oct  6 06:34:47 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 6 Oct 2014 14:34:47 +1000
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <5431DF3E.9000500@canterbury.ac.nz>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz> <m0sett$jht$1@ger.gmane.org>
 <5431DF3E.9000500@canterbury.ac.nz>
Message-ID: <CADiSq7f2GhO24VNMR=6F7KfW6xfFjnRDsYkn8HhokDRMjco6cQ@mail.gmail.com>

On 6 October 2014 10:15, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> I wrote:
>
>>> But you can't
>>> create an object that supports the buffer protocol by implementing
>>> Python methods.
>
>
> Another thing is that an object implementing the buffer
> interface doesn't have to look anything at all like a
> bytes object from Python, so calling it "bytes-like"
> could be rather confusing.

"bytes-like object" is already used that way - e.g. memoryview is a
bytes-like object, as is array.arrray. Even numpy.ndarray, PIL images,
etc can all be accessed as bytes-like objects.

The term itself isn't new - we've been using it to progressively
eliminate awkward docs references to "objects that support the buffer
protocol" for years. David's post was just to say that the process of
adopting it is now largely complete, as the exception messages that
mentioned the buffer protocol have also been updated.

As near as I can figure out, some of the the reasons it appears to
work better than relying on the "buffer" term include:

1. Anyone learning Python 3 will know "bytes", and "A bytes-like
object is one that can be treated like a bytes object by adapting it
with memoryview" is a relatively simple step to take (although we
could likely be more explicit about that in the glossary entry).
2. When reporting errors, it conveys more clearly that passing a bytes
object will work, since the assumption that "bytes instances are bytes
like objects" is both obvious and correct. It also isn't too hard to
figure out that "str instances are not bytes like objects". With those
two figured out as category anchors, it becomes easier to start
grouping other types by comparing them with the two archetypal
examples. By contrast, is not *at all* obvious why bytes supports the
buffer protocol, while str does not.
3. "buffer" is a completely new term for most users, and one that
refers to an implementation detail of memoryview, moreso than
something developers actually need to care about. Using it directly in
error messages and documentation is to make the abstraction leak in a
way that raises unnecessary barriers to entry.
4. As a term, "buffer" in Python 3 also suffers from meaning something
*different* from what the buffer builtin refers to in Python 2 (the
fact str implemented the buffer protocol in Python 2 doesn't help).

In many way, it's similar to "file-like object" - those vary wildly in
how much of the file API they support, based on underlying technical
details (like whether it's backed by a file descriptor or not, whether
it's a binary or text stream, whether it's seekable, etc). However, by
saying "file-like object", we make the interface easy to learn to use
productively, as there are some obvious immediate inclusions (like the
actual file objects returned by open()) and exclusions (like strings
and bytes objects). Navigating the grey area can then be postponed
until later (and for many users, it will never be necessary to
navigate it at all).

If anything, bytes-like object is *better* defined than file-like
object, since the one thing that is expected to work for *any*
bytes-like object is "raw_data = memoryview(obj)". Everything beyond
that is negotiable (including the rest of the bytes API, and whether
or not the object is immutable or not).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Mon Oct  6 08:17:39 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 06 Oct 2014 02:17:39 -0400
Subject: [Python-Dev] 3.4.2rc1 online docs: cannot access tkinter,
	idle pages
In-Reply-To: <CAPTjJmqEA-LsZviCx4TziZMeqeTf5ZYFasEbvde6crhRjKu2aQ@mail.gmail.com>
References: <m0s7ct$2r5$1@ger.gmane.org>
 <CAPTjJmqEA-LsZviCx4TziZMeqeTf5ZYFasEbvde6crhRjKu2aQ@mail.gmail.com>
Message-ID: <m0tc64$6nq$1@ger.gmane.org>

On 10/5/2014 11:13 PM, Chris Angelico wrote:
> On Mon, Oct 6, 2014 at 6:49 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I can access every page I have tried except for the tkinter and idle pages,
>> which either give interminable 'Connecting...' or in one try, 404.
>> https://docs.python.org/3/library/tkinter.html
>> https://docs.python.org/3/library/idle.html
>
> Is this still the case? I just tried them and they work for me.

Me too, now.

-- 
Terry Jan Reedy


From g.brandl at gmx.net  Mon Oct  6 10:33:03 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 06 Oct 2014 10:33:03 +0200
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <CADiSq7f2GhO24VNMR=6F7KfW6xfFjnRDsYkn8HhokDRMjco6cQ@mail.gmail.com>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz> <m0sett$jht$1@ger.gmane.org>
 <5431DF3E.9000500@canterbury.ac.nz>
 <CADiSq7f2GhO24VNMR=6F7KfW6xfFjnRDsYkn8HhokDRMjco6cQ@mail.gmail.com>
Message-ID: <m0tk40$2jq$1@ger.gmane.org>

On 10/06/2014 06:34 AM, Nick Coghlan wrote:

> 3. "buffer" is a completely new term for most users, and one that
> refers to an implementation detail of memoryview, moreso than
> something developers actually need to care about. Using it directly in
> error messages and documentation is to make the abstraction leak in a
> way that raises unnecessary barriers to entry.

Especially since buffer() is gone in Py3.

The only prominent mention of "buffer" in the docs is now the C API section
that explains the buffer protocol.  Since this is cross-referenced
from the "bytes-like object" glossary, it will lead people who want to
write such an object to the right place.

Georg


From tismer at stackless.com  Mon Oct  6 15:59:59 2014
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 06 Oct 2014 15:59:59 +0200
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <m0tk40$2jq$1@ger.gmane.org>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz> <m0sett$jht$1@ger.gmane.org>
 <5431DF3E.9000500@canterbury.ac.nz>
 <CADiSq7f2GhO24VNMR=6F7KfW6xfFjnRDsYkn8HhokDRMjco6cQ@mail.gmail.com>
 <m0tk40$2jq$1@ger.gmane.org>
Message-ID: <5432A05F.1050501@stackless.com>

On 06/10/14 10:33, Georg Brandl wrote:
> bytes-like object

Howdy,

two small comments:

1)
just as a quick check, I did a Python search for exactly
that phrase

https://docs.python.org/3/search.html?q=bytes-like+object&check_keywords=yes&area=default

with zero results.

Maybe it would be a good thing if glossary entries were always
found via the search page.

2)
And about this glossary entry:

"An object that supports the Buffer Protocol"
- can I take that for granted, as a real definition, meaning

"""an object is bytes-like iff it supports the buffer protocol"""?

cheers - Chris

-- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     http://www.pydica.net/
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023

From ethan at stoneleaf.us  Mon Oct  6 18:08:23 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 06 Oct 2014 09:08:23 -0700
Subject: [Python-Dev] Fixing 2.7.x
Message-ID: <5432BE77.40507@stoneleaf.us>

With the incredibly long life span of 2.7, which bugs should we *not* fix?

For example, in http://bugs.python.org/issue22297 I mentioned one reason to not fix that bug was that the fix was not in 
3.1-3.3, but 2.7 will outlive all those plus a couple more.

So, what are the current guidelines on what to fix?  Is it still security only, with the rest being carrots for switching?

--
~Ethan~

From solipsis at pitrou.net  Mon Oct  6 18:21:19 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 6 Oct 2014 18:21:19 +0200
Subject: [Python-Dev] Fixing 2.7.x
References: <5432BE77.40507@stoneleaf.us>
Message-ID: <20141006182119.6a6806da@fsol>

On Mon, 06 Oct 2014 09:08:23 -0700
Ethan Furman <ethan at stoneleaf.us> wrote:
> With the incredibly long life span of 2.7, which bugs should we *not* fix?*

Those that are not bugs but enhancement requests.
On that issue, you pointed out there was no regression and that enums
were never meant to be supported by the json module at that time, so
IMHO it's a won't fix.

Regards

Antoine.



From nad at acm.org  Mon Oct  6 19:24:07 2014
From: nad at acm.org (Ned Deily)
Date: Mon, 06 Oct 2014 10:24:07 -0700
Subject: [Python-Dev] Fixing 2.7.x
References: <5432BE77.40507@stoneleaf.us>
Message-ID: <nad-0134B3.10240706102014@news.gmane.org>

In article <5432BE77.40507 at stoneleaf.us>,
 Ethan Furman <ethan at stoneleaf.us> wrote:
> With the incredibly long life span of 2.7, which bugs should we *not* fix?
> 
> For example, in http://bugs.python.org/issue22297 I mentioned one reason to 
> not fix that bug was that the fix was not in 
> 3.1-3.3, but 2.7 will outlive all those plus a couple more.
> 
> So, what are the current guidelines on what to fix?  Is it still security 
> only, with the rest being carrots for switching?

Python release families are in one of four states:

1. in-development feature: the default branch, unreleased
   = 3.5

2. maintenance: currently released and actively maintained, bug fixes, 
no compatibility breaks, no new features without very compelling use 
cases, discussion, and release manager approval.
   = 2.7.x and 3.4.x

3. security: "fixing issues exploitable by attackers such as crashes, 
privilege escalation and, optionally, other issues such as denial of 
service attacks. Any other changes are not considered a security risk 
and thus not backported to a security branch."
   = 3.2.x and 3.3.x

4. retired: no fixes of any kind from python-dev
   = all other releases

So 2.7.x is not "security only" and wouldn't reach that stage until 2020 
under current policy.

http://legacy.python.org/dev/peps/pep-0373/#id5

https://docs.python.org/devguide/devcycle.html#branches

-- 
 Ned Deily,
 nad at acm.org


From skip.montanaro at gmail.com  Mon Oct  6 19:33:18 2014
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 6 Oct 2014 12:33:18 -0500
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <nad-0134B3.10240706102014@news.gmane.org>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
Message-ID: <CANc-5UyJ5ZkB8Mxha-80+BvREVYT745L16y8EuQ7PhdOm1A7Qw@mail.gmail.com>

On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
> So 2.7.x is not "security only" and wouldn't reach that stage until 2020
> under current policy.

Apparently no other 2.x release qualifies as "security only" at this
point? I would have expected at least 2.6 to fall into that category.

Skip

From rosuav at gmail.com  Mon Oct  6 19:38:45 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 7 Oct 2014 04:38:45 +1100
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <CANc-5UyJ5ZkB8Mxha-80+BvREVYT745L16y8EuQ7PhdOm1A7Qw@mail.gmail.com>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CANc-5UyJ5ZkB8Mxha-80+BvREVYT745L16y8EuQ7PhdOm1A7Qw@mail.gmail.com>
Message-ID: <CAPTjJmqEDqYOeH4KMe=u-SwsG_Vw1ReMbfZK8XFB3StJT_dhwQ@mail.gmail.com>

On Tue, Oct 7, 2014 at 4:33 AM, Skip Montanaro <skip.montanaro at gmail.com> wrote:
> On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
>> So 2.7.x is not "security only" and wouldn't reach that stage until 2020
>> under current policy.
>
> Apparently no other 2.x release qualifies as "security only" at this
> point? I would have expected at least 2.6 to fall into that category.

Was until about a year ago.

http://legacy.python.org/dev/peps/pep-0361/

ChrisA

From nad at acm.org  Mon Oct  6 19:38:54 2014
From: nad at acm.org (Ned Deily)
Date: Mon, 06 Oct 2014 10:38:54 -0700
Subject: [Python-Dev] Fixing 2.7.x
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CANc-5UyJ5ZkB8Mxha-80+BvREVYT745L16y8EuQ7PhdOm1A7Qw@mail.gmail.com>
Message-ID: <nad-D2A101.10385306102014@news.gmane.org>

In article 
<CANc-5UyJ5ZkB8Mxha-80+BvREVYT745L16y8EuQ7PhdOm1A7Qw at mail.gmail.com>,
 Skip Montanaro <skip.montanaro at gmail.com> wrote:
> Apparently no other 2.x release qualifies as "security only" at this
> point? I would have expected at least 2.6 to fall into that category.

2.6 had its five-year run.

"Python 2.6.9 is the final security-only source-only maintenance
 release of the Python 2.6 series.  With its release on October 29,
 2013, all official support for Python 2.6 has ended.  Python 2.6
 is no longer being maintained for any purpose."

http://legacy.python.org/dev/peps/pep-0361/

-- 
 Ned Deily,
 nad at acm.org


From zachary.ware+pydev at gmail.com  Mon Oct  6 20:55:42 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 6 Oct 2014 13:55:42 -0500
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <nad-0134B3.10240706102014@news.gmane.org>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
Message-ID: <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>

On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
> 3. security: "fixing issues exploitable by attackers such as crashes,
> privilege escalation and, optionally, other issues such as denial of
> service attacks. Any other changes are not considered a security risk
> and thus not backported to a security branch."
>    = 3.2.x and 3.3.x

3.1 is still in this category, is it not?  According to PEP 375, it's
a few months past due for its last release.

http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases

-- 
Zach

From tismer at stackless.com  Mon Oct  6 21:18:23 2014
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 06 Oct 2014 21:18:23 +0200
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
Message-ID: <5432EAFF.5000202@stackless.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 06.10.14 20:55, Zachary Ware wrote:
> On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
>> 3. security: "fixing issues exploitable by attackers such as crashes,
>> privilege escalation and, optionally, other issues such as denial of
>> service attacks. Any other changes are not considered a security risk
>> and thus not backported to a security branch."
>>    = 3.2.x and 3.3.x
>
> 3.1 is still in this category, is it not?  According to PEP 375, it's
> a few months past due for its last release.
>
> http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases
>

I don't think that the rules should be implicitly considered
compatible between the 2.X and 3.X series.

Python 2.X has a history that extends to X==6, X==5 and
even X==4, as a really conservative POV with an extent over more
than 10 years.

I believe, such a thing does not exist for the Python 3.X series
at all. My impression is that no 3.X user ever would want to stick
with any older version.

Is that true, or am I totally wrong?

cheers -- Chris

- -- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     http://www.pydica.net/
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - https://gpgtools.org

iQEcBAEBCgAGBQJUMur7AAoJEOcwEVD7e+4OW0UIAJk2ZTFX3OjBrt1G0RZc9nPb
hHVxGJNXNeBellM9BpmoW9t9hgk94lAIgmh5hop5uMt32o9CH47s97rKw7K1ekl5
sML/4hl5/BLRiHXgwSB1ZltqZrvG/xsE6AE1v37BcPf/X3T4UfPhW30z+43eaBJw
Q3b21EwxxUJGJ//GWwi2+buCfkfRuePBIB4MQiMm3/JI9h03EPbRoQ0/53huKLeW
I7oAemVzprQHw7coaTf6EOHFTlmUfHvm5K9ywpabX10/Ediz1suJfPMPdzUigHG3
G3ABMKAA1YockpfIDU/7rpmDcliblpjU5MT4BsZycuYXOyUesV6uDaLMOdO5fEY=
=ZuoT
-----END PGP SIGNATURE-----



From zachary.ware+pydev at gmail.com  Mon Oct  6 21:35:27 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 6 Oct 2014 14:35:27 -0500
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <5432EAFF.5000202@stackless.com>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
 <5432EAFF.5000202@stackless.com>
Message-ID: <CAKJDb-MP02fWhASiwzQhYYOxZ8tZ2puZ+4zLFJUOZ+K4Nx_g+w@mail.gmail.com>

On Mon, Oct 6, 2014 at 2:18 PM, Christian Tismer <tismer at stackless.com> wrote:
> My impression is that no 3.X user ever would want to stick
> with any older version.
>
> Is that true, or am I totally wrong?

My impression is that you're mostly right, but only because those who
would still be on 3.1 are actually still on 2.5 ;).  However as one
who uses 3.x exclusively, I do still use 3.2 on a regular basis simply
because it's the last release to support Windows 2000 and it does
everything I need to do on that platform.

Either way, that seems a bit tangential to my original point, which
was that 3.1 is still technically an open security branch which is due
to be closed (with or without it's last security release, which was
due in June).

-- 
Zach

From benjamin at python.org  Mon Oct  6 21:37:02 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 06 Oct 2014 15:37:02 -0400
Subject: [Python-Dev] Semi-official read-only Github mirror of the
 CPython Mercurial repository
In-Reply-To: <CAF-Rda-CdXM7Par=skFfwuD1YgctvrZ49d+FCr9Rq4aQNa32Yw@mail.gmail.com>
References: <CAF-Rda__36CpmDaVxLWo+iWhw5x9SqNFGjoQFXWM591PGNOzhg@mail.gmail.com>
 <CAF-Rda-CdXM7Par=skFfwuD1YgctvrZ49d+FCr9Rq4aQNa32Yw@mail.gmail.com>
Message-ID: <1412624222.1694450.175812217.20165796@webmail.messagingengine.com>

Eli,
Thanks for setting this up. People are evidently finding it quite useful
and are wondering if it could be more frequently run. We don't want you
to have to absorb the bandwidth costs yourself, though. Is the code you
use available somewhere? It shouldn't be hard to set up the cron job on
PSF infrastructure.

On Sat, Jul 12, 2014, at 09:15, Eli Bendersky wrote:
> Just a quick update on this. I've finally found time to set up a VPS at
> DigitalOcean of myself, and I'm moving the cronjob for updating the
> Github
> mirrors to it. This lets me ramp up the update frequency. For now I'll
> set
> it to every 4 hours, but in the future I may make it even more frequent.
> Hopefully this will not overrun my bandwidth allocation :)
> 
> The CPython mirror (https://github.com/python/cpython) has been pretty
> popular so far, with over 70 forks.
> 
> Eli
> 
> 
> 
> On Mon, Sep 30, 2013 at 6:09 AM, Eli Bendersky <eliben at gmail.com> wrote:
> 
> > Hi all,
> >
> > https://github.com/python/cpython is now live as a semi-official, *read
> > only* Github mirror of the CPython Mercurial repository. Let me know if you
> > have any problems/concerns.
> >
> > I still haven't decided how often to update it (considering either just N
> > times a day, or maybe use a Hg hook for batching). Suggestions are welcome.
> >
> > The methodology I used to create it is via hg-fast-export. I also tried to
> > pack and gc the git repo as much as possible before the initial Github push
> > - it went down from almost ~2GB to ~200MB (so this is the size of a fresh
> > clone right now).
> >
> > Eli
> >
> > P.S. thanks Jesse for the keys to https://github.com/python
> >
> >
> >
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/benjamin%40python.org

From rdmurray at bitdance.com  Mon Oct  6 21:45:52 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 06 Oct 2014 15:45:52 -0400
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <5432EAFF.5000202@stackless.com>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
 <5432EAFF.5000202@stackless.com>
Message-ID: <20141006194553.370152500B5@webabinitio.net>

On Mon, 06 Oct 2014 21:18:23 +0200, Christian Tismer <tismer at stackless.com> wrote:
> On 06.10.14 20:55, Zachary Ware wrote:
> > On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
> >> 3. security: "fixing issues exploitable by attackers such as crashes,
> >> privilege escalation and, optionally, other issues such as denial of
> >> service attacks. Any other changes are not considered a security risk
> >> and thus not backported to a security branch."
> >>    = 3.2.x and 3.3.x
> >
> > 3.1 is still in this category, is it not?  According to PEP 375, it's
> > a few months past due for its last release.
> >
> > http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases
> >
> 
> I don't think that the rules should be implicitly considered
> compatible between the 2.X and 3.X series.
> 
> Python 2.X has a history that extends to X==6, X==5 and
> even X==4, as a really conservative POV with an extent over more
> than 10 years.
> 
> I believe, such a thing does not exist for the Python 3.X series
> at all. My impression is that no 3.X user ever would want to stick
> with any older version.
> 
> Is that true, or am I totally wrong?

I don't think you are *totally* wrong, but I don't think you are really
right, either.  I myself have at least one system (I didn't check them
all) running 3.2 that I have intentionally only done security fixes on
rather than upgrade python3.  It's mostly laziness (given that my distro
provides the security updates), since I don't think there would be any
compatibility problems with the few python3 programs running on it, but
I'm likely not the only one.

So yes, the same rules should apply to python3 as apply to python2,
especially since more distros are about to start shipping python3 as
the system python (Arch has been since 2011).

3.1, however, is most likely a dead issue.

--David

From victor.stinner at gmail.com  Mon Oct  6 22:03:34 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 6 Oct 2014 22:03:34 +0200
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <5432BE77.40507@stoneleaf.us>
References: <5432BE77.40507@stoneleaf.us>
Message-ID: <CAMpsgwYG4VjR_DXQXxJtMnKmMVH2y_P4Y_SoFgbkRxQ_KG6Rjw@mail.gmail.com>

Hi,

2014-10-06 18:08 GMT+02:00 Ethan Furman <ethan at stoneleaf.us>:
> With the incredibly long life span of 2.7, which bugs should we *not* fix?

I started a list of Python 2 bugs that will not be fixed:
http://haypo-notes.readthedocs.org/python.html#bugs-that-won-t-be-fixed-in-python-2-anymore

It *is* possible to fix all bugs, but it requires a large amount of
work, and we decided to focus our energy on Python 3.

Victor

From brianvanderburg2 at aim.com  Mon Oct  6 22:36:01 2014
From: brianvanderburg2 at aim.com (Brian Allen Vanderburg II)
Date: Mon, 6 Oct 2014 16:36:01 -0400
Subject: [Python-Dev] Idea to support lazy loaded names.
Message-ID: <20141006163601.233e71f9@allen-pc>

I've got an idea for Python but don't know where to post it.  It seems
like this may be the right place.

When developing Python libraries (and any other language for that
matter), items are separated into modules for organization and other
purpose.  The following is an example fake framework to illustrate this:

  mini-framework/
    __init__.py
    app.py
      class BaseApplication
      class Application
      class ProxyApplication
    config.py
      class Config
      class ConfigLoader
    template.py
      class Template
      class TemplateCompiler
      class TemplateManager

In order to be used externally, one must reference the module:

  from miniframework.app import Application
  from miniframework.template import TemplateManager

This can be solved by importing these values directly in __init__.py.
However this requires fully loading those imported modules:

  __init__.py:
    from .app import BaseApplication, Application, ProxyApplication
    from .config import Config, ConfigLoader
    from .template import Template, TemplateCompiler, TemplateManager
  
One idea I had was to support lazy imported names.  I've seen some
frameworks implement this in various means, and figured the idea is good
to be part Python.

The idea is that for a new module attribute to exist: __lazyload__.
During the access of any attribute for a module, the following would
happen:

  * If the named attribute already exists, use it
  * If the named attribute does not already exist:
    * If a lazy load of the name has already been attempted, result in
      a NameError
    * If a lazy load of the name has not yet been attempted:
       * Check the __lazyload__ module attribute for the name, perform
         the loading operation and assign the module attribute the
         value if found or result in a NameError
       * Even if not found, set a flag that lazy load has been
         attempted so it will not be attempted again for the same name

The __lazyload__ attribute is intended to be a dictionary.  The key of
the dictionary is the name of the attribute that would be set/tested
for in the module.  The value of the dictionary is a string that
represents either the module name to load or the module name and
attribute to load.  If the value starts with a dot, then it is treated
as a relative import relative to the module/package containing the
__lazyload__ value.

With this idea, the packages __init__.py file could look like this:

  __lazyload__ = {
    'BaseApplication': '.app.BaseApplication',
    'Application': '.app.Application',
    ...
  }
       
The end use of the package (and even the developer) can then perform an
import as follows:

  from miniframework import Application

instead of:

  from miniframework.app import Application

This allows the public api be be cleaner, while still being efficient
by not loading all modules in __init__.py until the value is actually
accessed.



Brian Allen Vanderburg II

From benjamin at python.org  Mon Oct  6 22:49:28 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 06 Oct 2014 16:49:28 -0400
Subject: [Python-Dev] Idea to support lazy loaded names.
In-Reply-To: <20141006163601.233e71f9@allen-pc>
References: <20141006163601.233e71f9@allen-pc>
Message-ID: <1412628568.1715711.175842577.666D620B@webmail.messagingengine.com>

On Mon, Oct 6, 2014, at 16:36, Brian Allen Vanderburg II wrote:
> I've got an idea for Python but don't know where to post it.

python-ideas at python.org

From mcepl at cepl.eu  Mon Oct  6 23:26:42 2014
From: mcepl at cepl.eu (=?UTF-8?Q?Mat=C4=9Bj?= Cepl)
Date: Mon, 6 Oct 2014 23:26:42 +0200
Subject: [Python-Dev] Fixing 2.7.x
References: <5432BE77.40507@stoneleaf.us>
 <CAMpsgwYG4VjR_DXQXxJtMnKmMVH2y_P4Y_SoFgbkRxQ_KG6Rjw@mail.gmail.com>
Message-ID: <slrnm3628i.n3t.mcepl@wycliff.ceplovi.cz>

On 2014-10-06, 20:03 GMT, Victor Stinner wrote:
> I started a list of Python 2 bugs that will not be fixed:
> http://haypo-notes.readthedocs.org/python.html\
> #bugs-that-won-t-be-fixed-in-python-2-anymore
>
> It *is* possible to fix all bugs, but it requires a large amount of
> work, and we decided to focus our energy on Python 3.

What about bugs which have patch already devleoped for both py3k 
and py2k. Hint: I would love to have some movement on 
http://bugs.python.org/issue19494 :)

Best,

Mat?j


From nad at acm.org  Tue Oct  7 01:13:29 2014
From: nad at acm.org (Ned Deily)
Date: Mon, 06 Oct 2014 16:13:29 -0700
Subject: [Python-Dev] Fixing 2.7.x
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
Message-ID: <nad-D3476C.16132906102014@news.gmane.org>

In article 
<CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g at mail.gmail.com>,
 Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
> > 3. security: "fixing issues exploitable by attackers such as crashes,
> > privilege escalation and, optionally, other issues such as denial of
> > service attacks. Any other changes are not considered a security risk
> > and thus not backported to a security branch."
> >    = 3.2.x and 3.3.x
> 3.1 is still in this category, is it not?  According to PEP 375, it's
> a few months past due for its last release.
> http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases

I don't think Benjamin was planning any further security releases.  But, 
in any case, the PEP should be updated, I guess.  Benjamin?

-- 
 Ned Deily,
 nad at acm.org


From benjamin at python.org  Tue Oct  7 01:32:48 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 06 Oct 2014 19:32:48 -0400
Subject: [Python-Dev] Fixing 2.7.x
In-Reply-To: <nad-D3476C.16132906102014@news.gmane.org>
References: <5432BE77.40507@stoneleaf.us>
 <nad-0134B3.10240706102014@news.gmane.org>
 <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g@mail.gmail.com>
 <nad-D3476C.16132906102014@news.gmane.org>
Message-ID: <1412638368.1757115.175894093.4F4D9E14@webmail.messagingengine.com>

On Mon, Oct 6, 2014, at 19:13, Ned Deily wrote:
> In article 
> <CAKJDb-P6nx5szhBeuxv0fWSJ+-5=U1cD6aY--eD2YqeWwRK4-g at mail.gmail.com>,
>  Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> > On Mon, Oct 6, 2014 at 12:24 PM, Ned Deily <nad at acm.org> wrote:
> > > 3. security: "fixing issues exploitable by attackers such as crashes,
> > > privilege escalation and, optionally, other issues such as denial of
> > > service attacks. Any other changes are not considered a security risk
> > > and thus not backported to a security branch."
> > >    = 3.2.x and 3.3.x
> > 3.1 is still in this category, is it not?  According to PEP 375, it's
> > a few months past due for its last release.
> > http://legacy.python.org/dev/peps/pep-0375/#maintenance-releases
> 
> I don't think Benjamin was planning any further security releases.  But, 
> in any case, the PEP should be updated, I guess.  Benjamin?

Oh, yeah, I'm really tired of 3.1, so it's over.

From ncoghlan at gmail.com  Tue Oct  7 10:59:47 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 7 Oct 2014 18:59:47 +1000
Subject: [Python-Dev] bytes-like objects
In-Reply-To: <5432A05F.1050501@stackless.com>
References: <20141005161158.9A52F250069@webabinitio.net>
 <5431790C.1000702@gmx.net>
 <CAPkN8xJLMFbnG-dr9-Bqw=JHCUpAPrxi9FxQb4bzwoeSiinvyQ@mail.gmail.com>
 <5431B713.2060708@canterbury.ac.nz> <m0sett$jht$1@ger.gmane.org>
 <5431DF3E.9000500@canterbury.ac.nz>
 <CADiSq7f2GhO24VNMR=6F7KfW6xfFjnRDsYkn8HhokDRMjco6cQ@mail.gmail.com>
 <m0tk40$2jq$1@ger.gmane.org> <5432A05F.1050501@stackless.com>
Message-ID: <CADiSq7cWneGDBhSC9-OaJo7=q0qUeyfHcJngDSFyvT4ZTUhaEA@mail.gmail.com>

On 7 Oct 2014 00:31, "Christian Tismer" <tismer at stackless.com> wrote:

>
> 2)
> And about this glossary entry:
>
> "An object that supports the Buffer Protocol"
> - can I take that for granted, as a real definition, meaning
>
> """an object is bytes-like iff it supports the buffer protocol"""?

Yes, although we should likely update it to say the requirement is for
"memoryview(obj)" to work, with the C level buffer protocol being the only
current way to support that on CPython.

Cheers,
Nick.

>
> cheers - Chris
>
> --
> Christian Tismer             :^)   tismer at stackless.com
> Software Consulting          :     http://www.stackless.com/
> Karl-Liebknecht-Str. 121     :     http://www.pydica.net/
> 14482 Potsdam                :     GPG key -> 0xFB7BEE0E
> phone +49 173 24 18 776  fax +49 (30) 700143-0023
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/620564e6/attachment.html>

From larry at hastings.org  Tue Oct  7 11:48:53 2014
From: larry at hastings.org (Larry Hastings)
Date: Tue, 07 Oct 2014 02:48:53 -0700
Subject: [Python-Dev] 3.4.2 is slipping by a day
Message-ID: <5433B705.5000106@hastings.org>



We don't have Windows installers yet.  If we can't get 'em in the next 
24 hours or so, I'll release without them.  But for now... we wait.

Twiddling my thumbs,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/4b12b3f3/attachment.html>

From tshawver at quantopian.com  Tue Oct  7 17:41:12 2014
From: tshawver at quantopian.com (tshawver)
Date: Tue, 7 Oct 2014 08:41:12 -0700 (PDT)
Subject: [Python-Dev] Interactive Grid for Sorting,
 Filtering DataFrames in IPython Notebook
Message-ID: <1412696472668-5073931.post@n6.nabble.com>

As part of the work on our research environment at  Quantopian
<https://www.quantopian.com/research>  , I've been building an extension
which renders pandas DataFrames as interactive grids in the IPython
notebook.  The extension uses a Javascript library called SlickGrid to
render the grids, and the current state of the project can be found here on
GitHub: https://github.com/quantopian/qgrid

The extension is also viewable in nbviewer:
http://nbviewer.ipython.org/github/quantopian/qgrid/blob/master/qgrid_demo.ipynb

The GitHub repository contains some explanation of why I chose to implement
the grid as a Python package rather than a standard IPython notebook
extension, which might be interesting for other people who are looking to
add functionality to the notebook in a similar way.

-Tim



--
View this message in context: http://python.6.x6.nabble.com/Interactive-Grid-for-Sorting-Filtering-DataFrames-in-IPython-Notebook-tp5073931.html
Sent from the Python - python-dev mailing list archive at Nabble.com.

From brett at python.org  Tue Oct  7 17:55:37 2014
From: brett at python.org (Brett Cannon)
Date: Tue, 07 Oct 2014 15:55:37 +0000
Subject: [Python-Dev] Interactive Grid for Sorting,
 Filtering DataFrames in IPython Notebook
References: <1412696472668-5073931.post@n6.nabble.com>
Message-ID: <CAP1=2W7bJNas+o4PF2D21-5_kM1PSzF3Gdm5aZTRMjwaRNwgmA@mail.gmail.com>

Python-dev is for discussing the development *of* Python, not *with* it.
This kind of thing is more appropriate for python-list.

On Tue Oct 07 2014 at 11:49:37 AM tshawver <tshawver at quantopian.com> wrote:

> As part of the work on our research environment at  Quantopian
> <https://www.quantopian.com/research>  , I've been building an extension
> which renders pandas DataFrames as interactive grids in the IPython
> notebook.  The extension uses a Javascript library called SlickGrid to
> render the grids, and the current state of the project can be found here on
> GitHub: https://github.com/quantopian/qgrid
>
> The extension is also viewable in nbviewer:
> http://nbviewer.ipython.org/github/quantopian/qgrid/blob/
> master/qgrid_demo.ipynb
>
> The GitHub repository contains some explanation of why I chose to implement
> the grid as a Python package rather than a standard IPython notebook
> extension, which might be interesting for other people who are looking to
> add functionality to the notebook in a similar way.
>
> -Tim
>
>
>
> --
> View this message in context: http://python.6.x6.nabble.com/
> Interactive-Grid-for-Sorting-Filtering-DataFrames-in-
> IPython-Notebook-tp5073931.html
> Sent from the Python - python-dev mailing list archive at Nabble.com.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/7eb7f9c2/attachment.html>

From sansprivacy at gmail.com  Tue Oct  7 19:46:16 2014
From: sansprivacy at gmail.com (John Smith)
Date: Tue, 7 Oct 2014 12:46:16 -0500
Subject: [Python-Dev] performance delta with .py presence v.s. only pyc on
	python 2.7.x?
Message-ID: <CAB6HS2V48Pv0sqjfSrQgY9KKyOhTJf0ouzRpo89rWJT_09x+1Q@mail.gmail.com>

My team has a python app that appears to perform 10+% better when the
python source is present v.s. when only the pyc's are distributed.  This is
contradictory to my understanding of how python leverages pyc's.

Scenario

1. pyc-only install sees mediocre performance. (pyc's are built using
compileall.py, then source .py's removed before packaging)
2. adding the .py files to the installed filesystem results in 10+%
performance improvement
3. verified no difference in mod times, no new .pyc's generated
4. remove the .py's and again see drop in performance


Env info: centos 6.5, x86_64, however python used is 32bit from the i686
repos (this is how we deal with some 32bit only c libraries that the app
must use)


Has anyone got an idea of what I might be running into?  The app itself is
a black box to me.  I was not able to isolate this using a simple piece of
python code that worked on a math problem for a discernible amount of time,
so it could be something to do with how the app is using python.  The app
was originally written for python 2.4...

Any suggestions on what I can try is appreciated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/252b869d/attachment.html>

From skip.montanaro at gmail.com  Tue Oct  7 20:22:35 2014
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 7 Oct 2014 13:22:35 -0500
Subject: [Python-Dev] performance delta with .py presence v.s. only pyc
 on python 2.7.x?
In-Reply-To: <CAB6HS2V48Pv0sqjfSrQgY9KKyOhTJf0ouzRpo89rWJT_09x+1Q@mail.gmail.com>
References: <CAB6HS2V48Pv0sqjfSrQgY9KKyOhTJf0ouzRpo89rWJT_09x+1Q@mail.gmail.com>
Message-ID: <CANc-5UwZ-FuH4+=Q4SdYPqYiStWvhSCo+QJ9g1MkyQ9_4tgp2A@mail.gmail.com>

On Tue, Oct 7, 2014 at 12:46 PM, John Smith <sansprivacy at gmail.com> wrote:
> pyc-only install sees mediocre performance. (pyc's are built using
> compileall.py, then source .py's removed before packaging)

(Warning: it's been probably a decade since I looked at any of this
stuff, so treat this response as mere conjecture.)

I'd take a look at startup time and things like stat(2) calls in the
presence or absence of .py files. It's possible that it tries all
other possible file extensions before considering .pyc. If you have a
long sys.path, it would then run through all the other file extension
possibilities before trying the .pyc. OTOH, if the .py is present, it
might be found early in the search, then as an optimization, look for
a .pyc file it can use rather than compiling the .py file. How long is
sys.path?

Skip

From brett at python.org  Tue Oct  7 20:43:55 2014
From: brett at python.org (Brett Cannon)
Date: Tue, 07 Oct 2014 18:43:55 +0000
Subject: [Python-Dev] performance delta with .py presence v.s. only pyc
 on python 2.7.x?
References: <CAB6HS2V48Pv0sqjfSrQgY9KKyOhTJf0ouzRpo89rWJT_09x+1Q@mail.gmail.com>
 <CANc-5UwZ-FuH4+=Q4SdYPqYiStWvhSCo+QJ9g1MkyQ9_4tgp2A@mail.gmail.com>
Message-ID: <CAP1=2W7sAQ4NZNpAQOx7220+=qx7CPYFkOmsDWqJvFAFpXd3ag@mail.gmail.com>

On Tue Oct 07 2014 at 2:24:52 PM Skip Montanaro <skip.montanaro at gmail.com>
wrote:

> On Tue, Oct 7, 2014 at 12:46 PM, John Smith <sansprivacy at gmail.com> wrote:
> > pyc-only install sees mediocre performance. (pyc's are built using
> > compileall.py, then source .py's removed before packaging)
>
> (Warning: it's been probably a decade since I looked at any of this
> stuff, so treat this response as mere conjecture.)
>
> I'd take a look at startup time and things like stat(2) calls in the
> presence or absence of .py files. It's possible that it tries all
> other possible file extensions before considering .pyc. If you have a
> long sys.path, it would then run through all the other file extension
> possibilities before trying the .pyc. OTOH, if the .py is present, it
> might be found early in the search, then as an optimization, look for
> a .pyc file it can use rather than compiling the .py file. How long is
> sys.path?
>

The extension check is per sys.path entry so sys.path is the outer loop,
not the file extension list. The relevant code is all in
https://hg.python.org/cpython/file/05f70805f37f/Python/import.c for Python
2.7  and the search code is
https://hg.python.org/cpython/file/05f70805f37f/Python/import.c#l1291 .

But with the code being a black box there is no good way to answer this
question. E.g. if they have a custom finder that is very costly when there
is no source then that could explain this. But you're talking **app**
performance and not import performance, so either something on your system
or in that code is very quirky that is leading to an actual performance
loss to that level (import costs are usually washed out and bytecode is
literally just the internal representation of source after compilation so
there is no semantic difference at execution time if the same source is
used).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141007/039c3521/attachment.html>

From larry at hastings.org  Wed Oct  8 10:57:53 2014
From: larry at hastings.org (Larry Hastings)
Date: Wed, 08 Oct 2014 01:57:53 -0700
Subject: [Python-Dev] [RELEASE] Python 3.4.2 is now available
Message-ID: <5434FC91.9070306@hastings.org>



On behalf of the Python development community and the Python 3.4 release 
team, I'm pleased to announce the availability of Python 3.4.2. Python 
3.4.2 has many bugfixes and other small improvements over 3.4.1.  One 
new feature for Mac OS X users: the OS X installers are now distributed 
as signed installer package files compatible with the OS X Gatekeeper 
security feature.

You can download it here:

    https://www.python.org/download/releases/3.4.2


May the road rise up to meet you,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141008/13f70642/attachment.html>

From tismer at stackless.com  Wed Oct  8 12:16:42 2014
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 08 Oct 2014 12:16:42 +0200
Subject: [Python-Dev] shebang policy, and pip
Message-ID: <54350F0A.7060901@stackless.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Howdy,

this question is a bit about general policy which is not yet
covered in the python recommendations:

I see projects which do check-ins like "get rid of shebang lines"
and they remove those lines from non-script sources.

It is not always clear to me what to do, so I tend to leave those
lines in per default, in order not to waste time thinking about it,
but well, today I was confronted with that.

Digging a bit deeper shows the following:

python docs:
No mention of shebang, but for Windows.
https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default
https://docs.python.org/3/using/windows.html?highlight=shebang

Google's python style guide also says when a shebang is needed, but
does not forbid it.

Pep 394 explains how to use shebang, but still nothing about not using it.
http://legacy.python.org/dev/peps/pep-0394/

So is there anything officially preferred, and should that go into pep 8?


Special case with pip
- ------------------

I was looking through my installed packages and wondered quite
much about pip:

Pip has a shebang in the __init__ file, but no shebang in the
__main__ file.

I guess this is wrong and should be in the executable file,
which is __main__ .

cheers - Chris


- -- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     http://www.pydica.net/
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - https://gpgtools.org

iQEcBAEBCgAGBQJUNQ8FAAoJEOcwEVD7e+4Of5MH/13YyXRzPVXfsLDbfe/CRkFF
ORVAPkRB90OaEmbLTvBXPhFlAgEwcQnpdO+tmqigvlORd0cSXQKIxoPOiqq601gs
XV56aREqBCT26XMaKTuoPdu4DaW+TkwyWSn70eq4U/P7YjF3ZlNt8IkA5mteM7an
ycRYMnknEaIvP/xpZdGp+v4pq5LA42LWAY1awnBk4eMP04uDowSmcuLELpZrmSCr
iMkw6wPUdZxGVtQNwSses0mh3DuaQuwrubhHMnLoOKn/lqRjckG2Ii2BIHlWy9lQ
5X3y8IdWPh7awio8xaibNqsWaP+0DT97g2H7QZf8YIG7/DkHL/iacSr7NAPBXXQ=
=IODc
-----END PGP SIGNATURE-----



From victor.stinner at gmail.com  Wed Oct  8 12:21:44 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 8 Oct 2014 12:21:44 +0200
Subject: [Python-Dev] [python-committers] [RELEASE] Python 3.4.2 is now
	available
In-Reply-To: <5434FC91.9070306@hastings.org>
References: <5434FC91.9070306@hastings.org>
Message-ID: <CAMpsgwZmLoOTqrq6zn+FuQ3jPENSuWcrs+rh5GnYnLHZ0MF0mw@mail.gmail.com>

2014-10-08 10:57 GMT+02:00 Larry Hastings <larry at hastings.org>:
> You can download it here:
>
> https://www.python.org/download/releases/3.4.2

This page  redirect me to
https://www.python.org/download/releases/3.4.1 Maybe some web servers
of the CDN don't contain the latest version. I guess that the issue
will quickly disappears.

Victor

From breamoreboy at yahoo.co.uk  Wed Oct  8 14:06:00 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Wed, 08 Oct 2014 13:06:00 +0100
Subject: [Python-Dev] [python-committers] [RELEASE] Python 3.4.2 is now
	available
In-Reply-To: <CAMpsgwZmLoOTqrq6zn+FuQ3jPENSuWcrs+rh5GnYnLHZ0MF0mw@mail.gmail.com>
References: <5434FC91.9070306@hastings.org>
 <CAMpsgwZmLoOTqrq6zn+FuQ3jPENSuWcrs+rh5GnYnLHZ0MF0mw@mail.gmail.com>
Message-ID: <m139bb$7bg$1@ger.gmane.org>

On 08/10/2014 11:21, Victor Stinner wrote:
> 2014-10-08 10:57 GMT+02:00 Larry Hastings <larry at hastings.org>:
>> You can download it here:
>>
>> https://www.python.org/download/releases/3.4.2
>
> This page  redirect me to
> https://www.python.org/download/releases/3.4.1 Maybe some web servers
> of the CDN don't contain the latest version. I guess that the issue
> will quickly disappears.
>
> Victor
>

Further if you navigate from 3.4.1 to 3.4.2 it says "Python 3.4.2rc1 was 
released on October 8th, 2014.".  The download itself is correct.

Thanks as always to everybody who has contributed, another great piece 
of work.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From donald at stufft.io  Wed Oct  8 14:20:00 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 8 Oct 2014 08:20:00 -0400
Subject: [Python-Dev] shebang policy, and pip
In-Reply-To: <54350F0A.7060901@stackless.com>
References: <54350F0A.7060901@stackless.com>
Message-ID: <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>


> On Oct 8, 2014, at 6:16 AM, Christian Tismer <tismer at stackless.com> wrote:
> 
> 
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA512
> 
> Howdy,
> 
> this question is a bit about general policy which is not yet
> covered in the python recommendations:
> 
> I see projects which do check-ins like "get rid of shebang lines"
> and they remove those lines from non-script sources.
> 
> It is not always clear to me what to do, so I tend to leave those
> lines in per default, in order not to waste time thinking about it,
> but well, today I was confronted with that.
> 
> Digging a bit deeper shows the following:
> 
> python docs:
> No mention of shebang, but for Windows.
> https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default
> https://docs.python.org/3/using/windows.html?highlight=shebang
> 
> Google's python style guide also says when a shebang is needed, but
> does not forbid it.
> 
> Pep 394 explains how to use shebang, but still nothing about not using it.
> http://legacy.python.org/dev/peps/pep-0394/
> 
> So is there anything officially preferred, and should that go into pep 8?

Some editors can use shebang lines to control syntax highlighting or linting
(mine for example will lint different for python2 vs python3 shebangs).

> 
> 
> Special case with pip
> - ------------------
> 
> I was looking through my installed packages and wondered quite
> much about pip:
> 
> Pip has a shebang in the __init__ file, but no shebang in the
> __main__ file.
> 
> I guess this is wrong and should be in the executable file,
> which is __main__ .
> 

*puts on pip developer hat*

I?m guessing that comes from when pip used to be a single file and from before
we dropped support for Pythons that didn?t support python -m. It?s probably
nonsense cruft that?s just been left over and hadn?t been managed to be deleted
now.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From geoffspear at gmail.com  Wed Oct  8 17:10:36 2014
From: geoffspear at gmail.com (Geoffrey Spear)
Date: Wed, 8 Oct 2014 11:10:36 -0400
Subject: [Python-Dev] [RELEASE] Python 3.4.2 is now available
In-Reply-To: <5434FC91.9070306@hastings.org>
References: <5434FC91.9070306@hastings.org>
Message-ID: <CAGifb9HApSaJ89B+BzzNZ_S0dHkY66p1CSx-38+9TG=VYa+J8Q@mail.gmail.com>

It looks like the Download dropdown on python.org has a blank button
(when accessed from Windows) has an empty button that's supposed to be
for the 3.4.2 release by just links back to the current page; the 2.7
button is working fine.

On Wed, Oct 8, 2014 at 4:57 AM, Larry Hastings <larry at hastings.org> wrote:
>
>
> On behalf of the Python development community and the Python 3.4 release
> team, I'm pleased to announce the availability of Python 3.4.2.   Python
> 3.4.2 has many bugfixes and other small improvements over 3.4.1.  One new
> feature for Mac OS X users: the OS X installers are now distributed as
> signed installer package files compatible with the OS X Gatekeeper security
> feature.
>
> You can download it here:
>
> https://www.python.org/download/releases/3.4.2
>
>
> May the road rise up to meet you,
>
>
> /arry
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/geoffspear%40gmail.com
>

From nad at acm.org  Wed Oct  8 20:33:40 2014
From: nad at acm.org (Ned Deily)
Date: Wed, 08 Oct 2014 11:33:40 -0700
Subject: [Python-Dev] [RELEASE] Python 3.4.2 is now available
References: <5434FC91.9070306@hastings.org>
 <CAGifb9HApSaJ89B+BzzNZ_S0dHkY66p1CSx-38+9TG=VYa+J8Q@mail.gmail.com>
Message-ID: <nad-751D94.11334008102014@news.gmane.org>

In article 
<CAGifb9HApSaJ89B+BzzNZ_S0dHkY66p1CSx-38+9TG=VYa+J8Q at mail.gmail.com>,
 Geoffrey Spear <geoffspear at gmail.com> wrote:
> It looks like the Download dropdown on python.org has a blank button
> (when accessed from Windows) has an empty button that's supposed to be
> for the 3.4.2 release by just links back to the current page; the 2.7
> button is working fine.

https://github.com/python/pythondotorg/issues/167

-- 
 Ned Deily,
 nad at acm.org


From nad at acm.org  Wed Oct  8 20:35:48 2014
From: nad at acm.org (Ned Deily)
Date: Wed, 08 Oct 2014 11:35:48 -0700
Subject: [Python-Dev] [python-committers] [RELEASE] Python 3.4.2 is now
	available
References: <5434FC91.9070306@hastings.org>
 <CAMpsgwZmLoOTqrq6zn+FuQ3jPENSuWcrs+rh5GnYnLHZ0MF0mw@mail.gmail.com>
 <m139bb$7bg$1@ger.gmane.org>
Message-ID: <nad-1961BC.11354808102014@news.gmane.org>

In article <m139bb$7bg$1 at ger.gmane.org>,
 Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> On 08/10/2014 11:21, Victor Stinner wrote:
> > This page  redirect me to
> > https://www.python.org/download/releases/3.4.1 Maybe some web servers
> > of the CDN don't contain the latest version. I guess that the issue
> > will quickly disappears.
> Further if you navigate from 3.4.1 to 3.4.2 it says "Python 3.4.2rc1 was 
> released on October 8th, 2014.".  The download itself is correct.

Those two problems should be gone now.  Thanks.

-- 
 Ned Deily,
 nad at acm.org


From d4n1h4ck at gmail.com  Wed Oct  8 20:46:43 2014
From: d4n1h4ck at gmail.com (Daniel Pimentel (d4n1))
Date: Wed, 8 Oct 2014 15:46:43 -0300
Subject: [Python-Dev] [python-committers] [RELEASE] Python 3.4.2 is now
	available
In-Reply-To: <nad-1961BC.11354808102014@news.gmane.org>
References: <5434FC91.9070306@hastings.org>
 <CAMpsgwZmLoOTqrq6zn+FuQ3jPENSuWcrs+rh5GnYnLHZ0MF0mw@mail.gmail.com>
 <m139bb$7bg$1@ger.gmane.org>
 <nad-1961BC.11354808102014@news.gmane.org>
Message-ID: <CAG4m0SyOJJwqCqmmPYuVOg7dPMMAK_02WXQrd132Vzx=X2TEFA@mail.gmail.com>

Good news :)

2014-10-08 15:35 GMT-03:00 Ned Deily <nad at acm.org>:

> In article <m139bb$7bg$1 at ger.gmane.org>,
>  Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> > On 08/10/2014 11:21, Victor Stinner wrote:
> > > This page  redirect me to
> > > https://www.python.org/download/releases/3.4.1 Maybe some web servers
> > > of the CDN don't contain the latest version. I guess that the issue
> > > will quickly disappears.
> > Further if you navigate from 3.4.1 to 3.4.2 it says "Python 3.4.2rc1 was
> > released on October 8th, 2014.".  The download itself is correct.
>
> Those two problems should be gone now.  Thanks.
>
> --
>  Ned Deily,
>  nad at acm.org
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/d4n1%40member.fsf.org
>



-- 
Msc. Daniel Pimentel (d4n1 <http:/www.d4n1.org>)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141008/5e8bc56b/attachment.html>

From donald at stufft.io  Wed Oct  8 21:37:17 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 8 Oct 2014 15:37:17 -0400
Subject: [Python-Dev] shebang policy, and pip
In-Reply-To: <CACfEFw9_vrnNz5BmqddseptN41s=A8R2a-3qdhTzUv-MHh_DnA@mail.gmail.com>
References: <54350F0A.7060901@stackless.com>
 <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
 <CACfEFw9_vrnNz5BmqddseptN41s=A8R2a-3qdhTzUv-MHh_DnA@mail.gmail.com>
Message-ID: <35B79F54-2EC2-417C-A0EF-A013A27FF744@stufft.io>


> On Oct 8, 2014, at 3:36 PM, Wes Turner <wes.turner at gmail.com> wrote:
> 
> 
> On Oct 8, 2014 7:20 AM, "Donald Stufft" <donald at stufft.io <mailto:donald at stufft.io>> wrote:
> >
> >
> > > On Oct 8, 2014, at 6:16 AM, Christian Tismer <tismer at stackless.com <mailto:tismer at stackless.com>> wrote:
> > >
> > >
> > > -----BEGIN PGP SIGNED MESSAGE-----
> > > Hash: SHA512
> > >
> > > Howdy,
> > >
> > > this question is a bit about general policy which is not yet
> > > covered in the python recommendations:
> > >
> > > I see projects which do check-ins like "get rid of shebang lines"
> > > and they remove those lines from non-script sources.
> > >
> > > It is not always clear to me what to do, so I tend to leave those
> > > lines in per default, in order not to waste time thinking about it,
> > > but well, today I was confronted with that.
> > >
> > > Digging a bit deeper shows the following:
> > >
> > > python docs:
> > > No mention of shebang, but for Windows.
> > > https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default <https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default>
> > > https://docs.python.org/3/using/windows.html?highlight=shebang <https://docs.python.org/3/using/windows.html?highlight=shebang>
> > >
> > > Google's python style guide also says when a shebang is needed, but
> > > does not forbid it.
> > >
> > > Pep 394 explains how to use shebang, but still nothing about not using it.
> > > http://legacy.python.org/dev/peps/pep-0394/ <http://legacy.python.org/dev/peps/pep-0394/>
> > >
> > > So is there anything officially preferred, and should that go into pep 8?
> >
> > Some editors can use shebang lines to control syntax highlighting or linting
> > (mine for example will lint different for python2 vs python3 shebangs).
> 
> Does it support shebang lines like:
> 
> #!/usr/bin/env python
> 
> ?
> 

Sure. Though it doesn?t resolve it to determine what that would actually execute,
it just uses the name of the python in the shebang as heuristics.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141008/acd67269/attachment.html>

From tismer at stackless.com  Wed Oct  8 21:50:16 2014
From: tismer at stackless.com (Christian Tismer)
Date: Wed, 08 Oct 2014 21:50:16 +0200
Subject: [Python-Dev] shebang policy, and pip
In-Reply-To: <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
References: <54350F0A.7060901@stackless.com>
 <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
Message-ID: <54359578.1090903@stackless.com>


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

On 08.10.14 14:20, Donald Stufft wrote:
>
>> On Oct 8, 2014, at 6:16 AM, Christian Tismer <tismer at stackless.com>
wrote:
>>
>>
>> ...

>>
>> So is there anything officially preferred, and should that go into pep 8?
>
> Some editors can use shebang lines to control syntax highlighting or
linting
> (mine for example will lint different for python2 vs python3 shebangs).
>

Interesting use case, of course! At first sight, at least ;-)

But it becomes relatively awkward when virtualenv is used:
There is no textual distinction between python 2 and 3, and
the text editor would have to do more analysis to get things
right than to guess from the shebang line.

Which then makes it useless, and the editor would need to grab
the real python interpreter to find things out.

cheers - Chris

- -- 
Christian Tismer             :^)   tismer at stackless.com
Software Consulting          :     http://www.stackless.com/
Karl-Liebknecht-Str. 121     :     http://www.pydica.net/
14482 Potsdam                :     GPG key -> 0xFB7BEE0E
phone +49 173 24 18 776  fax +49 (30) 700143-0023
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.22 (Darwin)
Comment: GPGTools - https://gpgtools.org

iQEcBAEBCgAGBQJUNZV1AAoJEOcwEVD7e+4OlpcIAN3556PP4sBlFeDDPn4bq3fO
ScXRhB8tPkS3Ftgyux2swE91yxOZFSV8Zyae04wTB+GlzKZ6v9aXMKn5mt+b6sIr
aq1vfPxG7Qoz3q7zhaAfps8ErD9f1PriC7/4F2P4FOOko07eHt6/e8Sxg4qAgMPz
nADLj8/2MtvQYFiw3m4Zsqs31wjCTTICH52FnRgla9+u5IhStdQE7OFTiHZaPNK3
tsUohUoKqhMPIhwuB669JE7rwcRB9dA5Iatiy3uU0wqCYkekT3I4DwCtAS+DZkJX
Fe0wpLGGHOprwUTQu0SMFGQRxPX3HxL0RzTNLKjCcDNLDWRcwZRPOZ3K4DVoggQ=
=i5dg
-----END PGP SIGNATURE-----



From wes.turner at gmail.com  Wed Oct  8 21:36:07 2014
From: wes.turner at gmail.com (Wes Turner)
Date: Wed, 8 Oct 2014 14:36:07 -0500
Subject: [Python-Dev] shebang policy, and pip
In-Reply-To: <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
References: <54350F0A.7060901@stackless.com>
 <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
Message-ID: <CACfEFw9_vrnNz5BmqddseptN41s=A8R2a-3qdhTzUv-MHh_DnA@mail.gmail.com>

On Oct 8, 2014 7:20 AM, "Donald Stufft" <donald at stufft.io> wrote:
>
>
> > On Oct 8, 2014, at 6:16 AM, Christian Tismer <tismer at stackless.com>
wrote:
> >
> >
> > -----BEGIN PGP SIGNED MESSAGE-----
> > Hash: SHA512
> >
> > Howdy,
> >
> > this question is a bit about general policy which is not yet
> > covered in the python recommendations:
> >
> > I see projects which do check-ins like "get rid of shebang lines"
> > and they remove those lines from non-script sources.
> >
> > It is not always clear to me what to do, so I tend to leave those
> > lines in per default, in order not to waste time thinking about it,
> > but well, today I was confronted with that.
> >
> > Digging a bit deeper shows the following:
> >
> > python docs:
> > No mention of shebang, but for Windows.
> >
https://docs.python.org/3/search.html?q=shebang&check_keywords=yes&area=default
> > https://docs.python.org/3/using/windows.html?highlight=shebang
> >
> > Google's python style guide also says when a shebang is needed, but
> > does not forbid it.
> >
> > Pep 394 explains how to use shebang, but still nothing about not using
it.
> > http://legacy.python.org/dev/peps/pep-0394/
> >
> > So is there anything officially preferred, and should that go into pep
8?
>
> Some editors can use shebang lines to control syntax highlighting or
linting
> (mine for example will lint different for python2 vs python3 shebangs).

Does it support shebang lines like:

#!/usr/bin/env python

?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141008/c0410605/attachment-0001.html>

From barry at python.org  Thu Oct  9 00:00:20 2014
From: barry at python.org (Barry Warsaw)
Date: Wed, 8 Oct 2014 18:00:20 -0400
Subject: [Python-Dev] shebang policy, and pip
References: <54350F0A.7060901@stackless.com>
 <2C2136A6-5D1D-4DD9-A34E-6170C9ED1CFB@stufft.io>
Message-ID: <20141008180020.58161aa0@anarchist.wooz.org>

On Oct 08, 2014, at 08:20 AM, Donald Stufft wrote:

>Some editors can use shebang lines to control syntax highlighting or linting
>(mine for example will lint different for python2 vs python3 shebangs).

Some editors can also use `# -*- foo -*-` comments to set up editing modes and
there are other ways to specify which version linters to use.

I generally avoid shebangs where they aren't needed, and between entry points
and -m they rarely are these days.  I find most uses are in smaller one-off
scripts and such.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141008/a5818192/attachment.sig>

From jcea at jcea.es  Fri Oct 10 00:47:46 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 00:47:46 +0200
Subject: [Python-Dev] mUTF-7 support?
Message-ID: <54371092.2070403@jcea.es>

I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
in the codecs module. As an european with a language with 27 different
letters (instead of english 26), tildes, opening question marks, etc., I
find it very inconvenient.

This encoding is used basically only in IMAP4, I know. But IMAP4 is an
important protocol and all projects related to it needs mUTF-7 support
if they care about non-english alphabets. Everybody has already an
implementation, waste of effort.

We already support quite amusing encodings in
<https://docs.python.org/3.5/library/codecs.html#standard-encodings>.

What do you think?. Could be considered for Python 3.5?.

I volunteer for the job, of course.

PS: Do you think a Python implementation would be good enough?. I don't
think this need to be C-fast.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/82966820/attachment.sig>

From victor.stinner at gmail.com  Fri Oct 10 01:08:29 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 01:08:29 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371092.2070403@jcea.es>
References: <54371092.2070403@jcea.es>
Message-ID: <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>

Hi,

You can develop a codec and plug it into Python 3.4 right now using
codecs.register().

It's difficult to decide if a codec is important enough to be added to Python.

When you say "IMAP4", do you mean any IMAP4 server? Do you have a list
of server vendors known to use the encoding mUTF-7? Is it possible to
ask the server to speak a specific codec like UTF-8? I don't know the
protocol. Interesting article:
http://comments.gmane.org/gmane.mail.imap.general/3416

Python supports UTF-7, but this codec doesn't look to be used. Bugs
were fixed in this codec "recently".

Anyway, open an issue ;-)

How is mUTF-7 different than UTF-7? (Why yet another encoding while
standard UTF encodings exist???)

Requests of new encodings:

"missing vietnamese codec TCVN 5712:1993 in Python" (open)
http://bugs.python.org/issue21081

"add thai encoding aliases to encodings.aliases" (open)
http://bugs.python.org/issue17254

"Add "java modified utf-8" codec" (closed as wont fix 2 years ago)
http://bugs.python.org/issue2857

"Add support for CESU-8 encoding" (rejected 3 years ago)
http://bugs.python.org/issue12742

"Adding new CNS11643, a *huge* charset,    support in cjkcodecs"
(closed as wont fix 4 years ago)
http://bugs.python.org/issue2066

"Add KOI8-RU as a known encoding" (rejected 5 years ago)
http://bugs.python.org/issue5214
("This charset wasn't supported by Ukrainian Internet community due to
political reasons; KOI8-U was invented as opposition to KOI8-RU.")

Recently added codec:

"Add support of the cp1125 encoding" (1 year ago)
http://bugs.python.org/issue19668

"Add cp65001 codec" (3 years ago)
http://bugs.python.org/issue13216

Victor

2014-10-10 0:47 GMT+02:00 Jesus Cea <jcea at jcea.es>:
> I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
> in the codecs module. As an european with a language with 27 different
> letters (instead of english 26), tildes, opening question marks, etc., I
> find it very inconvenient.
>
> This encoding is used basically only in IMAP4, I know. But IMAP4 is an
> important protocol and all projects related to it needs mUTF-7 support
> if they care about non-english alphabets. Everybody has already an
> implementation, waste of effort.
>
> We already support quite amusing encodings in
> <https://docs.python.org/3.5/library/codecs.html#standard-encodings>.
>
> What do you think?. Could be considered for Python 3.5?.
>
> I volunteer for the job, of course.
>
> PS: Do you think a Python implementation would be good enough?. I don't
> think this need to be C-fast.
>
> --
> Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
> jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
> Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
> jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
> "Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
> "My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
> "El amor es poner tu felicidad en la felicidad de otro" - Leibniz
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>

From solipsis at pitrou.net  Fri Oct 10 01:10:33 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Oct 2014 01:10:33 +0200
Subject: [Python-Dev] mUTF-7 support?
References: <54371092.2070403@jcea.es>
Message-ID: <20141010011033.3eff7889@fsol>

On Fri, 10 Oct 2014 00:47:46 +0200
Jesus Cea <jcea at jcea.es> wrote:
> I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
> in the codecs module. As an european with a language with 27 different
> letters (instead of english 26), tildes, opening question marks, etc., I
> find it very inconvenient.
> 
> This encoding is used basically only in IMAP4, I know. But IMAP4 is an
> important protocol and all projects related to it needs mUTF-7 support
> if they care about non-english alphabets. Everybody has already an
> implementation, waste of effort.

This sounds good to me. Feel free to propose a patch for 3.5.

Regards

Antoine.



From jcea at jcea.es  Fri Oct 10 01:33:58 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 01:33:58 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
Message-ID: <54371B66.8080108@jcea.es>

On 10/10/14 01:08, Victor Stinner wrote:
> When you say "IMAP4", do you mean any IMAP4 server? Do you have a list
> of server vendors known to use the encoding mUTF-7?

All of them. IMAP4 protocol **REQUIRES** mUTF-7.

UTF-8 is optional in IMAP4, and even UTF-8 capable servers have to
support clients without that ability.

Check "5.1. Mailbox Naming" in IMAP4 RFC:
<http://tools.ietf.org/html/rfc3501>.

> Anyway, open an issue ;-)

I would like to gauge interest/resistance to the idea from my fellows first.

> How is mUTF-7 different than UTF-7? (Why yet another encoding while
> standard UTF encodings exist???)

As explained in section "5.1.3. Mailbox International Naming Convention":

"""
The purpose of these modifications is to correct the following
   problems with UTF-7:

      1) UTF-7 uses the "+" character for shifting; this conflicts with
         the common use of "+" in mailbox names, in particular USENET
         newsgroup names.

      2) UTF-7's encoding is BASE64 which uses the "/" character; this
         conflicts with the use of "/" as a popular hierarchy delimiter.

      3) UTF-7 prohibits the unencoded usage of "\"; this conflicts with
         the use of "\" as a popular hierarchy delimiter.

      4) UTF-7 prohibits the unencoded usage of "~"; this conflicts with
         the use of "~" in some servers as a home directory indicator.

      5) UTF-7 permits multiple alternate forms to represent the same
         string; in particular, printable US-ASCII characters can be
         represented in encoded form.
"""

> Requests of new encodings:

I am volunteering and can even do the mercurial PUSH myself :-p. That is
an advantage over some of those new encoding requests :-pp.

But then yes, I realize that this is a specialized tool (even if IMAP4
is probably the most popular mail access protocol in the world), we can
accommodate every use-case and there are tons of mUTF-7 libraries out
there already.

So, I am asking :).

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/224a4315/attachment.sig>

From ethan at stoneleaf.us  Fri Oct 10 01:50:32 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 09 Oct 2014 16:50:32 -0700
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371092.2070403@jcea.es>
References: <54371092.2070403@jcea.es>
Message-ID: <54371F48.6010809@stoneleaf.us>

On 10/09/2014 03:47 PM, Jesus Cea wrote:
> [....]  mUTF-7 support  [...]
>
> What do you think?. Could be considered for Python 3.5?.

+1

--
~Ethan~

From victor.stinner at gmail.com  Fri Oct 10 02:00:27 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 02:00:27 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371B66.8080108@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
Message-ID: <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>

2014-10-10 1:33 GMT+02:00 Jesus Cea <jcea at jcea.es>:
> The purpose of these modifications is to correct the following
>    problems with UTF-7:

If you need performances, I would be interested to see if it would be
possible to reuse the C codec for UTF-7 to share as much code as
possible.

What is the current behaviour of imaplib in Python 3.4 with non-ASCII
characters in mailbox names?

Victor

From victor.stinner at gmail.com  Fri Oct 10 02:29:29 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 02:29:29 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
Message-ID: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>

Hi,

Windows is not the primary target of Python developers, probably
because most of them work on Linux. Official Python binaries are
currently built by Microsoft Visual Studio. Even if Python developers
get free licenses thanks for Microsoft, I would prefer to use an open
source compiler if it would be possible. So *anyone* can build Python
from scatch. I don't like the requirement of having a license to build
Python. The free version (Visual Studio Express) only supports 32-bit
and doesn't support PGO build (Profile-Guided Optimizations, which are
disabled if I remember correctly because of compiler bugs).

I know that it's hard to replace Visual Studio. I don't want to do it
right now, but I would like to discuss that with you.


=== Open Watcom

Jeffrey Armstrong is working on the Python support of OpenWatcom(v2), see:
http://lightningpython.org/
https://bitbucket.org/ArmstrongJ/lightning-python

This compiler was initially written on MS-DOS in 32-bit, but it now
supports Windows and Linux as well. The 64-bit mode is new and
experimental. The Open Watcom "v2" project is actively developed at:

https://github.com/open-watcom/open-watcom-v2/

On Linux, Open Watcom don't support dynamic linking. On Windows, it
uses its own C library. I'm not sure that Open Watcom is the best
choice to build Python on Windows.


=== MinGW

Some people tried to compile Python. See for example:
https://bitbucket.org/puqing/python-mingw

We even got some patches:
http://bugs.python.org/issue3871 (rejected)

See also:
https://stackoverflow.com/questions/15365249/build-python-with-mingw-and-gcc

MinGW reuses the Microsoft C library and it is based on GCC which is
very stable, actively developed, supports a lot of archiectures, etc.
I guess that it should be possible to reuse third party GCC tools like
the famous GDB debugger?


=== Cywin

Cygwin was written compile POSIX applications on Windows and it
provides a DLL for that. Python doesn't need that, it uses directly
the Windows native API. I don't think that we should use Cygwin.


=== Clang

I have no idea of the support of Clang on Windows. Which C library is
used? I found some pages:
http://clang.llvm.org/docs/MSVCCompatibility.html

http://blog.llvm.org/2014/07/clangllvm-on-windows-update.html
This article starts with "It?s time for an update on Clang?s support
for building native Windows programs, compatible with Visual C++!".
Good.

I see binaries for Windows.


=== Other compilers?

Do you know other C compiler which can be used to build Python?


=== Requirements

A compiler alone is not enough. To develop, we need tools to automate
the compilation, we need a good debugger, and other similar tools.
(Personally, I don't need an IDE. I mostly write code on Linux and
only run Windows to ensure that my code works on Windows before
pushing it.)

IMO 64-bit support is simply required (we currently provide 64-bit
binaries on Windows). Supporting ARM can be interesting for Windows 8
and Windows 9.

Some parts of Python are low-level like ctypes with libffi. The
_decimal module uses libmpdec which is highly optimized. In short, the
goal is to have a full working standard Python library.

It's probably better to reuse the Microsoft C library instead of
having to embed a different C library. What do you think?

What about the Python stable ABI? Would it be broken if we use a
different compiler?

What about third party Python extensions?

What about external dependencies like gzip, bz2, Tk, Tcl, OpenSSL, etc.?

Victor

From rdmurray at bitdance.com  Fri Oct 10 02:34:08 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 09 Oct 2014 20:34:08 -0400
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371B66.8080108@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
Message-ID: <20141010003409.330B2250E03@webabinitio.net>

On Fri, 10 Oct 2014 01:33:58 +0200, Jesus Cea <jcea at jcea.es> wrote:
> On 10/10/14 01:08, Victor Stinner wrote:
> > When you say "IMAP4", do you mean any IMAP4 server? Do you have a list
> > of server vendors known to use the encoding mUTF-7?
> 
> All of them. IMAP4 protocol **REQUIRES** mUTF-7.

[...]

> I am volunteering and can even do the mercurial PUSH myself :-p. That is
> an advantage over some of those new encoding requests :-pp.
> 
> But then yes, I realize that this is a specialized tool (even if IMAP4
> is probably the most popular mail access protocol in the world), we can
> accommodate every use-case and there are tons of mUTF-7 libraries out
> there already.
> 
> So, I am asking :).

I see you are already nosy on issue 5305.  I don't think there is any
question that this is a useful and desired feature for python's imaplib.
If it makes sense to implement it as a codec, there is no reason *not*
to do that.

--David

From jcea at jcea.es  Fri Oct 10 02:34:47 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 02:34:47 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
Message-ID: <543729A7.1080304@jcea.es>

On 10/10/14 02:00, Victor Stinner wrote:
> 2014-10-10 1:33 GMT+02:00 Jesus Cea <jcea at jcea.es>:
>> The purpose of these modifications is to correct the following
>>    problems with UTF-7:
> 
> If you need performances, I would be interested to see if it would be
> possible to reuse the C codec for UTF-7 to share as much code as
> possible.

I don't need performance, and implementations I am studying are already
using UTF-7 as an intermediate step.

Example: <http://imapclient.freshfoo.com/browser/imapclient/imap_utf7.py>

> What is the current behaviour of imaplib in Python 3.4 with non-ASCII
> characters in mailbox names?

It breaks. Crash & burn.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/7f0e403d/attachment.sig>

From victor.stinner at gmail.com  Fri Oct 10 02:43:53 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 02:43:53 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <543729A7.1080304@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
Message-ID: <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>

2014-10-10 2:34 GMT+02:00 Jesus Cea <jcea at jcea.es>:
>> What is the current behaviour of imaplib in Python 3.4 with non-ASCII
>> characters in mailbox names?
>
> It breaks. Crash & burn.

Oh ok. So in short, imaplib doesn't work on Python 3: it's a bug and
it must be fixed. I agree that a new codec is good idea and I will
support it!

Victor

From jcea at jcea.es  Fri Oct 10 02:52:02 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 02:52:02 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
 <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
Message-ID: <54372DB2.70402@jcea.es>

On 10/10/14 02:43, Victor Stinner wrote:
> 2014-10-10 2:34 GMT+02:00 Jesus Cea <jcea at jcea.es>:
>>> What is the current behaviour of imaplib in Python 3.4 with non-ASCII
>>> characters in mailbox names?
>>
>> It breaks. Crash & burn.
> 
> Oh ok. So in short, imaplib doesn't work on Python 3: it's a bug and
> it must be fixed. I agree that a new codec is good idea and I will
> support it!

Actually, it doesn't work in Python 2 either. It never supported
international mailbox names.

Should I dare to suggest to port this to 2.7, since 2.7 is special and
will be supported for a long time?. Or maybe this is something like
"Yes, Python 2 is broken, the real deal is Python 3"? :).

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/d6a6633f/attachment.sig>

From njs at pobox.com  Fri Oct 10 02:53:42 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Fri, 10 Oct 2014 01:53:42 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <CAPJVwBkjjXY1VKospi9PGj1chPZFTo8=AYUSD6xwwSES-sn54g@mail.gmail.com>

On Fri, Oct 10, 2014 at 1:29 AM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> Windows is not the primary target of Python developers, probably
> because most of them work on Linux. Official Python binaries are
> currently built by Microsoft Visual Studio. Even if Python developers
> get free licenses thanks for Microsoft, I would prefer to use an open
> source compiler if it would be possible. So *anyone* can build Python
> from scatch. I don't like the requirement of having a license to build
> Python. The free version (Visual Studio Express) only supports 32-bit
> and doesn't support PGO build (Profile-Guided Optimizations, which are
> disabled if I remember correctly because of compiler bugs).
>
> I know that it's hard to replace Visual Studio. I don't want to do it
> right now, but I would like to discuss that with you.
>
>
> === Open Watcom
>
> Jeffrey Armstrong is working on the Python support of OpenWatcom(v2), see:
> http://lightningpython.org/
> https://bitbucket.org/ArmstrongJ/lightning-python
>
> This compiler was initially written on MS-DOS in 32-bit, but it now
> supports Windows and Linux as well. The 64-bit mode is new and
> experimental. The Open Watcom "v2" project is actively developed at:
>
> https://github.com/open-watcom/open-watcom-v2/
>
> On Linux, Open Watcom don't support dynamic linking. On Windows, it
> uses its own C library. I'm not sure that Open Watcom is the best
> choice to build Python on Windows.
>
>
> === MinGW
>
> Some people tried to compile Python. See for example:
> https://bitbucket.org/puqing/python-mingw
>
> We even got some patches:
> http://bugs.python.org/issue3871 (rejected)
>
> See also:
> https://stackoverflow.com/questions/15365249/build-python-with-mingw-and-gcc
>
> MinGW reuses the Microsoft C library and it is based on GCC which is
> very stable, actively developed, supports a lot of archiectures, etc.
> I guess that it should be possible to reuse third party GCC tools like
> the famous GDB debugger?

You may want to get in touch with Carl Kleffner -- he's done a bunch
of work lately on getting a mingw-based toolchain to the point where
it can build numpy and scipy. (This is pretty urgent for us because
(a) numerical work requires a BLAS library and the main competitive
open-source one -- OpenBLAS -- cannot be built by msvc because of asm
syntax issues, (b) msvc's fortran support is even worse than its C99
support.) Getting this working is non-trivial, since by default
mingw-compiled code depends on the GCC runtime libraries, the default
ABI doesn't match msvc, etc. But apparently these issues are all
fixable.

General info:
  https://github.com/numpy/numpy/wiki/Mingw-static-toolchain

The built toolchains etc.:
  https://bitbucket.org/carlkl/mingw-w64-for-python/downloads

Readme:
  https://bitbucket.org/carlkl/mingw-w64-for-python/downloads/readme.txt

The patch to the numpy sources -- this in particular includes the
various distutils hacks needed to enable the crucial ABI-compatibility
switches:
  https://bitbucket.org/carlkl/mingw-w64-for-python/downloads/numpy.patch

(Unfortunately he doesn't seem to have posted the build recipe for the
toolchain itself -- I'm sure he'd be happy to if you asked though.)

AFAICT the end result is a single free compiler toolchain that can
spit out 32- and 64-bit binaries using whichever MSVC runtime you
prefer.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From victor.stinner at gmail.com  Fri Oct 10 03:20:37 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 03:20:37 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54372DB2.70402@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
 <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
 <54372DB2.70402@jcea.es>
Message-ID: <CAMpsgwaCrMf3dJG5dLVNuHzECirCWuE=GVZL95_AQYXaTARQGQ@mail.gmail.com>

2014-10-10 2:52 GMT+02:00 Jesus Cea <jcea at jcea.es>:
> "Yes, Python 2 is broken, the real deal is Python 3"? :).

For Unicode, my favorite answer is "it's time to upgrade! Python 3 has
a much better Unicode support." and not fix the issue on Python 2.7.

I don't want to open the can of worm "unicode" in Python 2. I don't
want to redo all the work I already did on Python 3.

For the specific case of the new codec, I don't know. It will be
easier to decide when the bug will be fully fixed in Python 3.5, to
see the size of the changeset.

Victor

From drsalists at gmail.com  Fri Oct 10 04:12:29 2014
From: drsalists at gmail.com (Dan Stromberg)
Date: Thu, 9 Oct 2014 19:12:29 -0700
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371092.2070403@jcea.es>
References: <54371092.2070403@jcea.es>
Message-ID: <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>

On Thu, Oct 9, 2014 at 3:47 PM, Jesus Cea <jcea at jcea.es> wrote:
> I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
> in the codecs module. As an european with a language with 27 different
> letters (instead of english 26), tildes, opening question marks, etc., I
> find it very inconvenient.
>
> This encoding is used basically only in IMAP4, I know. But IMAP4 is an
> important protocol and all projects related to it needs mUTF-7 support
> if they care about non-english alphabets. Everybody has already an
> implementation, waste of effort.

I've been parsing up a huge gmail account with no encoding errors,
using CPython 2.x and CPython 3.x.  I'd be surprised if there are no
foreign characters in any of the thousands of messages there - but
maybe I'm just being very lucky.  I'm not specifying a codec, and I
don't see a way of specifying one offhand.

Does email.header.decode_header help you?

From solipsis at pitrou.net  Fri Oct 10 04:28:21 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Oct 2014 04:28:21 +0200
Subject: [Python-Dev] mUTF-7 support?
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
Message-ID: <20141010042821.61224fb6@fsol>

On Thu, 9 Oct 2014 19:12:29 -0700
Dan Stromberg <drsalists at gmail.com> wrote:
> On Thu, Oct 9, 2014 at 3:47 PM, Jesus Cea <jcea at jcea.es> wrote:
> > I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
> > in the codecs module. As an european with a language with 27 different
> > letters (instead of english 26), tildes, opening question marks, etc., I
> > find it very inconvenient.
> >
> > This encoding is used basically only in IMAP4, I know. But IMAP4 is an
> > important protocol and all projects related to it needs mUTF-7 support
> > if they care about non-english alphabets. Everybody has already an
> > implementation, waste of effort.
> 
> I've been parsing up a huge gmail account with no encoding errors,
> using CPython 2.x and CPython 3.x.  I'd be surprised if there are no
> foreign characters in any of the thousands of messages there - but
> maybe I'm just being very lucky.  I'm not specifying a codec, and I
> don't see a way of specifying one offhand.

AFAIU, this is specifically about mailbox names, not messages.

Regards

Antoine.



From rosuav at gmail.com  Fri Oct 10 04:36:49 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 10 Oct 2014 13:36:49 +1100
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54372DB2.70402@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
 <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
 <54372DB2.70402@jcea.es>
Message-ID: <CAPTjJmrfV7i38PcOYhKccOQFnKcZa0H=JmsTv85mTFH01iA7zQ@mail.gmail.com>

On Fri, Oct 10, 2014 at 11:52 AM, Jesus Cea <jcea at jcea.es> wrote:
> Actually, it doesn't work in Python 2 either. It never supported
> international mailbox names.
>
> Should I dare to suggest to port this to 2.7, since 2.7 is special and
> will be supported for a long time?. Or maybe this is something like
> "Yes, Python 2 is broken, the real deal is Python 3"? :).

That's ultimately up to the release manager, but IMO this sounds like
a bug to be fixed, more than a feature being added.

ChrisA

From rdmurray at bitdance.com  Fri Oct 10 04:41:52 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 09 Oct 2014 22:41:52 -0400
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <20141010042821.61224fb6@fsol>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol>
Message-ID: <20141010024153.899F2250EDC@webabinitio.net>

On Fri, 10 Oct 2014 04:28:21 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Thu, 9 Oct 2014 19:12:29 -0700
> Dan Stromberg <drsalists at gmail.com> wrote:
> > On Thu, Oct 9, 2014 at 3:47 PM, Jesus Cea <jcea at jcea.es> wrote:
> > > I miss mUTF-7 support (as used to encode IMAP4 mailbox names) in Python,
> > > in the codecs module. As an european with a language with 27 different
> > > letters (instead of english 26), tildes, opening question marks, etc., I
> > > find it very inconvenient.
> > >
> > > This encoding is used basically only in IMAP4, I know. But IMAP4 is an
> > > important protocol and all projects related to it needs mUTF-7 support
> > > if they care about non-english alphabets. Everybody has already an
> > > implementation, waste of effort.
> > 
> > I've been parsing up a huge gmail account with no encoding errors,
> > using CPython 2.x and CPython 3.x.  I'd be surprised if there are no
> > foreign characters in any of the thousands of messages there - but
> > maybe I'm just being very lucky.  I'm not specifying a codec, and I
> > don't see a way of specifying one offhand.
> 
> AFAIU, this is specifically about mailbox names, not messages.

Specifically, it is about what we might better term mailbox
*folders*...that is, not what you would normally think of as the
'mailbox name', which is usually understood to be the thing before the @
in the email address (and can't contain non-ASCII yet...we need RFC 6855
support for that, and I'm not sure *anybody* has that yet).

In this context it is the names you give to folders on the IMAP
server...starting (usually) with INBOX and adding from there.  These
names are used in IMAP commands (ex: the 'select' or 'create' commands),
and IMAP uses mUTF-7 for those.

--David

From solipsis at pitrou.net  Fri Oct 10 04:50:58 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Oct 2014 04:50:58 +0200
Subject: [Python-Dev] mUTF-7 support?
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
 <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
 <54372DB2.70402@jcea.es>
 <CAPTjJmrfV7i38PcOYhKccOQFnKcZa0H=JmsTv85mTFH01iA7zQ@mail.gmail.com>
Message-ID: <20141010045058.18abcb39@fsol>

On Fri, 10 Oct 2014 13:36:49 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> On Fri, Oct 10, 2014 at 11:52 AM, Jesus Cea <jcea at jcea.es> wrote:
> > Actually, it doesn't work in Python 2 either. It never supported
> > international mailbox names.
> >
> > Should I dare to suggest to port this to 2.7, since 2.7 is special and
> > will be supported for a long time?. Or maybe this is something like
> > "Yes, Python 2 is broken, the real deal is Python 3"? :).
> 
> That's ultimately up to the release manager, but IMO this sounds like
> a bug to be fixed, more than a feature being added.

Well, it would be a bug if we had claimed to support it.

Regards

Antoine.



From v+python at g.nevcal.com  Fri Oct 10 06:05:38 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 09 Oct 2014 21:05:38 -0700
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <20141010024153.899F2250EDC@webabinitio.net>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol> <20141010024153.899F2250EDC@webabinitio.net>
Message-ID: <54375B12.8030403@g.nevcal.com>

On 10/9/2014 7:41 PM, R. David Murray wrote:
> Specifically, it is about what we might better term mailbox
> *folders*...that is, not what you would normally think of as the
> 'mailbox name', which is usually understood to be the thing before the @
> in the email address (and can't contain non-ASCII yet...we need RFC 6855
> support for that, and I'm not sure*anybody*  has that yet).

There are still lots of idiotic web sites that assume everything in 
front of the @ must be a letter, digit, dot, or hyphen, and even some 
that only permit one dot after the @... even though for 30 years or so, 
the RFCs have permitted a nice variety of other special characters, 
although not all of them.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141009/1b812b34/attachment.html>

From rosuav at gmail.com  Fri Oct 10 08:55:55 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 10 Oct 2014 17:55:55 +1100
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54375B12.8030403@g.nevcal.com>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol>
 <20141010024153.899F2250EDC@webabinitio.net>
 <54375B12.8030403@g.nevcal.com>
Message-ID: <CAPTjJmquU1RmKWrAZb0ipDKpOzMUZe4Jxa0kXWFY9FRuEiqP2w@mail.gmail.com>

On Fri, Oct 10, 2014 at 3:05 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> There are still lots of idiotic web sites that assume everything in front of
> the @ must be a letter, digit, dot, or hyphen, and even some that only
> permit one dot after the @... even though for 30 years or so, the RFCs have
> permitted a nice variety of other special characters, although not all of
> them.

And heaps that require a dot after the @, which is definitely not a requirement.

ChrisA

From p.f.moore at gmail.com  Fri Oct 10 09:07:38 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Oct 2014 08:07:38 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>

On 10 October 2014 01:29, Victor Stinner <victor.stinner at gmail.com> wrote:
> What about the Python stable ABI? Would it be broken if we use a
> different compiler?
>
> What about third party Python extensions?
>
> What about external dependencies like gzip, bz2, Tk, Tcl, OpenSSL, etc.?

The key point for me is that any supported build on Windows supports
the exact same ABI. It is difficult for Windows users to set up a
build environment (and changing the compiler will not alter that fact)
so Windows users will rely on binary builds. If multiple ABIs exist,
users will have the problem of projects shipping only one ABI binary,
and if it doesn't match their Python, they are out of luck. It's
critical that we don't double the number of binary builds projects
need to ship.

Having said that, I'm personally not interested in this, as I am happy
with MSVC Express. Python 3.5 will be using MSVC 14, where the express
edition supports both 32 and 64 bit. The licensing doesn't bother me
personally, and the compiler is easy to install for people who want
it. Any competing build environment would have to be as easy to
install and use as MSVC Express to be worthwhile, IMO.

The only advantage[1] to a new compiler would be if it trivially
supported cross-compiling on Linux, as that would allow Limux
developers to easily ship Windows binaries.

Paul

[1] I am not commenting on philosophical advantages like licensing here.

From stephen at xemacs.org  Fri Oct 10 09:16:24 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 10 Oct 2014 16:16:24 +0900
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <20141010024153.899F2250EDC@webabinitio.net>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol>
 <20141010024153.899F2250EDC@webabinitio.net>
Message-ID: <87lhoopf7b.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:

 > in the email address (and can't contain non-ASCII yet...we need RFC 6855
 > support for that, and I'm not sure *anybody* has that yet).

If it's an RFC, *somebody* has it *somewhere*. :-)


From jcea at jcea.es  Fri Oct 10 11:08:24 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 11:08:24 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <20141010024153.899F2250EDC@webabinitio.net>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol> <20141010024153.899F2250EDC@webabinitio.net>
Message-ID: <5437A208.90902@jcea.es>

On 10/10/14 04:41, R. David Murray wrote:
> Specifically, it is about what we might better term mailbox
> *folders*...that is, not what you would normally think of as the
> 'mailbox name', which is usually understood to be the thing before the @
> in the email address (and can't contain non-ASCII yet...we need RFC 6855
> support for that, and I'm not sure *anybody* has that yet).
> 
> In this context it is the names you give to folders on the IMAP
> server...starting (usually) with INBOX and adding from there.  These
> names are used in IMAP commands (ex: the 'select' or 'create' commands),
> and IMAP uses mUTF-7 for those.

In IMAP4, mail folders are called "mailboxes". Yes, non universal
notation sucks.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/c095efd0/attachment.sig>

From sturla.molden at gmail.com  Fri Oct 10 11:12:51 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 09:12:51 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CAPJVwBkjjXY1VKospi9PGj1chPZFTo8=AYUSD6xwwSES-sn54g@mail.gmail.com>
Message-ID: <142452442434624467.660307sturla.molden-gmail.com@news.gmane.org>

Nathaniel Smith <njs at pobox.com> wrote:

> You may want to get in touch with Carl Kleffner -- he's done a bunch
> of work lately on getting a mingw-based toolchain to the point where
> it can build numpy and scipy. 

To build *Python extensions*, one can use Carl's toolchain or the VC9
compiler for Python 2.7 that Microsoft just released.

To build *Python* you need Visual Studio, Visual Studio Express, Windows
SDK, or Cygwin because there is no other build process available on
Windows. Python cannot be built with MinGW.

The official 64-bit Python installer from Python.org is built with the
Windows SDK compiler, not Visual Studio. The Windows SDK is a free
download. The 32-bit installer is built with Visual Studio.

Sturla


From sturla.molden at gmail.com  Fri Oct 10 11:18:22 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 09:18:22 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
Message-ID: <1328059165434625212.523390sturla.molden-gmail.com@news.gmane.org>

Paul Moore <p.f.moore at gmail.com> wrote:

> Having said that, I'm personally not interested in this, as I am happy
> with MSVC Express. Python 3.5 will be using MSVC 14, where the express
> edition supports both 32 and 64 bit.

If you build Python yourself, you can (more or less) use whichever version
of Visual Studio you want. There is nothing that prevents you from building
Python 2.7 or 3.4 with MSVC 14. But then you have to build all Python
extensions with this version of Visual Studio as well.

Sturla


From larry at hastings.org  Fri Oct 10 11:26:33 2014
From: larry at hastings.org (Larry Hastings)
Date: Fri, 10 Oct 2014 10:26:33 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
Message-ID: <5437A649.9080701@hastings.org>


On 10/10/2014 08:07 AM, Paul Moore wrote:
> On 10 October 2014 01:29, Victor Stinner <victor.stinner at gmail.com> wrote:
>> What about the Python stable ABI? Would it be broken if we use a
>> different compiler?
>>
>> What about third party Python extensions?
>>
>> What about external dependencies like gzip, bz2, Tk, Tcl, OpenSSL, etc.?
> The key point for me is that any supported build on Windows supports
> the exact same ABI.

Just to make something clear that may not be clear to non-Windows 
developers: the C library is implicitly part of the ABI.  So unless 
these other compilers bend over backwards to generate code / have a C 
library that behaves *exactly* like MSVC, their ABI will be different, 
and therefore shared libraries compiled with one compiler won't work 
with the next.  Heck, even shared libraries compiled with one version of 
MSVC won't work with any other version!  (This is something apparently 
being fixed by MSVC 15; apparently they are designing the ABI for 
forwards compatibility.  Huzzah!)

So if CPython officially said "we support MSVC and Compiler X", I worry 
that we'd have third-party modules compiled with either one or the 
other, leaving users unable to mix and match third-party extensions as 
they do today.  ("I want to use library X, which is only available 
compiled by MSVC.  I also want to use library Y, which is only available 
compiled by Compiler X.  What should I do?" "... install Linux?")


Here's my perspective.  Having your code compilable by more compilers is 
good.  But a maze of #ifdefs is bad.  We still have #ifdef's for Borland 
C--I'd be very surprised if anyone was compiling Python 3 with Borland 
C.  IMO the benefit from supporting other compilers on Windows is 
negligible, but the costs in maintaining these other compilers is 
tangible.  Or, worse, we accept changes to support these other 
compilers, but the support is incomplete, or goes unmaintained and 
breaks (and nobody notices).

So as a practical matter I think I'd prefer if we continued to only 
support MSVC.  In fact I'd prefer it if we removed support for other 
Windows compilers, instead asking those maintainers to publish their own 
patches / repos, in the way that Stackless does.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/1d6fe915/attachment-0001.html>

From victor.stinner at gmail.com  Fri Oct 10 11:29:03 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 11:29:03 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1328059165434625212.523390sturla.molden-gmail.com@news.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <1328059165434625212.523390sturla.molden-gmail.com@news.gmane.org>
Message-ID: <CAMpsgwYE+RDqC2u8L2Z8Z_77zRwpJSnfk3k7x55dsP82UD2mWg@mail.gmail.com>

2014-10-10 11:18 GMT+02:00 Sturla Molden <sturla.molden at gmail.com>:
> If you build Python yourself, you can (more or less) use whichever version
> of Visual Studio you want. There is nothing that prevents you from building
> Python 2.7 or 3.4 with MSVC 14.

Python 2.7 provides project files (PCbuild/*) for Visual Studio 2008.
Python 3.4 provides project files (PCbuild/*) for Visual Studio 2010.

Does it mean that VC 2010 supports VC 2008 project files? If I
remember correctly, I got a wizzard proposing to convert the project
files. I didn't try it, and I installed VS 2008 instead.

I guess that VS 2014 requires a similar conversion wizzard for VS 2010
and VS 2008 project files.

Victor

From victor.stinner at gmail.com  Fri Oct 10 11:50:09 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 10 Oct 2014 11:50:09 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <5437A649.9080701@hastings.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
Message-ID: <CAMpsgwa4x3wavT4TZg_GvZorXMALGxScFXHvwcmkhhN7F5XL6A@mail.gmail.com>

Hi,

Paul Moore wrote:
> The key point for me is that any supported build on Windows supports
> the exact same ABI.

It looks like ABI compatibility is a goal of Clang on Windows:

   http://clang.llvm.org/docs/MSVCCompatibility.html
   http://blog.llvm.org/2014/07/clangllvm-on-windows-update.html

I don't know the status of the compatibility for the C ABI with VS
2008 and VS 2010. (These articles look to be focused on C++.)


OpenWatcom and Cygwin are not compatible with VS.


Is MinGW fully compatible with MSVS ABI? I read that it reuses the
MSVCRT, but I don't know if it's enough. I guess that a full ABI
compatibility means more than just using the C library, calling
convention and much more. Clang documentation mentions for example
debug symbols compatible with the Microsoft debugger.


> ... therefore
> shared libraries compiled with one compiler won't work with the next.

I noticed this issue when I provided wheel packages for Python 2.7 and
3.3 using the same Windows SDK (7.1)...

Python 2.7 and 3.3 from python.org are built with different versions
of VS, and so require a different version of the Windows SDK (7.0 for
Python 2.7, 7.1 for Python 3.3).


> So if CPython officially said "we support MSVC and Compiler X", I worry that
> we'd have third-party modules compiled with either one or the other, leaving
> users unable to mix and match third-party extensions as they do today.

Ok, I understand and I agree. Currently, VS is the defacto standard,
at least for Python.


>  We still have #ifdef's for Borland C--I'd be very surprised if anyone was compiling Python 3 with Borland C.

I opened an issue yesterday to drop support of this compiler! Please
write your comment there to support my patch.
http://bugs.python.org/issue22592


> IMO the benefit from supporting other compilers on Windows is negligible,
> but the costs in maintaining these other compilers is tangible.  Or, worse,
> we accept changes to support these other compilers, but the support is
> incomplete, or goes unmaintained and breaks (and nobody notices).

If we decide to support officially a C compiler different than VS on
Windows, it should be a real support. It should be possible to build
Python without any patch, and we should have a buildbot. And someone
should maintain the support for this compiler (fix all bugs).

Untested code always break (later).

Victor

From mal at egenix.com  Fri Oct 10 12:21:41 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 10 Oct 2014 12:21:41 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <5437A649.9080701@hastings.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>	<CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
Message-ID: <5437B335.8030604@egenix.com>

On 10.10.2014 11:26, Larry Hastings wrote:
> 
> On 10/10/2014 08:07 AM, Paul Moore wrote:
>> On 10 October 2014 01:29, Victor Stinner <victor.stinner at gmail.com> wrote:
>>> What about the Python stable ABI? Would it be broken if we use a
>>> different compiler?
>>>
>>> What about third party Python extensions?
>>>
>>> What about external dependencies like gzip, bz2, Tk, Tcl, OpenSSL, etc.?
>> The key point for me is that any supported build on Windows supports
>> the exact same ABI.
> 
> Just to make something clear that may not be clear to non-Windows developers: the C library is
> implicitly part of the ABI.  So unless these other compilers bend over backwards to generate code /
> have a C library that behaves *exactly* like MSVC, their ABI will be different, and therefore shared
> libraries compiled with one compiler won't work with the next.  Heck, even shared libraries compiled
> with one version of MSVC won't work with any other version!  (This is something apparently being
> fixed by MSVC 15; apparently they are designing the ABI for forwards compatibility.  Huzzah!)
> 
> So if CPython officially said "we support MSVC and Compiler X", I worry that we'd have third-party
> modules compiled with either one or the other, leaving users unable to mix and match third-party
> extensions as they do today.  ("I want to use library X, which is only available compiled by MSVC. 
> I also want to use library Y, which is only available compiled by Compiler X.  What should I do?"
> "... install Linux?")
> 
> 
> Here's my perspective.  Having your code compilable by more compilers is good.  But a maze of
> #ifdefs is bad.  We still have #ifdef's for Borland C--I'd be very surprised if anyone was compiling
> Python 3 with Borland C.  IMO the benefit from supporting other compilers on Windows is negligible,
> but the costs in maintaining these other compilers is tangible.  Or, worse, we accept changes to
> support these other compilers, but the support is incomplete, or goes unmaintained and breaks (and
> nobody notices).
> 
> So as a practical matter I think I'd prefer if we continued to only support MSVC.  In fact I'd
> prefer it if we removed support for other Windows compilers, instead asking those maintainers to
> publish their own patches / repos, in the way that Stackless does.

I don't think this is special to the Windows platform. We already
do support quite a few compilers in CPython and for multiple
platforms, so keeping support for e.g. MinGW or adding Intel
C support wouldn't really make much difference in the overall
#ifdef picture ;-)

That said, We do need maintainers for this support, so if there
are no people willing to support these compilers in CPython,
we should use the external port hosting approach for these, IMO.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try our new mxODBC.Connect Python Database Interface for free ! ::::


   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From jcea at jcea.es  Fri Oct 10 12:26:05 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 12:26:05 +0200
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54371092.2070403@jcea.es>
References: <54371092.2070403@jcea.es>
Message-ID: <5437B43D.6070203@jcea.es>

I think the consensus so far is that this is a good idea. I just opened
<http://bugs.python.org/issue22598>. Thanks for your feedback.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/5b1de707/attachment.sig>

From valhallasw at arctus.nl  Fri Oct 10 13:00:47 2014
From: valhallasw at arctus.nl (Merlijn van Deen)
Date: Fri, 10 Oct 2014 13:00:47 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <CADJO0eLYxurTpUjkYLXe03msHM8ZAXPwROYLCGXyjvHqxob5SA@mail.gmail.com>

On 10 October 2014 02:29, Victor Stinner <victor.stinner at gmail.com> wrote:

> The free version (Visual Studio Express) only supports 32-bit
>

VC++ 2008/2010 EE do not *bundle* a 64-bit compiler, but it's certainly
possible to build 64-bit applications by using the compiler in the (also
free) Windows SDK:

http://jenshuebel.wordpress.com/2009/02/12/visual-c-2008-express-edition-and-64-bit-targets/
https://stackoverflow.com/questions/1865069/how-to-compile-a-64-bit-application-using-visual-c-2010-express
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/f5ea5224/attachment.html>

From sturla.molden at gmail.com  Fri Oct 10 13:20:54 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 11:20:54 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
Message-ID: <130575377434631811.861466sturla.molden-gmail.com@news.gmane.org>

Larry Hastings <larry at hastings.org> wrote:

> Just to make something clear that may not be clear to non-Windows
> developers: the C library is implicitly part of the ABI.  

MacOS X also has this issue, but it less known amon Mac developers! There
tends to be multiple versions of the C library, one for each SDK version.
If you link the wrong one when building a Python extension it can crash.
For example if you have Python built with the 10.6 SDK (e.g. Enthought) but
has XCode with the 10.9 SDK as default, you need to build with the flag
-mmacosx-version-min=10.6, and for C++ also -stdlib=libstdc++. Not doing so
will cause all sorts of mysterious errors.


Two other ABI problems on Windows is the stack alignment and the MinGW
runtime: On 32-bit applications, MSVC use 16 bit stack alignment whereas
MinGW uses 32-bit alignment. This is a common cause of segfaults for Python
extensions built with MinGW. Most developers just assume it is sufficient
to link the same CRT as Python. Another problem is the MinGW runtime
(mingw32.a or mingw32.dll) which conflicts with MSCV and can cause
segfaults unless it is statically linked. The vanlilla MinGW distro
defaults to dynamic linkage for this library. Because of this a special
MinGW toolchain was created for building SciPy on Windows:

https://github.com/numpy/numpy/wiki/Mingw-static-toolchain



Sturla


From sturla.molden at gmail.com  Fri Oct 10 13:24:43 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 11:24:43 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
 <CAMpsgwa4x3wavT4TZg_GvZorXMALGxScFXHvwcmkhhN7F5XL6A@mail.gmail.com>
Message-ID: <244241516434632882.503285sturla.molden-gmail.com@news.gmane.org>

Victor Stinner <victor.stinner at gmail.com> wrote:

> Is MinGW fully compatible with MSVS ABI? I read that it reuses the
> MSVCRT, but I don't know if it's enough. 

Not out of the box. See:

https://github.com/numpy/numpy/wiki/Mingw-static-toolchain



Sturla


From sturla.molden at gmail.com  Fri Oct 10 13:27:27 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 11:27:27 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
Message-ID: <1170155471434633128.251782sturla.molden-gmail.com@news.gmane.org>

Larry Hastings <larry at hastings.org> wrote:

> So as a practical matter I think I'd prefer if we continued to only
> support MSVC.  In fact I'd prefer it if we removed support for other
> Windows compilers, instead asking those maintainers to publish their own
> patches / repos, in the way that Stackless does.

The scientific community needs to use MinGW or Intel compilers because of
Fortran. So some support for other compilers will be good, at least for
building C extensions.

Sturla


From sturla.molden at gmail.com  Fri Oct 10 13:29:53 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Fri, 10 Oct 2014 11:29:53 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CADJO0eLYxurTpUjkYLXe03msHM8ZAXPwROYLCGXyjvHqxob5SA@mail.gmail.com>
Message-ID: <117392237434633299.707393sturla.molden-gmail.com@news.gmane.org>

Merlijn van Deen <valhallasw at arctus.nl> wrote:

> VC++ 2008/2010 EE do not *bundle* a 64-bit compiler, 

Actually it does, but it is not available from the UI. You can use it from
the command line, though.

Sturla


From pachi at rvburke.com  Fri Oct 10 13:49:54 2014
From: pachi at rvburke.com (Rafael Villar Burke)
Date: Fri, 10 Oct 2014 11:49:54 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <loom.20141010T134248-272@post.gmane.org>

Victor Stinner <victor.stinner <at> gmail.com> writes:

> 
> Hi,
> 
> Windows is not the primary target of Python developers, probably
> because most of them work on Linux. Official Python binaries are
> currently built by Microsoft Visual Studio. Even if Python developers
> get free licenses thanks for Microsoft, I would prefer to use an open
> source compiler if it would be possible.
> 
> === Other compilers?
> 

I'm happily using MSYS2 (sourceforge.net/projects/msys2/), which handles
Python and many Python and GNOME related projects using the mingw64 compiler
(which can target 32 and 64 bit).

MSYS2 provides a POSIX like environment to build native applications on
windows (using the win32 api). It features a source level package manager,
pacman, ported from ArchLinux, so lots of *nix applications and libraries
are available (see https://github.com/Alexpux/MINGW-packages and also
https://github.com/Alexpux/MSYS2-packages/ for core libs and apps).

I've used it with success in apps that use Python + GTK+3 + Numpy + SciPy +
Matplotlib.

Hope this information is useful.

Regards,

Rafael


From p.f.moore at gmail.com  Fri Oct 10 14:05:11 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Oct 2014 13:05:11 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwa4x3wavT4TZg_GvZorXMALGxScFXHvwcmkhhN7F5XL6A@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
 <CAMpsgwa4x3wavT4TZg_GvZorXMALGxScFXHvwcmkhhN7F5XL6A@mail.gmail.com>
Message-ID: <CACac1F8wfCnOCC3zpbp4adKNrivj0VH7pc5bHwN21XAHWTeefw@mail.gmail.com>

On 10 October 2014 10:50, Victor Stinner <victor.stinner at gmail.com> wrote:
> Is MinGW fully compatible with MSVS ABI? I read that it reuses the
> MSVCRT, but I don't know if it's enough. I guess that a full ABI
> compatibility means more than just using the C library, calling
> convention and much more.

MinGW can be made to build ABI-compatible extensions. Whether this
will continue with MSVC 15 I don't know, as it requires a change to
add an interface library for the relevant msvcrXX runtime. And the
MinGW community is somewhat fragmented these days, with the core
project not supporting 64-bit and various external projects doing so.

Having said all this, it *is* possible with some effort to use MinGW
to build Python extensions. As noted, the numpy developers have done a
lot of work on this as some of the libraries they need must be built
with mingw. And the state of distutils support for mingw is very sad,
as well, IIRC (last time I looked there were a number of open bugs,
and very little movement on them).

Rather than put effort into more build options for CPython, I think it
would be much more beneficial to the Windows community if effort was
put into:

1. Improving support for compiling *extensions* with mingw and its
64-bit cousins, in a way that is compatible with a standard MSVC-built
Python. This would involve sorting through and applying some of the
distutils changes, as well as working with the mingw projects to
ensure they will offer support for the MSVC15 runtime ina  timely
manner.
2. Looking at ways to support cross-compiling Windows extensions from
Linux using mingw. I've no idea how practical this would be, but if
Linux developers could provide Windows builds without having to
maintain a Windows environment, that would be great.

As I say, I have no personal interest here, as I use MSVC, but I know
the above two options are commonly asked for.

Paul

From rdmurray at bitdance.com  Fri Oct 10 15:00:37 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 10 Oct 2014 09:00:37 -0400
Subject: [Python-Dev] Internationalized email support (was mUTF-7
	support?)
In-Reply-To: <87lhoopf7b.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <54371092.2070403@jcea.es>
 <CAGGBd_qTMu-RZzXcpYrwbm5uUOFtsuXGydWME0qJdDdLaRV7iw@mail.gmail.com>
 <20141010042821.61224fb6@fsol> <20141010024153.899F2250EDC@webabinitio.net>
 <87lhoopf7b.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <20141010130037.8F88D250EE6@webabinitio.net>

On Fri, 10 Oct 2014 16:16:24 +0900, "Stephen J. Turnbull" <stephen at xemacs.org> wrote:
> R. David Murray writes:
> 
>  > in the email address (and can't contain non-ASCII yet...we need RFC 6855
>  > support for that, and I'm not sure *anybody* has that yet).
> 
> If it's an RFC, *somebody* has it *somewhere*. :-)

Ah, it looks like you may be right.  I didn't find any hits in the
first few pages on Google; the closest I got was:

https://groups.google.com/forum/?_escaped_fragment_=topic/mozilla.dev.apps.thunderbird/QBFibOEK5x4#!topic/mozilla.dev.apps.thunderbird/QBFibOEK5x4

I started writing a response that said "if so, it is hidden from
google".  Then I did some deeper digging, and found this:

http://googleblog.blogspot.com/2014/08/a-first-step-toward-more-global-email.html

So my search using the RFC number was just too narrow...since gmail is
IMAP based, Google itself must have *some* sort of RFC 6855 support.
(Unless they've done a hack job of this internationalization support
that only works for their own gmail clients, which given my past
experience working with the gmail IMAP API I would believe...although
I think the IMAP problems I found were mostly bugs rather than
intentional deviations from the standards).

We had a GSOC student (Milan Oberkirch) working on international email
support this past summer, and I believe you asked in the proposal review
phase why working on support for these RFCs was apropos.  At the time I
was kind of hoping Python could kickstart development of
internationalized email by providing tools to build experiments, but now
that Google has announced support, supporting it has become surprisingly
immediately relevant.  (The previously experimental postfix support for
RFC 6531 is targeted for official release in 2.12, which is another big
step forward.)

I'd better get to reviewing the rest of those patches sometime
soon....they include a preliminary patch for RFC 6855.  Unfortunately
the email package itself doesn't handle RFC 6532 email and there's no
proposed patch for that yet.

--David

From brian at python.org  Fri Oct 10 16:31:34 2014
From: brian at python.org (Brian Curtin)
Date: Fri, 10 Oct 2014 09:31:34 -0500
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <CAD+XWwqMCwxMfo5CtSPoh-BFyjV0R4FvSmNQ1g5cyZrxTYymbQ@mail.gmail.com>

On Thu, Oct 9, 2014 at 7:29 PM, Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
>
> Windows is not the primary target of Python developers, probably
> because most of them work on Linux. Official Python binaries are
> currently built by Microsoft Visual Studio. Even if Python developers
> get free licenses thanks for Microsoft, I would prefer to use an open
> source compiler if it would be possible. So *anyone* can build Python
> from scatch. I don't like the requirement of having a license to build
> Python. The free version (Visual Studio Express) only supports 32-bit
> and doesn't support PGO build (Profile-Guided Optimizations, which are
> disabled if I remember correctly because of compiler bugs).
>
> I know that it's hard to replace Visual Studio. I don't want to do it
> right now, but I would like to discuss that with you.

Although I'm not very active around here much anymore, I was primarily
working on Windows things within the last few years.

While we have a lot of Windows users, we don't have a lot of Windows
contributors. The huge amount of churn necessary to make a change away
from VS, or the more likely move to make it possible to support both
VS and <insert other compiler>, seems like a large amount of work that
doesn't turn up much of a benefit. Especially for a platform with
constrained developer availability, working software trumps all, so I
don't expect that a project like this is going to see the regular
contributors shifting their focus away from improving Python as it is.

With that said, I do see the benefit of being able to build Python
with a free compiler. It would be great for us to be able to say it's
always built with free tools, but I'm not sure who's going to make
this happen...

From tseaver at palladion.com  Fri Oct 10 16:36:38 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Fri, 10 Oct 2014 10:36:38 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <5437A649.9080701@hastings.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
Message-ID: <m18qtn$7bc$2@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/10/2014 05:26 AM, Larry Hastings wrote:
> IMO the benefit from supporting other compilers on Windows is 
> negligible

Did you miss the OP's point that OpenBLAS cannot be compiled with MSVC,
raising the priority of mingw-buildable extensions for numerical work?


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlQ37vYACgkQ+gerLs4ltQ6V9ACffcq9Cn+kHzDZxaWJ63NVb6Uk
SasAoK5kNfBrCupcBz3FcRsUjs2aZxvu
=LFqg
-----END PGP SIGNATURE-----


From matthieu.brucher at gmail.com  Fri Oct 10 16:45:10 2014
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Fri, 10 Oct 2014 15:45:10 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m18qtn$7bc$2@ger.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
Message-ID: <CAHCaCk+_nxa3gJvVf=omjcPuhGPZXeVO_77EUcW6Z0RF1ba=xA@mail.gmail.com>

I don't think this is exactly on the same axis. Being able Python to
build with a free compiler won't change this issue. Scientific Python
won't be only the free compiler version, Visual Studio would remain
the main citizen. It may fragment a little bit more the environment
with people needing to pay attention to which compiler/Python version
the Python extension/module is compatible with.
And jettisonning Visual Studio for building Python is not an option
because of the amount of applications relying on both (Visual Studio
and Python) in companies.

Cheers,

Matthieu

2014-10-10 15:36 GMT+01:00 Tres Seaver <tseaver at palladion.com>:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> On 10/10/2014 05:26 AM, Larry Hastings wrote:
>> IMO the benefit from supporting other compilers on Windows is
>> negligible
>
> Did you miss the OP's point that OpenBLAS cannot be compiled with MSVC,
> raising the priority of mingw-buildable extensions for numerical work?
>
>
> Tres.
> - --
> ===================================================================
> Tres Seaver          +1 540-429-0999          tseaver at palladion.com
> Palladion Software   "Excellence by Design"    http://palladion.com
> -----BEGIN PGP SIGNATURE-----
> Version: GnuPG v1.4.11 (GNU/Linux)
>
> iEYEARECAAYFAlQ37vYACgkQ+gerLs4ltQ6V9ACffcq9Cn+kHzDZxaWJ63NVb6Uk
> SasAoK5kNfBrCupcBz3FcRsUjs2aZxvu
> =LFqg
> -----END PGP SIGNATURE-----
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/matthieu.brucher%40gmail.com



-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher
Music band: http://liliejay.com/

From p.f.moore at gmail.com  Fri Oct 10 16:56:21 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Oct 2014 15:56:21 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m18qtn$7bc$2@ger.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
Message-ID: <CACac1F_tBbWcR7ih32FNLo4F+Fecqh_dEcDDG7Kk1Yh=_w4+Vg@mail.gmail.com>

On 10 October 2014 15:36, Tres Seaver <tseaver at palladion.com> wrote:
> On 10/10/2014 05:26 AM, Larry Hastings wrote:
>> IMO the benefit from supporting other compilers on Windows is
>> negligible
>
> Did you miss the OP's point that OpenBLAS cannot be compiled with MSVC,
> raising the priority of mingw-buildable extensions for numerical work?

The proposal here is to make it possible to build *python* with a
different compiler. Being able to compile extensions with mingw is
independent. It used to be relatively easy to build extensions with
mingw, although Python's support for that has bitrotted to the point
where it's not worth it unless you have a strong need (which is why
the SciPy folks are working on it).

In my view, anyone capable of modifying the Python build process to
support other compilers would also be more than capable of improving
the distutils support for building extensions with mingw in a way that
is compatible with the current MSVC-built Python.

While I can't dictate where anyone else chooses to spend their time,
it seems to me that working on allowing non-MSVC builds of Python is a
sad waste of limited resources while the mingw extension building
issues remain.

Paul

From jcea at jcea.es  Fri Oct 10 17:45:50 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 17:45:50 +0200
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <20140903023738.1ddc8aac@fsol>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol>
Message-ID: <5437FF2E.6000404@jcea.es>

On 03/09/14 02:37, Antoine Pitrou wrote:

> I'm not sure that's an answer to the problem. What we need is not more
> machines, but dedicated buildbot maintainers.

I would love to get an email if my buildbots are consistently RED for a
few hours.

In the past Antoine, Victor and others pinged me about issues in my
buildbots (OpenIndiana). I feel shame and I think it is not very
productive either.

If red state in the buildbots were really trustable, the committer of
that changelog should be bugged too. Since we have quite a few false
positives, that would be probably annoying and no productive. Decreasing
false positives should be a priority.

In my case another issue is that my buildbots are hosted as a Solaris
ZONE inside a OpenIndiana development machine I don't manage. Their own
usage varies and can impact buildbots resource limits like memory, swap
space, etc.

Thanks for your patience and for notifying me issues when you suffer them.

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/75c8d6a5/attachment.sig>

From rosuav at gmail.com  Fri Oct 10 17:56:18 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sat, 11 Oct 2014 02:56:18 +1100
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <5437FF2E.6000404@jcea.es>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
Message-ID: <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>

On Sat, Oct 11, 2014 at 2:45 AM, Jesus Cea <jcea at jcea.es> wrote:
> On 03/09/14 02:37, Antoine Pitrou wrote:
>
>> I'm not sure that's an answer to the problem. What we need is not more
>> machines, but dedicated buildbot maintainers.
>
> I would love to get an email if my buildbots are consistently RED for a
> few hours.
>
> In the past Antoine, Victor and others pinged me about issues in my
> buildbots (OpenIndiana). I feel shame and I think it is not very
> productive either.

Likewise. In the earliest days of my buildbots, I kept a very close
eye on them (which was good; they had some occasional network
glitches, which I had to resolve), but once they became stable, I
stopped watching. Now, I almost never check them - as long as the VM's
running, I basically assume all's well. It'd be very helpful if a
computer could auto-email any time they're down for any length of
time. Is the info available in some way? Could I write a little
monitor at my end that asks every hour if my buildbots can be seen?

ChrisA

From charlesc-lists-python-dev2 at pyropus.ca  Fri Oct 10 17:54:17 2014
From: charlesc-lists-python-dev2 at pyropus.ca (Charles Cazabon)
Date: Fri, 10 Oct 2014 09:54:17 -0600
Subject: [Python-Dev] mUTF-7 support?
In-Reply-To: <54372DB2.70402@jcea.es>
References: <54371092.2070403@jcea.es>
 <CAMpsgwZ8p4iu7xu+4f1pL2Pd_49UGA1OxVWUxRH1Ya8MkKR9Lg@mail.gmail.com>
 <54371B66.8080108@jcea.es>
 <CAMpsgwZKb+dB0On+8gy2CUjNk_XEN6R_eRy7zapdj2Su4dAxag@mail.gmail.com>
 <543729A7.1080304@jcea.es>
 <CAMpsgwb=7LM25Cyh_OQjFrQVspcoWT2kCQKXQkb7A5K+3rByBg@mail.gmail.com>
 <54372DB2.70402@jcea.es>
Message-ID: <20141010155417.GA29246@pyropus.ca>

Jesus Cea <jcea at jcea.es> wrote:
> On 10/10/14 02:43, Victor Stinner wrote:
> >>> What is the current behaviour of imaplib in Python 3.4 with non-ASCII
> >>> characters in mailbox names?
> >>
> >> It breaks. Crash & burn.
> > 
> > Oh ok. So in short, imaplib doesn't work on Python 3:
> 
> Actually, it doesn't work in Python 2 either. It never supported
> international mailbox names.

Correct.  I had to hack in mUTF-7 support in getmail to properly support
international IMAP users.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From jcea at jcea.es  Fri Oct 10 18:00:06 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 18:00:06 +0200
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
 <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>
Message-ID: <54380286.7010604@jcea.es>

On 10/10/14 17:56, Chris Angelico wrote:
> Could I write a little
> monitor at my end that asks every hour if my buildbots can be seen?

AFAIK maintainers already get an email if the buildbot vanishes long
enough. I am more interested in getting an email when my buildbot is
consistently red because somebody committed something it doesn't like
two months ago...

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/171f7eda/attachment.sig>

From jcea at jcea.es  Fri Oct 10 18:00:11 2014
From: jcea at jcea.es (Jesus Cea)
Date: Fri, 10 Oct 2014 18:00:11 +0200
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <5437FF2E.6000404@jcea.es>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
Message-ID: <5438028B.9060300@jcea.es>

On 10/10/14 17:45, Jesus Cea wrote:
> Thanks for your patience and for notifying me issues when you suffer them.

Another issue is changes that actually breaks buildbots and I don't
actually know where to start debugging. For instance, currently:

<http://buildbot.python.org/all/builders/x86%20OpenIndiana%202.7/builds/2541/steps/test/logs/stdio>

"""
======================================================================
FAIL: test_init (test.test_readline.TestReadline)
----------------------------------------------------------------------
Traceback (most recent call last):
  File
"/export/home/buildbot/32bits/2.7.cea-indiana-x86/build/Lib/test/test_readline.py",
line 52, in test_init
    self.assertEqual(stdout, b'')
AssertionError: '\x1b[?1034h' != ''
"""

No idea what to do now, beside forcing bisection builds and blaming
somebody else :).

-- 
Jes?s Cea Avi?n                         _/_/      _/_/_/        _/_/_/
jcea at jcea.es - http://www.jcea.es/     _/_/    _/_/  _/_/    _/_/  _/_/
Twitter: @jcea                        _/_/    _/_/          _/_/_/_/_/
jabber / xmpp:jcea at jabber.org  _/_/  _/_/    _/_/          _/_/  _/_/
"Things are not so easy"      _/_/  _/_/    _/_/  _/_/    _/_/  _/_/
"My name is Dump, Core Dump"   _/_/_/        _/_/_/      _/_/  _/_/
"El amor es poner tu felicidad en la felicidad de otro" - Leibniz

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 473 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141010/2a2b61dd/attachment.sig>

From status at bugs.python.org  Fri Oct 10 18:08:14 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 10 Oct 2014 18:08:14 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20141010160814.63DBC56409@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-10-03 - 2014-10-10)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4598 (-33)
  closed 29769 (+90)
  total  34367 (+57)

Open issues with patches: 2152 


Issues opened (34)
==================

#7830: Flatten nested functools.partial
http://bugs.python.org/issue7830  reopened by belopolsky

#22526: file iteration SystemError for huge lines (2GiB+)
http://bugs.python.org/issue22526  reopened by doko

#22542: Use arc4random under OpenBSD for os.urandom() if /dev/urandom 
http://bugs.python.org/issue22542  reopened by 700eb415

#22552: ctypes.CDLL returns singleton objects, resulting in usage conf
http://bugs.python.org/issue22552  opened by Ivan.Pozdeev

#22554: Idle: optionally auto-pop-up completion window for names
http://bugs.python.org/issue22554  opened by terry.reedy

#22555: Tracking issue for adjustments to binary/text boundary handlin
http://bugs.python.org/issue22555  opened by ncoghlan

#22557: Local import is too slow
http://bugs.python.org/issue22557  opened by serhiy.storchaka

#22558: Missing hint to source code - complete
http://bugs.python.org/issue22558  opened by Friedrich.Spee.von.Langenfeld

#22559: [backport] ssl.MemoryBIO
http://bugs.python.org/issue22559  opened by alex

#22560: Add loop-agnostic SSL implementation to asyncio
http://bugs.python.org/issue22560  opened by pitrou

#22568: Use of "utime" as variable name in Modules/posixmodule.c cause
http://bugs.python.org/issue22568  opened by Jeffrey.Armstrong

#22570: Better stdlib support for Path objects
http://bugs.python.org/issue22570  opened by barry

#22571: Remove import * recommendations and examples in doc?
http://bugs.python.org/issue22571  opened by terry.reedy

#22575: bytearray documentation confuses string for unicode objects
http://bugs.python.org/issue22575  opened by mjpieters

#22577: local variable changes lost after pdb jump command
http://bugs.python.org/issue22577  opened by xdegaye

#22578: Add additional attributes to re.error
http://bugs.python.org/issue22578  opened by serhiy.storchaka

#22581: Other mentions of the buffer protocol
http://bugs.python.org/issue22581  opened by serhiy.storchaka

#22583: C stack overflow in the Python 2.7 compiler
http://bugs.python.org/issue22583  opened by Kevin.Dyer

#22585: os.urandom() should use getentropy() of OpenBSD 5.6
http://bugs.python.org/issue22585  opened by haypo

#22586: urljoin allow_fragments doesn't work
http://bugs.python.org/issue22586  opened by ColonelThirtyTwo

#22587: os.path.abspath(None) behavior is inconsistent between platfor
http://bugs.python.org/issue22587  opened by KevKeating

#22589: mimetypes uses image/x-ms-bmp as the type for bmp files
http://bugs.python.org/issue22589  opened by brma

#22590: math.copysign buggy with nan under Windows
http://bugs.python.org/issue22590  opened by pitrou

#22592: Drop support of Borland C compiler
http://bugs.python.org/issue22592  opened by haypo

#22593: Automate update of doc references to UCD version when it chang
http://bugs.python.org/issue22593  opened by r.david.murray

#22594: Add a link to the regex module in re documentation
http://bugs.python.org/issue22594  opened by serhiy.storchaka

#22596: support.transient_internet() doesn't catch connection refused 
http://bugs.python.org/issue22596  opened by berker.peksag

#22598: Add mUTF-7 codec (UTF-7 modified for IMAP)
http://bugs.python.org/issue22598  opened by jcea

#22599: traceback: errors in the linecache module at exit
http://bugs.python.org/issue22599  opened by haypo

#22600: In Multiprocessing Queue() doesn't work with list : __.put(lis
http://bugs.python.org/issue22600  opened by AlainCALMET

#22601: asyncio: loop.run_forever() should consume exception of the te
http://bugs.python.org/issue22601  opened by haypo

#22602: UTF-7 codec decodes ill-formed sequences starting with "+"
http://bugs.python.org/issue22602  opened by jwilk

#22603: Fix a typo in the contextlib docs
http://bugs.python.org/issue22603  opened by Francisco.Fern??ndez.Casta??o

#22604: assertion error in complex division
http://bugs.python.org/issue22604  opened by pitrou



Most recent 15 issues with no replies (15)
==========================================

#22604: assertion error in complex division
http://bugs.python.org/issue22604

#22603: Fix a typo in the contextlib docs
http://bugs.python.org/issue22603

#22602: UTF-7 codec decodes ill-formed sequences starting with "+"
http://bugs.python.org/issue22602

#22601: asyncio: loop.run_forever() should consume exception of the te
http://bugs.python.org/issue22601

#22600: In Multiprocessing Queue() doesn't work with list : __.put(lis
http://bugs.python.org/issue22600

#22596: support.transient_internet() doesn't catch connection refused 
http://bugs.python.org/issue22596

#22594: Add a link to the regex module in re documentation
http://bugs.python.org/issue22594

#22593: Automate update of doc references to UCD version when it chang
http://bugs.python.org/issue22593

#22589: mimetypes uses image/x-ms-bmp as the type for bmp files
http://bugs.python.org/issue22589

#22585: os.urandom() should use getentropy() of OpenBSD 5.6
http://bugs.python.org/issue22585

#22581: Other mentions of the buffer protocol
http://bugs.python.org/issue22581

#22577: local variable changes lost after pdb jump command
http://bugs.python.org/issue22577

#22575: bytearray documentation confuses string for unicode objects
http://bugs.python.org/issue22575

#22539: Table formatting errors in pydoc
http://bugs.python.org/issue22539

#22538: turtledemo two_canvases reversion
http://bugs.python.org/issue22538



Most recent 15 issues waiting for review (15)
=============================================

#22603: Fix a typo in the contextlib docs
http://bugs.python.org/issue22603

#22601: asyncio: loop.run_forever() should consume exception of the te
http://bugs.python.org/issue22601

#22599: traceback: errors in the linecache module at exit
http://bugs.python.org/issue22599

#22596: support.transient_internet() doesn't catch connection refused 
http://bugs.python.org/issue22596

#22592: Drop support of Borland C compiler
http://bugs.python.org/issue22592

#22578: Add additional attributes to re.error
http://bugs.python.org/issue22578

#22568: Use of "utime" as variable name in Modules/posixmodule.c cause
http://bugs.python.org/issue22568

#22560: Add loop-agnostic SSL implementation to asyncio
http://bugs.python.org/issue22560

#22559: [backport] ssl.MemoryBIO
http://bugs.python.org/issue22559

#22557: Local import is too slow
http://bugs.python.org/issue22557

#22552: ctypes.CDLL returns singleton objects, resulting in usage conf
http://bugs.python.org/issue22552

#22526: file iteration SystemError for huge lines (2GiB+)
http://bugs.python.org/issue22526

#22525: ast.literal_eval() doesn't do what the documentation says
http://bugs.python.org/issue22525

#22524: PEP 471 implementation: os.scandir() directory scanning functi
http://bugs.python.org/issue22524

#22522: sys.excepthook doesn't receive the traceback when called from 
http://bugs.python.org/issue22522



Top 10 most discussed issues (10)
=================================

#22515: Implement partial order on Counter
http://bugs.python.org/issue22515  21 msgs

#22568: Use of "utime" as variable name in Modules/posixmodule.c cause
http://bugs.python.org/issue22568  17 msgs

#22524: PEP 471 implementation: os.scandir() directory scanning functi
http://bugs.python.org/issue22524  16 msgs

#20167: Exception on IDLE closing
http://bugs.python.org/issue20167  12 msgs

#22181: os.urandom() should use Linux 3.17 getrandom() syscall
http://bugs.python.org/issue22181  11 msgs

#22486: Add math.gcd()
http://bugs.python.org/issue22486  10 msgs

#22526: file iteration SystemError for huge lines (2GiB+)
http://bugs.python.org/issue22526  10 msgs

#22547: Implement an informative `BoundArguments.__repr__`
http://bugs.python.org/issue22547  10 msgs

#17636: Modify IMPORT_FROM to fallback on sys.modules
http://bugs.python.org/issue17636   8 msgs

#22559: [backport] ssl.MemoryBIO
http://bugs.python.org/issue22559   8 msgs



Issues closed (85)
==================

#4832: IDLE does not supply a default ext of .py on Windows or OS X f
http://bugs.python.org/issue4832  closed by terry.reedy

#6359: pyexpat.c calls trace function incorrectly for exceptions
http://bugs.python.org/issue6359  closed by georg.brandl

#6653: Potential memory leak in multiprocessing
http://bugs.python.org/issue6653  closed by pitrou

#8065: Memory leak in readline.get_current_history_length
http://bugs.python.org/issue8065  closed by terry.reedy

#9417: Declaring a class creates circular references
http://bugs.python.org/issue9417  closed by georg.brandl

#10031: Withdraw anti-recommendation of relative imports from document
http://bugs.python.org/issue10031  closed by python-dev

#10583: Encoding issue with chm help in 2.7.1
http://bugs.python.org/issue10583  closed by georg.brandl

#10712: 2to3 fixer for deprecated unittest method names
http://bugs.python.org/issue10712  closed by r.david.murray

#11491: dbm.open(..., flag="n") raises dbm.error if file exists and is
http://bugs.python.org/issue11491  closed by r.david.murray

#11693: memory leak in email.generator.Generator().flatten() method
http://bugs.python.org/issue11693  closed by terry.reedy

#11866: race condition in threading._newname()
http://bugs.python.org/issue11866  closed by r.david.murray

#12148: Clarify "or-ing together" doctest option flags
http://bugs.python.org/issue12148  closed by python-dev

#13101: Module Doc viewer closes when browser window closes on Windows
http://bugs.python.org/issue13101  closed by georg.brandl

#14056: Misc doc changes for tarfile
http://bugs.python.org/issue14056  closed by r.david.murray

#14201: Documented caching for shared library's __getattr__ and __geti
http://bugs.python.org/issue14201  closed by r.david.murray

#14303: Incorrect documentation for socket.py on linux
http://bugs.python.org/issue14303  closed by python-dev

#15855: memoryview methods and data members are missing docstrings
http://bugs.python.org/issue15855  closed by r.david.murray

#16155: Some minor doc fixes in Doc/faq
http://bugs.python.org/issue16155  closed by python-dev

#16177: Typing left parenthesis in IDLE causes intermittent Cocoa Tk c
http://bugs.python.org/issue16177  closed by ned.deily

#16518: add "buffer protocol" to glossary
http://bugs.python.org/issue16518  closed by r.david.murray

#17057: Data model documentation grammar
http://bugs.python.org/issue17057  closed by python-dev

#17078: string.Template.safe_substitute hard-wires "braces" as {}
http://bugs.python.org/issue17078  closed by georg.brandl

#17444: multiprocessing.cpu_count() should use hw.availcpu on Mac OS X
http://bugs.python.org/issue17444  closed by pitrou

#18043: No mention of `match.regs` in `re` documentation
http://bugs.python.org/issue18043  closed by georg.brandl

#18176: Builtins documentation refers to old version of UCD.
http://bugs.python.org/issue18176  closed by r.david.murray

#18348: Additional code pages for EBCDIC
http://bugs.python.org/issue18348  closed by haypo

#18494: PyType_GenericSet/GetDict functions misnamed in docs?
http://bugs.python.org/issue18494  closed by python-dev

#18615: sndhdr.whathdr could return a namedtuple
http://bugs.python.org/issue18615  closed by r.david.murray

#19071: Documentation on what self is for module-level functions is mi
http://bugs.python.org/issue19071  closed by python-dev

#19380: Optimize parsing of regular expressions
http://bugs.python.org/issue19380  closed by serhiy.storchaka

#19472: inspect.getsource() raises a wrong exception type
http://bugs.python.org/issue19472  closed by yselivanov

#19477: document tp_print() as being dead in Py3
http://bugs.python.org/issue19477  closed by python-dev

#21052: Consider dropping ImportWarning for empty sys.path_hooks and s
http://bugs.python.org/issue21052  closed by brett.cannon

#21072: Python docs and downloads not available for Egypt
http://bugs.python.org/issue21072  closed by georg.brandl

#21173: WeakKeyDictionary.__len__ fragile w/ _IterationGuards
http://bugs.python.org/issue21173  closed by python-dev

#21428: Python suddenly cares about EOLs formats on windows
http://bugs.python.org/issue21428  closed by georg.brandl

#21456: skip 2 tests in test_urllib2net.py if _ssl module not present
http://bugs.python.org/issue21456  closed by berker.peksag

#21480: A build now requires...
http://bugs.python.org/issue21480  closed by python-dev

#21715: Chaining exceptions at C level
http://bugs.python.org/issue21715  closed by serhiy.storchaka

#21782: hashable documentation error: shouldn't mention id
http://bugs.python.org/issue21782  closed by python-dev

#21883: relpath: Provide better errors when mixing bytes and strings
http://bugs.python.org/issue21883  closed by serhiy.storchaka

#21905: RuntimeError in pickle.whichmodule  when sys.modules if mutate
http://bugs.python.org/issue21905  closed by pitrou

#21965: Add support for Memory BIO to _ssl
http://bugs.python.org/issue21965  closed by pitrou

#22219: python -mzipfile fails to add empty folders to created zip
http://bugs.python.org/issue22219  closed by serhiy.storchaka

#22222: dtoa.c: remove custom memory allocator
http://bugs.python.org/issue22222  closed by haypo

#22247: More incomplete module.__all__ lists
http://bugs.python.org/issue22247  closed by benjamin.peterson

#22290: "AMD64 OpenIndiana 3.x" buildbot: assertion failed in PyObject
http://bugs.python.org/issue22290  closed by haypo

#22292: pickle whichmodule RuntimeError
http://bugs.python.org/issue22292  closed by attilio.dinisio

#22331: test_io.test_interrupted_write_text() hangs on the buildbot Fr
http://bugs.python.org/issue22331  closed by haypo

#22332: test_multiprocessing_main_handling fail on buildbot "x86 FreeB
http://bugs.python.org/issue22332  closed by haypo

#22423: Errors in printing exceptions raised in a thread
http://bugs.python.org/issue22423  closed by serhiy.storchaka

#22449: SSLContext.load_verify_locations behavior on Windows and OSX
http://bugs.python.org/issue22449  closed by python-dev

#22462: Modules/pyexpat.c violates PEP 384
http://bugs.python.org/issue22462  closed by pitrou

#22470: Possible integer overflow in error handlers
http://bugs.python.org/issue22470  closed by serhiy.storchaka

#22482: logging: fileConfig doesn't support formatter styles
http://bugs.python.org/issue22482  closed by vinay.sajip

#22507: PyType_IsSubtype doesn't call __subclasscheck__
http://bugs.python.org/issue22507  closed by georg.brandl

#22508: Remove __version__ string from email
http://bugs.python.org/issue22508  closed by r.david.murray

#22546: Wrong default precision in documentation for format
http://bugs.python.org/issue22546  closed by terry.reedy

#22548: Bogus parsing of negative zeros in complex literals
http://bugs.python.org/issue22548  closed by mark.dickinson

#22549: bug in accessing bytes, inconsistent with normal strings and p
http://bugs.python.org/issue22549  closed by r.david.murray

#22550: issubclass can fail after module reloading
http://bugs.python.org/issue22550  closed by serhiy.storchaka

#22551: Anything results in a SyntaxError after -u on 2.7.8 on Windows
http://bugs.python.org/issue22551  closed by Taylor.Marks

#22553: Diagram issue
http://bugs.python.org/issue22553  closed by georg.brandl

#22556: datetime comparison with 'None' returning a TypeError
http://bugs.python.org/issue22556  closed by eric.smith

#22561: PyUnicode_InternInPlace crashes
http://bugs.python.org/issue22561  closed by ned.deily

#22562: Singleton pattern for namedtuple
http://bugs.python.org/issue22562  closed by r.david.murray

#22563: namedtuple: raise an exception when the given class name would
http://bugs.python.org/issue22563  closed by mark.dickinson

#22564: ssl: post-commit review of the new memory BIO API
http://bugs.python.org/issue22564  closed by haypo

#22565: missing const in declaration of PyErr_WarnEx in C-API docs
http://bugs.python.org/issue22565  closed by python-dev

#22566: International keyboard makes IDLE crash on OSX
http://bugs.python.org/issue22566  closed by ned.deily

#22567: doctest handle ignored execption
http://bugs.python.org/issue22567  closed by r.david.murray

#22569: Add support for weakrefs to _socket.socket
http://bugs.python.org/issue22569  closed by python-dev

#22572: NoneType object is not iterable error when asyncio Server.clos
http://bugs.python.org/issue22572  closed by r.david.murray

#22573: AttributeErrors in statements split up into multiple linse wit
http://bugs.python.org/issue22573  closed by r.david.murray

#22574: super() and del in the same method leads to UnboundLocalError
http://bugs.python.org/issue22574  closed by Miaou

#22576: ftplib documentation gives a wrong argument name for storbinar
http://bugs.python.org/issue22576  closed by berker.peksag

#22579: Posix module init function name should not be compiler-depende
http://bugs.python.org/issue22579  closed by haypo

#22580: PyUnicode_Tailmatch documentation does not match signature
http://bugs.python.org/issue22580  closed by python-dev

#22582: RuntimeError with string concatenation
http://bugs.python.org/issue22582  closed by Kevin.Dyer

#22584: Get rid of SRE character tables
http://bugs.python.org/issue22584  closed by serhiy.storchaka

#22588: memory corrupted in test_capi refleaks test
http://bugs.python.org/issue22588  closed by haypo

#22591: Drop support of MS-DOS (DJGPP compiler)
http://bugs.python.org/issue22591  closed by haypo

#22595: F5 shortcut key not working in IDLE for OSX
http://bugs.python.org/issue22595  closed by mc128k

#22597: printf-style formatting allows mixing keyed and keyless specif
http://bugs.python.org/issue22597  closed by eric.smith

#1519638: Unmatched Group issue - workaround
http://bugs.python.org/issue1519638  closed by serhiy.storchaka

From rdmurray at bitdance.com  Fri Oct 10 18:08:54 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 10 Oct 2014 12:08:54 -0400
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <54380286.7010604@jcea.es>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
 <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>
 <54380286.7010604@jcea.es>
Message-ID: <20141010160854.96CE8250EF7@webabinitio.net>

On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea <jcea at jcea.es> wrote:
> On 10/10/14 17:56, Chris Angelico wrote:
> > Could I write a little
> > monitor at my end that asks every hour if my buildbots can be seen?
> 
> AFAIK maintainers already get an email if the buildbot vanishes long
> enough. I am more interested in getting an email when my buildbot is
> consistently red because somebody committed something it doesn't like
> two months ago...

I think they are supposed to, but I've never gotten one, not even when
my gentoo buildbots suffered a hardware failure.  A month or so later
Antoine emailed me, and I told him to remove them at least for now,
since I don't currently have replacement hardware.  (I'm hoping to fix
that, but I have to find the time...)

That said, there has never as far as I know been an active hook to email
the maintainer when the buildbot is red.  I don't know if buildbot
supports that or not, so that would be the first thing to check.

--David

From benjamin at python.org  Fri Oct 10 18:11:12 2014
From: benjamin at python.org (Benjamin Peterson)
Date: Fri, 10 Oct 2014 12:11:12 -0400
Subject: [Python-Dev] Sad status of Python 3.x buildbots
In-Reply-To: <20141010160854.96CE8250EF7@webabinitio.net>
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
 <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>
 <54380286.7010604@jcea.es> <20141010160854.96CE8250EF7@webabinitio.net>
Message-ID: <1412957472.3622803.177525253.52317B73@webmail.messagingengine.com>

It's https://bugs.python.org/issue19884

On Fri, Oct 10, 2014, at 12:08, R. David Murray wrote:
> On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea <jcea at jcea.es> wrote:
> > On 10/10/14 17:56, Chris Angelico wrote:
> > > Could I write a little
> > > monitor at my end that asks every hour if my buildbots can be seen?
> > 
> > AFAIK maintainers already get an email if the buildbot vanishes long
> > enough. I am more interested in getting an email when my buildbot is
> > consistently red because somebody committed something it doesn't like
> > two months ago...
> 
> I think they are supposed to, but I've never gotten one, not even when
> my gentoo buildbots suffered a hardware failure.  A month or so later
> Antoine emailed me, and I told him to remove them at least for now,
> since I don't currently have replacement hardware.  (I'm hoping to fix
> that, but I have to find the time...)
> 
> That said, there has never as far as I know been an active hook to email
> the maintainer when the buildbot is red.  I don't know if buildbot
> supports that or not, so that would be the first thing to check.
> 
> --David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/benjamin%40python.org

From solipsis at pitrou.net  Fri Oct 10 18:16:10 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 10 Oct 2014 18:16:10 +0200
Subject: [Python-Dev] Sad status of Python 3.x buildbots
References: <CAMpsgwb-9cfOmuY9BeiCfeWF6cZ+yw1rnxx_EF_83RH6HYfH_A@mail.gmail.com>
 <CADiSq7cTr_fswv-59bUZwqmTPJm5tucsOJ3mRQzWQeFKKMpgCg@mail.gmail.com>
 <20140903023738.1ddc8aac@fsol> <5437FF2E.6000404@jcea.es>
 <CAPTjJmoXgLa-8=WXKGZSby-sMhQg8_jJTV5p66FeQ32sU7gxzw@mail.gmail.com>
 <54380286.7010604@jcea.es>
 <20141010160854.96CE8250EF7@webabinitio.net>
Message-ID: <20141010181610.6120a84b@fsol>

On Fri, 10 Oct 2014 12:08:54 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:
> On Fri, 10 Oct 2014 18:00:06 +0200, Jesus Cea <jcea at jcea.es> wrote:
> > On 10/10/14 17:56, Chris Angelico wrote:
> > > Could I write a little
> > > monitor at my end that asks every hour if my buildbots can be seen?
> > 
> > AFAIK maintainers already get an email if the buildbot vanishes long
> > enough. I am more interested in getting an email when my buildbot is
> > consistently red because somebody committed something it doesn't like
> > two months ago...
> 
> I think they are supposed to, but I've never gotten one, not even when
> my gentoo buildbots suffered a hardware failure.

As far as I remember it's a relaying problem on the SMTP server.
Perhaps Benjamin can confirm.

> That said, there has never as far as I know been an active hook to email
> the maintainer when the buildbot is red.  I don't know if buildbot
> supports that or not, so that would be the first thing to check.

I don't know if it would support e-mailing the maintainer rather than
whoever committed the last changes.
Besides, I'm afraid our test suite isn't stable enough for such e-mail
notifications to be bearable...

Regards

Antoine.



From breamoreboy at yahoo.co.uk  Fri Oct 10 18:28:00 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Fri, 10 Oct 2014 17:28:00 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <m191ej$ov5$1@ger.gmane.org>

On 10/10/2014 01:29, Victor Stinner wrote:

> === MinGW
>
> Some people tried to compile Python. See for example:
> https://bitbucket.org/puqing/python-mingw
>
> We even got some patches:
> http://bugs.python.org/issue3871 (rejected)
>

There are 55 open issues on the bug tracker with mingw in the title.

> See also:
> https://stackoverflow.com/questions/15365249/build-python-with-mingw-and-gcc
>
> MinGW reuses the Microsoft C library and it is based on GCC which is
> very stable, actively developed, supports a lot of archiectures, etc.
> I guess that it should be possible to reuse third party GCC tools like
> the famous GDB debugger?
>
-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From jtaylor.debian at googlemail.com  Fri Oct 10 18:49:33 2014
From: jtaylor.debian at googlemail.com (Julian Taylor)
Date: Fri, 10 Oct 2014 18:49:33 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8wfCnOCC3zpbp4adKNrivj0VH7pc5bHwN21XAHWTeefw@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org>
 <CAMpsgwa4x3wavT4TZg_GvZorXMALGxScFXHvwcmkhhN7F5XL6A@mail.gmail.com>
 <CACac1F8wfCnOCC3zpbp4adKNrivj0VH7pc5bHwN21XAHWTeefw@mail.gmail.com>
Message-ID: <54380E1D.6090001@googlemail.com>

On 10.10.2014 14:05, Paul Moore wrote:
> On 10 October 2014 10:50, Victor Stinner <victor.stinner at gmail.com> wrote:
>> Is MinGW fully compatible with MSVS ABI? I read that it reuses the
>> MSVCRT, but I don't know if it's enough. I guess that a full ABI
>> compatibility means more than just using the C library, calling
>> convention and much more.
> 
> MinGW can be made to build ABI-compatible extensions. Whether this
> will continue with MSVC 15 I don't know, as it requires a change to
> add an interface library for the relevant msvcrXX runtime. And the
> MinGW community is somewhat fragmented these days, with the core
> project not supporting 64-bit and various external projects doing so.
> 
> Having said all this, it *is* possible with some effort to use MinGW
> to build Python extensions. As noted, the numpy developers have done a
> lot of work on this as some of the libraries they need must be built
> with mingw. And the state of distutils support for mingw is very sad,
> as well, IIRC (last time I looked there were a number of open bugs,
> and very little movement on them).
> 
> Rather than put effort into more build options for CPython, I think it
> would be much more beneficial to the Windows community if effort was
> put into:
> ....
> 2. Looking at ways to support cross-compiling Windows extensions from
> Linux using mingw. I've no idea how practical this would be, but if
> Linux developers could provide Windows builds without having to
> maintain a Windows environment, that would be great.

It is practical. Numpy Windows binaries are built on linux using mingw
3.4.5 and wine.
The (vagrant based) setup which is currently used is available here:
https://github.com/juliantaylor/numpy-vendor
For the next release we do want to look into providing "official" win64
binaries based on the mingw64 toolchain that has been mentioned a few
times already. An attempt to do so in the last released failed due to
test issues and there were no experienced debuggers available to solve them.

>From my perspective cross building for windows is easier than cross
building for mac, but thats probably just because I never seriously
looked into that.

From p.f.moore at gmail.com  Fri Oct 10 19:01:04 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 10 Oct 2014 18:01:04 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m191ej$ov5$1@ger.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <m191ej$ov5$1@ger.gmane.org>
Message-ID: <CACac1F8-f7WZ_Nzybb4iUE6SuawsOcLs63qXxLky3nufSBe4Ng@mail.gmail.com>

On 10 October 2014 17:28, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
> There are 55 open issues on the bug tracker with mingw in the title.

It's not easy to tell, but on a spot check a fair proportion of them
seem to be about distutils/extension builds. And a lot of the rest are
related to http://bugs.python.org/issue3871 which is a rejected issue
about adding build support for mingw.

Paul

From Steve.Dower at microsoft.com  Sat Oct 11 00:59:05 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Fri, 10 Oct 2014 22:59:05 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <f49bdb4b9cf74499813dae682f0a50ab@DM2PR0301MB0734.namprd03.prod.outlook.com>

From Victor Stinner:
> I know that it's hard to replace Visual Studio. I don't want to do it right now, but I would like to discuss that with you.


I have read the rest of the thread, but I want to start from this point. I'm probably going to run off in random directions since there are a lot of issues raised here (all of which I've heard before and given serious thought and discussion to), so please excuse the brain dump.

As one of the driving forces behind keeping MSVC a viable choice for building CPython, I'd be disappointed if we were to change away from it (as would my employer). That said, I'm involved with Python because I want to be involved - I was developing in Python long before Microsoft hired me and I'm incredibly lucky that my day job permits me to be involved in this community - and it's not going to directly hurt my career if everyone decides to move away from MSVC, so I don't believe I'm conflicted here.

The main way I'm currently trying to keep MSVC viable is by porting CPython 3.5 to build with the latest version (VC14 - yet to be released, but there are previews available). My progress is in my sandbox at https://hg.python.org/sandbox/steve.dower (VC14 branch), I've been regularly testing with the latest (internal) builds, fixing issues in Python and getting issues fixed in OpenSSL, Tcl and VC. The known PGO bugs have been fixed for VC14, and one of my plans is to do some thorough testing to see if it's safe and worth re-enabling.

Some answers to points that were raised in this discussion:

* VC 14 Express for Desktop can build both 32-bit and 64-bit versions of CPython
 - I'm also making changes to Python's VC projects so they can build correctly without VS, though I don't know if we'll have a compiler only package for VC14

* The main advantage of VC14 is that the C runtime will be binary compatible into the future (rather than *mostly* source compatible)
 - most of the macros are now function calls and opaque types like FILE* are genuinely opaque

* Extensions built with VC14 or later will work with CPython built with VC14 or later
 - If other compilers begin to support the new CRT ABI and can match the calling convention, then it won't matter which compiler is used for extensions
 - The MS CRT has public sources - they ship with any version of VC14 - which should help 

* There are plenty of issues that prevent you from building with VC14 currently, just like with any other compiler that isn't VC10.
 - Platforms and compilers need dedicated maintainers

I don't have any official confirmation, but my guess would be that the 64-bit compilers were omitted from the VC 2008 Express to save space (bearing in mind that WinXP was the main target at that time, which had poor 64-bit support, and very few people cared about building 64-bit binaries) and were not available in the IDE for VC 2010 Express by mistake. For building extensions, the former is resolved by the package at http://aka.ms/vcpython27, and the latter works fine since the 64-bit compiler is there, just not exposed in the IDE. Neither of these will be an issue with VC14 - 64-bit is far too important these days.

Cross compilation is a valid issue, but I hope that build services like Appveyor also help out here. There is regular talk about the PSF/PyPI providing something similar, though I have doubts about its feasibility under any model other than renting a preconfigured VM. I don't see any reason why projects couldn't apply to the PSF for a grant to cover Appveyor costs or the expenses of similar services.

As for extensions with Fortran or BLAS dependencies - the Python ABI requirements only extend to the Python interface. You can easily define a second interface within your project and build dependencies with whatever tools you like (and in theory if your compiler emits COFF modules, you can link objects from separate compilers, though that's probably easier said than done). When you control both sides of the ABI, the tooling is less important. It's just unfortunate that right now the ABI for Python is defined by the tooling - hopefully the VC14 CRT will help stabilise this.

Currently the official CPython 2.7 build from python.org is built with VS 2008 Ultimate SP1 and I assume 3.4 is built with VS 2010 SP1, since I know Martin has access to the full version of VS. As Paul mentioned a few times, I'd also be quite happy to see all the effort towards building CPython with other compilers go towards building extensions with other compilers. On Windows, very very few people ever build their own binaries for anything - it's a totally different culture to *nix - so it's far more important that the development teams can build their own product. In this scenario, it's more important that extension developers know exactly what the ABI is so they can match it, than whether it's exactly the same as what their tools use by default. 

I don't think there's anything actionable in there, but here's a few things we could do (discussion points - I'm not necessarily +1 on all of them):

* Deprecate/remove support for compiling CPython itself with compilers other than MSVC on Windows
* Add preprocessor magic to pyconfig.h to fail extension builds with mismatched ABIs
* Improve distutils to automatically detect and support more compilers and cross-compilation where possible
* Help out on distutils-sig to finish the PEPs that are holding up improvements to binary distribution
* Reach out to companies to help specific projects (e.g. ask Company X if they're willing/able to produce official Y builds on Windows on behalf of the Y team)
* Help projects apply to the PSF for grants for Appveyor/similar services

Cheers,
Steve

From donald at stufft.io  Sat Oct 11 01:39:10 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 10 Oct 2014 19:39:10 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <f49bdb4b9cf74499813dae682f0a50ab@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <f49bdb4b9cf74499813dae682f0a50ab@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <720EDE52-ACAD-445A-AE46-7CD9041C5A9A@stufft.io>


> On Oct 10, 2014, at 6:59 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> 
> Cross compilation is a valid issue, but I hope that build services like Appveyor also help out here. There is regular talk about the PSF/PyPI providing something similar, though I have doubts about its feasibility under any model other than renting a preconfigured VM. I don't see any reason why projects couldn't apply to the PSF for a grant to cover Appveyor costs or the expenses of similar services.

I know very little about Windows, but we can spin up arbitrary Windows boxes, build something on them, and then shut them down as long as there?s some way to automate logging into a Windows box and running some commands? Normally I?d do this with SSH but I don?t know if Windows has anything like that.

IOW we can totally spin up preconfigured VMs for a Windows build service.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From larry at hastings.org  Sat Oct 11 01:45:11 2014
From: larry at hastings.org (Larry Hastings)
Date: Sat, 11 Oct 2014 00:45:11 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m18qtn$7bc$2@ger.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
Message-ID: <54386F87.9090503@hastings.org>


On 10/10/2014 03:36 PM, Tres Seaver wrote:
> On 10/10/2014 05:26 AM, Larry Hastings wrote:
>> IMO the benefit from supporting other compilers on Windows is
>> negligible
> Did you miss the OP's point that OpenBLAS cannot be compiled with MSVC,
> raising the priority of mingw-buildable extensions for numerical work?

CPython doesn't require OpenBLAS.  Not that I am not receptive to the 
needs of the numeric community... but, on the other hand, who in the 
hell releases a library with Windows support that doesn't work with MSVC?!


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141011/19582991/attachment.html>

From sturla.molden at gmail.com  Sat Oct 11 02:23:17 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 00:23:17 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <f49bdb4b9cf74499813dae682f0a50ab@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <551820588434679008.599292sturla.molden-gmail.com@news.gmane.org>

Steve Dower <Steve.Dower at microsoft.com> wrote:

> I don't have any official confirmation, but my guess would be that the
> 64-bit compilers were omitted from the VC 2008 Express to save space
> (bearing in mind that WinXP was the main target at that time, which had
> poor 64-bit support, and very few people cared about building 64-bit
> binaries) and were not available in the IDE for VC 2010 Express by
> mistake. For building extensions, the former is resolved by the package at
> http://aka.ms/vcpython27, and the latter works fine since the 64-bit
> compiler is there, just not exposed in the IDE. Neither of these will be
> an issue with VC14 - 64-bit is far too important these days.


The 64-bit compiler is in VC 2008 Express as well, just not exposed in the
IDE. I know this because when I got the Absoft Fortran compiler I was told
to download VC 2008 Express, because Absoft uses the VC9 linker. And indeed
there was a 64-bit compiler in VC 2008 Express as well, just not available
from the IDE. If I remeber correctly, some fiddling with vcvars.bat was
required to turn it on. I never tried to build Python extensions with it,
though. In the beginning I thought Absoft had given me the wrong product,
because I had ordered a 64-bit Fortran compiler and I "knew" VC 2008
Express was only 32-bit. But they assured me the 64-bit VC9 compiler was
there as well, and indeed it was.

Sturla


From sturla.molden at gmail.com  Sat Oct 11 02:30:51 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 00:30:51 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
Message-ID: <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>

Larry Hastings <larry at hastings.org> wrote:

> CPython doesn't require OpenBLAS.  Not that I am not receptive to the
> needs of the numeric community... but, on the other hand, who in the
> hell releases a library with Windows support that doesn't work with MSVC?!

It uses AT&T assembly syntax instead of Intel assembly syntax. 

Sturla


From solipsis at pitrou.net  Sat Oct 11 15:41:37 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 11 Oct 2014 15:41:37 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
Message-ID: <20141011154137.0597141a@fsol>

On Sat, 11 Oct 2014 00:30:51 +0000 (UTC)
Sturla Molden <sturla.molden at gmail.com> wrote:
> Larry Hastings <larry at hastings.org> wrote:
> 
> > CPython doesn't require OpenBLAS.  Not that I am not receptive to the
> > needs of the numeric community... but, on the other hand, who in the
> > hell releases a library with Windows support that doesn't work with MSVC?!
> 
> It uses AT&T assembly syntax instead of Intel assembly syntax. 

But you can compile OpenBLAS with one compiler and then link it to
Python using another compiler, right? There is a single C ABI.

Regards

Antoine.



From sturla.molden at gmail.com  Sat Oct 11 15:59:52 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 13:59:52 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
Message-ID: <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:

> But you can compile OpenBLAS with one compiler and then link it to
> Python using another compiler, right? There is a single C ABI.

BLAS and LAPACK are actually Fortran, which does not have a single C ABI.
The ABI depends on the Fortran compiler. g77 and gfortran will produce
different C ABIs. This is a consistent source of PITA  in any scientific
programming that combines C and Fortran.

There is cblas though, which is a C API, but it does not include LAPACK.

Another thing is that libraries are different. MSVC wants a .lib file, but
MinGW produces .a files like GCC does on Linux. Perhaps you can generate a
.lib file from a .a file, but I have never tried.

Sturla


From solipsis at pitrou.net  Sat Oct 11 16:08:17 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 11 Oct 2014 16:08:17 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
Message-ID: <20141011160817.2f3b0ebe@fsol>

On Sat, 11 Oct 2014 13:59:52 +0000 (UTC)
Sturla Molden <sturla.molden at gmail.com> wrote:
> Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> > But you can compile OpenBLAS with one compiler and then link it to
> > Python using another compiler, right? There is a single C ABI.
> 
> BLAS and LAPACK are actually Fortran, which does not have a single C ABI.
> The ABI depends on the Fortran compiler. g77 and gfortran will produce
> different C ABIs. This is a consistent source of PITA  in any scientific
> programming that combines C and Fortran.

But is that CPython's fault? I don't think so.

> Another thing is that libraries are different. MSVC wants a .lib file, but
> MinGW produces .a files like GCC does on Linux.

It sound like whatever MSVC produces should be the defacto standard
under Windows.

If Microsoft released a (GNU/)Linux compiler which produced
incompatible library files, I don't think anyone would regard them very
highly.

Regards

Antoine.



From sturla.molden at gmail.com  Sat Oct 11 16:15:13 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 14:15:13 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
Message-ID: <1764215136434728996.097758sturla.molden-gmail.com@news.gmane.org>

Sturla Molden <sturla.molden at gmail.com> wrote:

> BLAS and LAPACK are actually Fortran, which does not have a single C ABI.
> The ABI depends on the Fortran compiler. g77 and gfortran will produce
> different C ABIs. This is a consistent source of PITA  in any scientific
> programming that combines C and Fortran.
> 
> There is cblas though, which is a C API, but it does not include LAPACK.
> 
> Another thing is that libraries are different. MSVC wants a .lib file, but
> MinGW produces .a files like GCC does on Linux. Perhaps you can generate a
> .lib file from a .a file, but I have never tried.

And not to mention that the Fortran run-time depends on the C runtime...
What Carl Keffner did for SciPy was to use a static libgfortran, which is
not liked against any specific CRT, so it could be linked with msvcr90.dll
when the Python extension is built. The vanilla libgfortran.dll from MinGW
is linked with msvcrt.dll. However, not linking with msvcrt.dll broke the
pthreads library, which in turn broke OpenMP, so he had to patch the
pthreads library for this... This just shows some of the difficulties of
trying to combine the GNU and Microsoft compilers. There are many others,
like different stack alignment, differenr exception handling, and the mingw
runtime (which causes segfaults when linked dynamically to MSVC
executables). It's not just getting the CRT right.

Sturla


From sturla.molden at gmail.com  Sat Oct 11 16:21:33 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 14:21:33 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
Message-ID: <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>

Antoine Pitrou <solipsis at pitrou.net> wrote:

> It sound like whatever MSVC produces should be the defacto standard
> under Windows.

Yes, and that is what Clang does on Windows. It is not as usable as MinGW
yet, but soon it will be. Clang also suffers fronthe lack of a Fortran
compiler, though.

Sturla


From Steve.Dower at microsoft.com  Sat Oct 11 17:33:45 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 11 Oct 2014 15:33:45 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>,
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
Message-ID: <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>

Is there some reason the Fortran part can't be separated out into a DLL? That's the C ABI Antoine was referring to, and most compilers can generate import libraries from binaries, even if the original compiler produced then in a different format.

Top-posted from my Windows Phone
________________________________
From: Sturla Molden<mailto:sturla.molden at gmail.com>
Sent: ?10/?11/?2014 7:22
To: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Status of C compilers for Python on Windows

Antoine Pitrou <solipsis at pitrou.net> wrote:

> It sound like whatever MSVC produces should be the defacto standard
> under Windows.

Yes, and that is what Clang does on Windows. It is not as usable as MinGW
yet, but soon it will be. Clang also suffers fronthe lack of a Fortran
compiler, though.

Sturla

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141011/48bb6a33/attachment.html>

From njs at pobox.com  Sat Oct 11 16:01:07 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 11 Oct 2014 15:01:07 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141011154137.0597141a@fsol>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
Message-ID: <CAPJVwBkr3x5F_b19cA4OTkBjf7tBC5CV+W2ceKBS_d7TQhzi4w@mail.gmail.com>

On 11 Oct 2014 14:42, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> On Sat, 11 Oct 2014 00:30:51 +0000 (UTC)
> Sturla Molden <sturla.molden at gmail.com> wrote:
> > Larry Hastings <larry at hastings.org> wrote:
> >
> > > CPython doesn't require OpenBLAS.  Not that I am not receptive to the
> > > needs of the numeric community... but, on the other hand, who in the
> > > hell releases a library with Windows support that doesn't work with
MSVC?!
> >
> > It uses AT&T assembly syntax instead of Intel assembly syntax.
>
> But you can compile OpenBLAS with one compiler and then link it to
> Python using another compiler, right? There is a single C ABI.

In theory, yes, but this is pretty misleading. The theory has been known
for years. In practice we've only managed to pull this off for the first
time within the last few months, and it requires one specific build of one
specific mingw fork with one specific set of build options, and only one
person understands the details. Hopefully all those things will continue to
be available and there aren't any showstopper bugs we haven't noticed yet...

(Before that we've spent the last 5+ years using a carefully preserved
build of an ancient 32 bit mingw and substandard BLAS .dll that were handed
down from our ancestors; we've had no capability to produce 64 bit official
builds at all. And a large proportion of users have been using 3rd party
proprietary builds.)

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141011/2aa25a1c/attachment.html>

From sturla.molden at gmail.com  Sat Oct 11 18:58:14 2014
From: sturla.molden at gmail.com (Sturla Molden)
Date: Sat, 11 Oct 2014 16:58:14 +0000 (UTC)
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
 <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>
Message-ID: <1266336123434737411.887465sturla.molden-gmail.com@news.gmane.org>

Steve Dower <Steve.Dower at microsoft.com> wrote:

> Is there some reason the Fortran part can't be separated out into a DLL? 

DLL hell, I assume. Using the Python extension module loader makes it less
of a problem. If we stick with .pyd files where everything is statically
linked we can rely on the Python dev team to make sure that DLL hell does
not bite us. Most of the contributors to projects like NumPy and SciPy are
not computer scientists. So the KISS principle is important, which is why
scientific programmers often use Fortran in the first place. Making sure
DLLs are resolved and loaded correctly, or using stuff like COM or .NET to
mitigate DLL hell, is just in a different league. That is for computer
engineers to take care of, but we are trained as physicists, matematicians,
astronomers, chemists, biologists, or what ever... I am sure that engineers
at Microsoft could do this correctly, but we are not the kind of guys you
would hire :-)


OT: Contrary to common belief, there is no speed advantage of using Fortran
on a modern CPU, because the long pipeline and the hierarchical memory
alleviates the problems with pointer aliasing. C code tends to run faster
then Fortran, often 10 to 20 % faster, and C++ tends to be slightly faster
than C. In 2014, Fortran is only used because it is easier to program for
non-specialists. And besides, correctness is far more important than speed,
which is why we prefer Python or MATLAB in the first place. If you ever see
the argument that Fortran is used because of pointer aliasing, please feel
free to ignore it.


Sturla


From Steve.Dower at microsoft.com  Sat Oct 11 19:44:14 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 11 Oct 2014 17:44:14 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1266336123434737411.887465sturla.molden-gmail.com@news.gmane.org>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
 <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>,
 <1266336123434737411.887465sturla.molden-gmail.com@news.gmane.org>
Message-ID: <7c73341b866e4804abd734eac2ed27b5@DM2PR0301MB0734.namprd03.prod.outlook.com>

DLLs linked by import library at compile time (ie. not using LoadLibrary calls) and placed in the same directory as the .pyd should be exempt from DLL hell - Python already creates an activation context when importing pyds to let them load their own dependencies. Multiple CRTs are also okay as long as they don't try and share state (such as file descriptors) across boundaries.

I do understand the lack of knowledge and experience though. I've helped out in the past but I personally don't have the bandwidth to contribute more, nor the persuasiveness to get someone else to.

Top-posted from my Windows Phone
________________________________
From: Sturla Molden<mailto:sturla.molden at gmail.com>
Sent: ?10/?11/?2014 9:59
To: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Status of C compilers for Python on Windows

Steve Dower <Steve.Dower at microsoft.com> wrote:

> Is there some reason the Fortran part can't be separated out into a DLL?

DLL hell, I assume. Using the Python extension module loader makes it less
of a problem. If we stick with .pyd files where everything is statically
linked we can rely on the Python dev team to make sure that DLL hell does
not bite us. Most of the contributors to projects like NumPy and SciPy are
not computer scientists. So the KISS principle is important, which is why
scientific programmers often use Fortran in the first place. Making sure
DLLs are resolved and loaded correctly, or using stuff like COM or .NET to
mitigate DLL hell, is just in a different league. That is for computer
engineers to take care of, but we are trained as physicists, matematicians,
astronomers, chemists, biologists, or what ever... I am sure that engineers
at Microsoft could do this correctly, but we are not the kind of guys you
would hire :-)


OT: Contrary to common belief, there is no speed advantage of using Fortran
on a modern CPU, because the long pipeline and the hierarchical memory
alleviates the problems with pointer aliasing. C code tends to run faster
then Fortran, often 10 to 20 % faster, and C++ tends to be slightly faster
than C. In 2014, Fortran is only used because it is easier to program for
non-specialists. And besides, correctness is far more important than speed,
which is why we prefer Python or MATLAB in the first place. If you ever see
the argument that Fortran is used because of pointer aliasing, please feel
free to ignore it.


Sturla

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141011/fe2f2f97/attachment.html>

From njs at pobox.com  Sat Oct 11 20:32:22 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 11 Oct 2014 19:32:22 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
 <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>
Message-ID: <CAPJVwBkGpvFfzofwSX_KiBe8=Q0Ljx=NROMtVBRVUJHiBrAFCw@mail.gmail.com>

I'm not at all an expert on Fortran ABIs, but I think there are two
distinct issues being conflated here.

The first is that there is no standard way to look at some Fortran source
code and figure out the corresponding C API. When trying to call a Fortran
routine from C, then different Fortran compilers require different sorts of
name mangling, different ways of mapping Fortran concepts like "output
arguments" onto C concepts like "pointers", etc., so your C code needs to
have explicit knowledge of which Fortran compiler is in use. That's what
Sturla was referring to with the differences between g77 versus gfortran
etc. This is all very annoying, but it isn't a *deep* bad - it can be
solved by unilateral action by whatever project wants to link the Fortran
library.

The bigger problem is that getting a usable DLL at all is a serious
challenge. Some of the issues we deal with: (a) the classic, stable mingw
has no 64-bit support, (b) the only portable way to compile fortran (f2c)
only works for the ancient fortran 77, (c) getting even mingw-w64 to use a
msvc-compatible ABI is not trivial (I have no idea why this is the case,
but it is), (d)
<https://github.com/rust-lang/rust/issues/1768#issuecomment-4007553>mingw-built
dlls normally depend on the mingw runtime dlls. Because these aren't
shipped globally with Python, they have to be either linked statically or
else a separate copy of them has to be placed into every directory that
contains any mingw-compiled extension module.

All the runtime and ABI issues do mean that it would be much easier to use
mingw(-w64) to build extension modules if Python itself were built with
mingw(-w64). Obviously this would in turn make it harder to build
extensions with MSVC, though, which would be a huge transition. I don't
know whether gcc's advantages (support for more modern C, better
cross-platform compatibility, better accessibility to non-windows-experts,
etc.) would outweigh the transition and other costs.

As an intermediate step, there are almost certainly things that could be
done to make it easier to use mingw-w64 to build python extensions, e.g.
teaching setuptools about how to handle the ABI issues. Maybe it would even
be possible to ship the mingw runtimes in some globally available location.

-n
On 11 Oct 2014 17:07, "Steve Dower" <Steve.Dower at microsoft.com> wrote:

>   Is there some reason the Fortran part can't be separated out into a
> DLL? That's the C ABI Antoine was referring to, and most compilers can
> generate import libraries from binaries, even if the original compiler
> produced then in a different format.
>
> Top-posted from my Windows Phone
>  ------------------------------
> From: Sturla Molden <sturla.molden at gmail.com>
> Sent: ?10/?11/?2014 7:22
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Status of C compilers for Python on Windows
>
>   Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> > It sound like whatever MSVC produces should be the defacto standard
> > under Windows.
>
> Yes, and that is what Clang does on Windows. It is not as usable as MinGW
> yet, but soon it will be. Clang also suffers fronthe lack of a Fortran
> compiler, though.
>
> Sturla
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/njs%40pobox.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141011/25239f69/attachment-0001.html>

From p.f.moore at gmail.com  Sat Oct 11 23:12:25 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 11 Oct 2014 22:12:25 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAPJVwBkGpvFfzofwSX_KiBe8=Q0Ljx=NROMtVBRVUJHiBrAFCw@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
 <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>
 <CAPJVwBkGpvFfzofwSX_KiBe8=Q0Ljx=NROMtVBRVUJHiBrAFCw@mail.gmail.com>
Message-ID: <CACac1F_ne5E7_h_qxd-BpRCfkHWp3i3cs+ftL3JgG51RWGKBng@mail.gmail.com>

On 11 October 2014 19:32, Nathaniel Smith <njs at pobox.com> wrote:
> The bigger problem is that getting a usable DLL at all is a serious
> challenge. Some of the issues we deal with: (a) the classic, stable mingw
> has no 64-bit support, (b) the only portable way to compile fortran (f2c)
> only works for the ancient fortran 77, (c) getting even mingw-w64 to use a
> msvc-compatible ABI is not trivial (I have no idea why this is the case, but
> it is), (d) mingw-built dlls normally depend on the mingw runtime dlls.
> Because these aren't shipped globally with Python, they have to be either
> linked statically or else a separate copy of them has to be placed into
> every directory that contains any mingw-compiled extension module.

These are all genuine and difficult issues. I have some knowledge of
the mingw environment, although only as a user, and I can confirm that
it is not in an ideal state. As you mention, the core project only
offers 32-bit compilers, and the 64-bit versions are separate, not
always reliable, projects. It can be the case that getting a
successful build relies on using a precise distributed archive of the
relevant mingw binaries.

By default, mingw uses msvcrt, for essentially licensing reasons (to
do with the fact that msvcrt is the only version they can consider as
being "shipped with the OS" which is relevant to their use of the
GPL). To get it to link to an alternative VC runtime requires a symbol
library for that runtime, which needs someone to build it. I don't
know whether mingw includes runtime symbol libraries for later than
msvcr10[1] - I asked for that to be added when Python switched to
VC10, and it was fairly difficult to get that done, even at that point
when the mingw community was much less fragmented than now.

It is possible to build C extensions with mingw in such a way that
they don't depend on the mingw runtime DLLs - at least the distutils
support made that happen for the average extension when I last worked
on it which was pre-Python 3... I'm pretty sure that C++ needs runtime
support DLLs which would be tricky to avoid, and I have no idea what
sorts of difficulty Fortran might add (although your comments sound
about what I expect).

> All the runtime and ABI issues do mean that it would be much easier to use
> mingw(-w64) to build extension modules if Python itself were built with
> mingw(-w64). Obviously this would in turn make it harder to build extensions
> with MSVC, though, which would be a huge transition. I don't know whether
> gcc's advantages (support for more modern C, better cross-platform
> compatibility, better accessibility to non-windows-experts, etc.) would
> outweigh the transition and other costs.

As mentioned above, I don't think the mingw environment is that
reliable, which would be an issue if Python were built with it. Would
it really be a positive step if we had to say "to build Python you
need to download this precise personal build from a specific mingw64
spin-off project"? And yes, I have code for which that is *precisely*
what I need to do.

> As an intermediate step, there are almost certainly things that could be
> done to make it easier to use mingw-w64 to build python extensions, e.g.
> teaching setuptools about how to handle the ABI issues. Maybe it would even
> be possible to ship the mingw runtimes in some globally available location.

As I've said a number of times now, I think this is much more likely
to be a useful avenue. For example, shipping the appropriate
libmsvcrXXX.a and static libraries for the relevant runtimes with
distutils, instead of relying on the user having a version of mingw
that does so. And testing (and fixing if necessary) the distutils
MingwCompiler class with a wider range of mingw builds. Note that
where I say distutils here, it would ideally be something that we do
in setuptools, so that it won't be tied to the stdlib release cycles.
But AFAIK, setuptools doesn't yet include any compiler classes, so
that'd be a bigger change. I have no idea what the most appropriate
direction to take would be here.

By the way, Steve Dower - distutils\cygwinccompiler will need the new
MSVC runtime added to get_msvcr() for 3.5/VC15. It won't help
unless/until mingw ships the runtime symbol library, of course, but if
it's not added to the shipped Python 3.5, it'll be a pain to add
later... It doesn't seem to be in your VC14 branch.

Paul

[1] I just checked and it seems that there's a msvcr110 library
shipped with the mingw I have, but nothing later.

From ncoghlan at gmail.com  Sun Oct 12 03:04:48 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 12 Oct 2014 11:04:48 +1000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <7c73341b866e4804abd734eac2ed27b5@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <CACac1F_m3_nNxRwy7G=VZThxtKJo5ySO-ZRKa_NuNwpYs98q0g@mail.gmail.com>
 <5437A649.9080701@hastings.org> <m18qtn$7bc$2@ger.gmane.org>
 <54386F87.9090503@hastings.org>
 <2019567660434679885.159353sturla.molden-gmail.com@news.gmane.org>
 <20141011154137.0597141a@fsol>
 <1155124416434728166.922817sturla.molden-gmail.com@news.gmane.org>
 <20141011160817.2f3b0ebe@fsol>
 <1267413183434729768.261328sturla.molden-gmail.com@news.gmane.org>
 <07131b9c921b480aa8c6c680639a319e@BN1PR0301MB0723.namprd03.prod.outlook.com>
 <1266336123434737411.887465sturla.molden-gmail.com@news.gmane.org>
 <7c73341b866e4804abd734eac2ed27b5@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CADiSq7fXkf8nY=+qbuoZMg7AK42i_u5aSo_KwcGpNYy1rA7cNA@mail.gmail.com>

On 12 Oct 2014 04:20, "Steve Dower" <Steve.Dower at microsoft.com> wrote:
>
> DLLs linked by import library at compile time (ie. not using LoadLibrary
calls) and placed in the same directory as the .pyd should be exempt from
DLL hell - Python already creates an activation context when importing pyds
to let them load their own dependencies. Multiple CRTs are also okay as
long as they don't try and share state (such as file descriptors) across
boundaries.
>
> I do understand the lack of knowledge and experience though. I've helped
out in the past but I personally don't have the bandwidth to contribute
more, nor the persuasiveness to get someone else to.

The current key phrase in getting potential corporate sponsors interested
in the scientific Python stack is "big data analytics". The professional
programming world is actually quite bad at using modern mathematical
techniques effectively - the scientific community are way ahead of us.
Python is relatively unique in being a language used extensively by both
professional programmers *and* research scientists, so we're well
positioned to serve as a basis for effective communication between the two
groups.

AMPLab at Berkeley are one of the groups at the forefront of that. MS are
sponsors, so if you can find the group behind that sponsorship, you may
find folks in a position to help out with the scientific Python toolchain
challenges.

Cheers,
Nick.

>
>
> Top-posted from my Windows Phone
> ________________________________
> From: Sturla Molden
> Sent: ?10/?11/?2014 9:59
>
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Status of C compilers for Python on Windows
>
> Steve Dower <Steve.Dower at microsoft.com> wrote:
>
> > Is there some reason the Fortran part can't be separated out into a
DLL?
>
> DLL hell, I assume. Using the Python extension module loader makes it less
> of a problem. If we stick with .pyd files where everything is statically
> linked we can rely on the Python dev team to make sure that DLL hell does
> not bite us. Most of the contributors to projects like NumPy and SciPy are
> not computer scientists. So the KISS principle is important, which is why
> scientific programmers often use Fortran in the first place. Making sure
> DLLs are resolved and loaded correctly, or using stuff like COM or .NET to
> mitigate DLL hell, is just in a different league. That is for computer
> engineers to take care of, but we are trained as physicists,
matematicians,
> astronomers, chemists, biologists, or what ever... I am sure that
engineers
> at Microsoft could do this correctly, but we are not the kind of guys you
> would hire :-)
>
>
> OT: Contrary to common belief, there is no speed advantage of using
Fortran
> on a modern CPU, because the long pipeline and the hierarchical memory
> alleviates the problems with pointer aliasing. C code tends to run faster
> then Fortran, often 10 to 20 % faster, and C++ tends to be slightly faster
> than C. In 2014, Fortran is only used because it is easier to program for
> non-specialists. And besides, correctness is far more important than
speed,
> which is why we prefer Python or MATLAB in the first place. If you ever
see
> the argument that Fortran is used because of pointer aliasing, please feel
> free to ignore it.
>
>
> Sturla
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141012/73406f47/attachment.html>

From georg at python.org  Sun Oct 12 09:38:31 2014
From: georg at python.org (Georg Brandl)
Date: Sun, 12 Oct 2014 09:38:31 +0200
Subject: [Python-Dev] [RELEASED] Python 3.2.6, Python 3.3.6
Message-ID: <543A2FF7.5060207@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I'm happy to announce
the release of Python 3.2.6 and 3.3.6.  Both are security-fix releases,
which are provided source-only on python.org.

The list of security-related issues fixed in the releases is given in
the changelogs:

    https://hg.python.org/cpython/raw-file/v3.2.6/Misc/NEWS
    https://hg.python.org/cpython/raw-file/v3.3.6/Misc/NEWS

To download the releases visit one of:

    https://www.python.org/downloads/release/python-326/
    https://www.python.org/downloads/release/python-336/

These are production versions, please report any bugs to

     http://bugs.python.org/

Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlQ6L/cACgkQN9GcIYhpnLBxIwCeLqjXeIOxGA2vkjbkN5Ic6j2u
7WcAoKgFaB4drMX5ZOVUJ4VLyNTcfycN
=KLlw
-----END PGP SIGNATURE-----

From bugtrack at roumenpetrov.info  Sun Oct 12 10:43:33 2014
From: bugtrack at roumenpetrov.info (Roumen Petrov)
Date: Sun, 12 Oct 2014 11:43:33 +0300
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
Message-ID: <543A3F35.3010603@roumenpetrov.info>

Victor Stinner wrote:
> Hi,
[SKIP]
> === MinGW
>
> Some people tried to compile Python. See for example:
> https://bitbucket.org/puqing/python-mingw
>
> We even got some patches:
> http://bugs.python.org/issue3871 (rejected)
[SNIP]

As "all in one" patch it was rejected , but you could find splits:
17605 - mingw-meta: build interpeter core
18653 - mingw-meta: build core modules

Lot of people post links to possible issues using GCC windows compiler. 
A lot of them are not real issues for CPython.


In addition for those why would like to cross compile C-extensions for 
MS Windows either from linux of cygwin then could use this set:
18654 - modernize mingw&cygwin compiler classes


I could step in as maintainer for Cygwin and builds based on GCC using 
mingw* APIs.


Regards,
Roumen Petrov


From bugtrack at roumenpetrov.info  Sun Oct 12 11:21:56 2014
From: bugtrack at roumenpetrov.info (Roumen Petrov)
Date: Sun, 12 Oct 2014 12:21:56 +0300
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8-f7WZ_Nzybb4iUE6SuawsOcLs63qXxLky3nufSBe4Ng@mail.gmail.com>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <m191ej$ov5$1@ger.gmane.org>
 <CACac1F8-f7WZ_Nzybb4iUE6SuawsOcLs63qXxLky3nufSBe4Ng@mail.gmail.com>
Message-ID: <543A4834.9050505@roumenpetrov.info>

Paul Moore wrote:
> On 10 October 2014 17:28, Mark Lawrence <breamoreboy at yahoo.co.uk> wrote:
>> There are 55 open issues on the bug tracker with mingw in the title.
>
> It's not easy to tell, but on a spot check a fair proportion of them
> seem to be about distutils/extension builds. And a lot of the rest are
> related to http://bugs.python.org/issue3871 which is a rejected issue
> about adding build support for mingw.

Rejection is for "all in one". It was requested to split in separate 
patches to be able developers to review them more easy.

> Paul

Roumen

From mingw.android at gmail.com  Sun Oct 12 16:31:58 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 12 Oct 2014 15:31:58 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <543A3F35.3010603@roumenpetrov.info>
References: <CAMpsgwaZZ+ks6VOosdZ8ra+DA=Ma5p+i2R+RSTFyiDr1XNpUOQ@mail.gmail.com>
 <543A3F35.3010603@roumenpetrov.info>
Message-ID: <CAOYw7duW_n7iFvPdW2izZtKtJ2hSC=_aBa9q7kQ90rykzxGJmA@mail.gmail.com>

On Sun, Oct 12, 2014 at 9:43 AM, Roumen Petrov
<bugtrack at roumenpetrov.info> wrote:
> Victor Stinner wrote:
>>
>> Hi,
>
> [SKIP]
>>
>> === MinGW
>>
>> Some people tried to compile Python. See for example:
>> https://bitbucket.org/puqing/python-mingw
>>
>> We even got some patches:
>> http://bugs.python.org/issue3871 (rejected)
>
> [SNIP]
>
> As "all in one" patch it was rejected , but you could find splits:
> 17605 - mingw-meta: build interpeter core
> 18653 - mingw-meta: build core modules
>
> Lot of people post links to possible issues using GCC windows compiler. A
> lot of them are not real issues for CPython.
>
>
> In addition for those why would like to cross compile C-extensions for MS
> Windows either from linux of cygwin then could use this set:
> 18654 - modernize mingw&cygwin compiler classes
>
>
> I could step in as maintainer for Cygwin and builds based on GCC using
> mingw* APIs.
>

+1 for Roumen maintaining GCC cross builds using mingw*.

As Rafael Villar Burke mentioned, the MSYS2 project has Native Windows
Python builds (for both 3.4.2 and 2.7.8). We use Roumen's split
patches (and then our own on top):
https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python3
and https://github.com/Alexpux/MINGW-packages/tree/master/mingw-w64-python2.

To install MSYS2, build then test 32bit and 64bit mingw-w64-python3 on
a fresh 64bit Windows installation:
Download http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20141003.exe/download
Run msys2-x86_64-20141003.exe and install to a (short) path without
spaces or non-ascii characters (C:\msys64 is good), keep "Run MSYS2
64bit now." ticked. The remaining commands are to be entered in the
MSYS2 mintty shell.
# Install the packages necessary to build mingw-w64-python*
#  using Pacman package manager (answer Y or press enter when prompted):
pacman -S base-devel mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain
# Download the source recipes and patches
#  that are used to build all of MSYS2's mingw-w64 packages:
git clone https://github.com/Alexpux/MINGW-packages
cd MINGW-packages/mingw-w64-python3
# Build it:
# (s == sync (install) necessary {make,}dependencies
#  L == write log files)
# answer Y or press enter when prompted
# (remove --nocheck if you want to run the testsuite before packaging)
makepkg-mingw -sL --nocheck

# To install the newly built packages:
pacman -U mingw-w64-*.xz

# To run them, you should add /mingw64/bin or /mingw32/bin to your PATH
# (or launch a new shell via mingw32_shell.bat or mingw64_shell.bat)
# Of course, if you don't want to build it from source you can simply issue:
pacman -S mingw-w64-python3

.. all of the above applies equally to mingw-w64-python2.

If anyone would like to help us to get our work into shape and then
merged we would be extremely grateful. Unfortunately Python is one of
our most patched packages.

In response to Steve Dower's request for discussion:

Having an alternative, fully Open Source build system for Python on
Windows using a stable Win32 ABI which is compatible all the way back
to Windows XP SP3 and Windows XP 64 and can interoperate out of the
box with many other tools and libraries (numpy, GNU Fortran - yes we
have this, pyQt, pyGTK etc) is something many people would dearly
like. We don't wish to usurp Visual Studio as the recommended build
system for Windows, we simply want to enable an Open Source choice.
For philosophical and practical reasons there are many people who wish
to limit their exposure to proprietary, closed build tools such as
Visual Studio. That Windows has always been a much more difficult
platform for Open Source development is not something that the Open
Source community should accept and then work around, rather something
we should try to fix. For information on contributing to MSYS2 please
see https://sourceforge.net/p/msys2/wiki/Contributing%20to%20MSYS2/

Finally, this thread has contained many references to mingw, care
should be taken to be explicit about which of MinGW-w64 or mingw is
being referred to, since they are two different projects. MinGW-w64
supports 64bit and a lot of work is being done to support ARM.

Best regards,

Ray Donnelly.

>
> Regards,
> Roumen Petrov
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From guido at python.org  Mon Oct 13 00:55:36 2014
From: guido at python.org (Guido van Rossum)
Date: Sun, 12 Oct 2014 15:55:36 -0700
Subject: [Python-Dev] PEP476: Enabling certificate validation by default
In-Reply-To: <loom.20141003T235603-112@post.gmane.org>
References: <loom.20140919T185218-488@post.gmane.org>
 <CAP7+vJLwO2HPWJd2WGXFCtyA-c33D6dK-=QYVxLAEpF1svUP1w@mail.gmail.com>
 <CAFRnB2UJFErffyZTAJPM8f=_DDAbD5aGN-e9G7DacBfYUqb8Og@mail.gmail.com>
 <CADiSq7fLz80aV2g7hVCU4K3w==tU6smid+C292L1N5m-KwsWfw@mail.gmail.com>
 <CAP7+vJKQwrjPJydcys+UFOty0xzzOuTgrYg=WLnAfUNQjY4nJA@mail.gmail.com>
 <CAFRnB2Vs9bPOuR1HRsvbYoVT4=URqsa1xMF+2gH9qyUf4imavA@mail.gmail.com>
 <CAP7+vJKioet=Bm-BvXakG9pBB7eYiF1F=Z7FHf=2=Z8UtvAu8g@mail.gmail.com>
 <CAFRnB2WviuB-=gKgFAKsW6a0vZvXpMWp36knDoSqqM4c42TPQw@mail.gmail.com>
 <CADiSq7cCM-YNZFO_1Vkss+Ja6zmPB-CvT34hOFg1eWKSwipW6g@mail.gmail.com>
 <CAP7+vJ+C897KLH7XNMcQwTYBH2CNeSu8NfH6=r0d9474RCjDBw@mail.gmail.com>
 <CADiSq7dXtDxZM14WvTH-0ZbrRES6Mh=9sqfJrf9O8VpBJtEtWw@mail.gmail.com>
 <CAP7+vJ+1D=b=4r3qL4B2+BBkJPSpoji2R4Ds=R+h73JErDrkqw@mail.gmail.com>
 <loom.20141003T235603-112@post.gmane.org>
Message-ID: <CAP7+vJK1-McBTJF2h9WZxot7ULUhNzwsU3MtYx9zKJsExwo1Mg@mail.gmail.com>

I see no reason to hold up this PEP's approval any longer, so I hereby
approve PEP 476. It looks like a fair amount of work is still needed to
backport this to Python 2.7 (and a smaller amount for 3.4) but I trust that
this will all happen before the next releases of these two. Congrats Alex!

On Fri, Oct 3, 2014 at 2:57 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:

> Guido van Rossum <guido <at> python.org> writes:
>
> >
> > OK, I'll hold off a bit on approving the PEP, but my intention is to
> approve
> > it. Go Alex go!
> >
>
> A patch for the environmental variable overrides on Windows has landed;
> thanks
> Benjamin!
>
> Alex
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141012/0e59bcf2/attachment.html>

From ncoghlan at gmail.com  Mon Oct 13 01:39:29 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 13 Oct 2014 09:39:29 +1000
Subject: [Python-Dev] PEP476: Enabling certificate validation by default
In-Reply-To: <CAP7+vJK1-McBTJF2h9WZxot7ULUhNzwsU3MtYx9zKJsExwo1Mg@mail.gmail.com>
References: <loom.20140919T185218-488@post.gmane.org>
 <CAP7+vJLwO2HPWJd2WGXFCtyA-c33D6dK-=QYVxLAEpF1svUP1w@mail.gmail.com>
 <CAFRnB2UJFErffyZTAJPM8f=_DDAbD5aGN-e9G7DacBfYUqb8Og@mail.gmail.com>
 <CADiSq7fLz80aV2g7hVCU4K3w==tU6smid+C292L1N5m-KwsWfw@mail.gmail.com>
 <CAP7+vJKQwrjPJydcys+UFOty0xzzOuTgrYg=WLnAfUNQjY4nJA@mail.gmail.com>
 <CAFRnB2Vs9bPOuR1HRsvbYoVT4=URqsa1xMF+2gH9qyUf4imavA@mail.gmail.com>
 <CAP7+vJKioet=Bm-BvXakG9pBB7eYiF1F=Z7FHf=2=Z8UtvAu8g@mail.gmail.com>
 <CAFRnB2WviuB-=gKgFAKsW6a0vZvXpMWp36knDoSqqM4c42TPQw@mail.gmail.com>
 <CADiSq7cCM-YNZFO_1Vkss+Ja6zmPB-CvT34hOFg1eWKSwipW6g@mail.gmail.com>
 <CAP7+vJ+C897KLH7XNMcQwTYBH2CNeSu8NfH6=r0d9474RCjDBw@mail.gmail.com>
 <CADiSq7dXtDxZM14WvTH-0ZbrRES6Mh=9sqfJrf9O8VpBJtEtWw@mail.gmail.com>
 <CAP7+vJ+1D=b=4r3qL4B2+BBkJPSpoji2R4Ds=R+h73JErDrkqw@mail.gmail.com>
 <loom.20141003T235603-112@post.gmane.org>
 <CAP7+vJK1-McBTJF2h9WZxot7ULUhNzwsU3MtYx9zKJsExwo1Mg@mail.gmail.com>
Message-ID: <CADiSq7eyorTjF4Hm84W39VYmN7LnHXo6Ef23TTHVswKpG63ckQ@mail.gmail.com>

On 13 Oct 2014 08:58, "Guido van Rossum" <guido at python.org> wrote:
>
> I see no reason to hold up this PEP's approval any longer, so I hereby
approve PEP 476. It looks like a fair amount of work is still needed to
backport this to Python 2.7 (and a smaller amount for 3.4) but I trust that
this will all happen before the next releases of these two. Congrats Alex!

Huzzah! Thanks to everyone involved in getting this one through to
acceptance.

Cheers,
Nick.

>
> On Fri, Oct 3, 2014 at 2:57 PM, Alex Gaynor <alex.gaynor at gmail.com> wrote:
>>
>> Guido van Rossum <guido <at> python.org> writes:
>>
>> >
>> > OK, I'll hold off a bit on approving the PEP, but my intention is to
approve
>> > it. Go Alex go!
>> >
>>
>> A patch for the environmental variable overrides on Windows has landed;
thanks
>> Benjamin!
>>
>> Alex
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141013/69aa791e/attachment.html>

From bblais at gmail.com  Mon Oct 13 17:28:29 2014
From: bblais at gmail.com (Brian Blais)
Date: Mon, 13 Oct 2014 11:28:29 -0400
Subject: [Python-Dev] [IPython-User] proper animation in notebook?
In-Reply-To: <78D0AA12-21F5-48AE-99D5-D6FA389A6FF9@holdenweb.com>
References: <CADzuzGM2XyvVV-QdFSrc+KtVLHYNZaSJsmO7dan5oxkv4AWaKQ@mail.gmail.com>
 <78D0AA12-21F5-48AE-99D5-D6FA389A6FF9@holdenweb.com>
Message-ID: <CADzuzGMAzD8_vEKEVQ4XOuSmkxaaZJnq3kQHOf7FpFUUrsrDTw@mail.gmail.com>

On Sat, Oct 11, 2014 at 4:07 AM, Steve Holden <steve at holdenweb.com> wrote:
> I found the following code appeared to work without any issues:
>
> %matplotlib inline
> import numpy as np
> import matplotlib.pyplot as plt
>
> import sys
> import time
> from IPython.display import display, clear_output
>
> x=np.linspace(0,6,100)
> for t in np.linspace(0,20,100):
>    clear_output(wait=True)
>    f=plt.figure(figsize=(10,10))
>    plt.plot(np.sin(x)*np.sin(t),'-o')
>    plt.gca().set_ylim([-1,1])
>    display(f)
>    plt.close()
>
> Further queries should be sent to the python-dev list, as this one is being
> phased out.
>

Running this code makes a working animation, but watching the memory
usage of the python2.7 process it is clear there is a memory leak - it
starts around 150 M and climbs to 300 M.  Running for longer keeps
this going.  I think the figures are being generated, cleared in the
notebook, but still existing somehow in the background.  Is there a
way to determine if this is happening?  Is there a proper way to write
the animation to avoid this?

thanks!

Brian Blais

-----------------

             bblais at gmail.com
             http://web.bryant.edu/~bblais

From donald at stufft.io  Wed Oct 15 01:00:34 2014
From: donald at stufft.io (Donald Stufft)
Date: Tue, 14 Oct 2014 19:00:34 -0400
Subject: [Python-Dev] Disabling SSL 3.0
Message-ID: <9240FA61-321F-4D78-97A4-5C02F1792441@stufft.io>

A big security breach of SSL 3.0 just dropped a little while ago (named POODLE).
With this there is now no ability to securely connect via SSL 3.0. I believe
that we should disable SSL 3.0 in Python similarly to how SSL 2.0 is disabled,
where it is disabled by default unless the user has explicitly re-enabled it.

The new attack essentially allows reading the sensitive data from within a SSL
3.0 connection stream. It takes roughly 256 requests to break a single byte so
the attack is very practical. You can read more about the attack here at the
google announcement [1] or the whitepaper [2].

[1] http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html
[2] https://www.openssl.org/~bodo/ssl-poodle.pdf

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From victor.stinner at gmail.com  Wed Oct 15 01:16:26 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 15 Oct 2014 01:16:26 +0200
Subject: [Python-Dev] Disabling SSL 3.0
In-Reply-To: <9240FA61-321F-4D78-97A4-5C02F1792441@stufft.io>
References: <9240FA61-321F-4D78-97A4-5C02F1792441@stufft.io>
Message-ID: <CAMpsgwZisKtYGPrx3-dV33mttuoY3X4kMYx3t_qC2uTAG-oRVA@mail.gmail.com>

Hi,

I opened an issue to track this vulnerability:
http://bugs.python.org/issue22638

SSL 3.0 is 8 years old, I guess that TLS is now widely deployed and
well supported?

I guess that Linux vendors will have to fix the issues directly in
OpenSSL directly. Should Python only be changed on Windows?

Or do you want to modify Python to disable SSLv3 in the ssl module?
OpenSSL provides a SSL_OP_NO_SSLv2 option for SSL context. Is there a
SSL_OP_NO_SSLv3 option? Or only change the constructor of
ssl.SSLContext?

Victor

2014-10-15 1:00 GMT+02:00 Donald Stufft <donald at stufft.io>:
> A big security breach of SSL 3.0 just dropped a little while ago (named POODLE).
> With this there is now no ability to securely connect via SSL 3.0. I believe
> that we should disable SSL 3.0 in Python similarly to how SSL 2.0 is disabled,
> where it is disabled by default unless the user has explicitly re-enabled it.
>
> The new attack essentially allows reading the sensitive data from within a SSL
> 3.0 connection stream. It takes roughly 256 requests to break a single byte so
> the attack is very practical. You can read more about the attack here at the
> google announcement [1] or the whitepaper [2].
>
> [1] http://googleonlinesecurity.blogspot.com/2014/10/this-poodle-bites-exploiting-ssl-30.html
> [2] https://www.openssl.org/~bodo/ssl-poodle.pdf
>
> ---
> Donald Stufft
> PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com

From solipsis at pitrou.net  Wed Oct 15 01:20:14 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 15 Oct 2014 01:20:14 +0200
Subject: [Python-Dev] Disabling SSL 3.0
References: <9240FA61-321F-4D78-97A4-5C02F1792441@stufft.io>
 <CAMpsgwZisKtYGPrx3-dV33mttuoY3X4kMYx3t_qC2uTAG-oRVA@mail.gmail.com>
Message-ID: <20141015012014.3840d398@fsol>

On Wed, 15 Oct 2014 01:16:26 +0200
Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
> 
> I opened an issue to track this vulnerability:
> http://bugs.python.org/issue22638
> 
> SSL 3.0 is 8 years old, I guess that TLS is now widely deployed and
> well supported?
> 
> I guess that Linux vendors will have to fix the issues directly in
> OpenSSL directly. Should Python only be changed on Windows?

If OpenSSL gets a patch, we can simply update the OpenSSL version used
for Windows installers.

> Or do you want to modify Python to disable SSLv3 in the ssl module?
> OpenSSL provides a SSL_OP_NO_SSLv2 option for SSL context. Is there a
> SSL_OP_NO_SSLv3 option? Or only change the constructor of
> ssl.SSLContext?

Please let's not have this discussion on two different channels.
*Either* the bug tracker or the mailing-list.

Thank you

Antoine.



From saimadhavheblikar at gmail.com  Wed Oct 15 02:24:21 2014
From: saimadhavheblikar at gmail.com (Saimadhav Heblikar)
Date: Wed, 15 Oct 2014 05:54:21 +0530
Subject: [Python-Dev] Review tool not detecting all changed files
Message-ID: <CAO3PiBjvuQc=ot_ne2esoNDNNPpMAN-wJd97JELA6_dYQmrABg@mail.gmail.com>

Hi,

We were working on IDLE related issue [1] , when I noticed that the
review tool does not detect all affected files for the
cfg-ext-34-2.diff patch uploaded by Terry Reedy. Version 1 of the same
patch does not have this issue - the only difference between the two
files being line endings and time stamps. Also see Terry Reedy's
message in the same issue. [3]

Could someone please let me know if this is normal behavior or not?


[1] - http://bugs.python.org/issue3068
[2] - http://bugs.python.org/file36904/cfg-ext-34-2.diff
[3] - http://bugs.python.org/issue3068#msg229315
-- 
Regards
Saimadhav Heblikar

From tjreedy at udel.edu  Wed Oct 15 05:16:14 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 14 Oct 2014 23:16:14 -0400
Subject: [Python-Dev] Review tool not detecting all changed files
In-Reply-To: <CAO3PiBjvuQc=ot_ne2esoNDNNPpMAN-wJd97JELA6_dYQmrABg@mail.gmail.com>
References: <CAO3PiBjvuQc=ot_ne2esoNDNNPpMAN-wJd97JELA6_dYQmrABg@mail.gmail.com>
Message-ID: <m1kou7$7ib$1@ger.gmane.org>

On 10/14/2014 8:24 PM, Saimadhav Heblikar wrote:
> Hi,
>
> We were working on IDLE related issue [1] , when I noticed that the
> review tool does not detect all affected files for the
> cfg-ext-34-2.diff patch uploaded by Terry Reedy. Version 1 of the same
> patch does not have this issue - the only difference between the two
> files being line endings and time stamps. Also see Terry Reedy's
> message in the same issue. [3]

Version 3 and 4 also works fine.

> Could someone please let me know if this is normal behavior or not?
>
>
> [1] - http://bugs.python.org/issue3068
> [2] - http://bugs.python.org/file36904/cfg-ext-34-2.diff
> [3] - http://bugs.python.org/issue3068#msg229315
>


-- 
Terry Jan Reedy


From eliben at gmail.com  Wed Oct 15 22:21:35 2014
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 15 Oct 2014 13:21:35 -0700
Subject: [Python-Dev] Semi-official read-only Github mirror of the
 CPython Mercurial repository
In-Reply-To: <1412624222.1694450.175812217.20165796@webmail.messagingengine.com>
References: <CAF-Rda__36CpmDaVxLWo+iWhw5x9SqNFGjoQFXWM591PGNOzhg@mail.gmail.com>
 <CAF-Rda-CdXM7Par=skFfwuD1YgctvrZ49d+FCr9Rq4aQNa32Yw@mail.gmail.com>
 <1412624222.1694450.175812217.20165796@webmail.messagingengine.com>
Message-ID: <CAF-Rda8s_WeYCwGrLCzyUs=tnzVKYAEJBjVy61wNVjV+S_hNkg@mail.gmail.com>

On Mon, Oct 6, 2014 at 12:37 PM, Benjamin Peterson <benjamin at python.org>
wrote:

> Eli,
> Thanks for setting this up. People are evidently finding it quite useful
> and are wondering if it could be more frequently run. We don't want you
> to have to absorb the bandwidth costs yourself, though. Is the code you
> use available somewhere? It shouldn't be hard to set up the cron job on
> PSF infrastructure.
>
>
Sorry I missed this. Alex reached out directly a few days ago and I updated
it to hourly; I don't think there are significant costs involved, but if
someone at the PSF wants to run this, I have no objections to that either.

It's using the hg-fast-export tool, though, so its whole cache is required
(incremental updates are done and it has its own hg hash -> git hash
mapping).

Eli





> On Sat, Jul 12, 2014, at 09:15, Eli Bendersky wrote:
> > Just a quick update on this. I've finally found time to set up a VPS at
> > DigitalOcean of myself, and I'm moving the cronjob for updating the
> > Github
> > mirrors to it. This lets me ramp up the update frequency. For now I'll
> > set
> > it to every 4 hours, but in the future I may make it even more frequent.
> > Hopefully this will not overrun my bandwidth allocation :)
> >
> > The CPython mirror (https://github.com/python/cpython) has been pretty
> > popular so far, with over 70 forks.
> >
> > Eli
> >
> >
> >
> > On Mon, Sep 30, 2013 at 6:09 AM, Eli Bendersky <eliben at gmail.com> wrote:
> >
> > > Hi all,
> > >
> > > https://github.com/python/cpython is now live as a semi-official,
> *read
> > > only* Github mirror of the CPython Mercurial repository. Let me know
> if you
> > > have any problems/concerns.
> > >
> > > I still haven't decided how often to update it (considering either
> just N
> > > times a day, or maybe use a Hg hook for batching). Suggestions are
> welcome.
> > >
> > > The methodology I used to create it is via hg-fast-export. I also
> tried to
> > > pack and gc the git repo as much as possible before the initial Github
> push
> > > - it went down from almost ~2GB to ~200MB (so this is the size of a
> fresh
> > > clone right now).
> > >
> > > Eli
> > >
> > > P.S. thanks Jesse for the keys to https://github.com/python
> > >
> > >
> > >
> > >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > https://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141015/c6ef3adb/attachment.html>

From pmiscml at gmail.com  Thu Oct 16 02:54:32 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Thu, 16 Oct 2014 03:54:32 +0300
Subject: [Python-Dev] How io.IOBase.readline() should behave when used on
 non-blocking obj and no data available?
Message-ID: <20141016035432.05cd914f@x230>

Hello,

io.RawIOBase.read() is well specified for behavior in case it
immediately gets a would-block condition: "If the object is in
non-blocking mode and no bytes are available, None is returned."
(https://docs.python.org/3/library/io.html#io.RawIOBase.read).

However, nothing is said about such condition for io.IOBase.readline(),
which is mixin method in a base class, default implementation of which
thus would use io.RawIOBase.read(). Looking at 3.4.0 source, iobase.c:
iobase_readline() has:

        b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead);
[...]
        if (!PyBytes_Check(b)) {
            PyErr_Format(PyExc_IOError,
                         "read() should have returned a bytes object, "
                         "not '%.200s'", Py_TYPE(b)->tp_name);

I.e. it's not even ready to receive legitimate return value of None
from read(). I didn't try to write a testcase though, so may be missing
something.


So, how readline() should behave in this case, and can that be
specified in the Library Reference?


Thanks,
 Paul                          mailto:pmiscml at gmail.com

From solipsis at pitrou.net  Thu Oct 16 13:34:06 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 16 Oct 2014 13:34:06 +0200
Subject: [Python-Dev] How io.IOBase.readline() should behave when used
 on non-blocking obj and no data available?
References: <20141016035432.05cd914f@x230>
Message-ID: <20141016133406.07559453@fsol>

On Thu, 16 Oct 2014 03:54:32 +0300
Paul Sokolovsky <pmiscml at gmail.com> wrote:
> Hello,
> 
> io.RawIOBase.read() is well specified for behavior in case it
> immediately gets a would-block condition: "If the object is in
> non-blocking mode and no bytes are available, None is returned."
> (https://docs.python.org/3/library/io.html#io.RawIOBase.read).
> 
> However, nothing is said about such condition for io.IOBase.readline(),
> which is mixin method in a base class, default implementation of which
> thus would use io.RawIOBase.read(). Looking at 3.4.0 source, iobase.c:
> iobase_readline() has:
> 
>         b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead);
> [...]
>         if (!PyBytes_Check(b)) {
>             PyErr_Format(PyExc_IOError,
>                          "read() should have returned a bytes object, "
>                          "not '%.200s'", Py_TYPE(b)->tp_name);
> 
> I.e. it's not even ready to receive legitimate return value of None
> from read(). I didn't try to write a testcase though, so may be missing
> something.
> 
> So, how readline() should behave in this case, and can that be
> specified in the Library Reference?

Well, the problem is that it's not obvious how to implement such methods
in a non-blocking context.

Let's says some data is received but there isn't a complete line.
Should readline() return just that data (an incomplete line)? That
breaks the API's contract. Should readline() buffer the incomplete line
and keep it for the next readline() call? But then the internal buffer
becomes unbounded: perhaps there is no new line in the next 4GB of
incoming data...

And besides, raw I/O objects *shouldn't* have an internal buffer. That's
the role of the buffered I/O layer.

Regards

Antoine.



From guido at python.org  Thu Oct 16 16:50:02 2014
From: guido at python.org (Guido van Rossum)
Date: Thu, 16 Oct 2014 07:50:02 -0700
Subject: [Python-Dev] How io.IOBase.readline() should behave when used
 on non-blocking obj and no data available?
In-Reply-To: <20141016133406.07559453@fsol>
References: <20141016035432.05cd914f@x230> <20141016133406.07559453@fsol>
Message-ID: <CAP7+vJKMqps0C-HzpqvRgNjE2csqok9_anwfNi84PPfPA_VKDA@mail.gmail.com>

On Thu, Oct 16, 2014 at 4:34 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Thu, 16 Oct 2014 03:54:32 +0300
> Paul Sokolovsky <pmiscml at gmail.com> wrote:
> > Hello,
> >
> > io.RawIOBase.read() is well specified for behavior in case it
> > immediately gets a would-block condition: "If the object is in
> > non-blocking mode and no bytes are available, None is returned."
> > (https://docs.python.org/3/library/io.html#io.RawIOBase.read).
> >
> > However, nothing is said about such condition for io.IOBase.readline(),
> > which is mixin method in a base class, default implementation of which
> > thus would use io.RawIOBase.read(). Looking at 3.4.0 source, iobase.c:
> > iobase_readline() has:
> >
> >         b = _PyObject_CallMethodId(self, &PyId_read, "n", nreadahead);
> > [...]
> >         if (!PyBytes_Check(b)) {
> >             PyErr_Format(PyExc_IOError,
> >                          "read() should have returned a bytes object, "
> >                          "not '%.200s'", Py_TYPE(b)->tp_name);
> >
> > I.e. it's not even ready to receive legitimate return value of None
> > from read(). I didn't try to write a testcase though, so may be missing
> > something.
> >
> > So, how readline() should behave in this case, and can that be
> > specified in the Library Reference?
>
> Well, the problem is that it's not obvious how to implement such methods
> in a non-blocking context.
>
> Let's says some data is received but there isn't a complete line.
> Should readline() return just that data (an incomplete line)? That
> breaks the API's contract. Should readline() buffer the incomplete line
> and keep it for the next readline() call? But then the internal buffer
> becomes unbounded: perhaps there is no new line in the next 4GB of
> incoming data...
>
> And besides, raw I/O objects *shouldn't* have an internal buffer. That's
> the role of the buffered I/O layer.
>

Well, occasionally this occurs, and I think it's reasonable for readline()
to deal with it.

The argument about a 4 GB buffer is irrelevant -- this can happen with a
blocking underlying stream too.

I think that at the point where the readline() function says to itself "I
need more data" it should ask the underlying stream for data. If that
returns an empty string, meaning EOF, readline() is satisfied and return
whatever it has buffered (even if it's empty). If that returns some bytes
containing a newline, readline() is satisfied, returns the data up to that
point, and buffers the rest (if any). If the underlying stream returns
None, I think it makes sense for readline() to return None too --
attempting to read more will just turn into a busy-wait loop, and that's
the opposite of what should happen.

You may argue that the caller of readline() doesn't expect this. Sure. But
in the end, if the stream is unbuffered and the caller isn't prepared for
that, the caller will always get in trouble. Maybe it'll treat the None as
EOF. That's fine -- it would be the same if it was calling read() on the
underlying stream and it got None (the EOF signalling is the same in both
cases).

At least, by being prepared for the None from the underlying read() in the
readline() code, someone who knows what they are doing can use readline()
on a non-blocking stream -- when they receive None they will have to ask
their selector (or whatever they use) to wait for the underlying FD and
then they can try again.

(Alternatively, we could raise BlockingIOError, which is that the OS level
read() raises if there's no data immediately available on a non-blocking
FD; but it seems that streams have already gotten a convention of returning
None instead, so I think that should be propagated up the stack.)

Oh, BTW, I tested this a little bit. Currently readline() returns an empty
string (or empty bytes, depending on which level you use) when the stream
is nonblocking. I think returning None makes muck more sense.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141016/664adcac/attachment.html>

From status at bugs.python.org  Fri Oct 17 18:08:13 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 17 Oct 2014 18:08:13 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20141017160813.614B0561CB@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-10-10 - 2014-10-17)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4587 (-11)
  closed 29833 (+64)
  total  34420 (+53)

Open issues with patches: 2146 


Issues opened (38)
==================

#13664: UnicodeEncodeError in gzip when filename contains non-ascii
http://bugs.python.org/issue13664  reopened by ezio.melotti

#15859: PyUnicode_EncodeFSDefault win32 inconsistancy.
http://bugs.python.org/issue15859  reopened by ideasman42

#22606: Inconsistency between Python 2 and PyPy regarding future impor
http://bugs.python.org/issue22606  opened by Claudiu.Popa

#22607: find by dichotomy the failing test
http://bugs.python.org/issue22607  opened by xdegaye

#22608: test_socket fails with sem_init: Too many open files
http://bugs.python.org/issue22608  opened by Urs.Traber

#22609: Constructors of some mapping classes don't accept `self` keywo
http://bugs.python.org/issue22609  opened by abacabadabacaba

#22610: test_ftplib fails with sem_init: Too many open files
http://bugs.python.org/issue22610  opened by Urs.Traber

#22612: Add block info to unicodedata
http://bugs.python.org/issue22612  opened by flying sheep

#22613: Several minor doc issues
http://bugs.python.org/issue22613  opened by zach.ware

#22616: Allow connecting AST nodes with corresponding source ranges
http://bugs.python.org/issue22616  opened by Aivar.Annamaa

#22618: urllib.parse.parse_qsl different results after urllib.parse.un
http://bugs.python.org/issue22618  opened by Alex.Vaystikh

#22619: Possible implementation of negative limit for traceback functi
http://bugs.python.org/issue22619  opened by vlth

#22622: ElementTree only writes declaration when passed encoding
http://bugs.python.org/issue22622  opened by towb

#22623: Missing guards for some POSIX functions
http://bugs.python.org/issue22623  opened by Link Mauve

#22624: Bogus usage of floatclock in timemodule
http://bugs.python.org/issue22624  opened by Link Mauve

#22625: When cross-compiling, don???t try to execute binaries
http://bugs.python.org/issue22625  opened by Link Mauve

#22629: Idle: update htest.py and htests
http://bugs.python.org/issue22629  opened by terry.reedy

#22630: `concurrent.futures.Future.set_running_or_notify_cancel` does 
http://bugs.python.org/issue22630  opened by bwhmather

#22631: Feature Request CAN_RAW_FD_FRAMES
http://bugs.python.org/issue22631  opened by rumpelsepp

#22633: Memory disclosure/buffer overread via bug in Py_FrozenMain
http://bugs.python.org/issue22633  opened by Guido

#22634: importing _ctypes failed: undefined symbol: ffi_call_win32
http://bugs.python.org/issue22634  opened by siyuan

#22635: subprocess.getstatusoutput changed behavior in 3.4 (maybe 3.3.
http://bugs.python.org/issue22635  opened by josh.r

#22636: avoid using a shell in ctypes.util: replace os.popen with subp
http://bugs.python.org/issue22636  opened by haypo

#22637: avoid using a shell in uuid: replce os.popen with subprocess.P
http://bugs.python.org/issue22637  opened by haypo

#22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack)
http://bugs.python.org/issue22638  opened by haypo

#22640: Add silent mode for py_compile
http://bugs.python.org/issue22640  opened by berker.peksag

#22642: trace module: unclear error message
http://bugs.python.org/issue22642  opened by giampaolo.rodola

#22644: Update Windows installers to OpenSSL 1.0.1j
http://bugs.python.org/issue22644  opened by alex

#22645: Unable to install Python 3.4.2 amd64 on Windows 8.1 Update 1
http://bugs.python.org/issue22645  opened by Zac.Greve

#22648: Unable to install Python 3.4.2 amd64 on Windows 8.1
http://bugs.python.org/issue22648  opened by brp-log

#22649: Use _PyUnicodeWriter in case_operation()
http://bugs.python.org/issue22649  opened by haypo

#22650: set up and use VM for net access in the test suite
http://bugs.python.org/issue22650  opened by georg.brandl

#22651: Open file in a+ mode reads from end of file in Python 3.4
http://bugs.python.org/issue22651  opened by nicksjacobson

#22652: Add suggestion about keyword arguments to this error message: 
http://bugs.python.org/issue22652  opened by cool-RR

#22653: Crash in insertdict
http://bugs.python.org/issue22653  opened by pitrou

#22654: issue with PYTHON_FOR_BUILD
http://bugs.python.org/issue22654  opened by bill9889

#22655: Build error on CentOS 5.4
http://bugs.python.org/issue22655  opened by maorui

#22656: `help` ignores `__doc__` of descriptors
http://bugs.python.org/issue22656  opened by cool-RR



Most recent 15 issues with no replies (15)
==========================================

#22656: `help` ignores `__doc__` of descriptors
http://bugs.python.org/issue22656

#22655: Build error on CentOS 5.4
http://bugs.python.org/issue22655

#22650: set up and use VM for net access in the test suite
http://bugs.python.org/issue22650

#22644: Update Windows installers to OpenSSL 1.0.1j
http://bugs.python.org/issue22644

#22642: trace module: unclear error message
http://bugs.python.org/issue22642

#22640: Add silent mode for py_compile
http://bugs.python.org/issue22640

#22633: Memory disclosure/buffer overread via bug in Py_FrozenMain
http://bugs.python.org/issue22633

#22630: `concurrent.futures.Future.set_running_or_notify_cancel` does 
http://bugs.python.org/issue22630

#22623: Missing guards for some POSIX functions
http://bugs.python.org/issue22623

#22622: ElementTree only writes declaration when passed encoding
http://bugs.python.org/issue22622

#22612: Add block info to unicodedata
http://bugs.python.org/issue22612

#22610: test_ftplib fails with sem_init: Too many open files
http://bugs.python.org/issue22610

#22606: Inconsistency between Python 2 and PyPy regarding future impor
http://bugs.python.org/issue22606

#22602: UTF-7 codec decodes ill-formed sequences starting with "+"
http://bugs.python.org/issue22602

#22600: In Multiprocessing Queue() doesn't work with list : __.put(lis
http://bugs.python.org/issue22600



Most recent 15 issues waiting for review (15)
=============================================

#22653: Crash in insertdict
http://bugs.python.org/issue22653

#22649: Use _PyUnicodeWriter in case_operation()
http://bugs.python.org/issue22649

#22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack)
http://bugs.python.org/issue22638

#22637: avoid using a shell in uuid: replce os.popen with subprocess.P
http://bugs.python.org/issue22637

#22636: avoid using a shell in ctypes.util: replace os.popen with subp
http://bugs.python.org/issue22636

#22630: `concurrent.futures.Future.set_running_or_notify_cancel` does 
http://bugs.python.org/issue22630

#22629: Idle: update htest.py and htests
http://bugs.python.org/issue22629

#22625: When cross-compiling, don???t try to execute binaries
http://bugs.python.org/issue22625

#22623: Missing guards for some POSIX functions
http://bugs.python.org/issue22623

#22619: Possible implementation of negative limit for traceback functi
http://bugs.python.org/issue22619

#22610: test_ftplib fails with sem_init: Too many open files
http://bugs.python.org/issue22610

#22609: Constructors of some mapping classes don't accept `self` keywo
http://bugs.python.org/issue22609

#22608: test_socket fails with sem_init: Too many open files
http://bugs.python.org/issue22608

#22607: find by dichotomy the failing test
http://bugs.python.org/issue22607

#22599: traceback: errors in the linecache module at exit
http://bugs.python.org/issue22599



Top 10 most discussed issues (10)
=================================

#22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack)
http://bugs.python.org/issue22638  28 msgs

#21991: The new email API should use MappingProxyType instead of retur
http://bugs.python.org/issue21991  13 msgs

#3068: IDLE - Add an extension configuration dialog
http://bugs.python.org/issue3068  11 msgs

#13664: UnicodeEncodeError in gzip when filename contains non-ascii
http://bugs.python.org/issue13664   9 msgs

#22607: find by dichotomy the failing test
http://bugs.python.org/issue22607   9 msgs

#22653: Crash in insertdict
http://bugs.python.org/issue22653   9 msgs

#22634: importing _ctypes failed: undefined symbol: ffi_call_win32
http://bugs.python.org/issue22634   8 msgs

#22651: Open file in a+ mode reads from end of file in Python 3.4
http://bugs.python.org/issue22651   8 msgs

#22654: issue with PYTHON_FOR_BUILD
http://bugs.python.org/issue22654   8 msgs

#12067: Doc: remove errors about mixed-type comparisons.
http://bugs.python.org/issue12067   7 msgs



Issues closed (64)
==================

#7442: _localemodule.c: str2uni() with different LC_NUMERIC and LC_CT
http://bugs.python.org/issue7442  closed by haypo

#9311: os.access can return bogus values when run as superuser
http://bugs.python.org/issue9311  closed by georg.brandl

#10769: ast: provide more useful range information
http://bugs.python.org/issue10769  closed by scummos

#11694: xdrlib raises ConversionError in inconsistent way
http://bugs.python.org/issue11694  closed by petri.lehtinen

#11973: kevent does not accept KQ_NOTE_EXIT (and other (f)flags)
http://bugs.python.org/issue11973  closed by r.david.murray

#12834: memoryview.to_bytes() and PyBuffer_ToContiguous() incorrect fo
http://bugs.python.org/issue12834  closed by skrah

#12965: longobject: documentation improvements
http://bugs.python.org/issue12965  closed by skrah

#13090: test_multiprocessing: memory leaks
http://bugs.python.org/issue13090  closed by skrah

#13096: ctypes: segfault with large POINTER type names
http://bugs.python.org/issue13096  closed by r.david.murray

#14537: "Fatal Python error: Cannot recover from stack overflow."  wit
http://bugs.python.org/issue14537  closed by r.david.murray

#15414: os.path.join behavior on Windows (ntpath.join) is not well doc
http://bugs.python.org/issue15414  closed by python-dev

#15569: Doc doc: incorrect description of some roles as format-only
http://bugs.python.org/issue15569  closed by python-dev

#15672: PEP 3121, 384 Refactoring applied to testbuffer module
http://bugs.python.org/issue15672  closed by skrah

#15722: PEP 3121, 384 Refactoring applied to decimal module
http://bugs.python.org/issue15722  closed by skrah

#15857: memoryview: complete support for struct packing/unpacking
http://bugs.python.org/issue15857  closed by skrah

#16233: IDLE: conceptual problems with *Class browser*
http://bugs.python.org/issue16233  closed by terry.reedy

#16728: Missing cross-reference in sequence glossary entry
http://bugs.python.org/issue16728  closed by r.david.murray

#17325: improve organization of the PyPI distutils docs
http://bugs.python.org/issue17325  closed by r.david.murray

#17636: Modify IMPORT_FROM to fallback on sys.modules
http://bugs.python.org/issue17636  closed by pitrou

#18643: add a fallback socketpair() implementation to the socket modul
http://bugs.python.org/issue18643  closed by neologix

#18873: "Encoding" detected in non-comment lines
http://bugs.python.org/issue18873  closed by serhiy.storchaka

#18959: Create a "Superseded modules" section in standard library ToC
http://bugs.python.org/issue18959  closed by python-dev

#20167: Exception on IDLE closing
http://bugs.python.org/issue20167  closed by terry.reedy

#20386: socket.SocketType enum overwrites import of _socket.SocketType
http://bugs.python.org/issue20386  closed by ethan.furman

#20815: ipaddress unit tests PEP8
http://bugs.python.org/issue20815  closed by r.david.murray

#21061: Is contextlib.redirect_stdout reentrant or not?
http://bugs.python.org/issue21061  closed by ncoghlan

#21189: Broken link to patch
http://bugs.python.org/issue21189  closed by barry

#21227: Decimal class error messages for integer division aren't good
http://bugs.python.org/issue21227  closed by skrah

#21338: Silent mode for compileall
http://bugs.python.org/issue21338  closed by berker.peksag

#21675: Library - Introduction - paragraph 5 - wrong ordering
http://bugs.python.org/issue21675  closed by python-dev

#21687: Py_SetPath: Path components separated by colons
http://bugs.python.org/issue21687  closed by python-dev

#21855: Fix decimal in unicodeless build
http://bugs.python.org/issue21855  closed by serhiy.storchaka

#21907: Update Windows build batch scripts
http://bugs.python.org/issue21907  closed by zach.ware

#21917: Python 2.7.7 Tests fail, and math is faulty
http://bugs.python.org/issue21917  closed by skrah

#21986: Idle: disable pickleability of user code objects
http://bugs.python.org/issue21986  closed by terry.reedy

#22480: SystemExit out of run_until_complete causes AttributeError whe
http://bugs.python.org/issue22480  closed by haypo

#22489: .gitignore file
http://bugs.python.org/issue22489  closed by zach.ware

#22506: `dir` on Enum subclass doesn't expose parent class attributes
http://bugs.python.org/issue22506  closed by ethan.furman

#22515: Implement partial order on Counter
http://bugs.python.org/issue22515  closed by rhettinger

#22533: Counter with no keys does not compare equal to Counter with ke
http://bugs.python.org/issue22533  closed by rhettinger

#22568: Use of "utime" as variable name in Modules/posixmodule.c cause
http://bugs.python.org/issue22568  closed by python-dev

#22575: bytearray documentation confuses string for unicode objects
http://bugs.python.org/issue22575  closed by terry.reedy

#22586: urljoin allow_fragments doesn't work
http://bugs.python.org/issue22586  closed by python-dev

#22590: math.copysign buggy with nan under Windows
http://bugs.python.org/issue22590  closed by mark.dickinson

#22601: asyncio: loop.run_forever() should consume exception of the te
http://bugs.python.org/issue22601  closed by haypo

#22603: Fix a typo in the contextlib docs
http://bugs.python.org/issue22603  closed by terry.reedy

#22604: assertion error in complex division
http://bugs.python.org/issue22604  closed by pitrou

#22605: memcpy(NULL, NULL, 0) in array_new()
http://bugs.python.org/issue22605  closed by python-dev

#22611: pip needs ctypes
http://bugs.python.org/issue22611  closed by dstufft

#22614: Idle: problem in PyShellEditorWindow.color_breakpoint_text
http://bugs.python.org/issue22614  closed by terry.reedy

#22615: Argument Clinic doesn't support the "type" argument for the in
http://bugs.python.org/issue22615  closed by larry

#22617: spam
http://bugs.python.org/issue22617  closed by georg.brandl

#22620: pythonw does not open on windows 8.1 x64
http://bugs.python.org/issue22620  closed by r.david.murray

#22621: Please make it possible to make the output of hash() equal bet
http://bugs.python.org/issue22621  closed by georg.brandl

#22626: Documentation should point people to bugs. over HTTPS
http://bugs.python.org/issue22626  closed by alex

#22627: Calling timestamp() on a datetime object modifies the timestam
http://bugs.python.org/issue22627  closed by pitrou

#22628: Idle: Tree lines are spaced too close together.
http://bugs.python.org/issue22628  closed by terry.reedy

#22632: Official IDLE web page address is not valid
http://bugs.python.org/issue22632  closed by terry.reedy

#22639: test "test_bad_address" fails on Python 3.4.2 under Linux Mint
http://bugs.python.org/issue22639  closed by ned.deily

#22641: Use better default context in asyncio
http://bugs.python.org/issue22641  closed by pitrou

#22643: Integer overflow in case_operation
http://bugs.python.org/issue22643  closed by python-dev

#22646: Set SMTPHandler's "credentials" through "logging.config.dictCo
http://bugs.python.org/issue22646  closed by python-dev

#22647: test_readline failed on ScientificLinux 6.5
http://bugs.python.org/issue22647  closed by ned.deily

#22657: Bad link to packages specification in language reference 2.x s
http://bugs.python.org/issue22657  closed by python-dev

From tjreedy at udel.edu  Sun Oct 19 01:05:09 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 18 Oct 2014 19:05:09 -0400
Subject: [Python-Dev] Half the 'stable' buildbots are not running.
Message-ID: <m1urni$8qs$1@ger.gmane.org>

4 offline, 2 not compiling, 1 not compiling completely (no _ctypes == 
venv fail)

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Mon Oct 20 22:01:02 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 20 Oct 2014 16:01:02 -0400
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not linking
	correctly
Message-ID: <m23plv$2gq$1@ger.gmane.org>

If I go to https://docs.python.org/3/using/index.html and click on any 
of the TOC entries, I get 'connecting' indefinitely.  This problem seems 
unique to this file. I tried several other index files and clicking am 
entry brings up the corresponding page almost immediately.

-- 
Terry Jan Reedy


From g.brandl at gmx.net  Mon Oct 20 22:17:50 2014
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 20 Oct 2014 22:17:50 +0200
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
	linking correctly
In-Reply-To: <m23plv$2gq$1@ger.gmane.org>
References: <m23plv$2gq$1@ger.gmane.org>
Message-ID: <m23qle$bee$1@ger.gmane.org>

On 10/20/2014 10:01 PM, Terry Reedy wrote:
> If I go to https://docs.python.org/3/using/index.html and click on any 
> of the TOC entries, I get 'connecting' indefinitely.  This problem seems 
> unique to this file. I tried several other index files and clicking am 
> entry brings up the corresponding page almost immediately.

Sounds like a similar problem to this:
https://mail.python.org/pipermail/docs/2014-October/020440.html

(There, the OP reported that she saw a different result in different
browsers.)

Georg



From phd at phdru.name  Mon Oct 20 22:11:04 2014
From: phd at phdru.name (Oleg Broytman)
Date: Mon, 20 Oct 2014 22:11:04 +0200
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <m23plv$2gq$1@ger.gmane.org>
References: <m23plv$2gq$1@ger.gmane.org>
Message-ID: <20141020201104.GA5115@phdru.name>

Hi!

On Mon, Oct 20, 2014 at 04:01:02PM -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> If I go to https://docs.python.org/3/using/index.html and click on
> any of the TOC entries, I get 'connecting' indefinitely.  This
> problem seems unique to this file. I tried several other index files
> and clicking am entry brings up the corresponding page almost
> immediately.
> 
> -- 
> Terry Jan Reedy

   Works for me. I clicked "2. Using Python on Unix platforms" and got
to https://docs.python.org/3/using/unix.html without any problem.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From eliben at gmail.com  Tue Oct 21 01:09:45 2014
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 20 Oct 2014 16:09:45 -0700
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <m23plv$2gq$1@ger.gmane.org>
References: <m23plv$2gq$1@ger.gmane.org>
Message-ID: <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>

On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> If I go to https://docs.python.org/3/using/index.html and click on any of
> the TOC entries, I get 'connecting' indefinitely.  This problem seems
> unique to this file. I tried several other index files and clicking am
> entry brings up the corresponding page almost immediately.


Works fine for me, Terry, in both Chrome and Firefox. Could be something in
your browser/caching? Try "private mode" or whatever that's called in your
browser of choice.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141020/b45c0b1e/attachment.html>

From python at mrabarnett.plus.com  Tue Oct 21 01:29:45 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 21 Oct 2014 00:29:45 +0100
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
Message-ID: <54459AE9.6060005@mrabarnett.plus.com>

On 2014-10-21 00:09, Eli Bendersky wrote:
>
>
> On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:
>
>     If I go to https://docs.python.org/3/using/index.html and click on
>     any of the TOC entries, I get 'connecting' indefinitely.  This
>     problem seems unique to this file. I tried several other index files
>     and clicking am entry brings up the corresponding page almost
>     immediately.
>
>
> Works fine for me, Terry, in both Chrome and Firefox. Could be something
> in your browser/caching? Try "private mode" or whatever that's called in
> your browser of choice.
>
Just tried the first link:

https://docs.python.org/3/using/cmdline.html

I also get 'Connecting' indefinitely.

So I tested the link here:

http://www.downforeveryoneorjustme.com/

It said: "It's not just you! http://https looks down from here."


From phd at phdru.name  Tue Oct 21 01:38:24 2014
From: phd at phdru.name (Oleg Broytman)
Date: Tue, 21 Oct 2014 01:38:24 +0200
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <54459AE9.6060005@mrabarnett.plus.com>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com>
Message-ID: <20141020233824.GA7647@phdru.name>

On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB <python at mrabarnett.plus.com> wrote:
> On 2014-10-21 00:09, Eli Bendersky wrote:
> >
> >On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
> ><mailto:tjreedy at udel.edu>> wrote:
> >
> >    If I go to https://docs.python.org/3/using/index.html and click on
> >    any of the TOC entries, I get 'connecting' indefinitely.  This
> >    problem seems unique to this file. I tried several other index files
> >    and clicking am entry brings up the corresponding page almost
> >    immediately.
> >
> >Works fine for me, Terry, in both Chrome and Firefox. Could be something
> >in your browser/caching? Try "private mode" or whatever that's called in
> >your browser of choice.
> >
> Just tried the first link:
> 
> https://docs.python.org/3/using/cmdline.html
> 
> I also get 'Connecting' indefinitely.
> 
> So I tested the link here:
> 
> http://www.downforeveryoneorjustme.com/
> 
> It said: "It's not just you! http://https looks down from here."

   I think this is a limitation of isup.me: it doesn't like https://
URLs -- either use http:// or no protocol prefix at all.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From donald at stufft.io  Tue Oct 21 01:43:08 2014
From: donald at stufft.io (Donald Stufft)
Date: Mon, 20 Oct 2014 19:43:08 -0400
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
	linking correctly
In-Reply-To: <20141020233824.GA7647@phdru.name>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com> <20141020233824.GA7647@phdru.name>
Message-ID: <7C0972C5-346E-449E-A4E3-0226334FFFC3@stufft.io>

I'm on my phone but docs is served via fastly. Issues could be POP specific. 


> On Oct 20, 2014, at 7:38 PM, Oleg Broytman <phd at phdru.name> wrote:
> 
>> On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB <python at mrabarnett.plus.com> wrote:
>>> On 2014-10-21 00:09, Eli Bendersky wrote:
>>> 
>>> On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
>>> <mailto:tjreedy at udel.edu>> wrote:
>>> 
>>>   If I go to https://docs.python.org/3/using/index.html and click on
>>>   any of the TOC entries, I get 'connecting' indefinitely.  This
>>>   problem seems unique to this file. I tried several other index files
>>>   and clicking am entry brings up the corresponding page almost
>>>   immediately.
>>> 
>>> Works fine for me, Terry, in both Chrome and Firefox. Could be something
>>> in your browser/caching? Try "private mode" or whatever that's called in
>>> your browser of choice.
>> Just tried the first link:
>> 
>> https://docs.python.org/3/using/cmdline.html
>> 
>> I also get 'Connecting' indefinitely.
>> 
>> So I tested the link here:
>> 
>> http://www.downforeveryoneorjustme.com/
>> 
>> It said: "It's not just you! http://https looks down from here."
> 
>   I think this is a limitation of isup.me: it doesn't like https://
> URLs -- either use http:// or no protocol prefix at all.
> 
> Oleg.
> -- 
>     Oleg Broytman            http://phdru.name/            phd at phdru.name
>           Programmers don't die, they just GOSUB without RETURN.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

From tjreedy at udel.edu  Tue Oct 21 02:39:31 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 20 Oct 2014 20:39:31 -0400
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
	linking correctly
In-Reply-To: <54459AE9.6060005@mrabarnett.plus.com>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com>
Message-ID: <m24a04$o9i$1@ger.gmane.org>

On 10/20/2014 7:29 PM, MRAB wrote:
> On 2014-10-21 00:09, Eli Bendersky wrote:
>>
>>
>> On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
>> <mailto:tjreedy at udel.edu>> wrote:
>>
>>     If I go to https://docs.python.org/3/using/index.html and click on
>>     any of the TOC entries, I get 'connecting' indefinitely.  This
>>     problem seems unique to this file. I tried several other index files
>>     and clicking am entry brings up the corresponding page almost
>>     immediately.
>>
>>
>> Works fine for me, Terry, in both Chrome and Firefox. Could be something
>> in your browser/caching? Try "private mode" or whatever that's called in
>> your browser of choice.
>>
> Just tried the first link:
>
> https://docs.python.org/3/using/cmdline.html
>
> I also get 'Connecting' indefinitely.
>
> So I tested the link here:
>
> http://www.downforeveryoneorjustme.com/
>
> It said: "It's not just you! http://https looks down from here."

I am using Firefox, but the difference is that by happenstance, I was in 
privacy mode (perhaps for the first time with python.org -- I want the 
docs in the cache and history list -- because I accessed the docs to 
answer a StackOverflow question.) It is not just that document but also 
Execution model and Expression in Reference.  (I tried each 3 times 
minutes apart.)  I have not had a similiar problem with any Arstechnica 
or StackOverflow pages, or similar sites.

So the solution for me is to use normal mode, but something seems not 
quite right in the interaction between privacy mode Firefox and python.org.

No problem I could find with Internet Explorer's 'In Privacy' mode.

-- 
Terry Jan Reedy


From python at mrabarnett.plus.com  Tue Oct 21 03:17:10 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 21 Oct 2014 02:17:10 +0100
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <20141020233824.GA7647@phdru.name>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com> <20141020233824.GA7647@phdru.name>
Message-ID: <5445B416.7060507@mrabarnett.plus.com>

On 2014-10-21 00:38, Oleg Broytman wrote:
> On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB <python at mrabarnett.plus.com> wrote:
>> On 2014-10-21 00:09, Eli Bendersky wrote:
>> >
>> >On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
>> ><mailto:tjreedy at udel.edu>> wrote:
>> >
>> >    If I go to https://docs.python.org/3/using/index.html and click on
>> >    any of the TOC entries, I get 'connecting' indefinitely.  This
>> >    problem seems unique to this file. I tried several other index files
>> >    and clicking am entry brings up the corresponding page almost
>> >    immediately.
>> >
>> >Works fine for me, Terry, in both Chrome and Firefox. Could be something
>> >in your browser/caching? Try "private mode" or whatever that's called in
>> >your browser of choice.
>> >
>> Just tried the first link:
>>
>> https://docs.python.org/3/using/cmdline.html
>>
>> I also get 'Connecting' indefinitely.
>>
>> So I tested the link here:
>>
>> http://www.downforeveryoneorjustme.com/
>>
>> It said: "It's not just you! http://https looks down from here."
>
>     I think this is a limitation of isup.me: it doesn't like https://
> URLs -- either use http:// or no protocol prefix at all.
>
Well, this:

https://docs.python.org/3/using/cmdline.html

is working for me now, but this:

https://docs.python.org/3/whatsnew/index.html

isn't. It looks like some kind of intermittent glitch somewhere...


From python at mrabarnett.plus.com  Tue Oct 21 03:20:43 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 21 Oct 2014 02:20:43 +0100
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <5445B416.7060507@mrabarnett.plus.com>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com> <20141020233824.GA7647@phdru.name>
 <5445B416.7060507@mrabarnett.plus.com>
Message-ID: <5445B4EB.4090204@mrabarnett.plus.com>

On 2014-10-21 02:17, MRAB wrote:
> On 2014-10-21 00:38, Oleg Broytman wrote:
>> On Tue, Oct 21, 2014 at 12:29:45AM +0100, MRAB <python at mrabarnett.plus.com> wrote:
>>> On 2014-10-21 00:09, Eli Bendersky wrote:
>>> >
>>> >On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
>>> ><mailto:tjreedy at udel.edu>> wrote:
>>> >
>>> >    If I go to https://docs.python.org/3/using/index.html and click on
>>> >    any of the TOC entries, I get 'connecting' indefinitely.  This
>>> >    problem seems unique to this file. I tried several other index files
>>> >    and clicking am entry brings up the corresponding page almost
>>> >    immediately.
>>> >
>>> >Works fine for me, Terry, in both Chrome and Firefox. Could be something
>>> >in your browser/caching? Try "private mode" or whatever that's called in
>>> >your browser of choice.
>>> >
>>> Just tried the first link:
>>>
>>> https://docs.python.org/3/using/cmdline.html
>>>
>>> I also get 'Connecting' indefinitely.
>>>
>>> So I tested the link here:
>>>
>>> http://www.downforeveryoneorjustme.com/
>>>
>>> It said: "It's not just you! http://https looks down from here."
>>
>>     I think this is a limitation of isup.me: it doesn't like https://
>> URLs -- either use http:// or no protocol prefix at all.
>>
> Well, this:
>
> https://docs.python.org/3/using/cmdline.html
>
> is working for me now, but this:
>
> https://docs.python.org/3/whatsnew/index.html
>
> isn't. It looks like some kind of intermittent glitch somewhere...
>
And now:

https://docs.python.org/3/whatsnew/index.html

is working.

All on Firefox, BTW.

From MAIERA at de.ibm.com  Tue Oct 21 18:43:09 2014
From: MAIERA at de.ibm.com (Andreas Maier)
Date: Tue, 21 Oct 2014 18:43:09 +0200
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
Message-ID: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>


Hi. Today, I ran across this, in Python 2.7.6:

>>> class C:
...   pass
...
>>> issubclass(C,object)
False
>>> isinstance(C(),object)
True   <-- ???

The description of isinstance() in Python 2.7 does not reveal this result
(to my reading).

>From a duck-typing perspective, one would also not guess that an instance
of C would be considered an instance of object:

>>> dir(C())
['__doc__', '__module__']
>>> dir(object())
['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__',
'__hash__', '__init__', '__new__', '__reduce__
', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__',
'__subclasshook__']

-> What is the motivation for isinstance(C,object) to return True in Python
2.7?

Andy

Andreas Maier
IBM Senior Technical Staff Member, Systems Management Architecture & Design
IBM Research & Development Laboratory Boeblingen, Germany
maiera at de.ibm.com, +49-7031-16-3654
________________________________________________________________________
IBM Deutschland Research & Development GmbH
Vorsitzende des Aufsichtsrats: Martina Koederitz
Geschaeftsfuehrung: Dirk Wittkopp
Sitz der Gesellschaft: Boeblingen
Registergericht: Amtsgericht Stuttgart, HRB 243294


From pjenvey at underboss.org  Tue Oct 21 19:03:05 2014
From: pjenvey at underboss.org (Philip Jenvey)
Date: Tue, 21 Oct 2014 10:03:05 -0700
Subject: [Python-Dev] PyPy3 2.4.0 released
Message-ID: <6A935F6F-9790-4F8E-80F4-66D52259B76F@underboss.org>

=================================================
PyPy3 2.4 - Snow White
=================================================

We're pleased to announce PyPy3 2.4, which contains significant performance
enhancements and bug fixes.

You can download the PyPy3 2.4.0 release here:

    http://pypy.org/download.html

PyPy3 Highlights
================

Issues reported with our previous release were fixed after reports from users on
our new issue tracker at https://bitbucket.org/pypy/pypy/issues or on IRC at
#pypy. Here is a summary of the user-facing PyPy3 specific changes:

* Better Windows compatibility, e.g. the nt module functions _getfinalpathname
  & _getfileinformation are now supported (the former is required for the
  popular pathlib library for example)

* Various fsencode PEP 383 related fixes to the posix module (readlink, uname,
  ttyname and ctermid) and improved locale handling

* Switched default binary name os POSIX distributions to 'pypy3' (which
  symlinks to to 'pypy3.2')

* Fixed a couple different crashes related to parsing Python 3 source code

Further Highlights (shared w/ PyPy2)
====================================

Benchmarks improved after internal enhancements in string and
bytearray handling, and a major rewrite of the GIL handling. This means
that external calls are now a lot faster, especially the CFFI ones. It also
means better performance in a lot of corner cases with handling strings or
bytearrays. The main bugfix is handling of many socket objects in your
program which in the long run used to "leak" memory.

We fixed a memory leak in IO in the sandbox_ code

We welcomed more than 12 new contributors, and conducted two Google
Summer of Code projects, as well as other student projects not
directly related to Summer of Code.

* Reduced internal copying of bytearray operations

* Tweak the internal structure of StringBuilder to speed up large string
  handling, which becomes advantageous on large programs at the cost of slightly
  slower small *benchmark* type programs.

* Boost performance of thread-local variables in both unjitted and jitted code,
  this mostly affects errno handling on linux, which makes external calls
  faster.

* Move to a mixed polling and mutex GIL model that make mutlithreaded jitted
  code run *much* faster

* Optimize errno handling in linux (x86 and x86-64 only)

* Remove ctypes pythonapi and ctypes.PyDLL, which never worked on PyPy

* Classes in the ast module are now distinct from structures used by
  the compiler, which simplifies and speeds up translation of our
  source code to the PyPy binary interpreter

* Win32 now links statically to zlib, expat, bzip, and openssl-1.0.1i.
  No more missing DLLs

* Many issues were resolved_ since the 2.3.1 release in June

.. _`whats-new`: http://doc.pypy.org/en/latest/whatsnew-2.4.0.html
.. _resolved: https://bitbucket.org/pypy/pypy/issues?status=resolved
.. _sandbox: http://doc.pypy.org/en/latest/sandbox.html

We have further improvements on the way: rpython file handling,
numpy linalg compatibility, as well
as improved GC and many smaller improvements.

Please try it out and let us know what you think. We especially welcome
success stories, we know you are using PyPy, please tell us about it!

Cheers

The PyPy Team

From guido at python.org  Tue Oct 21 19:13:30 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Oct 2014 10:13:30 -0700
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
In-Reply-To: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
References: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
Message-ID: <CAP7+vJJzFP8F_YK52BpnyYk4nH8NOm5zc0RV0_HME5tak1Ta_A@mail.gmail.com>

This is one of the unfortunate effects of the existence of "old-style"
classes in Python 2. The old-style class hierarchy is distinct from the
new-style class hierarchy, but instances of old-style classes are still
objects (since in Python, *everything* is an object).

For new code, and whenever you have an opportunity to refactor old code,
you should use new-style classes, by inheriting your class from object (or
from another class that inherits from object).

On Tue, Oct 21, 2014 at 9:43 AM, Andreas Maier <MAIERA at de.ibm.com> wrote:

>
> Hi. Today, I ran across this, in Python 2.7.6:
>
> >>> class C:
> ...   pass
> ...
> >>> issubclass(C,object)
> False
> >>> isinstance(C(),object)
> True   <-- ???
>
> The description of isinstance() in Python 2.7 does not reveal this result
> (to my reading).
>
> From a duck-typing perspective, one would also not guess that an instance
> of C would be considered an instance of object:
>
> >>> dir(C())
> ['__doc__', '__module__']
> >>> dir(object())
> ['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__',
> '__hash__', '__init__', '__new__', '__reduce__
> ', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__',
> '__subclasshook__']
>
> -> What is the motivation for isinstance(C,object) to return True in Python
> 2.7?
>
> Andy
>
> Andreas Maier
> IBM Senior Technical Staff Member, Systems Management Architecture & Design
> IBM Research & Development Laboratory Boeblingen, Germany
> maiera at de.ibm.com, +49-7031-16-3654
> ________________________________________________________________________
> IBM Deutschland Research & Development GmbH
> Vorsitzende des Aufsichtsrats: Martina Koederitz
> Geschaeftsfuehrung: Dirk Wittkopp
> Sitz der Gesellschaft: Boeblingen
> Registergericht: Amtsgericht Stuttgart, HRB 243294
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141021/28df1dfb/attachment.html>

From mark at hotpy.org  Tue Oct 21 19:25:33 2014
From: mark at hotpy.org (Mark Shannon)
Date: Tue, 21 Oct 2014 18:25:33 +0100
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
In-Reply-To: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
References: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
Message-ID: <5446970D.7050200@hotpy.org>

Hi,
The problem is a side effect of the fact that old-style classes are 
implemented on top of new-style meta-classes.
Consequently although C is the "class" of C() it is not its "type".

 >>> type(C())
<type 'instance'>

 >>> type(C()).__mro__
(<type 'instance'>, <type 'object'>)

therefore
 >>> issubclass(type(C()), object)
True

which implies
 >>> isinstance(C(),object)
True

Cheers,
Mark.


On 21/10/14 17:43, Andreas Maier wrote:
>
> Hi. Today, I ran across this, in Python 2.7.6:
>
>>>> class C:
> ...   pass
> ...
>>>> issubclass(C,object)
> False
>>>> isinstance(C(),object)
> True   <-- ???
>
> The description of isinstance() in Python 2.7 does not reveal this result
> (to my reading).
>
>  From a duck-typing perspective, one would also not guess that an instance
> of C would be considered an instance of object:
>
>>>> dir(C())
> ['__doc__', '__module__']
>>>> dir(object())
> ['__class__', '__delattr__', '__doc__', '__format__', '__getattribute__',
> '__hash__', '__init__', '__new__', '__reduce__
> ', '__reduce_ex__', '__repr__', '__setattr__', '__sizeof__', '__str__',
> '__subclasshook__']
>
> -> What is the motivation for isinstance(C,object) to return True in Python
> 2.7?
>
> Andy
>
> Andreas Maier
> IBM Senior Technical Staff Member, Systems Management Architecture & Design
> IBM Research & Development Laboratory Boeblingen, Germany
> maiera at de.ibm.com, +49-7031-16-3654
> ________________________________________________________________________
> IBM Deutschland Research & Development GmbH
> Vorsitzende des Aufsichtsrats: Martina Koederitz
> Geschaeftsfuehrung: Dirk Wittkopp
> Sitz der Gesellschaft: Boeblingen
> Registergericht: Amtsgericht Stuttgart, HRB 243294
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mark%40hotpy.org
>

From barry at python.org  Tue Oct 21 19:53:52 2014
From: barry at python.org (Barry Warsaw)
Date: Tue, 21 Oct 2014 13:53:52 -0400
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
In-Reply-To: <CAP7+vJJzFP8F_YK52BpnyYk4nH8NOm5zc0RV0_HME5tak1Ta_A@mail.gmail.com>
References: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
 <CAP7+vJJzFP8F_YK52BpnyYk4nH8NOm5zc0RV0_HME5tak1Ta_A@mail.gmail.com>
Message-ID: <20141021135352.410b72ab@anarchist>

On Oct 21, 2014, at 10:13 AM, Guido van Rossum wrote:

>For new code, and whenever you have an opportunity to refactor old code,
>you should use new-style classes, by inheriting your class from object (or
>from another class that inherits from object).

One nice way to do this module-globally is to set:

__metaclass__ = type

at the top of your file.  Then when you're ready to drop Python 2, it's an
easy clean up.

Cheers,
-Barry

From guido at python.org  Tue Oct 21 20:22:34 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 21 Oct 2014 11:22:34 -0700
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
In-Reply-To: <20141021135352.410b72ab@anarchist>
References: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
 <CAP7+vJJzFP8F_YK52BpnyYk4nH8NOm5zc0RV0_HME5tak1Ta_A@mail.gmail.com>
 <20141021135352.410b72ab@anarchist>
Message-ID: <CAP7+vJKpXURdtVAYQnRcAj56qoX=_i75nr5wHKziOBYsPXULyQ@mail.gmail.com>

Hm. I've never been a fan of that. EIBTI and such...

On Tue, Oct 21, 2014 at 10:53 AM, Barry Warsaw <barry at python.org> wrote:

> On Oct 21, 2014, at 10:13 AM, Guido van Rossum wrote:
>
> >For new code, and whenever you have an opportunity to refactor old code,
> >you should use new-style classes, by inheriting your class from object (or
> >from another class that inherits from object).
>
> One nice way to do this module-globally is to set:
>
> __metaclass__ = type
>
> at the top of your file.  Then when you're ready to drop Python 2, it's an
> easy clean up.
>
> Cheers,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141021/634500ec/attachment.html>

From barry at python.org  Tue Oct 21 20:28:54 2014
From: barry at python.org (Barry Warsaw)
Date: Tue, 21 Oct 2014 14:28:54 -0400
Subject: [Python-Dev] isinstance() on old-style classes in Py 2.7
In-Reply-To: <CAP7+vJKpXURdtVAYQnRcAj56qoX=_i75nr5wHKziOBYsPXULyQ@mail.gmail.com>
References: <OF14DA1E13.A7A6F1B3-ONC1257D78.005914A5-C1257D78.005BD783@de.ibm.com>
 <CAP7+vJJzFP8F_YK52BpnyYk4nH8NOm5zc0RV0_HME5tak1Ta_A@mail.gmail.com>
 <20141021135352.410b72ab@anarchist>
 <CAP7+vJKpXURdtVAYQnRcAj56qoX=_i75nr5wHKziOBYsPXULyQ@mail.gmail.com>
Message-ID: <20141021142854.2bf3b5cd@anarchist>

On Oct 21, 2014, at 11:22 AM, Guido van Rossum wrote:

>Hm. I've never been a fan of that. EIBTI and such...

Yeah, I just hate seeing `class Foo(object)` in Python 3 and am too lazy to
clean up every class definition. ;)  YMMV!

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141021/2b3bac82/attachment.sig>

From python at mrabarnett.plus.com  Tue Oct 21 21:37:33 2014
From: python at mrabarnett.plus.com (MRAB)
Date: Tue, 21 Oct 2014 20:37:33 +0100
Subject: [Python-Dev] https://docs.python.org/3/using/index.html not
 linking correctly
In-Reply-To: <m24a04$o9i$1@ger.gmane.org>
References: <m23plv$2gq$1@ger.gmane.org>
 <CAF-Rda9qmzW2Oh0vgPRCs-9mzCpL_rRm9mD+4d1xbr6Mgae00g@mail.gmail.com>
 <54459AE9.6060005@mrabarnett.plus.com> <m24a04$o9i$1@ger.gmane.org>
Message-ID: <5446B5FD.5050404@mrabarnett.plus.com>

On 2014-10-21 01:39, Terry Reedy wrote:
> On 10/20/2014 7:29 PM, MRAB wrote:
>> On 2014-10-21 00:09, Eli Bendersky wrote:
>>>
>>>
>>> On Mon, Oct 20, 2014 at 1:01 PM, Terry Reedy <tjreedy at udel.edu
>>> <mailto:tjreedy at udel.edu>> wrote:
>>>
>>>     If I go to https://docs.python.org/3/using/index.html and click on
>>>     any of the TOC entries, I get 'connecting' indefinitely.  This
>>>     problem seems unique to this file. I tried several other index files
>>>     and clicking am entry brings up the corresponding page almost
>>>     immediately.
>>>
>>>
>>> Works fine for me, Terry, in both Chrome and Firefox. Could be something
>>> in your browser/caching? Try "private mode" or whatever that's called in
>>> your browser of choice.
>>>
>> Just tried the first link:
>>
>> https://docs.python.org/3/using/cmdline.html
>>
>> I also get 'Connecting' indefinitely.
>>
>> So I tested the link here:
>>
>> http://www.downforeveryoneorjustme.com/
>>
>> It said: "It's not just you! http://https looks down from here."
>
> I am using Firefox, but the difference is that by happenstance, I was in
> privacy mode (perhaps for the first time with python.org -- I want the
> docs in the cache and history list -- because I accessed the docs to
> answer a StackOverflow question.) It is not just that document but also
> Execution model and Expression in Reference.  (I tried each 3 times
> minutes apart.)  I have not had a similiar problem with any Arstechnica
> or StackOverflow pages, or similar sites.
>
> So the solution for me is to use normal mode, but something seems not
> quite right in the interaction between privacy mode Firefox and python.org.
>
> No problem I could find with Internet Explorer's 'In Privacy' mode.
>
I have/had occasional problems even though I was using normal mode.


From peke at iki.fi  Wed Oct 22 12:15:14 2014
From: peke at iki.fi (=?UTF-8?Q?Pekka_Kl=C3=A4rck?=)
Date: Wed, 22 Oct 2014 13:15:14 +0300
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
Message-ID: <CACXFLVcdp1iYvnG5MsacSaHsB+asgcivcK1yoqVa_rifC=GMzw@mail.gmail.com>

[Replying to a mail that was sent before I joined this list. Quoting,
headers, etc. aren't exactly right.]

Nick Coghlan wrote:
>On 4 October 2014 10:51, Donald Stufft <donald at stufft.io> wrote:
>> Whoops, I misred.
>>
>> So to be clear, you think:
>>
>> install -> pip, pip2, pip2.7
>> altinstall -> pip2.7
>
> To spell out the assumption I didn't make clear when helping with the
> backport PEP, the difference comes from PEP 394, which specifies the
> following behaviour when installing Python itself:
>
> Python 2.7: python, python2, python2.7
> Python 3.4: python3, python3.4
>
> That maps to ensurepip as:
>
> Python 2.7: pip, pip2, pip2.7
> Python 3.4: pip3, pip3.4

I just installed Python 3.4.2 on Windows and noticed that its Scripts
directory has these files out-of-the-box:

easy_install.exe
easy_install-3.4.exe.
pip.exe
pip3.exe
pip3.4.exe

Based on Nick's explanation above, having pip.exe there looks like bug
in the installer and could easily cause a conflict with other pip
installations. I don't understand why easy_install is included there
in the first place, but easy_install.exe can obviously cause a similar
conflict.

Cheers,
    .peke
-- 
Agile Tester/Developer/Consultant :: http://eliga.fi
Lead Developer of Robot Framework :: http://robotframework.org

From geoffspear at gmail.com  Wed Oct 22 13:10:16 2014
From: geoffspear at gmail.com (Geoffrey Spear)
Date: Wed, 22 Oct 2014 07:10:16 -0400
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <CACXFLVcdp1iYvnG5MsacSaHsB+asgcivcK1yoqVa_rifC=GMzw@mail.gmail.com>
References: <CACXFLVcdp1iYvnG5MsacSaHsB+asgcivcK1yoqVa_rifC=GMzw@mail.gmail.com>
Message-ID: <CAGifb9FVNXzD6VHKf_KVgqwvMhRCES0ZiRbcOxvoeT0ziY6+mQ@mail.gmail.com>

On Wed, Oct 22, 2014 at 6:15 AM, Pekka Kl?rck <peke at iki.fi> wrote:
> [Replying to a mail that was sent before I joined this list. Quoting,
> headers, etc. aren't exactly right.]
>
> Nick Coghlan wrote:
>>On 4 October 2014 10:51, Donald Stufft <donald at stufft.io> wrote:
>>> Whoops, I misred.
>>>
>>> So to be clear, you think:
>>>
>>> install -> pip, pip2, pip2.7
>>> altinstall -> pip2.7
>>
>> To spell out the assumption I didn't make clear when helping with the
>> backport PEP, the difference comes from PEP 394, which specifies the
>> following behaviour when installing Python itself:
>>
>> Python 2.7: python, python2, python2.7
>> Python 3.4: python3, python3.4
>>
>> That maps to ensurepip as:
>>
>> Python 2.7: pip, pip2, pip2.7
>> Python 3.4: pip3, pip3.4
>
> I just installed Python 3.4.2 on Windows and noticed that its Scripts
> directory has these files out-of-the-box:
>
> easy_install.exe
> easy_install-3.4.exe.
> pip.exe
> pip3.exe
> pip3.4.exe
>
> Based on Nick's explanation above, having pip.exe there looks like bug
> in the installer and could easily cause a conflict with other pip
> installations. I don't understand why easy_install is included there
> in the first place, but easy_install.exe can obviously cause a similar
> conflict.

Nick's explanation is based on PEP 394, which explicitly does not
apply to Windows. The Windows Python executables are called
"python.exe" and "pythonw.exe"; no "3" has ever been part of the name
and it's not surprising that there's a matching "pip.exe". The
pip3.exe and pip3.4.exe being installed are actually the anomalies
here, but I wouldn't call them a bug.

From ncoghlan at gmail.com  Wed Oct 22 13:45:31 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 22 Oct 2014 21:45:31 +1000
Subject: [Python-Dev] Backporting ensurepip to 2.7,
	Which commands to install?
In-Reply-To: <CAGifb9FVNXzD6VHKf_KVgqwvMhRCES0ZiRbcOxvoeT0ziY6+mQ@mail.gmail.com>
References: <CACXFLVcdp1iYvnG5MsacSaHsB+asgcivcK1yoqVa_rifC=GMzw@mail.gmail.com>
 <CAGifb9FVNXzD6VHKf_KVgqwvMhRCES0ZiRbcOxvoeT0ziY6+mQ@mail.gmail.com>
Message-ID: <CADiSq7ft23sZ5KPfa0y7sPxuznjrjzK8xTvSbx573fWUXzRkEQ@mail.gmail.com>

On 22 October 2014 21:10, Geoffrey Spear <geoffspear at gmail.com> wrote:
> On Wed, Oct 22, 2014 at 6:15 AM, Pekka Kl?rck <peke at iki.fi> wrote:
>> I just installed Python 3.4.2 on Windows and noticed that its Scripts
>> directory has these files out-of-the-box:
>>
>> easy_install.exe
>> easy_install-3.4.exe.
>> pip.exe
>> pip3.exe
>> pip3.4.exe
>>
>> Based on Nick's explanation above, having pip.exe there looks like bug
>> in the installer and could easily cause a conflict with other pip
>> installations. I don't understand why easy_install is included there
>> in the first place, but easy_install.exe can obviously cause a similar
>> conflict.
>
> Nick's explanation is based on PEP 394, which explicitly does not
> apply to Windows. The Windows Python executables are called
> "python.exe" and "pythonw.exe"; no "3" has ever been part of the name
> and it's not surprising that there's a matching "pip.exe". The
> pip3.exe and pip3.4.exe being installed are actually the anomalies
> here, but I wouldn't call them a bug.

Right, the design on Windows is a little different, as the Python 3
executable has always remained "python.exe" there. Originally we
followed PEP 394 style naming on Windows as well, but then realised
during the 3.4 beta cycle that doing so didn't actually make sense
(see http://bugs.python.org/issue20568 and
http://bugs.python.org/issue20139)

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From pmiscml at gmail.com  Wed Oct 22 22:32:26 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Wed, 22 Oct 2014 23:32:26 +0300
Subject: [Python-Dev] How io.IOBase.readline() should behave when used
 on non-blocking obj and no data available?
In-Reply-To: <20141016133406.07559453@fsol>
References: <20141016035432.05cd914f@x230>
	<20141016133406.07559453@fsol>
Message-ID: <20141022233226.67d9b58e@x230>

Hello,

On Thu, 16 Oct 2014 13:34:06 +0200
Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Thu, 16 Oct 2014 03:54:32 +0300
> Paul Sokolovsky <pmiscml at gmail.com> wrote:
> > Hello,
> > 
> > io.RawIOBase.read() is well specified for behavior in case it
> > immediately gets a would-block condition: "If the object is in
> > non-blocking mode and no bytes are available, None is returned."
> > (https://docs.python.org/3/library/io.html#io.RawIOBase.read).
> > 
> > However, nothing is said about such condition for
> > io.IOBase.readline(), which is mixin method in a base class,
> > default implementation of which thus would use io.RawIOBase.read().
> > Looking at 3.4.0 source, iobase.c: iobase_readline() has:
> > 
> >         b = _PyObject_CallMethodId(self, &PyId_read, "n",
> > nreadahead); [...]
> >         if (!PyBytes_Check(b)) {
> >             PyErr_Format(PyExc_IOError,
> >                          "read() should have returned a bytes
> > object, " "not '%.200s'", Py_TYPE(b)->tp_name);
> > 
> > I.e. it's not even ready to receive legitimate return value of None
> > from read(). I didn't try to write a testcase though, so may be
> > missing something.
> > 
> > So, how readline() should behave in this case, and can that be
> > specified in the Library Reference?
> 
> Well, the problem is that it's not obvious how to implement such
> methods in a non-blocking context.
> 
> Let's says some data is received but there isn't a complete line.
> Should readline() return just that data (an incomplete line)? That
> breaks the API's contract. Should readline() buffer the incomplete
> line and keep it for the next readline() call? But then the internal
> buffer becomes unbounded: perhaps there is no new line in the next
> 4GB of incoming data...
> 
> And besides, raw I/O objects *shouldn't* have an internal buffer.
> That's the role of the buffered I/O layer.

Yes, sure, readline() is defined on io.IOBase which is underspecified
for buffered-ness, so should have behavior which can be implemented for
both buffered and unbuffered case.

You're right also in saying that readline on non-blocking stream can't
work always the same way as blocking version, and that it "breaks the
API's contract". But it should be possible to extend that contract
for non-blocking readline() in pretty natural way:

1) An invariant of readline() is that it doesn't modify stream data,
it just segments it. So, readline() + write() looped until EOF will
produce the same result as read(N) + write(). Non-blocking readline()
will still satisfy this.

2) Even with blocking readline(), it can return a string not ending
with end-of-line character(s). For blocking readline, this may happen
with just the last line of a stream, with non-blocking, it may happen
for any call. The point is that even with blocking readline(), the
caller should be ready to check that a line satisfies its "complete
line" criteria, for non-blocking case, it's just will be different set
of criteria and actions to satisfy them.


I guess, defining non-blocking readline() in such way is better then
let it be underspecified whether it's supported or not (and if yes,
then how), or prohibit it.


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From pmiscml at gmail.com  Wed Oct 22 22:36:35 2014
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Wed, 22 Oct 2014 23:36:35 +0300
Subject: [Python-Dev] How io.IOBase.readline() should behave when used
 on non-blocking obj and no data available?
In-Reply-To: <CAP7+vJKMqps0C-HzpqvRgNjE2csqok9_anwfNi84PPfPA_VKDA@mail.gmail.com>
References: <20141016035432.05cd914f@x230> <20141016133406.07559453@fsol>
 <CAP7+vJKMqps0C-HzpqvRgNjE2csqok9_anwfNi84PPfPA_VKDA@mail.gmail.com>
Message-ID: <20141022233635.711c227f@x230>

Hello,

On Thu, 16 Oct 2014 07:50:02 -0700
Guido van Rossum <guido at python.org> wrote:

[]
> > > So, how readline() should behave in this case, and can that be
> > > specified in the Library Reference?
> >
> > Well, the problem is that it's not obvious how to implement such
> > methods in a non-blocking context.
> >
> > Let's says some data is received but there isn't a complete line.
> > Should readline() return just that data (an incomplete line)? That
> > breaks the API's contract. Should readline() buffer the incomplete
> > line and keep it for the next readline() call? But then the
> > internal buffer becomes unbounded: perhaps there is no new line in
> > the next 4GB of incoming data...
> >
> > And besides, raw I/O objects *shouldn't* have an internal buffer.
> > That's the role of the buffered I/O layer.
> >
> 
[]

> (if any). If the underlying stream returns None, I think it makes
> sense for readline() to return None too -- attempting to read more
> will just turn into a busy-wait loop, and that's the opposite of what
> should happen.
> 
[]
> 
> (Alternatively, we could raise BlockingIOError, which is that the OS
> level read() raises if there's no data immediately available on a
> non-blocking FD; but it seems that streams have already gotten a

Yes, that's the choices I had in mind - either return None, or raise
an exception. Thanks for confirming that None is a better choice here,
that's what I implemented in MicroPython. Thanks for other points
and commentary too.


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From matthew.i.frank at intel.com  Thu Oct 23 21:22:57 2014
From: matthew.i.frank at intel.com (Frank, Matthew I)
Date: Thu, 23 Oct 2014 19:22:57 +0000
Subject: [Python-Dev] Cross compiling Python (for Android)
Message-ID: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E0CDA2E@FMSMSX106.amr.corp.intel.com>

This email is about my experience getting CPython (3.4.1) to
cross-compile and run on x86 Android (4.4.2 with sdk 19 and ndk-r9).
I know that Android is not a supported architecture (and I won't
regale you with stories about the complete locale and mbstowcs support
I had to borrow from FreeBSD to get it working).  The purpose of this
email is that several things I found are arguably "bugs" in the Python
build system or code when it comes to cross-compiling that are exposed
by Android's poor Posix support.  I'd like some advice about what kind
of patch (if any) would be most suitable for fixing the problems on
the Python side.

Just to be complete:  I'm configuring with

     CPPFLAGS=-I../my-locale ../Python-3.4.1/configure --enable-shared
     --prefix=/path/to/install/dir --build=x86_64-linux-gnu
     --host=i686-linux-android --disable-ipv6 ac_cv_file__dev_ptmx=no
     ac_cv_file__dev_ptc=no ac_cv_little_endian_double=yes

(The CPPFLAGS addition is to get the headers for my locale fixes
instead of the default Android ones.  ac_cv_file__dev_ptmx=no and
ac_cv_file__dev_ptc=no are because I don't have /dev/whatever on my
build machine.  ac_cv_little_endian_double is because configure for
cross builds can't figure out the endianness of doubles on the host
(because it is running on the build machine not the host.)  (For ARM
it would be ac_cv_mixed_endian_double=yes.)

I've gotten to the point where `make; make install` succeeds up to the
point of building something that runs on my Android system (from the
command line) and `python -m test` runs 388 tests, with 321 ok, 24
test failures and 43 tests skipped (the skips mostly due, I think, to
me not yet having installed the right cross-building support for
things like bz2 and dbm.)

1. `make` succeeds but `make install` always fails at the end with
   something having to do with being unable to run "ensurepip"
   (presumably because ensurepip requires modules that only run on the
   host, not the build module.)  So it seems this should be wrapped in
   a test for cross compilation, but I haven't looked at exactly what
   yet.  The error is:

   /linux-python/bin/python3.4: Error while finding spec for
   'ensurepip.__main__' (<class 'ImportError'>:
   /build-directory/build/lib.linux-i686-3.4/time.cpython-34m.so:
   wrong ELF class: ELFCLASS32); 'ensurepip' is a package and cannot
   be directly executed make: *** [install] Error 1

2. setup.py is missing -lm flag for several modules.  On Linux this
   problem is hidden because libm is already loaded by the executable
   calling dlopen(), but Android's loader deals with unknown symbols
   differently (searches only the libs explicitly linked against the
   module being loaded.)  http://bugs.python.org/issue21668 reports
   the problem for selectmodule (can't find ceil()) and timemodule
   (fmod() and floor()).  But there are at least two more: audioop
   fails to load because it needs floor() and ctypes_test fails to
   load because it needs sqrt().  I'll happily update the patch in
   21668.

   Is there any fundamental objection to adding the -lm flag to the
   link step where it is necessary?

3. What is ossaudiodev?  It tries to include "sys/soundcard.h", which
   I don't have on my system.   (The rule in setup.py is
   wrapped in a test for host of Linux/FreeBSD/Darwin, but Android x86
   gets configured with --host=i686-linux-android so to turn it off
   requires an extra test for "and not cross_compiling".)

   Can I just turn off ossaudiodev for cross compiling or might
   someone want it in a different type of cross build?  (In which case
   I think I'll have to write some kind autoconf rule for it, which I
   don't quite know how to do yet.)

4. Module _decimal is failing to compile.  The problem is that it has
   a header called memory.h.  Android's libc has the problem that
   /usr/include/stdlib.h includes <memory.h>.  But the build system
   puts -I. on the include path before the system dirs (as it should)
   so when compiling _decimal, Modules/_decimal/libmpdec/memory.h gets
   found instead of /usr/include/memory.h.  Shiz has a patch here:
   https://github.com/rave-engine/python3-android/blob/master/mk/python/3.3.5/p\
ython-3.3.5-android-libmpdec.patch
   (which renames memory.h -> mpmemory.h) but I don't know

   a.  Is there a tracker for this yet?  and
   b.  Is Shiz's fix the desired one or should I be looking for
       another approach?  (Maybe modifying the -I flags for the build
       of just the build of _decimal or something?)

5. I'm not sure what test configure is actually doing for gethostby*()
   in a cross-compile environment.  In any case Android has a bug
   where gethostbyaddr_r() is declared in the headers, but not
   actually implemented in libc.  So I have to modify my pyconfig.h by
   hand to define HAVE_GETHOSTBYNAME and undef HAVE_GETHOSTBYNAME_R
   and HAVE_GETHOSTBYNAME_R_6_ARG.

   Is there a variable (like ac_cv_little_endian_double) that I can
   give to `configure` to make it set HAVE_GETHOSTBYNAME* the way I
   need?  If so I've been unable to figure it out.

6. Android's <pwd.h> header mysteriously leaves the pw_gecos field out
   of struct passwd.  Is a fix like defining a new variable
   HAVE_BROKEN_GECOS_FIELD the appropriate way to go with this?  (If
   this is an okay solution then the patch to Modules/pwdmodule.c is
   shown below, but I still have to figure out how to patch
   configure.ac to test for the condition and set the variable
   appropriately, so a pointer to a similar block of code in
   configure.ac would be appreciated.)

Sorry for the TL;DR.  I appreciate your having taken the time to read
this far.

Thanks,
-Matt

Proposed patch for pwdmodule.c:

--- a/Modules/pwdmodule.c 2014-05-19 00:19:39.000000000 -0500
+++ b/Modules/pwdmodule.c       2014-10-21 18:00:35.676331205 -0500
@@ -57,6 +57,10 @@
   }
}

+#if defined(HAVE_BROKEN_GECOS_FIELD)
+static char fakePwGecos[256] = "";
+#endif
+
static PyObject *
mkpwent(struct passwd *p)
{
@@ -72,7 +76,11 @@
     SETS(setIndex++, p->pw_passwd);
     PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromUid(p->pw_uid));
     PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromGid(p->pw_gid));
+#if !defined(HAVE_BROKEN_GECOS_FIELD)
     SETS(setIndex++, p->pw_gecos);
+#else
+    SETS(setIndex++, fakePwGecos);
+#endif
     SETS(setIndex++, p->pw_dir);
     SETS(setIndex++, p->pw_shell);

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141023/314dd74b/attachment.html>

From status at bugs.python.org  Fri Oct 24 18:08:11 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 24 Oct 2014 18:08:11 +0200 (CEST)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20141024160811.38FFB561EF@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-10-17 - 2014-10-24)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4604 (+17)
  closed 29877 (+44)
  total  34481 (+61)

Open issues with patches: 2144 


Issues opened (40)
==================

#17401: io.FileIO closefd parameter is not documented nor shown in rep
http://bugs.python.org/issue17401  reopened by serhiy.storchaka

#22659: SyntaxError in the configure_ctypes
http://bugs.python.org/issue22659  opened by bill9889

#22662: subprocess.Popen.communicate causing local tty terminal settin
http://bugs.python.org/issue22662  opened by kflavin

#22665: shutil.__all__ incomplete
http://bugs.python.org/issue22665  opened by vadmium

#22666: email.Header no encoding of unicode strings containing newline
http://bugs.python.org/issue22666  opened by flavio

#22668: memoryview.format is corrupted due to dangling shared pointer
http://bugs.python.org/issue22668  opened by Knio

#22669: Test_venv fails when _ctypes is not available.
http://bugs.python.org/issue22669  opened by terry.reedy

#22671: Typo in class io.BufferedIOBase docs
http://bugs.python.org/issue22671  opened by gigaplastik

#22672: float arguments in scientific notation not supported by argpar
http://bugs.python.org/issue22672  opened by jnespolo

#22673: document the special features (eg: fdclose=False) of the stand
http://bugs.python.org/issue22673  opened by snaphat

#22674: strsignal() missing from signal module
http://bugs.python.org/issue22674  opened by Dolda2000

#22678: An OSError subclass for "no space left on device" would be nic
http://bugs.python.org/issue22678  opened by bochecha

#22679: Add encodings of supported in glibc locales
http://bugs.python.org/issue22679  opened by serhiy.storchaka

#22680: unittest discovery is fragile
http://bugs.python.org/issue22680  opened by pitrou

#22681: Add support of KOI8-T encoding
http://bugs.python.org/issue22681  opened by serhiy.storchaka

#22682: Add support of KZ1048 (RK1048) encoding
http://bugs.python.org/issue22682  opened by serhiy.storchaka

#22683: bisect index out of bounds issue
http://bugs.python.org/issue22683  opened by Paul.Ianas

#22684: message.as_bytes() produces recursion depth exceeded
http://bugs.python.org/issue22684  opened by pas

#22685: memory leak: no transport for pipes by create_subprocess_exec/
http://bugs.python.org/issue22685  opened by wabu

#22687: horrible performance of textwrap.wrap() with a long word
http://bugs.python.org/issue22687  opened by inkerman

#22689: Posix getenv makes no guarantee of lifetime of returned string
http://bugs.python.org/issue22689  opened by aidanhs

#22695: open() declared deprecated in python 3 docs
http://bugs.python.org/issue22695  opened by ??????????????.??????????????

#22696: Add a function to know about interpreter shutdown
http://bugs.python.org/issue22696  opened by pitrou

#22697: Deadlock with writing to stderr from forked process
http://bugs.python.org/issue22697  opened by ionel.mc

#22698: Add constants for ioctl request codes
http://bugs.python.org/issue22698  opened by serhiy.storchaka

#22699: cross-compilation of Python3.4
http://bugs.python.org/issue22699  opened by bill9889

#22700: email's header_value_parser missing defect report for 'abc at xyz
http://bugs.python.org/issue22700  opened by r.david.murray

#22701: Write unescaped unicode characters (Japanese, Chinese, etc) in
http://bugs.python.org/issue22701  opened by Michael.Kuss

#22702: to improve documentation for join() (str method)
http://bugs.python.org/issue22702  opened by vy0123

#22703: Idle Code Context: separate changing current and future editor
http://bugs.python.org/issue22703  opened by terry.reedy

#22704: Review extension enable options
http://bugs.python.org/issue22704  opened by terry.reedy

#22705: Idle extension configuration: add option-help option
http://bugs.python.org/issue22705  opened by terry.reedy

#22706: Idle extension configuration and key bindings
http://bugs.python.org/issue22706  opened by terry.reedy

#22707: Idle: changed options should take effect immediately
http://bugs.python.org/issue22707  opened by terry.reedy

#22708: httplib/http.client in method _tunnel used HTTP/1.0 CONNECT me
http://bugs.python.org/issue22708  opened by vova

#22709: restore accepting detached stdin in fileinput binary mode
http://bugs.python.org/issue22709  opened by akira

#22711: "legacy" distutils docs better than packaging guide
http://bugs.python.org/issue22711  opened by pitrou

#22714: target of 'import statement' entry in general index for 'i' is
http://bugs.python.org/issue22714  opened by vy0123

#22718: pprint not handline uncomparable dictionary keys, set members 
http://bugs.python.org/issue22718  opened by Andreas.Kostyrka

#22719: os.path.isfile & os.path.exists bug in while loop
http://bugs.python.org/issue22719  opened by hosford42



Most recent 15 issues with no replies (15)
==========================================

#22719: os.path.isfile & os.path.exists bug in while loop
http://bugs.python.org/issue22719

#22714: target of 'import statement' entry in general index for 'i' is
http://bugs.python.org/issue22714

#22706: Idle extension configuration and key bindings
http://bugs.python.org/issue22706

#22704: Review extension enable options
http://bugs.python.org/issue22704

#22703: Idle Code Context: separate changing current and future editor
http://bugs.python.org/issue22703

#22700: email's header_value_parser missing defect report for 'abc at xyz
http://bugs.python.org/issue22700

#22699: cross-compilation of Python3.4
http://bugs.python.org/issue22699

#22697: Deadlock with writing to stderr from forked process
http://bugs.python.org/issue22697

#22682: Add support of KZ1048 (RK1048) encoding
http://bugs.python.org/issue22682

#22679: Add encodings of supported in glibc locales
http://bugs.python.org/issue22679

#22678: An OSError subclass for "no space left on device" would be nic
http://bugs.python.org/issue22678

#22672: float arguments in scientific notation not supported by argpar
http://bugs.python.org/issue22672

#22671: Typo in class io.BufferedIOBase docs
http://bugs.python.org/issue22671

#22666: email.Header no encoding of unicode strings containing newline
http://bugs.python.org/issue22666

#22662: subprocess.Popen.communicate causing local tty terminal settin
http://bugs.python.org/issue22662



Most recent 15 issues waiting for review (15)
=============================================

#22709: restore accepting detached stdin in fileinput binary mode
http://bugs.python.org/issue22709

#22708: httplib/http.client in method _tunnel used HTTP/1.0 CONNECT me
http://bugs.python.org/issue22708

#22696: Add a function to know about interpreter shutdown
http://bugs.python.org/issue22696

#22685: memory leak: no transport for pipes by create_subprocess_exec/
http://bugs.python.org/issue22685

#22682: Add support of KZ1048 (RK1048) encoding
http://bugs.python.org/issue22682

#22681: Add support of KOI8-T encoding
http://bugs.python.org/issue22681

#22678: An OSError subclass for "no space left on device" would be nic
http://bugs.python.org/issue22678

#22672: float arguments in scientific notation not supported by argpar
http://bugs.python.org/issue22672

#22668: memoryview.format is corrupted due to dangling shared pointer
http://bugs.python.org/issue22668

#22666: email.Header no encoding of unicode strings containing newline
http://bugs.python.org/issue22666

#22665: shutil.__all__ incomplete
http://bugs.python.org/issue22665

#22649: Use _PyUnicodeWriter in case_operation()
http://bugs.python.org/issue22649

#22642: trace module: unclear error message
http://bugs.python.org/issue22642

#22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack)
http://bugs.python.org/issue22638

#22636: avoid using a shell in ctypes.util: replace os.popen with subp
http://bugs.python.org/issue22636



Top 10 most discussed issues (10)
=================================

#22685: memory leak: no transport for pipes by create_subprocess_exec/
http://bugs.python.org/issue22685  12 msgs

#17401: io.FileIO closefd parameter is not documented nor shown in rep
http://bugs.python.org/issue17401   6 msgs

#22599: traceback: errors in the linecache module at exit
http://bugs.python.org/issue22599   6 msgs

#22669: Test_venv fails when _ctypes is not available.
http://bugs.python.org/issue22669   6 msgs

#22673: document the special features (eg: fdclose=False) of the stand
http://bugs.python.org/issue22673   6 msgs

#22674: strsignal() missing from signal module
http://bugs.python.org/issue22674   6 msgs

#22711: "legacy" distutils docs better than packaging guide
http://bugs.python.org/issue22711   6 msgs

#10548: Error in setUp not reported as expectedFailure (unittest)
http://bugs.python.org/issue10548   5 msgs

#11820: idle3 shell os.system swallows shell command output
http://bugs.python.org/issue11820   5 msgs

#22680: unittest discovery is fragile
http://bugs.python.org/issue22680   5 msgs



Issues closed (40)
==================

#7186: Document specialness of __doc__, and possibly other "special" 
http://bugs.python.org/issue7186  closed by ethan.furman

#9351: argparse set_defaults on subcommands should override top level
http://bugs.python.org/issue9351  closed by r.david.murray

#16000: test_curses should use unittest
http://bugs.python.org/issue16000  closed by zach.ware

#16863: Python 2 error in Argparse tutorial
http://bugs.python.org/issue16863  closed by terry.reedy

#18853: Got ResourceWarning unclosed file when running Lib/shlex.py de
http://bugs.python.org/issue18853  closed by r.david.murray

#18976: distutils/command/build_ext passes wrong linker flags
http://bugs.python.org/issue18976  closed by Benedikt.Morbach

#19746: No introspective way to detect ModuleImportFailure in unittest
http://bugs.python.org/issue19746  closed by python-dev

#20689: socket.AddressFamily is absent in pydoc output
http://bugs.python.org/issue20689  closed by ethan.furman

#21298: Cheese shop registration stopped working for me recently
http://bugs.python.org/issue21298  closed by georg.brandl

#21991: The new email API should use MappingProxyType instead of retur
http://bugs.python.org/issue21991  closed by r.david.murray

#22160: Windows installers need to be updated following OpenSSL securi
http://bugs.python.org/issue22160  closed by zach.ware

#22186: Typos in .py files
http://bugs.python.org/issue22186  closed by berker.peksag

#22592: Drop support of Borland C compiler
http://bugs.python.org/issue22592  closed by haypo

#22637: avoid using a shell in uuid: replce os.popen with subprocess.P
http://bugs.python.org/issue22637  closed by haypo

#22644: Update Windows installers to OpenSSL 1.0.1j
http://bugs.python.org/issue22644  closed by zach.ware

#22648: Unable to install Python 3.4.2 amd64 on Windows 8.1
http://bugs.python.org/issue22648  closed by brp-log

#22653: Crash in insertdict
http://bugs.python.org/issue22653  closed by pitrou

#22655: Build error on CentOS 5.4
http://bugs.python.org/issue22655  closed by skrah

#22658: IMAP4 Example lacking host information
http://bugs.python.org/issue22658  closed by r.david.murray

#22660: Review ssl docs for security recommendations
http://bugs.python.org/issue22660  closed by pitrou

#22661: WinXP concerns
http://bugs.python.org/issue22661  closed by r.david.murray

#22663: patchcheck alters content of .png files
http://bugs.python.org/issue22663  closed by python-dev

#22664: IDLE: Standard output and error from multiprocessing vanishes
http://bugs.python.org/issue22664  closed by serhiy.storchaka

#22667: Incorrect evaluation of variables with names containing supple
http://bugs.python.org/issue22667  closed by benjamin.peterson

#22670: wrong site-package installation even if correct libdir passed
http://bugs.python.org/issue22670  closed by georg.brandl

#22675: typo in argparse.py
http://bugs.python.org/issue22675  closed by python-dev

#22676: _pickle's whichmodule() is slow
http://bugs.python.org/issue22676  closed by brett.cannon

#22677: IDLE: icon not loaded
http://bugs.python.org/issue22677  closed by Ahmad.El-Komey

#22686: random.randint does not include endpoint
http://bugs.python.org/issue22686  closed by mark.dickinson

#22688: Use the subprocess module in the uuid module
http://bugs.python.org/issue22688  closed by serhiy.storchaka

#22690: importing Gtk breaks strptime
http://bugs.python.org/issue22690  closed by sigzegv

#22691: A Better Help File
http://bugs.python.org/issue22691  closed by r.david.murray

#22692: Problems with Python's help()
http://bugs.python.org/issue22692  closed by r.david.murray

#22694: The help file issue I'm having.
http://bugs.python.org/issue22694  closed by r.david.murray

#22710: doctest exceptions include nondeterministic namespace
http://bugs.python.org/issue22710  closed by r.david.murray

#22712: Add 'input' argument to subprocess.check_call and subprocess.c
http://bugs.python.org/issue22712  closed by serhiy.storchaka

#22713: print()
http://bugs.python.org/issue22713  closed by georg.brandl

#22715: PEP 257: drop the recommendation for a blank line between the 
http://bugs.python.org/issue22715  closed by r.david.murray

#22716: Add reference to the object missing an attribute to AttributeE
http://bugs.python.org/issue22716  closed by serhiy.storchaka

#22717: PySSL segmentation fault
http://bugs.python.org/issue22717  closed by r.david.murray

From guido at python.org  Sat Oct 25 05:15:42 2014
From: guido at python.org (Guido van Rossum)
Date: Fri, 24 Oct 2014 20:15:42 -0700
Subject: [Python-Dev] Cross compiling Python (for Android)
In-Reply-To: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E0CDA2E@FMSMSX106.amr.corp.intel.com>
References: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E0CDA2E@FMSMSX106.amr.corp.intel.com>
Message-ID: <CAP7+vJ+LCePsshqkP_b5ziaBeMfSx6DAVe-oF4LGo3f=-VN-gw@mail.gmail.com>

Hi Frank,

Nobody has responded to you yet, but don't feel discouraged by that! The
core Python development group consists mostly of people who don't develop
mobile apps in their day jobs, and they may feel reluctant to maintain
changes that they can't personally test. (And I don't expect it would be
easy to set up Android or iOS buildbots either.)

But there are definitely people trying to use Python to develop mobile apps
(e.g. Kivy). Today there was a similar post to python-ideas about this. I
think the world may soon be eager to develop mobile apps in Python (the
platforms are maturing and the processors are getting faster), and Python
should be ready for this change in attitude.

Hopefully we can get some good patches in for the next bugfix releases of
Python 2.7 and 3.4, as well as the upcoming 3.5 alphas and betas.

Of course, the Kivy approach might work for some time yet (they have a set
of patches for Python 2.7.1 or 2.7.2), but it would be better if that
wasn't necessary, and Python could be build (with the right dev
environment) for iOS and Android.

A word of advice: the specific patches you have should probably be
submitted to the Python issue tracker (bugs.python.org). Also, several
smaller patches are more likely to be reviewed and checked in timely than
one mega-patch.

Good luck!

--Guido

On Thu, Oct 23, 2014 at 12:22 PM, Frank, Matthew I <
matthew.i.frank at intel.com> wrote:

>  This email is about my experience getting CPython (3.4.1) to
>
> cross-compile and run on x86 Android (4.4.2 with sdk 19 and ndk-r9).
>
> I know that Android is not a supported architecture (and I won't
>
> regale you with stories about the complete locale and mbstowcs support
>
> I had to borrow from FreeBSD to get it working).  The purpose of this
>
> email is that several things I found are arguably "bugs" in the Python
>
> build system or code when it comes to cross-compiling that are exposed
>
> by Android's poor Posix support.  I'd like some advice about what kind
>
> of patch (if any) would be most suitable for fixing the problems on
>
> the Python side.
>
>
>
> Just to be complete:  I'm configuring with
>
>
>
>      CPPFLAGS=-I../my-locale ../Python-3.4.1/configure --enable-shared
>
>      --prefix=/path/to/install/dir --build=x86_64-linux-gnu
>
>      --host=i686-linux-android --disable-ipv6 ac_cv_file__dev_ptmx=no
>
>      ac_cv_file__dev_ptc=no ac_cv_little_endian_double=yes
>
>
>
> (The CPPFLAGS addition is to get the headers for my locale fixes
>
> instead of the default Android ones.  ac_cv_file__dev_ptmx=no and
>
> ac_cv_file__dev_ptc=no are because I don't have /dev/whatever on my
>
> build machine.  ac_cv_little_endian_double is because configure for
>
> cross builds can't figure out the endianness of doubles on the host
>
> (because it is running on the build machine not the host.)  (For ARM
>
> it would be ac_cv_mixed_endian_double=yes.)
>
>
>
> I've gotten to the point where `make; make install` succeeds up to the
>
> point of building something that runs on my Android system (from the
>
> command line) and `python -m test` runs 388 tests, with 321 ok, 24
>
> test failures and 43 tests skipped (the skips mostly due, I think, to
>
> me not yet having installed the right cross-building support for
>
> things like bz2 and dbm.)
>
>
>
> 1. `make` succeeds but `make install` always fails at the end with
>
>    something having to do with being unable to run "ensurepip"
>
>    (presumably because ensurepip requires modules that only run on the
>
>    host, not the build module.)  So it seems this should be wrapped in
>
>    a test for cross compilation, but I haven't looked at exactly what
>
>    yet.  The error is:
>
>
>
>    /linux-python/bin/python3.4: Error while finding spec for
>
>    'ensurepip.__main__' (<class 'ImportError'>:
>
>    /build-directory/build/lib.linux-i686-3.4/time.cpython-34m.so:
>
>    wrong ELF class: ELFCLASS32); 'ensurepip' is a package and cannot
>
>    be directly executed make: *** [install] Error 1
>
>
>
> 2. setup.py is missing -lm flag for several modules.  On Linux this
>
>    problem is hidden because libm is already loaded by the executable
>
>    calling dlopen(), but Android's loader deals with unknown symbols
>
>    differently (searches only the libs explicitly linked against the
>
>    module being loaded.)  http://bugs.python.org/issue21668 reports
>
>    the problem for selectmodule (can't find ceil()) and timemodule
>
>    (fmod() and floor()).  But there are at least two more: audioop
>
>    fails to load because it needs floor() and ctypes_test fails to
>
>    load because it needs sqrt().  I'll happily update the patch in
>
>    21668.
>
>
>
>    Is there any fundamental objection to adding the -lm flag to the
>
>    link step where it is necessary?
>
>
>
> 3. What is ossaudiodev?  It tries to include "sys/soundcard.h", which
>
>    I don't have on my system.   (The rule in setup.py is
>
>    wrapped in a test for host of Linux/FreeBSD/Darwin, but Android x86
>
>    gets configured with --host=i686-linux-android so to turn it off
>
>    requires an extra test for "and not cross_compiling".)
>
>
>
>    Can I just turn off ossaudiodev for cross compiling or might
>
>    someone want it in a different type of cross build?  (In which case
>
>    I think I'll have to write some kind autoconf rule for it, which I
>
>    don't quite know how to do yet.)
>
>
>
> 4. Module _decimal is failing to compile.  The problem is that it has
>
>    a header called memory.h.  Android's libc has the problem that
>
>    /usr/include/stdlib.h includes <memory.h>.  But the build system
>
>    puts -I. on the include path before the system dirs (as it should)
>
>    so when compiling _decimal, Modules/_decimal/libmpdec/memory.h gets
>
>    found instead of /usr/include/memory.h.  Shiz has a patch here:
>
>
> https://github.com/rave-engine/python3-android/blob/master/mk/python/3.3.5/p\
>
> ython-3.3.5-android-libmpdec.patch
>
>    (which renames memory.h -> mpmemory.h) but I don't know
>
>
>
>    a.  Is there a tracker for this yet?  and
>
>    b.  Is Shiz's fix the desired one or should I be looking for
>
>        another approach?  (Maybe modifying the -I flags for the build
>
>        of just the build of _decimal or something?)
>
>
>
> 5. I'm not sure what test configure is actually doing for gethostby*()
>
>    in a cross-compile environment.  In any case Android has a bug
>
>    where gethostbyaddr_r() is declared in the headers, but not
>
>    actually implemented in libc.  So I have to modify my pyconfig.h by
>
>    hand to define HAVE_GETHOSTBYNAME and undef HAVE_GETHOSTBYNAME_R
>
>    and HAVE_GETHOSTBYNAME_R_6_ARG.
>
>
>
>    Is there a variable (like ac_cv_little_endian_double) that I can
>
>    give to `configure` to make it set HAVE_GETHOSTBYNAME* the way I
>
>    need?  If so I've been unable to figure it out.
>
>
>
> 6. Android's <pwd.h> header mysteriously leaves the pw_gecos field out
>
>    of struct passwd.  Is a fix like defining a new variable
>
>    HAVE_BROKEN_GECOS_FIELD the appropriate way to go with this?  (If
>
>    this is an okay solution then the patch to Modules/pwdmodule.c is
>
>    shown below, but I still have to figure out how to patch
>
>    configure.ac to test for the condition and set the variable
>
>    appropriately, so a pointer to a similar block of code in
>
>    configure.ac would be appreciated.)
>
>
>
> Sorry for the TL;DR.  I appreciate your having taken the time to read
>
> this far.
>
>
>
> Thanks,
>
> -Matt
>
>
>
> Proposed patch for pwdmodule.c:
>
>
>
> --- a/Modules/pwdmodule.c 2014-05-19 00:19:39.000000000 -0500
>
> +++ b/Modules/pwdmodule.c       2014-10-21 18:00:35.676331205 -0500
>
> @@ -57,6 +57,10 @@
>
>    }
>
> }
>
>
>
> +#if defined(HAVE_BROKEN_GECOS_FIELD)
>
> +static char fakePwGecos[256] = "";
>
> +#endif
>
> +
>
> static PyObject *
>
> mkpwent(struct passwd *p)
>
> {
>
> @@ -72,7 +76,11 @@
>
>      SETS(setIndex++, p->pw_passwd);
>
>      PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromUid(p->pw_uid));
>
>      PyStructSequence_SET_ITEM(v, setIndex++, _PyLong_FromGid(p->pw_gid));
>
> +#if !defined(HAVE_BROKEN_GECOS_FIELD)
>
>      SETS(setIndex++, p->pw_gecos);
>
> +#else
>
> +    SETS(setIndex++, fakePwGecos);
>
> +#endif
>
>      SETS(setIndex++, p->pw_dir);
>
>      SETS(setIndex++, p->pw_shell);
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141024/292a4aad/attachment-0001.html>

From db3l.net at gmail.com  Sat Oct 25 05:47:05 2014
From: db3l.net at gmail.com (David Bolen)
Date: Fri, 24 Oct 2014 23:47:05 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
Message-ID: <m2oat0x13a.fsf@valheru.db3l.homeip.net>

Starting yesterday, my XP buildbot began failing to execute clone
operations against hg.python.org.  There's not a lot of data being
given aside from a transaction abort message (and my buildbot log
showing the hg command exiting), and I'm wondering if something may be
amiss on the server or its configuration?

Note that this is a full clone (which for some reason the Windows
buildbots seem to fall back on with some frequency) and can take quite
a while.  My Windows 7 buildbot is ok so far but it's still doing
incremental pulls over the same time period.

I've got two separate Internet connections here and have tried routing
over both so I don't think it's a network issue.  I've completely
flushed the local build trees and rebooted the buildbot.

Is there anything that might be available on the server to see if
there are errors being logged?  Or anything that could have changed
configuration wise recently (maybe timeout related or something)?  I'm
running a bit low of items to try to change or reset on the buildbot
side.

Thanks.

-- David


From donald at stufft.io  Sat Oct 25 05:55:06 2014
From: donald at stufft.io (Donald Stufft)
Date: Fri, 24 Oct 2014 23:55:06 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
Message-ID: <FCACA585-4DD4-439C-A0DE-5A2F9F9C413C@stufft.io>

Is this using HTTPS or SSH.

> On Oct 24, 2014, at 11:47 PM, David Bolen <db3l.net at gmail.com> wrote:
> 
> Starting yesterday, my XP buildbot began failing to execute clone
> operations against hg.python.org.  There's not a lot of data being
> given aside from a transaction abort message (and my buildbot log
> showing the hg command exiting), and I'm wondering if something may be
> amiss on the server or its configuration?
> 
> Note that this is a full clone (which for some reason the Windows
> buildbots seem to fall back on with some frequency) and can take quite
> a while.  My Windows 7 buildbot is ok so far but it's still doing
> incremental pulls over the same time period.
> 
> I've got two separate Internet connections here and have tried routing
> over both so I don't think it's a network issue.  I've completely
> flushed the local build trees and rebooted the buildbot.
> 
> Is there anything that might be available on the server to see if
> there are errors being logged?  Or anything that could have changed
> configuration wise recently (maybe timeout related or something)?  I'm
> running a bit low of items to try to change or reset on the buildbot
> side.
> 
> Thanks.
> 
> -- David
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


From solipsis at pitrou.net  Sat Oct 25 05:57:16 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 25 Oct 2014 05:57:16 +0200
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
Message-ID: <20141025055716.7e252f85@fsol>

On Fri, 24 Oct 2014 23:47:05 -0400
David Bolen <db3l.net at gmail.com> wrote:
> Starting yesterday, my XP buildbot began failing to execute clone
> operations against hg.python.org.  There's not a lot of data being
> given aside from a transaction abort message (and my buildbot log
> showing the hg command exiting), and I'm wondering if something may be
> amiss on the server or its configuration?

Have you tried running the hg clone manually from the buildbot?
You could try to add --debug to get more info where the thing breaks.

Regards

Antoine.



From db3l.net at gmail.com  Sat Oct 25 06:01:51 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 00:01:51 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <FCACA585-4DD4-439C-A0DE-5A2F9F9C413C@stufft.io>
Message-ID: <m2k33ox0eo.fsf@valheru.db3l.homeip.net>

Donald Stufft <donald at stufft.io> writes:

> Is this using HTTPS or SSH.

Um, good question - whatever the buildbot build process uses.

Looking at the slave log on buildbot.python.org (I don't get the hg
output locally), appears to be http (it's cloning
http://hg.python.org/cpython) - though I thought I saw it using https
(port 443) in some traffic monitoring I was doing, so maybe it gets
redirected?

Oh yeah, the log also shows "real URL is
https://hg.python.org/cpython" as the first output from hg.

-- David


From db3l.net at gmail.com  Sat Oct 25 06:25:13 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 00:25:13 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol>
Message-ID: <m2d29gwzbq.fsf@valheru.db3l.homeip.net>

Antoine Pitrou <solipsis at pitrou.net> writes:

> Have you tried running the hg clone manually from the buildbot?
> You could try to add --debug to get more info where the thing breaks.

Yes, I had but pretty much got the same output as the buildbot slave.
But I just tried --traceback and it's definitely complaining about the
connection being terminated.

Regular test:

> hg clone --verbose --noupdate http://hg.python.org/cpython test
    real URL is https://hg.python.org/cpython
    requesting all changes
    adding changesets
    adding manifests
    transaction abort!
    rollback completed
    abort: connection ended unexpectedly

Traceback:

> hg clone --traceback --verbose --noupdate http://hg.python.org/cpython test
    real URL is https://hg.python.org/cpython
    requesting all changes
    adding changesets
    adding manifests
    transaction abort!
    rollback completed
    Traceback (most recent call last):
      File "mercurial\dispatch.pyc", line 54, in _runcatch
      File "mercurial\dispatch.pyc", line 490, in _dispatch
      File "mercurial\dispatch.pyc", line 351, in runcommand
      File "mercurial\dispatch.pyc", line 541, in _runcommand
      File "mercurial\dispatch.pyc", line 495, in checkargs
      File "mercurial\dispatch.pyc", line 488, in <lambda>
      File "mercurial\util.pyc", line 420, in check
      File "mercurial\commands.pyc", line 725, in clone
      File "mercurial\hg.pyc", line 334, in clone
      File "mercurial\localrepo.pyc", line 1853, in clone
      File "mercurial\localrepo.pyc", line 1206, in pull
      File "mercurial\localrepo.pyc", line 1695, in addchangegroup
      File "mercurial\revlog.pyc", line 1239, in addgroup
      File "mercurial\changegroup.pyc", line 31, in chunkiter
      File "mercurial\changegroup.pyc", line 20, in getchunk
      File "mercurial\util.pyc", line 924, in read
      File "mercurial\httprepo.pyc", line 22, in zgenerator
    IOError: [Errno None] connection ended unexpectedly
    abort: connection ended unexpectedly


I also stuck on --debug which generates a metric ton of output, but
the final portion is:

> hg clone --debug --traceback --verbose --noupdate http://hg.python.org/cpython test

    (...)
    manifests: 5271/93170 chunks (5.66%)
    manifests: 5272/93170 chunks (5.66%)
    manifests: 5273/93170 chunks (5.66%)
    manifests: 5274/93170 chunks (5.66%)
    manifests: 5275/93170 chunks (5.66%)
    manifests: 5276/93170 chunks (5.66%)
    manifests: 5277/93170 chunks (5.66%)
    manifests: 5278/93170 chunks (5.66%)
    transaction abort!
    rollback completed
    Traceback (most recent call last):
      File "mercurial\dispatch.pyc", line 54, in _runcatch
      File "mercurial\dispatch.pyc", line 490, in _dispatch
      File "mercurial\dispatch.pyc", line 351, in runcommand
      File "mercurial\dispatch.pyc", line 541, in _runcommand
      File "mercurial\dispatch.pyc", line 495, in checkargs
      File "mercurial\dispatch.pyc", line 488, in <lambda>
      File "mercurial\util.pyc", line 420, in check
      File "mercurial\commands.pyc", line 725, in clone
      File "mercurial\hg.pyc", line 334, in clone
      File "mercurial\localrepo.pyc", line 1853, in clone
      File "mercurial\localrepo.pyc", line 1206, in pull
      File "mercurial\localrepo.pyc", line 1695, in addchangegroup
      File "mercurial\revlog.pyc", line 1239, in addgroup
      File "mercurial\changegroup.pyc", line 31, in chunkiter
      File "mercurial\changegroup.pyc", line 20, in getchunk
      File "mercurial\util.pyc", line 924, in read
      File "mercurial\httprepo.pyc", line 22, in zgenerator
    IOError: [Errno None] connection ended unexpectedly
    abort: connection ended unexpectedly

which appears to die mid-stream while receiving the manifests.

So I'm sort of hoping there might be some record server-side as to why
things are falling apart mid-way.

-- David


From db3l.net at gmail.com  Sat Oct 25 07:00:50 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 01:00:50 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
Message-ID: <m28uk4wxod.fsf@valheru.db3l.homeip.net>

David Bolen <db3l.net at gmail.com> writes:

> which appears to die mid-stream while receiving the manifests.
>
> So I'm sort of hoping there might be some record server-side as to why
> things are falling apart mid-way.

Just to follow-up to myself, I get the same same error trying to do a
clone from my own personal XP machine rather than the buildbot (which
is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.

However, the same clones completely successfully under OSX and Linux.

So that's sort of strange.

-- David


From donald at stufft.io  Sat Oct 25 07:33:16 2014
From: donald at stufft.io (Donald Stufft)
Date: Sat, 25 Oct 2014 01:33:16 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <m28uk4wxod.fsf@valheru.db3l.homeip.net>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
Message-ID: <D2477F05-D25A-43BF-86AE-FDD699AF968F@stufft.io>

What version of OpenSSL is it using.

> On Oct 25, 2014, at 1:00 AM, David Bolen <db3l.net at gmail.com> wrote:
> 
> David Bolen <db3l.net at gmail.com> writes:
> 
>> which appears to die mid-stream while receiving the manifests.
>> 
>> So I'm sort of hoping there might be some record server-side as to why
>> things are falling apart mid-way.
> 
> Just to follow-up to myself, I get the same same error trying to do a
> clone from my own personal XP machine rather than the buildbot (which
> is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.
> 
> However, the same clones completely successfully under OSX and Linux.
> 
> So that's sort of strange.
> 
> -- David
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io


From db3l.net at gmail.com  Sat Oct 25 08:14:27 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 02:14:27 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <D2477F05-D25A-43BF-86AE-FDD699AF968F@stufft.io>
Message-ID: <m24muswu9o.fsf@valheru.db3l.homeip.net>

Donald Stufft <donald at stufft.io> writes:

> What version of OpenSSL is it using.

I'm using the pre-built Windows Mercurial installer, but if I unpack
the included library.zip, the SSLEAY32.DLL shows version 0.9.8r.

This is from the 3.1.2 install I just did a few hours ago.  It appears
that hg 2.5.2 on my other XP box also has 0.9.8r.  The prior buildbot
version (1.6.2) looks like it had 0.9.8o.

I also got around to trying a manual clone on the Windows 7 buildbot,
and it worked fine, even with the older hg 1.6.2.

So it seems to correlate with XP more than anything else at the moment.

-- David


From db3l.net at gmail.com  Sat Oct 25 08:17:23 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 02:17:23 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <2a6a228adf3e4f00af3a0c7d0cf8c92c@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol>
 <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <2a6a228adf3e4f00af3a0c7d0cf8c92c@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CAFirtYWiHdiDURzLoDM1Tan-7d5ZXhfdy4=UBge=WdvEU=H_hQ@mail.gmail.com>

Do you mean your local repo?  If so, I don't have a local repo at this
point - the failure is during the first clone.

-- David

On Sat, Oct 25, 2014 at 1:19 AM, Steve Dower <Steve.Dower at microsoft.com>
wrote:

>   I was seeing this recently and had to run recover on my repo (not sure
> what the command line is for that - TortoiseHg had a menu). YMMV, but the
> symptoms sound the same.
>
> Cheers,
> Steve
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141025/15fd3db1/attachment.html>

From Steve.Dower at microsoft.com  Sat Oct 25 07:19:18 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 25 Oct 2014 05:19:18 +0000
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <m28uk4wxod.fsf@valheru.db3l.homeip.net>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol>
 <m2d29gwzbq.fsf@valheru.db3l.homeip.net>,
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
Message-ID: <2a6a228adf3e4f00af3a0c7d0cf8c92c@DM2PR0301MB0734.namprd03.prod.outlook.com>

I was seeing this recently and had to run recover on my repo (not sure what the command line is for that - TortoiseHg had a menu). YMMV, but the symptoms sound the same.

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: David Bolen<mailto:db3l.net at gmail.com>
Sent: ?10/?24/?2014 22:01
To: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] XP buildbot problem cloning from hg.python.org

David Bolen <db3l.net at gmail.com> writes:

> which appears to die mid-stream while receiving the manifests.
>
> So I'm sort of hoping there might be some record server-side as to why
> things are falling apart mid-way.

Just to follow-up to myself, I get the same same error trying to do a
clone from my own personal XP machine rather than the buildbot (which
is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.

However, the same clones completely successfully under OSX and Linux.

So that's sort of strange.

-- David

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141025/c8903464/attachment.html>

From donald at stufft.io  Sat Oct 25 15:53:26 2014
From: donald at stufft.io (Donald Stufft)
Date: Sat, 25 Oct 2014 09:53:26 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <m24muswu9o.fsf@valheru.db3l.homeip.net>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <D2477F05-D25A-43BF-86AE-FDD699AF968F@stufft.io>
 <m24muswu9o.fsf@valheru.db3l.homeip.net>
Message-ID: <D5A1F0B9-2B29-46FC-A8B6-D83E7EA8351A@stufft.io>

I have an idea, can you run https://bpaste.net/show/c5d7cd102f5b and
tell me what it outputs? Both on a machine that works and one that
doesn?t.

> On Oct 25, 2014, at 2:14 AM, David Bolen <db3l.net at gmail.com> wrote:
> 
> Donald Stufft <donald at stufft.io> writes:
> 
>> What version of OpenSSL is it using.
> 
> I'm using the pre-built Windows Mercurial installer, but if I unpack
> the included library.zip, the SSLEAY32.DLL shows version 0.9.8r.
> 
> This is from the 3.1.2 install I just did a few hours ago.  It appears
> that hg 2.5.2 on my other XP box also has 0.9.8r.  The prior buildbot
> version (1.6.2) looks like it had 0.9.8o.
> 
> I also got around to trying a manual clone on the Windows 7 buildbot,
> and it worked fine, even with the older hg 1.6.2.
> 
> So it seems to correlate with XP more than anything else at the moment.
> 
> -- David
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From kelman at berkeley.edu  Sat Oct 25 14:45:24 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Sat, 25 Oct 2014 05:45:24 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
Message-ID: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>

I'm several weeks late to this discussion, but I'm glad to see that it
happened. I'm not a Python developer, and barely a user, but I have several
years of daily experience compiling complicated scientific software cross-
platform, particularly with MinGW-w64 for Windows. The Python community,
both core language and scientific package developers and users, needs to
act here. The problem is bad and getting worse. Luckily much of the work
to start solving it has already been done in bits and pieces, it needs
coordination and participation to come to a conclusion.

> Cross compilation is a valid issue, but I hope that build services like
> Appveyor also help out here. There is regular talk about the PSF/PyPI
> providing something similar

AppVeyor is better than nothing (I've been using it since beta), but it's
a far cry from build services and package management as the Linux world
knows them. Obtaining and setting up build dependencies quickly and
repeatably, and finishing the build of a complicated piece of software
such as CPython, or NumPy, SciPy, Julia (where most of my recent expertise
lies), etc. on a small single-core VM with limited memory and a restrictive
time limit is often not possible. These problems are solved within Linux
infrastructure like Koji, Open Build Service, buildd, etc.

MinGW-w64 is a mature, well-tested toolchain that is very capable of cross-
compiling a wide variety of libraries from Linux to Windows, in addition to
building conventionally on Windows for Windows. The MSYS2 collection of
MinGW-w64-compiled packages (https://github.com/Alexpux/MINGW-packages) has
been mentioned. Linux distributions including
- Fedora https://admin.fedoraproject.org/pkgdb/packages/mingw%2A/
- openSUSE https://build.opensuse.org/project/show/windows:mingw:win32
- Arch https://aur.archlinux.org/packages/?K=mingw
and others have projects for providing many hundreds of open-source
packages compiled for Windows. Debian has the cross-compilers available but
not many packages yet (https://packages.debian.org/search?keywords=mingw).

As a developer of a (compiled) open-source library or application, wouldn't
you love to be able to build binaries on Linux for Windows? With some work
and build system patches, you can. For many projects it's a simple matter of
./configure --host=x86_64-w64-mingw32. Not with CPython though. CPython is
only included in 2 of the above MinGW-w64 distribution projects, MSYS2 and
Arch. This is possible with a very, very long set of patches, many of which
have been submitted by Roumen Petrov to the Python bug tracker - see
http://bugs.python.org/issue17605 and other issues linked therein. Roumen
has done a huge amount of work, and he and others who consider the MinGW-w64
compiler important will continue to do so. (Thanks a million Roumen!)

> I could step in as maintainer for Cygwin and builds based on GCC using
> mingw* APIs.
>
> Regards,
> Roumen Petrov

A maintainer has volunteered. Others will help. Can any core developers
please begin reviewing some of his patches? Even if just to say they need
to be rebased. The lack of responses on the bug tracker is disheartening
from an outside perspective. The pile of patches accumulating in external
MinGW packaging projects is tantamount to a fork of CPython. It won't go
away since there are dedicated packagers working to keep their MinGW-w64
builds functional, even in the ad-hoc current state. The patches continue
piling up, making it more difficult for everyone - except for the core
Python developers if they continue to ignore the problem. Bring the people
working on these patches into the fold as contributors. Review the patches.
It would make Python as a language and a community even more diverse and
welcoming.

> Deprecate/remove support for compiling CPython itself with compilers
> other than MSVC on Windows

I'm not alone in thinking that this would be a bad idea. MSVC can continue
to be the default compiler used for Python on Windows, none of Roumen's
patches change that. They would merely open up the choice for packagers and
users to build CPython (and extension modules, thanks to separate patches)
with alternate compilers, in cross-compilation or otherwise.

Sincerely,
Tony


From Steve.Dower at microsoft.com  Sat Oct 25 19:13:20 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 25 Oct 2014 17:13:20 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
Message-ID: <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>

(Apologies for the short reply, posting from my phone.)

"MSVC can continue
to be the default compiler used for Python on Windows, none of Roumen's
patches change that. They would merely open up the choice for packagers and
users to build CPython (and extension modules, thanks to separate patches)
with alternate compilers, in cross-compilation or otherwise."

Building CPython for Windows is not something that needs solving. The culture on Windows is to redistribute binaries, not source, and both the core team and a number of redistributors have this figured out (and it will only become easier with VC14 and Python 3.5).

I'd rather see this effort thrown behind compiling extensions, including cross compilation. The ABI is well defined enough that any compiler should be usable, especially once the new CRT is in use. However, there is work needed to update the various tool chains to link to VC14's CRT and we need to figure out the inconsistencies between tools so we can document and work through them.

Having different builds of CPython out there will only fragment the community and hurt extension authors far more than it may seem to help.

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Tony Kelman<mailto:kelman at berkeley.edu>
Sent: ?10/?25/?2014 9:06
To: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Status of C compilers for Python on Windows

I'm several weeks late to this discussion, but I'm glad to see that it
happened. I'm not a Python developer, and barely a user, but I have several
years of daily experience compiling complicated scientific software cross-
platform, particularly with MinGW-w64 for Windows. The Python community,
both core language and scientific package developers and users, needs to
act here. The problem is bad and getting worse. Luckily much of the work
to start solving it has already been done in bits and pieces, it needs
coordination and participation to come to a conclusion.

> Cross compilation is a valid issue, but I hope that build services like
> Appveyor also help out here. There is regular talk about the PSF/PyPI
> providing something similar

AppVeyor is better than nothing (I've been using it since beta), but it's
a far cry from build services and package management as the Linux world
knows them. Obtaining and setting up build dependencies quickly and
repeatably, and finishing the build of a complicated piece of software
such as CPython, or NumPy, SciPy, Julia (where most of my recent expertise
lies), etc. on a small single-core VM with limited memory and a restrictive
time limit is often not possible. These problems are solved within Linux
infrastructure like Koji, Open Build Service, buildd, etc.

MinGW-w64 is a mature, well-tested toolchain that is very capable of cross-
compiling a wide variety of libraries from Linux to Windows, in addition to
building conventionally on Windows for Windows. The MSYS2 collection of
MinGW-w64-compiled packages (https://github.com/Alexpux/MINGW-packages) has
been mentioned. Linux distributions including
- Fedora https://admin.fedoraproject.org/pkgdb/packages/mingw%2A/
- openSUSE https://build.opensuse.org/project/show/windows:mingw:win32
- Arch https://aur.archlinux.org/packages/?K=mingw
and others have projects for providing many hundreds of open-source
packages compiled for Windows. Debian has the cross-compilers available but
not many packages yet (https://packages.debian.org/search?keywords=mingw).

As a developer of a (compiled) open-source library or application, wouldn't
you love to be able to build binaries on Linux for Windows? With some work
and build system patches, you can. For many projects it's a simple matter of
./configure --host=x86_64-w64-mingw32. Not with CPython though. CPython is
only included in 2 of the above MinGW-w64 distribution projects, MSYS2 and
Arch. This is possible with a very, very long set of patches, many of which
have been submitted by Roumen Petrov to the Python bug tracker - see
http://bugs.python.org/issue17605 and other issues linked therein. Roumen
has done a huge amount of work, and he and others who consider the MinGW-w64
compiler important will continue to do so. (Thanks a million Roumen!)

> I could step in as maintainer for Cygwin and builds based on GCC using
> mingw* APIs.
>
> Regards,
> Roumen Petrov

A maintainer has volunteered. Others will help. Can any core developers
please begin reviewing some of his patches? Even if just to say they need
to be rebased. The lack of responses on the bug tracker is disheartening
from an outside perspective. The pile of patches accumulating in external
MinGW packaging projects is tantamount to a fork of CPython. It won't go
away since there are dedicated packagers working to keep their MinGW-w64
builds functional, even in the ad-hoc current state. The patches continue
piling up, making it more difficult for everyone - except for the core
Python developers if they continue to ignore the problem. Bring the people
working on these patches into the fold as contributors. Review the patches.
It would make Python as a language and a community even more diverse and
welcoming.

> Deprecate/remove support for compiling CPython itself with compilers
> other than MSVC on Windows

I'm not alone in thinking that this would be a bad idea. MSVC can continue
to be the default compiler used for Python on Windows, none of Roumen's
patches change that. They would merely open up the choice for packagers and
users to build CPython (and extension modules, thanks to separate patches)
with alternate compilers, in cross-compilation or otherwise.

Sincerely,
Tony

_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141025/fd0f9744/attachment.html>

From rdmurray at bitdance.com  Sat Oct 25 19:27:39 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 25 Oct 2014 13:27:39 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
Message-ID: <20141025172740.41865250D4D@webabinitio.net>

On Sat, 25 Oct 2014 05:45:24 -0700, "Tony Kelman" <kelman at berkeley.edu> wrote:
> As a developer of a (compiled) open-source library or application, wouldn't
> you love to be able to build binaries on Linux for Windows? With some work
> and build system patches, you can. For many projects it's a simple matter of
> ./configure --host=x86_64-w64-mingw32. Not with CPython though. CPython is
> only included in 2 of the above MinGW-w64 distribution projects, MSYS2 and
> Arch. This is possible with a very, very long set of patches, many of which
> have been submitted by Roumen Petrov to the Python bug tracker - see
> http://bugs.python.org/issue17605 and other issues linked therein. Roumen
> has done a huge amount of work, and he and others who consider the MinGW-w64
> compiler important will continue to do so. (Thanks a million Roumen!)

Yes, I for one appreciate Roumen's work, even though I'm not currently
in a position to review the patches.

> > I could step in as maintainer for Cygwin and builds based on GCC using
> > mingw* APIs.
> >
> > Regards,
> > Roumen Petrov
> 
> A maintainer has volunteered. Others will help. Can any core developers
> please begin reviewing some of his patches? Even if just to say they need
> to be rebased. The lack of responses on the bug tracker is disheartening
> from an outside perspective. The pile of patches accumulating in external
> MinGW packaging projects is tantamount to a fork of CPython. It won't go
> away since there are dedicated packagers working to keep their MinGW-w64
> builds functional, even in the ad-hoc current state. The patches continue
> piling up, making it more difficult for everyone - except for the core
> Python developers if they continue to ignore the problem. Bring the people
> working on these patches into the fold as contributors. Review the patches.
> It would make Python as a language and a community even more diverse and
> welcoming.

IIUC, our general policy for bringing new platforms into core support,
as opposed to continuing to be a separately maintained "set of patches",
is that there be a "big enough" community of interest (I don't remember
the definition of "big enough") and that there be both committed
maintainers *and* at least one buildbot.

Being able to build windows packages on linux is a compelling argument,
but that only applies to building extensions, not the interpreter.

I would recommend starting with any patches that help MinGW that are not
MinGW specific but instead generally improve the build system and cross
compilation.  There have been a number of such issues opened and
improvements made beyond the MinGW related ones, some coming from
Debian, some coming from the Android community.

So  target those first.  My suggestion would be to pick a patch that is
believed to be commit ready, and come to #python-dev on freenode and
gently bug us until it gets committed.  Then pick the next one, and
repeat.  Working from simplest to more complex would also probably be a
good strategy :)

>From there I'd move on to patches that support using MinGW for building
extensions.  There will probably be useful to also get engaged with the
packaging folks.  And, at this point, we would NEED A BUILDBOT.  That
is, a machine that has whatever tools are required installed such that
tests added to the test suite to test MinGW support in distutils would
run, so we can be sure we don't break anything when making other
changes.

For compiling CPython itself with MinGW, we'd first need to develop a
consensus that we want to support it in core.  I'd say get building
extensions working first, and then make that argument.

By the time a bunch of patches get committed, we should be ready (read:
eager :) to promote someone to do it themselves.  Even if we never
decide to support compiling CPython itself with MinGW, I would hope that
getting it to work for extensions would greatly reduce the number of
patches needed to be maintained outside the tree in order to do so.  And
once at least one MinGW advocate is part of the core team, advocacy
becomes easier :)

--David

PS: one meta comment about the MinGW issues: I scanned a few from the
linked bug, and while the issues split the patches out, which is a great
start, there is no discussion on the issues for the individual patches
to give background and motivation and an explanation of what the patch
is trying to accomplish.  But you probably don't want to waste time on
improving ones that apply *only* to compiling CPython itself unless we
get consensus that we want to support that.

From db3l.net at gmail.com  Sat Oct 25 21:43:02 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 15:43:02 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <D2477F05-D25A-43BF-86AE-FDD699AF968F@stufft.io>
 <m24muswu9o.fsf@valheru.db3l.homeip.net>
 <D5A1F0B9-2B29-46FC-A8B6-D83E7EA8351A@stufft.io>
Message-ID: <m2wq7oue9l.fsf@valheru.db3l.homeip.net>

Donald Stufft <donald at stufft.io> writes:

> I have an idea, can you run https://bpaste.net/show/c5d7cd102f5b and
> tell me what it outputs? Both on a machine that works and one that
> doesn?t.

All but Linux (so XP/7 buildbots, XP standalone, OSX) return:
   ('DHE-RSA-AES128-SHA', 'TLSv1/SSLv3', 128)
My Linux (Ubuntu 12.04) returns:
   ('ECDHE-RSA-AES128-SHA', 'TLSv1/SSLv3', 128)

The script was run under a default Python on each box (2.6 on Windows,
2.7 on OSX and Linux).  I tried 2.6 through 3.1 on my standalone XP with
no change, so I don't think it differs by Python version.  Its not
precisely the same as running hg, since it has its own embedded Python
under Windows, but I installed a source install on my XP box under 2.7
and it fails a clone the same way.

In new news though, I just the same failure on the Win7 buildbot in a
clone test.  In repeated attempts, that's the only one so far.

I also realized that one shared feature is that the XP boxes were using
IPv4 while the other boxes were all IPv6 (an HE tunnel on my side).
Though my earlier Win7 failure was also IPv6.  I manually forced the
Win7 box to use IPv4, but didn't see much difference.  It certainly didn't
start failing like the XP boxes.

Anecdotally, the failing XP attempts appear to be running slower in
general (with lower transfer rates as monitored by my router).  I have
had slow clones work on other boxes, so that's not automatically bad.
But I wonder if it's still some sort of timeout somewhere.

I don't think I currently have an active ssh account, but if there were
a way to test a clone over ssh rather than http perhaps that would be a
useful data point, in terms of eliminating some middlemen processing.

-- David



From db3l.net at gmail.com  Sat Oct 25 22:44:42 2014
From: db3l.net at gmail.com (David Bolen)
Date: Sat, 25 Oct 2014 16:44:42 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <D2477F05-D25A-43BF-86AE-FDD699AF968F@stufft.io>
 <m24muswu9o.fsf@valheru.db3l.homeip.net>
 <D5A1F0B9-2B29-46FC-A8B6-D83E7EA8351A@stufft.io>
 <m2wq7oue9l.fsf@valheru.db3l.homeip.net>
Message-ID: <m2siibvpz9.fsf@valheru.db3l.homeip.net>

As another data point, I've tried cloning randomly selected other
repositories from hg.python.org, and smaller repositories (distutils2,
peps, jython to name a few) are all working fine under XP, even though
with jython for example, the clone takes longer in terms of wall time
than I'll often see cpython fail.(*)

A test of what I presumed was a more comparably sized repository
(features/cdecimal) dies like cpython.

-- David

(*) Overall clone time is probably unrelated anyway since the XP
buildbot traditionally needed 10+min for clones in the past (such as
when the new build script changes were in place and every test used a
clone) and was working fine with that.


From Steve.Dower at microsoft.com  Sat Oct 25 22:50:56 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 25 Oct 2014 20:50:56 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>,
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
Message-ID: <1414270256914.92610@microsoft.com>

Ray Donnelly wrote:
> What is it that you
> are afraid of if CPython can be compiled out of the box using
> mingw/MinGW-w64? Why are you fighting so hard against having option.

I'm afraid of users having numpy crash because they're using an MSVC CPython instead of a mingw CPython. I'm afraid of users not being able to use library A and library B at the same time because A requires MSVC CPython and B requires mingw CPython. (I can produce more examples if you like, but the general concern is having a fragmented community, as I said in my previous post.)

I'm fighting against "having options" because it will suck up the precious volunteer time we have and direct it away from where it would be more useful, which is making it easier to build extensions with other compilers.

I would love to see extensions for Windows built on all platforms. I see no value in building Python itself for Windows from different platforms.

If other core developers agree with you that a more "pure" build of Python is worthwhile, then they can go ahead and merge the patches (though I suspect most would like to see some traction achieved on a fork first). I think it's important that I (as Windows build manager) make my own position clear, that's all.

(The rest of your email is purely unsubstantiated opinion, which is okay to have, but it doesn't demand any reply so I've omitted it.)

Cheers,
Steve

From mingw.android at gmail.com  Sat Oct 25 22:10:23 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sat, 25 Oct 2014 21:10:23 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>

On Sat, Oct 25, 2014 at 6:13 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> (Apologies for the short reply, posting from my phone.)
>
> "MSVC can continue
> to be the default compiler used for Python on Windows, none of Roumen's
> patches change that. They would merely open up the choice for packagers and
> users to build CPython (and extension modules, thanks to separate patches)
> with alternate compilers, in cross-compilation or otherwise."
>
> Building CPython for Windows is not something that needs solving. The
> culture on Windows is to redistribute binaries, not source, and both the
> core team and a number of redistributors have this figured out (and it will
> only become easier with VC14 and Python 3.5).

This is the second time you've used the vacuous "culture on Windows"
argument, now with an added appeal to (vague) authority. That may be
your opinion and that of some others, but there's a large number of
people who don't care for using non-Free tools. IMHO building CPython
on Windows using Open Source toolchains is very much something that
needs merging upstream and supporting by default. What is it that you
are afraid of if CPython can be compiled out of the box using
mingw/MinGW-w64? Why are you fighting so hard against having option.
If CPython wants to truly call itself an Open Source project then I
consider being able to compile and cross-compile it with capable Open
Source toolchains on all major platforms a requirement.

>
> I'd rather see this effort thrown behind compiling extensions, including
> cross compilation. The ABI is well defined enough that any compiler should
> be usable, especially once the new CRT is in use. However, there is work
> needed to update the various tool chains to link to VC14's CRT and we need
> to figure out the inconsistencies between tools so we can document and work
> through them.
>
> Having different builds of CPython out there will only fragment the
> community and hurt extension authors far more than it may seem to help.
>
> Cheers,
> Steve
>
> Top-posted from my Windows Phone
> ________________________________
> From: Tony Kelman
> Sent: ?10/?25/?2014 9:06
> To: python-dev at python.org
> Subject: Re: [Python-Dev] Status of C compilers for Python on Windows
>
> I'm several weeks late to this discussion, but I'm glad to see that it
> happened. I'm not a Python developer, and barely a user, but I have several
> years of daily experience compiling complicated scientific software cross-
> platform, particularly with MinGW-w64 for Windows. The Python community,
> both core language and scientific package developers and users, needs to
> act here. The problem is bad and getting worse. Luckily much of the work
> to start solving it has already been done in bits and pieces, it needs
> coordination and participation to come to a conclusion.
>
>> Cross compilation is a valid issue, but I hope that build services like
>> Appveyor also help out here. There is regular talk about the PSF/PyPI
>> providing something similar
>
> AppVeyor is better than nothing (I've been using it since beta), but it's
> a far cry from build services and package management as the Linux world
> knows them. Obtaining and setting up build dependencies quickly and
> repeatably, and finishing the build of a complicated piece of software
> such as CPython, or NumPy, SciPy, Julia (where most of my recent expertise
> lies), etc. on a small single-core VM with limited memory and a restrictive
> time limit is often not possible. These problems are solved within Linux
> infrastructure like Koji, Open Build Service, buildd, etc.
>
> MinGW-w64 is a mature, well-tested toolchain that is very capable of cross-
> compiling a wide variety of libraries from Linux to Windows, in addition to
> building conventionally on Windows for Windows. The MSYS2 collection of
> MinGW-w64-compiled packages (https://github.com/Alexpux/MINGW-packages) has
> been mentioned. Linux distributions including
> - Fedora https://admin.fedoraproject.org/pkgdb/packages/mingw%2A/
> - openSUSE https://build.opensuse.org/project/show/windows:mingw:win32
> - Arch https://aur.archlinux.org/packages/?K=mingw
> and others have projects for providing many hundreds of open-source
> packages compiled for Windows. Debian has the cross-compilers available but
> not many packages yet (https://packages.debian.org/search?keywords=mingw).
>
> As a developer of a (compiled) open-source library or application, wouldn't
> you love to be able to build binaries on Linux for Windows? With some work
> and build system patches, you can. For many projects it's a simple matter of
> ./configure --host=x86_64-w64-mingw32. Not with CPython though. CPython is
> only included in 2 of the above MinGW-w64 distribution projects, MSYS2 and
> Arch. This is possible with a very, very long set of patches, many of which
> have been submitted by Roumen Petrov to the Python bug tracker - see
> http://bugs.python.org/issue17605 and other issues linked therein. Roumen
> has done a huge amount of work, and he and others who consider the MinGW-w64
> compiler important will continue to do so. (Thanks a million Roumen!)
>
>> I could step in as maintainer for Cygwin and builds based on GCC using
>> mingw* APIs.
>>
>> Regards,
>> Roumen Petrov
>
> A maintainer has volunteered. Others will help. Can any core developers
> please begin reviewing some of his patches? Even if just to say they need
> to be rebased. The lack of responses on the bug tracker is disheartening
> from an outside perspective. The pile of patches accumulating in external
> MinGW packaging projects is tantamount to a fork of CPython. It won't go
> away since there are dedicated packagers working to keep their MinGW-w64
> builds functional, even in the ad-hoc current state. The patches continue
> piling up, making it more difficult for everyone - except for the core
> Python developers if they continue to ignore the problem. Bring the people
> working on these patches into the fold as contributors. Review the patches.
> It would make Python as a language and a community even more diverse and
> welcoming.
>
>> Deprecate/remove support for compiling CPython itself with compilers
>> other than MSVC on Windows
>
> I'm not alone in thinking that this would be a bad idea. MSVC can continue
> to be the default compiler used for Python on Windows, none of Roumen's
> patches change that. They would merely open up the choice for packagers and
> users to build CPython (and extension modules, thanks to separate patches)
> with alternate compilers, in cross-compilation or otherwise.
>
> Sincerely,
> Tony
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com
>

From rosuav at gmail.com  Sat Oct 25 23:11:39 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Oct 2014 08:11:39 +1100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1414270256914.92610@microsoft.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
Message-ID: <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>

On Sun, Oct 26, 2014 at 7:50 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Ray Donnelly wrote:
>> What is it that you
>> are afraid of if CPython can be compiled out of the box using
>> mingw/MinGW-w64? Why are you fighting so hard against having option.
>
> I'm afraid of users having numpy crash because they're using an MSVC CPython instead of a mingw CPython. I'm afraid of users not being able to use library A and library B at the same time because A requires MSVC CPython and B requires mingw CPython. (I can produce more examples if you like, but the general concern is having a fragmented community, as I said in my previous post.)
>

It might fragment the community to have multiple different binary
distributions. But it ought to be possible for any person/organization
to say "We're going to make our own build of Python, with these
extension modules, built with this compiler, targeting this platform",
and do everything from source. That might mean they can no longer take
the short-cut of "download someone's MSVC-built extension and use it
as-is", but they should be able to take anyone's extension and build
it on their chosen compiler. Having MinGW as a formally supported
platform would make life a lot easier for people who want to test
CPython patches, for instance - my building and testing of PEP
463-enhanced Python was Linux-only, because I didn't want to try to
set up an entire new buildchain just to try to get a Windows binary
going. There's absolutely no need for that to be binary-compatible
with anything else; as long as it'll run the standard library, it'll
do.

ChrisA

From solipsis at pitrou.net  Sat Oct 25 23:47:42 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 25 Oct 2014 23:47:42 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
Message-ID: <20141025234742.0119e061@fsol>

On Sun, 26 Oct 2014 08:11:39 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> 
> It might fragment the community to have multiple different binary
> distributions. But it ought to be possible for any person/organization
> to say "We're going to make our own build of Python, with these
> extension modules, built with this compiler, targeting this platform",
> and do everything from source. That might mean they can no longer take
> the short-cut of "download someone's MSVC-built extension and use it
> as-is", but they should be able to take anyone's extension and build
> it on their chosen compiler. Having MinGW as a formally supported
> platform would make life a lot easier for people who want to test
> CPython patches, for instance - my building and testing of PEP
> 463-enhanced Python was Linux-only,

And how do you know that it would have worked with MSVC if you only use
MinGW?
If you want to ensure compatibility with MSVC, you must build with MSVC.
There's no working around that.

Regards

Antoine.



From rosuav at gmail.com  Sat Oct 25 23:53:29 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Oct 2014 08:53:29 +1100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025234742.0119e061@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
Message-ID: <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>

On Sun, Oct 26, 2014 at 8:47 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> And how do you know that it would have worked with MSVC if you only use
> MinGW?
> If you want to ensure compatibility with MSVC, you must build with MSVC.
> There's no working around that.

Precisely. If you build with MinGW, you can't ensure compatibility
with MSVC. Reread my post: I gave two examples of situations where
that isn't a problem.

ChrisA

From solipsis at pitrou.net  Sat Oct 25 23:52:14 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 25 Oct 2014 23:52:14 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
Message-ID: <20141025235214.0b807816@fsol>

On Sat, 25 Oct 2014 21:10:23 +0100
Ray Donnelly <mingw.android at gmail.com> wrote:
> 
> This is the second time you've used the vacuous "culture on Windows"
> argument, now with an added appeal to (vague) authority.
[...]
> Why are you fighting so hard against having option.
> If CPython wants to truly call itself an Open Source project then I
> consider being able to compile and cross-compile it with capable Open
> Source toolchains on all major platforms a requirement.

Now *that* sounds vacuous.

Regarding open source, there's a clear and official definition of it,
which Python satisfies:
http://opensource.org/osd-annotated

Regards

Antoine.



From solipsis at pitrou.net  Sat Oct 25 23:59:45 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 25 Oct 2014 23:59:45 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
Message-ID: <20141025235945.2a3a7ddd@fsol>

On Sun, 26 Oct 2014 08:53:29 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 8:47 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > And how do you know that it would have worked with MSVC if you only use
> > MinGW?
> > If you want to ensure compatibility with MSVC, you must build with MSVC.
> > There's no working around that.
> 
> Precisely. If you build with MinGW, you can't ensure compatibility
> with MSVC. Reread my post: I gave two examples of situations where
> that isn't a problem.

How do you know this isn't a problem, since you haven't *tested* with
MSVC?
Why on Earth would you want to test your PEP work with an unsupported
Windows compiler and runtime, rather than with the officially supported
compiler and runtime?

Regards

Antoine.



From mingw.android at gmail.com  Sun Oct 26 00:01:15 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sat, 25 Oct 2014 23:01:15 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025235214.0b807816@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <20141025235214.0b807816@fsol>
Message-ID: <CAOYw7dvLivQ67bzzbeGz=j4pCarT8RJCCqgxrx7Wx_7OMrLGuA@mail.gmail.com>

On Sat, Oct 25, 2014 at 10:52 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 25 Oct 2014 21:10:23 +0100
> Ray Donnelly <mingw.android at gmail.com> wrote:
>>
>> This is the second time you've used the vacuous "culture on Windows"
>> argument, now with an added appeal to (vague) authority.
> [...]
>> Why are you fighting so hard against having option.
>> If CPython wants to truly call itself an Open Source project then I
>> consider being able to compile and cross-compile it with capable Open
>> Source toolchains on all major platforms a requirement.
>
> Now *that* sounds vacuous.
>

Maybe you missed where I said "I consider".

> Regarding open source, there's a clear and official definition of it,
> which Python satisfies:
> http://opensource.org/osd-annotated
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From rosuav at gmail.com  Sun Oct 26 00:06:36 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Oct 2014 09:06:36 +1100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025235945.2a3a7ddd@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
Message-ID: <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>

On Sun, Oct 26, 2014 at 8:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> How do you know this isn't a problem, since you haven't *tested* with
> MSVC?
> Why on Earth would you want to test your PEP work with an unsupported
> Windows compiler and runtime, rather than with the officially supported
> compiler and runtime?

This discussion revolved around supporting MinGW in addition to MSVC.
If it had been supported when I was doing that, I could have spun
myself up a Windows build and tested it. Since it was (and so far
still is) not, the hassle of hunting down a valid MSVC that could
build for Win XP (as that's what my test box runs) was simply not
worthwhile. My point is that there is no community fragmentation
happening here; the only fragmentation is of binary distribution of
extension modules, and there are several ways in which this needn't be
a problem.

ChrisA

From nad at acm.org  Sun Oct 26 00:14:03 2014
From: nad at acm.org (Ned Deily)
Date: Sat, 25 Oct 2014 15:14:03 -0700
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
Message-ID: <nad-95699A.15140225102014@news.gmane.org>

In article <m28uk4wxod.fsf at valheru.db3l.homeip.net>,
 David Bolen <db3l.net at gmail.com> wrote:

> David Bolen <db3l.net at gmail.com> writes:
> 
> > which appears to die mid-stream while receiving the manifests.
> >
> > So I'm sort of hoping there might be some record server-side as to why
> > things are falling apart mid-way.
> 
> Just to follow-up to myself, I get the same same error trying to do a
> clone from my own personal XP machine rather than the buildbot (which
> is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.
> 
> However, the same clones completely successfully under OSX and Linux.
> 
> So that's sort of strange.

Very interesting!  I had been doing some housekeeping on some of my 
older OS X build systems over the past few days and I've run into the 
same problem.  In particular, I am seeing this failure on an OS X 10.5.8 
system (running in a Fusion VM) which I've used for years and from which 
I have regularly cloned repos from hg.python.org.  I spent some time 
yesterday trying to isolate it.  I came to the conclusion that it was 
independent of the version of OpenSSL (identical failures occurred with 
the system's ancient Apple 0.9.7 as well as a newly-build 1.0.1j) and 
independent of the version of hg (at least with two data points, current 
and a year-old version) and seemingly independent of the network 
connection.  I was not able to reproduce the failure on the host OS X 
system (10.10) and I didn't have problems a few days earlier with 
various other OS X releases (10.6.x through 10.9.x) also running in VMs 
on the same host.  I stumbled across a workaround for the problem as I 
was experiencing it:  adding --uncompressed to hg clone eliminated 
failures.  You can get more info on the hg failures by adding 
--traceback and --debugger to the clone command.  After spending way too 
much time on the issue, I was not in the mood to spend more time 
isolating the problem after finding a workaround but if others are also 
seeing it, it might be worth doing.  Sigh.

      $ hg --version
      Mercurial Distributed SCM (version 3.1.2)
      $ hg clone -U http://hg.python.org/cpython cpython
      real URL is https://hg.python.org/cpython
      requesting all changes
      adding changesets
      adding manifests
      transaction abort!
      rollback completed
      abort: connection ended unexpectedly
      $ hg clone --uncompressed -U https://hg.python.org/cpython cpython
      streaming all changes
      10404 files to transfer, 248 MB of data
      transferred 248 MB in 44.4 seconds (5.58 MB/sec)

-- 
 Ned Deily,
 nad at acm.org


From solipsis at pitrou.net  Sun Oct 26 00:19:44 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 26 Oct 2014 00:19:44 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
Message-ID: <20141026001944.0373cb92@fsol>

On Sun, 26 Oct 2014 09:06:36 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 8:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > How do you know this isn't a problem, since you haven't *tested* with
> > MSVC?
> > Why on Earth would you want to test your PEP work with an unsupported
> > Windows compiler and runtime, rather than with the officially supported
> > compiler and runtime?
> 
> This discussion revolved around supporting MinGW in addition to MSVC.
> If it had been supported when I was doing that, I could have spun
> myself up a Windows build and tested it.

My point is that your "Windows build" would not have the same behaviour
as a MSVC-produced Windows build, and so testing it with it would not
certify that your code would actually be compatible with genuine
MSVC builds of CPython, which we will not stop supporting.

Therefore, what you and the OP are proposing would not make it
*easier* to ensure cross-platform compatibility but rather *harder*, by
adding another incompatible build configuration to the mix of supported
configurations.

The only remaining question is whether it is worthwhile adding support
for such an additional platform, and given that MinGW is extremely
marginal amongst Windows developers, the answer is IMHO no.

Regards

Antoine.



From rosuav at gmail.com  Sun Oct 26 00:22:18 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 26 Oct 2014 09:22:18 +1100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141026001944.0373cb92@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
Message-ID: <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>

On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> My point is that your "Windows build" would not have the same behaviour
> as a MSVC-produced Windows build, and so testing it with it would not
> certify that your code would actually be compatible with genuine
> MSVC builds of CPython, which we will not stop supporting.
>

So you're saying it's impossible to support two compilers?

ChrisA

From rdmurray at bitdance.com  Sun Oct 26 00:23:06 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 25 Oct 2014 18:23:06 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
Message-ID: <20141025222306.BFD19250D5B@webabinitio.net>

On Sat, 25 Oct 2014 21:10:23 +0100, Ray Donnelly <mingw.android at gmail.com> wrote:
> On Sat, Oct 25, 2014 at 6:13 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> > (Apologies for the short reply, posting from my phone.)
> >
> > "MSVC can continue
> > to be the default compiler used for Python on Windows, none of Roumen's
> > patches change that. They would merely open up the choice for packagers and
> > users to build CPython (and extension modules, thanks to separate patches)
> > with alternate compilers, in cross-compilation or otherwise."
> >
> > Building CPython for Windows is not something that needs solving. The
> > culture on Windows is to redistribute binaries, not source, and both the
> > core team and a number of redistributors have this figured out (and it will
> > only become easier with VC14 and Python 3.5).
> 
> This is the second time you've used the vacuous "culture on Windows"
> argument, now with an added appeal to (vague) authority. That may be
> your opinion and that of some others, but there's a large number of
> people who don't care for using non-Free tools. IMHO building CPython
> on Windows using Open Source toolchains is very much something that
> needs merging upstream and supporting by default. What is it that you
> are afraid of if CPython can be compiled out of the box using
> mingw/MinGW-w64? Why are you fighting so hard against having option.
> If CPython wants to truly call itself an Open Source project then I
> consider being able to compile and cross-compile it with capable Open
> Source toolchains on all major platforms a requirement.

You are doing yourself a disservice by this last statement.  There
really can't be any question that Python is an open source project,
so insinuating that the CPython community is "doing something wrong"
is not going to win you friends and helpers.

A better approach would be to acknowledge that what we are currently
doing works well for supporting Windows (especially since we actually
have some engagement from Microsoft that is *getting some problems
fixed* in ways that help make things more open).

And then say, "wouldn't it be *really cool* if we could also build
CPython using an open source toolchain on Windows out of the box?".  You
might not get instant agreement on that (well, clearly you won't), but
it'd be much more likely you'd start garnering support.

Assume that people are well intentioned, and convince them your
suggestions will make things *even better* using positive arguments.
You might not succeed, but you'll have a much better chance.

--David

From tjreedy at udel.edu  Sun Oct 26 00:32:26 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 25 Oct 2014 18:32:26 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
Message-ID: <m2h8du$9h3$1@ger.gmane.org>

On 10/25/2014 5:11 PM, Chris Angelico wrote:

> It might fragment the community to have multiple different binary
> distributions. But it ought to be possible for any person/organization
> to say "We're going to make our own build of Python, with these
> extension modules, built with this compiler, targeting this platform",
> and do everything from source. That might mean they can no longer take
> the short-cut of "download someone's MSVC-built extension and use it
> as-is", but they should be able to take anyone's extension and build
> it on their chosen compiler. Having MinGW as a formally supported
> platform would make life a lot easier for people who want to test
> CPython patches, for instance - my building and testing of PEP
> 463-enhanced Python was Linux-only, because I didn't want to try to
> set up an entire new buildchain just to try to get a Windows binary
> going. There's absolutely no need for that to be binary-compatible
> with anything else; as long as it'll run the standard library, it'll
> do.

David Murray's unanswered post laid out the path to move in the 
direction you want.  Either take it yourself or try to persuade other 
MinGW fans to do so.

-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Sun Oct 26 00:36:59 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 25 Oct 2014 23:36:59 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1414270256914.92610@microsoft.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
Message-ID: <CACac1F-BM7StO7VvO-xqdf8gjVg75eF6t5_XHyj8Kd8-PyPFmw@mail.gmail.com>

On 25 October 2014 21:50, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Ray Donnelly wrote:
>> What is it that you
>> are afraid of if CPython can be compiled out of the box using
>> mingw/MinGW-w64? Why are you fighting so hard against having option.
>
> I'm afraid of users having numpy crash because they're using an MSVC
> CPython instead of a mingw CPython. I'm afraid of users not being able
> to use library A and library B at the same time because A requires MSVC
> CPython and B requires mingw CPython. (I can produce more examples
> if you like, but the general concern is having a fragmented community,
> as I said in my previous post.)

Precisely. Either developers test with *all* supported compilers, or
there is a risk of incompatibilities. Advocates of supporting mingw as
a build system for Python generally do *not* suggest that they are
willing to test for, and deal with, cross-version compatibility
issues. Typically mingw is seen as "another platform" in some sense,
by such advocates, having its own set of supporters and maintainers.

The possibility of extensions built with a mingw-compiled Python
failing when used under a MSVC-built Python is real. It's difficult to
say how big that risk is, but it's certainly there. And I see no-one
offering to be responsible for such compatibility issues (the mingw
supporters generally don't want to set up a MSVC build chain, so it's
difficult to see how they could credibly offer).

> I'm fighting against "having options" because it will suck up the precious
> volunteer time we have and direct it away from where it would be more
> useful, which is making it easier to build extensions with other compilers.

And claiming that it doesn't suck up developer time ignores the point
I made above - *someone* has to deal with any compatibility issues
that come up. As a starter, does the wheel format need to include tags
to distinguish whether the target Python is MSVC-built and
mingw-built? Who will check that? If there is a need, who will work on
the code needed to incorporate that change into wheel, pip, and the
relevant PEPs?

As Steve says, the Python community has a genuine, strong need for
people with mingw expertise working on making it easier to build
*extensions* using mingw, that work with a MSVC-built CPython. If it
were possible to cross-compile compatible extensions on Linux,
projects developed on Linux could supply Windows binaries much more
easily, which would be a huge benefit to the whole Windows Python
community. But the mingw experts don't want to work on that,
preferring to develop patches allowing CPython to be built with mingw.
No objection from me, it's your free time, use it as you wish, but as
a Windows user of Python I can confirm that it's not what I'd like you
to be doing as your contribution to Python.

> I would love to see extensions for Windows built on all platforms. I see no
> value in building Python itself for Windows from different platforms.

Exactly.

> If other core developers agree with you that a more "pure" build of Python is
> worthwhile, then they can go ahead and merge the patches (though I suspect
> most would like to see some traction achieved on a fork first). I think it's
> important that I (as Windows build manager) make my own position clear,
> that's all.

Personally, I'm not a core developer, just a long-time member of this
list and occasional contributor to discussions. But I am also a core
pip developer and a Windows user, and from that perspective I am
acutely aware of the common pain points for Python users on Windows.
And they are all about binary extensions, and none at all about
building Python itself. So in my view, being able to build CPython
using mingw is somewhat interesting from a theoretical perspective,
but of little or no practical value[1] and disruptive in a number of
ways, as mentioned above, to improving the overall experience of
Python users on Windows.

Paul

[1] I note, without trying to make a judgement, that many of the
benefits cited for building with mingw are around "being able to use
free tools" or similar essentially ideological issues.

From nad at acm.org  Sun Oct 26 00:42:00 2014
From: nad at acm.org (Ned Deily)
Date: Sat, 25 Oct 2014 15:42:00 -0700
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <nad-95699A.15140225102014@news.gmane.org>
Message-ID: <nad-987DF6.15420025102014@news.gmane.org>

In article <nad-95699A.15140225102014 at news.gmane.org>,
 Ned Deily <nad at acm.org> wrote:

> In article <m28uk4wxod.fsf at valheru.db3l.homeip.net>,
>  David Bolen <db3l.net at gmail.com> wrote:
> > So that's sort of strange.
> Very interesting!  I had been doing some housekeeping on some of my 
> older OS X build systems over the past few days and I've run into the 
> same problem.  In particular, I am seeing this failure on an OS X 10.5.8 
> system (running in a Fusion VM) which I've used for years and from which 
> I have regularly cloned repos from hg.python.org. [...]

Update: after consulting with Donald on IRC, it appears that the problem 
was on the python.org end and is now fixed.  David, is it now working 
again for you?

-- 
 Ned Deily,
 nad at acm.org


From solipsis at pitrou.net  Sun Oct 26 00:42:45 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 26 Oct 2014 00:42:45 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
Message-ID: <20141026004245.3118d066@fsol>

On Sun, 26 Oct 2014 09:22:18 +1100
Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > My point is that your "Windows build" would not have the same behaviour
> > as a MSVC-produced Windows build, and so testing it with it would not
> > certify that your code would actually be compatible with genuine
> > MSVC builds of CPython, which we will not stop supporting.
> >
> 
> So you're saying it's impossible to support two compilers?

???


From p.f.moore at gmail.com  Sun Oct 26 00:44:02 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 25 Oct 2014 23:44:02 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
Message-ID: <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>

On 25 October 2014 23:22, Chris Angelico <rosuav at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> My point is that your "Windows build" would not have the same behaviour
>> as a MSVC-produced Windows build, and so testing it with it would not
>> certify that your code would actually be compatible with genuine
>> MSVC builds of CPython, which we will not stop supporting.
>>
>
> So you're saying it's impossible to support two compilers?

No, rather that Windows currently only has a single supported compiler
(excluding cygwin, which is essentially a different OS). Adding a
second compiler doesn't just involve adding support for it - which is
all that the people offering mingw patches are doing - but also
involves going through the whole CPython ecosystem locating the places
where there is an implicit assumption that "all Windows builds use the
same compiler" and fixing them. I've already pointed out where this is
a question for pip and wheel. Whoever wants to add support for a
second compiler needs to be willing to do this part of the job as
well.

Handwaving arguments that "it's binary compatible" aren't enough. Prove it.

Paul

From Stefan.Richthofer at gmx.de  Sun Oct 26 00:20:47 2014
From: Stefan.Richthofer at gmx.de (Stefan Richthofer)
Date: Sun, 26 Oct 2014 00:20:47 +0200
Subject: [Python-Dev] results of id() and weakref.getweakrefs() sometimes
 break on object resurrection
Message-ID: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>

Hello developers,

I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
regarding object resurrection.
Yes, resurrection is evil, but it is a valid scenario. If an object is
resurrected via its finalizer __del__, sometimes its unique id value as
returned from id() changes. Additionally the list of weak references
pointing to it as returned by weakref.getweakrefs(...) breaks (i.e. is
suddenly empty, assuming it wasn't before).
These issues only arise if the resurrection occurs during
cyclic garbage collection. If the object is finalized because its refcount
drops to zero, everything is fine. See the attached test-file.

Is this behaviour intended or is it a bug? If so, which variant is the bug
and which is right? I can hardly believe that whether id() is preserved
should depend on whether the garbage was cyclic or not.

This might appear of low relevance to you, since no sane program intentionally
performs resurrection. However I originally became aware of the issue
in Jython (where it not only occurs for cyclic garbage but in every
resurrection-case), c.f. http://bugs.jython.org/issue2224.
I am interested in this because I am implementing native gc support
in JyNI and need to understand these details to do it right.


Thanks in advance!

Stefan
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_resurrection_attr_preserve.py
Type: text/x-python
Size: 2456 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141026/f60b2cc0/attachment.py>

From solipsis at pitrou.net  Sun Oct 26 01:22:58 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 26 Oct 2014 01:22:58 +0200
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
Message-ID: <20141026012258.6deea3aa@fsol>


Hello Stefan,

On Sun, 26 Oct 2014 00:20:47 +0200
"Stefan Richthofer" <Stefan.Richthofer at gmx.de> wrote:
> Hello developers,
> 
> I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
> regarding object resurrection.

Your runGC() function is buggy, it does not run the GC under CPython.
Fix it and the first problem (with id()) disappears.

The second problem (with weakref) is different: weakrefs are cleared
before __del__ is called, so resurrection doesn't affect the whole
process. Add a callback to the weakref and you'll see it is getting
called.

In other words, CPython behaves as expected. Your concern is
appreciated, though.

Regards

Antoine.



From rdmurray at bitdance.com  Sun Oct 26 01:24:38 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sat, 25 Oct 2014 19:24:38 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141026001944.0373cb92@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
Message-ID: <20141025232439.C6214250D5B@webabinitio.net>

On Sun, 26 Oct 2014 00:19:44 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 26 Oct 2014 09:06:36 +1100
> Chris Angelico <rosuav at gmail.com> wrote:
> > On Sun, Oct 26, 2014 at 8:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > > How do you know this isn't a problem, since you haven't *tested* with
> > > MSVC?
> > > Why on Earth would you want to test your PEP work with an unsupported
> > > Windows compiler and runtime, rather than with the officially supported
> > > compiler and runtime?
> > 
> > This discussion revolved around supporting MinGW in addition to MSVC.
> > If it had been supported when I was doing that, I could have spun
> > myself up a Windows build and tested it.
> 
> My point is that your "Windows build" would not have the same behaviour
> as a MSVC-produced Windows build, and so testing it with it would not
> certify that your code would actually be compatible with genuine
> MSVC builds of CPython, which we will not stop supporting.

While true, I don't think that matters for Chris' point.  Given only the
ability to build with the MSVC toolchain, his code (which might even be
pure python for the purposes of this discussion) would not get tested on
Windows until committed and run by the buildbots.  If he could build
CPython using MinGW, he would, and would test his code on Windows.  Even
if there are C components and MSVC/MinGW compatibility issues are
revealed when the buildbots eventually run the code, still the number of
bugs present would probably be lower if he had tested it on Windows
than if he hadn't.

I know I for one do not generally test patches on Windows because I
haven't taken the time to learn how to build CPython on it.  Sure, I
could test pure python changes by applying patches to an installed
Python, but that's an ongoing pain and I'd rather learn to build CPython
on Windows and get to use the normal hg tools.

If I could use a more linux-like toolchain to build CPython on windows,
I would doubtless do much more testing on windows for stuff where I
think windows might behave differently (and I might look at more Windows
bugs...though frankly there are plenty of bugs for me to look at without
looking at Windows bugs).

This is not necessarily a compelling argument for MinGW support.
However, it *is* a valid argument, IMO.

Note: it can be made even less compelling by making it a lot easier to
build CPython on Windows without having an MSVC license (which I think
means not using the GUI, for which I say *yay* :).  I think Zach Ware
has been working on improving the Windows build process, and I keep
meaning to give it a try...

--David

From solipsis at pitrou.net  Sun Oct 26 01:30:36 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 26 Oct 2014 01:30:36 +0200
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <20141025232439.C6214250D5B@webabinitio.net>
Message-ID: <20141026013036.4a6be805@fsol>

On Sat, 25 Oct 2014 19:24:38 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:
> 
> I know I for one do not generally test patches on Windows because I
> haven't taken the time to learn how to build CPython on it.  Sure, I
> could test pure python changes by applying patches to an installed
> Python, but that's an ongoing pain and I'd rather learn to build CPython
> on Windows and get to use the normal hg tools.
> 
> If I could use a more linux-like toolchain to build CPython on windows,

Well, I don't know how "linux-like" you want your toolchain, but FTR
you should be able to build from the command line by running
"Tools\buildbot\build.bat".

I doubt the MinGW case can be much simpler :-)

Regards

Antoine.



From guido at python.org  Sun Oct 26 01:51:44 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 25 Oct 2014 16:51:44 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
Message-ID: <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>

On Sat, Oct 25, 2014 at 1:10 PM, Ray Donnelly <mingw.android at gmail.com>
wrote:

> On Sat, Oct 25, 2014 at 6:13 PM, Steve Dower <Steve.Dower at microsoft.com>
> wrote:
> > Building CPython for Windows is not something that needs solving. The
> > culture on Windows is to redistribute binaries, not source, and both the
> > core team and a number of redistributors have this figured out (and it
> will
> > only become easier with VC14 and Python 3.5).
>
> This is the second time you've used the vacuous "culture on Windows"
> argument, now with an added appeal to (vague) authority. That may be
> your opinion and that of some others, but there's a large number of
> people who don't care for using non-Free tools. IMHO building CPython
> on Windows using Open Source toolchains is very much something that
> needs merging upstream and supporting by default. What is it that you
> are afraid of if CPython can be compiled out of the box using
> mingw/MinGW-w64? Why are you fighting so hard against having option.
> If CPython wants to truly call itself an Open Source project then I
> consider being able to compile and cross-compile it with capable Open
> Source toolchains on all major platforms a requirement.
>

Please stop this ridiculous argument. There's no definition of "truly open
source project" that has such a requirement, and if you took it to the
extreme you should not be using Windows at all. I appreciate your concern
that building Python for your favorite platform using your favorite
toolchain doesn't work, and if you have patches (or even bug reports) those
are appreciated. But please take your rhetoric about open source elsewhere.


> > I'd rather see this effort thrown behind compiling extensions, including
> > cross compilation. The ABI is well defined enough that any compiler
> should
> > be usable, especially once the new CRT is in use. However, there is work
> > needed to update the various tool chains to link to VC14's CRT and we
> need
> > to figure out the inconsistencies between tools so we can document and
> work
> > through them.
> >
> > Having different builds of CPython out there will only fragment the
> > community and hurt extension authors far more than it may seem to help.
>

Here's the crux of the matter. We want compiled extension modules
distributed via PyPI to work with the binaries distributed from python.org.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141025/728b8341/attachment-0001.html>

From breamoreboy at yahoo.co.uk  Sun Oct 26 01:58:57 2014
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Sun, 26 Oct 2014 00:58:57 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025232439.C6214250D5B@webabinitio.net>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol> <20141025232439.C6214250D5B@webabinitio.net>
Message-ID: <m2hdg3$flo$1@ger.gmane.org>

On 26/10/2014 00:24, R. David Murray wrote:
> On Sun, 26 Oct 2014 00:19:44 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sun, 26 Oct 2014 09:06:36 +1100
>> Chris Angelico <rosuav at gmail.com> wrote:
>>> On Sun, Oct 26, 2014 at 8:59 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>> How do you know this isn't a problem, since you haven't *tested* with
>>>> MSVC?
>>>> Why on Earth would you want to test your PEP work with an unsupported
>>>> Windows compiler and runtime, rather than with the officially supported
>>>> compiler and runtime?
>>>
>>> This discussion revolved around supporting MinGW in addition to MSVC.
>>> If it had been supported when I was doing that, I could have spun
>>> myself up a Windows build and tested it.
>>
>> My point is that your "Windows build" would not have the same behaviour
>> as a MSVC-produced Windows build, and so testing it with it would not
>> certify that your code would actually be compatible with genuine
>> MSVC builds of CPython, which we will not stop supporting.
>
> While true, I don't think that matters for Chris' point.  Given only the
> ability to build with the MSVC toolchain, his code (which might even be
> pure python for the purposes of this discussion) would not get tested on
> Windows until committed and run by the buildbots.  If he could build
> CPython using MinGW, he would, and would test his code on Windows.  Even
> if there are C components and MSVC/MinGW compatibility issues are
> revealed when the buildbots eventually run the code, still the number of
> bugs present would probably be lower if he had tested it on Windows
> than if he hadn't.
>
> I know I for one do not generally test patches on Windows because I
> haven't taken the time to learn how to build CPython on it.  Sure, I
> could test pure python changes by applying patches to an installed
> Python, but that's an ongoing pain and I'd rather learn to build CPython
> on Windows and get to use the normal hg tools.
>
> If I could use a more linux-like toolchain to build CPython on windows,
> I would doubtless do much more testing on windows for stuff where I
> think windows might behave differently (and I might look at more Windows
> bugs...though frankly there are plenty of bugs for me to look at without
> looking at Windows bugs).
>
> This is not necessarily a compelling argument for MinGW support.
> However, it *is* a valid argument, IMO.
>
> Note: it can be made even less compelling by making it a lot easier to
> build CPython on Windows without having an MSVC license (which I think
> means not using the GUI, for which I say *yay* :).  I think Zach Ware
> has been working on improving the Windows build process, and I keep
> meaning to give it a try...
>
> --David
>

MSVC Express Edition 2010 works perfectly for building 3.5 so no license 
needed.  Links to older versions have been pointed out on other threads, 
either here or python-ideas, maybe both?  Or use the command line as 
Antoine pointed out elsewhere.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From mingw.android at gmail.com  Sun Oct 26 02:05:41 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 01:05:41 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141026013036.4a6be805@fsol>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <20141025232439.C6214250D5B@webabinitio.net>
 <20141026013036.4a6be805@fsol>
Message-ID: <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>

On Sun, Oct 26, 2014 at 12:30 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 25 Oct 2014 19:24:38 -0400
> "R. David Murray" <rdmurray at bitdance.com> wrote:
>>
>> I know I for one do not generally test patches on Windows because I
>> haven't taken the time to learn how to build CPython on it.  Sure, I
>> could test pure python changes by applying patches to an installed
>> Python, but that's an ongoing pain and I'd rather learn to build CPython
>> on Windows and get to use the normal hg tools.
>>
>> If I could use a more linux-like toolchain to build CPython on windows,
>
> Well, I don't know how "linux-like" you want your toolchain, but FTR
> you should be able to build from the command line by running
> "Tools\buildbot\build.bat".
>
> I doubt the MinGW case can be much simpler :-)
>

I beg to differ:

"Tools\buildbot\build.bat" contains:
@rem Used by the buildbot "compile" step.
cmd /c Tools\buildbot\external.bat
call "%VS100COMNTOOLS%vsvars32.bat"
cmd /c Tools\buildbot\clean.bat
msbuild PCbuild\pcbuild.sln /p:Configuration=Debug /p:Platform=Win32

^ this involves purchasing and installing MS Visual Studio (I'm not
sure if the Express Edition can be used).

Without explanations for what each step is doing (I posted those last
week), on MSYS2:
Download and run:
http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20141003.exe/download
>From the MSYS2 shell:
    pacman -S git base-devel mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain
    git clone https://github.com/Alexpux/MINGW-packages
    cd MINGW-packages/mingw-w64-python3
    makepkg-mingw -sL --nocheck
(remove --nocheck to run the testsuite. Also you could put this into a
batch file if you were so inclined.)

To install the newly built packages, from the MSYS2 shell again:
    pacman -U mingw-w64-*.xz

To run them, you should add /mingw64/bin or /mingw32/bin to your PATH
(or launch a new shell via mingw32_shell.bat or mingw64_shell.bat)
Of course, if you don't want to build it from source you can simply issue:
    pacman -S mingw-w64-python3

.. all of the above applies equally to mingw-w64-python2.

> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From mingw.android at gmail.com  Sun Oct 26 02:24:23 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 01:24:23 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
 <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>
Message-ID: <CAOYw7dt5AnwPSk0N6xpo_+F-miUQ_E1-V19AokfNUEj+vsuQ+w@mail.gmail.com>

On Sat, Oct 25, 2014 at 11:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 25 October 2014 23:22, Chris Angelico <rosuav at gmail.com> wrote:
>> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>> My point is that your "Windows build" would not have the same behaviour
>>> as a MSVC-produced Windows build, and so testing it with it would not
>>> certify that your code would actually be compatible with genuine
>>> MSVC builds of CPython, which we will not stop supporting.
>>>
>>
>> So you're saying it's impossible to support two compilers?
>
> No, rather that Windows currently only has a single supported compiler
> (excluding cygwin, which is essentially a different OS). Adding a
> second compiler doesn't just involve adding support for it - which is
> all that the people offering mingw patches are doing - but also
> involves going through the whole CPython ecosystem locating the places
> where there is an implicit assumption that "all Windows builds use the
> same compiler" and fixing them. I've already pointed out where this is
> a question for pip and wheel. Whoever wants to add support for a
> second compiler needs to be willing to do this part of the job as
> well.
>
> Handwaving arguments that "it's binary compatible" aren't enough. Prove it.

The msvcrt.dlls that MinGW-w64 depends on are those dating back to
Windows XP SP3 / XP64. Ironically, the official Windows CPython
doesn't come with any such crt guarantees and you must ensure that the
same msvcr??.dll is used for *all* extensions. This puts considerable
strain on extension developers to use the correct (or any) version of
Visual Studio to build their extensions for CPython on Windows. Also,
where are the publicly accessible specifications and other technical
descriptions that MinGW-w64 would need to implement strong binary
compatibility with MSVC? As a random example, those for C++ name
mangling and the PDB file format would be very helpful.

>
> Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From Stefan.Richthofer at gmx.de  Sun Oct 26 02:50:39 2014
From: Stefan.Richthofer at gmx.de (Stefan Richthofer)
Date: Sun, 26 Oct 2014 02:50:39 +0200
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <20141026012258.6deea3aa@fsol>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>, 
 <20141026012258.6deea3aa@fsol>
Message-ID: <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>

Okay, sorry, I was thinking too Jython-like. I fixed runGC() just to
see now that it does not even trigger resurrection, since under
CPython there are no finalizers executed in ref cycles (i.e. I find my
objects in gc.garbage).
So I realize, my xy_cyclic tests are pointless anyway since in cyclic
gc no resurrection can happen.

> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process.
It appears weakrefs are only cleared if this is done by gc (where no
resurrection can happen anyway). If a resurrection-performing-__del__ is
just called by ref-count-drop-to-0, weakrefs persist - a behavior that is
very difficult and inefficient to emulate in Jython, but I'll give it
some more thoughts...

However thanks for the help!

-Stefan


> Gesendet: Sonntag, 26. Oktober 2014 um 01:22 Uhr
> Von: "Antoine Pitrou" <solipsis at pitrou.net>
> An: python-dev at python.org
> Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection
>
> 
> Hello Stefan,
> 
> On Sun, 26 Oct 2014 00:20:47 +0200
> "Stefan Richthofer" <Stefan.Richthofer at gmx.de> wrote:
> > Hello developers,
> > 
> > I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
> > regarding object resurrection.
> 
> Your runGC() function is buggy, it does not run the GC under CPython.
> Fix it and the first problem (with id()) disappears.
> 
> The second problem (with weakref) is different: weakrefs are cleared
> before __del__ is called, so resurrection doesn't affect the whole
> process. Add a callback to the weakref and you'll see it is getting
> called.
> 
> In other words, CPython behaves as expected. Your concern is
> appreciated, though.
> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
> 

From solipsis at pitrou.net  Sun Oct 26 02:18:01 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 26 Oct 2014 02:18:01 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
Message-ID: <20141026021801.626735f7@fsol>

On Sun, 26 Oct 2014 02:50:39 +0200
"Stefan Richthofer" <Stefan.Richthofer at gmx.de> wrote:
> Okay, sorry, I was thinking too Jython-like. I fixed runGC() just to
> see now that it does not even trigger resurrection, since under
> CPython there are no finalizers executed in ref cycles (i.e. I find my
> objects in gc.garbage).

Try CPython 3.4 :-)

Regards

Antoine.



From mingw.android at gmail.com  Sun Oct 26 02:06:29 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 01:06:29 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <1414284354729.57145@microsoft.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
 <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>
 <CAOYw7dt5AnwPSk0N6xpo_+F-miUQ_E1-V19AokfNUEj+vsuQ+w@mail.gmail.com>
 <1414284354729.57145@microsoft.com>
Message-ID: <CAOYw7dv-Qqd-Lgg9bCmHvVH=C5fBNZYkZ3Q--ba1BF-teM+n5w@mail.gmail.com>

On Sun, Oct 26, 2014 at 1:45 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Ray Donnelly wrote:
>> On Sat, Oct 25, 2014 at 11:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>>> On 25 October 2014 23:22, Chris Angelico <rosuav at gmail.com> wrote:
>>>> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>>> My point is that your "Windows build" would not have the same behaviour
>>>>> as a MSVC-produced Windows build, and so testing it with it would not
>>>>> certify that your code would actually be compatible with genuine
>>>>> MSVC builds of CPython, which we will not stop supporting.
>>>>>
>>>>
>>>> So you're saying it's impossible to support two compilers?
>>>
>>> No, rather that Windows currently only has a single supported compiler
>>> (excluding cygwin, which is essentially a different OS). Adding a
>>> second compiler doesn't just involve adding support for it - which is
>>> all that the people offering mingw patches are doing - but also
>>> involves going through the whole CPython ecosystem locating the places
>>> where there is an implicit assumption that "all Windows builds use the
>>> same compiler" and fixing them. I've already pointed out where this is
>>> a question for pip and wheel. Whoever wants to add support for a
>>> second compiler needs to be willing to do this part of the job as
>>> well.
>>>
>>> Handwaving arguments that "it's binary compatible" aren't enough. Prove it.
>>
>> The msvcrt.dlls that MinGW-w64 depends on are those dating back to
>> Windows XP SP3 / XP64. Ironically, the official Windows CPython
>> doesn't come with any such crt guarantees and you must ensure that the
>> same msvcr??.dll is used for *all* extensions. This puts considerable
>> strain on extension developers to use the correct (or any) version of
>> Visual Studio to build their extensions for CPython on Windows.
>
> We're well aware of this, and it's a big part of why I'm currently migrating CPython to build with VC14, which will not have the same binary compatibility issues. For VC14, the entire CRT has been cleaned up and mostly hidden behind calls into DLLs, so provided the calling conventions match (which they must or everything would crash very quickly), it should be relatively easy to build compatible extensions with MinGW-w64.

Compatibility going forwards though, right? Still it's great to see
positive steps being made for the future of the Windows platform.

>
>> Also, where are the publicly accessible specifications and other technical
>> descriptions that MinGW-w64 would need to implement strong binary
>> compatibility with MSVC? As a random example, those for C++ name
>> mangling and the PDB file format would be very helpful.
>
> C++ name mangling is always an implementation detail and it changes from version to version. Luckily, CPython is entirely in C, so that doesn't matter. PDBs are another red herring - you can build a loadable PE file without PDBs.

Of course C++ can be called from C and that is done in some CPython
extensions, so it's not a red herring. If we want to talk about strong
binary compatibility I'd expect the aim would be to intermix freely
between compilers. We'd like people to be able to debug MinGW-w64 code
using CDB in Visual Studio if they want to, and on the flipside, to
have GDB able to read PDB files built by MSVC (actually there's a long
standing problem when debugging MinGW-w64 code in GDB that stack
unwinding out of MS built dlls is flaky at best) - so again this is
not really a red herring. I'm also led to believe that MSVC has a very
good optimizer so if some project wanted to build certain libraries or
objects with that for their performance critical paths then I can see
that as being useful to those projects and their users'.

>
> The full source code for the MSVCRT is available with any version of Visual Studio (including the free editions, last time I checked), so feel free to check whatever you need to ensure compatibility. I've suggested to the VC team that they could get in touch with the MinGW projects and offer to help them improve compatibility with MSVC, but unfortunately I don't think anyone will take me up on that. I'm happy to research what I can to answer specific questions, but there's very little that isn't already publicly available other than direct access to the devs.

Under what license? We'd rather have open specifications that
copyrighted, strictly licensed code that we can't look at for various
tainting reasons.

>
> Cheers,
> Steve
>
>>> Paul

From Steve.Dower at microsoft.com  Sun Oct 26 03:03:11 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 26 Oct 2014 02:03:11 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dv-Qqd-Lgg9bCmHvVH=C5fBNZYkZ3Q--ba1BF-teM+n5w@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
 <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>
 <CAOYw7dt5AnwPSk0N6xpo_+F-miUQ_E1-V19AokfNUEj+vsuQ+w@mail.gmail.com>
 <1414284354729.57145@microsoft.com>,
 <CAOYw7dv-Qqd-Lgg9bCmHvVH=C5fBNZYkZ3Q--ba1BF-teM+n5w@mail.gmail.com>
Message-ID: <1414288990487.90928@microsoft.com>

Ray Donnelly wrote:
> On Sun, Oct 26, 2014 at 1:45 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> Ray Donnelly wrote:
>>> Also, where are the publicly accessible specifications and other technical
>>> descriptions that MinGW-w64 would need to implement strong binary
>>> compatibility with MSVC? As a random example, those for C++ name
>>> mangling and the PDB file format would be very helpful.
>>
>> C++ name mangling is always an implementation detail and it changes from
>> version to version. Luckily, CPython is entirely in C, so that doesn't matter.
>> PDBs are another red herring - you can build a loadable PE file without PDBs.
>
> Of course C++ can be called from C and that is done in some CPython
> extensions, so it's not a red herring. If we want to talk about strong
> binary compatibility I'd expect the aim would be to intermix freely
> between compilers. We'd like people to be able to debug MinGW-w64 code
> using CDB in Visual Studio if they want to, and on the flipside, to
> have GDB able to read PDB files built by MSVC (actually there's a long
> standing problem when debugging MinGW-w64 code in GDB that stack
> unwinding out of MS built dlls is flaky at best) - so again this is
> not really a red herring. I'm also led to believe that MSVC has a very
> good optimizer so if some project wanted to build certain libraries or
> objects with that for their performance critical paths then I can see
> that as being useful to those projects and their users'.

Binary compatibility that strong is very unlikely to ever happen, and certainly
not with versions of compilers that are being actively developed. It would be far
too restrictive to both development teams.

The weaker compatibility of C DLL boundaries is far more achievable - we already
mostly have it, as evidenced by some Python packages working correctly with
mismatched compilers. Soon the CRT will be isolated along the same boundaries,
which is short-term pain for long-term gain.

>>
>> The full source code for the MSVCRT is available with any version of Visual
>> Studio (including the free editions, last time I checked), so feel free to
check
>> whatever you need to ensure compatibility. I've suggested to the VC team that
>> they could get in touch with the MinGW projects and offer to help them improve
>> compatibility with MSVC, but unfortunately I don't think anyone will take me up
>> on that. I'm happy to research what I can to answer specific questions, but
>> there's very little that isn't already publicly available other than direct
>> access to the devs.
>
> Under what license? We'd rather have open specifications that
> copyrighted, strictly licensed code that we can't look at for various
> tainting reasons.

As far as I can tell, it's covered by the Visual Studio license, which basically
means you can't redistribute the files (I'm not a lawyer, but I've spent plenty of
time talking about licenses to lawyers... not sure how much that counts for :) ).

Most closed-source Microsoft code is not released under open-source-like licenses,
so there's no concept of derivative work, attribution or reciprocation, and that's
what appears to cover the CRT sources. "Using" the sources probably counts as using
VS, which may trigger some non-commercial clauses if you've got the free version
(but probably not the 30 day trial of the paid version... licenses are weird), but
reading them is well within the granted permissions.

The intention of including the sources is to help people with debugging... I don't
think it's even possible to rebuild the CRT from them. I do understand the taint
concerns though - until recently, I was operating under rules that made even some
documentation "unsafe"...

>> Cheers,
>> Steve
>>
>>>> Paul

From zachary.ware+pydev at gmail.com  Sun Oct 26 05:31:47 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sat, 25 Oct 2014 23:31:47 -0500
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol> <20141025232439.C6214250D5B@webabinitio.net>
 <20141026013036.4a6be805@fsol>
 <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>
Message-ID: <CAKJDb-P8-yQjhCds_TH4HQ+XhpBW7vM=fu5Va9tfuDe_kaMEpA@mail.gmail.com>

On Sat, Oct 25, 2014 at 7:05 PM, Ray Donnelly <mingw.android at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 12:30 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 25 Oct 2014 19:24:38 -0400
>> "R. David Murray" <rdmurray at bitdance.com> wrote:
>>>
>>> I know I for one do not generally test patches on Windows because I
>>> haven't taken the time to learn how to build CPython on it.  Sure, I
>>> could test pure python changes by applying patches to an installed
>>> Python, but that's an ongoing pain and I'd rather learn to build CPython
>>> on Windows and get to use the normal hg tools.
>>>
>>> If I could use a more linux-like toolchain to build CPython on windows,
>>
>> Well, I don't know how "linux-like" you want your toolchain, but FTR
>> you should be able to build from the command line by running
>> "Tools\buildbot\build.bat".
>>
>> I doubt the MinGW case can be much simpler :-)
>>
>
> I beg to differ:
>
> "Tools\buildbot\build.bat" contains:
> @rem Used by the buildbot "compile" step.
> cmd /c Tools\buildbot\external.bat
> call "%VS100COMNTOOLS%vsvars32.bat"
> cmd /c Tools\buildbot\clean.bat
> msbuild PCbuild\pcbuild.sln /p:Configuration=Debug /p:Platform=Win32
>
> ^ this involves purchasing and installing MS Visual Studio (I'm not
> sure if the Express Edition can be used).

The Express Edition is fine for 32-bit builds.  PCbuild\readme.txt has
full details on which editions are needed for what, and in 3.5 also
has a "quick start guide" (absent from older versions due to a
rewriting of the batch scripts that I did a while back):

1.  Install Microsoft Visual C++ 2010 SP1, any edition.
2.  Install Subversion, and make sure 'svn.exe' is on your PATH.
3.  Install NASM, and make sure 'nasm.exe' is on your PATH.
4.  Run "build.bat -e" to build Python in 32-bit Release configuration.
5.  (Optional, but recommended) Run the test suite with "rt.bat -q".

And really, you can skip step 5 if you don't want to run the tests;
you can skip step 3 if you don't want/need ssl; and you can skip step
2 if you don't want/need bz2, lzma, sqlite3, ssl, or tkinter.
Skipping steps 2 or 3 will cause a lot of angry red text in your build
output, but I hope to solve that problem eventually.

> Without explanations for what each step is doing (I posted those last
> week), on MSYS2:
> Download and run:
> http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20141003.exe/download
> From the MSYS2 shell:
>     pacman -S git base-devel mingw-w64-x86_64-toolchain mingw-w64-i686-toolchain
>     git clone https://github.com/Alexpux/MINGW-packages
>     cd MINGW-packages/mingw-w64-python3
>     makepkg-mingw -sL --nocheck
> (remove --nocheck to run the testsuite. Also you could put this into a
> batch file if you were so inclined.)
>
> To install the newly built packages, from the MSYS2 shell again:
>     pacman -U mingw-w64-*.xz
>
> To run them, you should add /mingw64/bin or /mingw32/bin to your PATH
> (or launch a new shell via mingw32_shell.bat or mingw64_shell.bat)
> Of course, if you don't want to build it from source you can simply issue:
>     pacman -S mingw-w64-python3
>
> .. all of the above applies equally to mingw-w64-python2.

I'm failing to see where that's simpler :)

For the record, on the issue at hand, I agree that any effort should
go toward making it possible to build extensions without MSVC.  Once
that problem has been completely solved, then we can consider (long
and hard) building Python itself with something else.  I personally
have not been convinced that there's any good reason to do so, but I'm
not unconvincible :)

-- 
Zach

From zachary.ware+pydev at gmail.com  Sun Oct 26 06:08:47 2014
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Sun, 26 Oct 2014 00:08:47 -0500
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025232439.C6214250D5B@webabinitio.net>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol> <20141025232439.C6214250D5B@webabinitio.net>
Message-ID: <CAKJDb-OjURFxEb3dTtLvo8nLmgzfnpoyhm8dPteV51m+BvzVOQ@mail.gmail.com>

On Sat, Oct 25, 2014 at 6:24 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> Note: it can be made even less compelling by making it a lot easier to
> build CPython on Windows without having an MSVC license (which I think
> means not using the GUI, for which I say *yay* :).  I think Zach Ware
> has been working on improving the Windows build process, and I keep
> meaning to give it a try...

So far my improvements have been limited to what it takes to build
after installing prerequisites (and documenting what exactly those
prerequisites are), but on that front things are significantly better
in 3.5, I think.  I will note that it's been possible to build Python
entirely without using the VS GUI (though it still has to be
installed, I think) for quite some time, but hasn't been especially
easy to remember the incantations to do so.  3.5 now has a fairly nice
set of batch scripts (I think; but I (re)wrote them :) that work well
together and are even documented in the PCbuild readme.  I've had
dreams of a set of configure.bat/make.bat scripts (issue16895) to make
things even simpler, but I've put that on hold until after Steve
Dower's major overhaul for VC14 has happened.

One thing I'd like to look into more eventually is seeing what it
would take to build with only the Windows SDK installed (which is free
and has no GUI at all), but I think Steve has mentioned something
about that in connection with the work he's doing -- either way,
things will be different when we switch to VC14 so I'm also putting
that off.

-- 
Zach

From guido at python.org  Sun Oct 26 06:53:51 2014
From: guido at python.org (Guido van Rossum)
Date: Sat, 25 Oct 2014 22:53:51 -0700
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
Message-ID: <CAP7+vJKJt-j-AzxmZOcrjdHu8xr_UR6uKYVgT-TF2xDskTHgZw@mail.gmail.com>

On Saturday, October 25, 2014, Stefan Richthofer <Stefan.Richthofer at gmx.de>
wrote:

> Okay, sorry, I was thinking too Jython-like. I fixed runGC() just to
> see now that it does not even trigger resurrection, since under
> CPython there are no finalizers executed in ref cycles (i.e. I find my
> objects in gc.garbage).
> So I realize, my xy_cyclic tests are pointless anyway since in cyclic
> gc no resurrection can happen.
>
> > The second problem (with weakref) is different: weakrefs are cleared
> > before __del__ is called, so resurrection doesn't affect the whole
> > process.
> It appears weakrefs are only cleared if this is done by gc (where no
> resurrection can happen anyway). If a resurrection-performing-__del__ is
> just called by ref-count-drop-to-0, weakrefs persist - a behavior that is
> very difficult and inefficient to emulate in Jython, but I'll give it
> some more thoughts...
>
> You shouldn't have to emulate that. The exact behavior of GC is allowed to
vary between systems.


> However thanks for the help!
>
> -Stefan
>
>
> > Gesendet: Sonntag, 26. Oktober 2014 um 01:22 Uhr
> > Von: "Antoine Pitrou" <solipsis at pitrou.net <javascript:;>>
> > An: python-dev at python.org <javascript:;>
> > Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs()
> sometimes break on object resurrection
> >
> >
> > Hello Stefan,
> >
> > On Sun, 26 Oct 2014 00:20:47 +0200
> > "Stefan Richthofer" <Stefan.Richthofer at gmx.de <javascript:;>> wrote:
> > > Hello developers,
> > >
> > > I observed strange behaviour in CPython (tested in 2.7.5 and 3.3.3)
> > > regarding object resurrection.
> >
> > Your runGC() function is buggy, it does not run the GC under CPython.
> > Fix it and the first problem (with id()) disappears.
> >
> > The second problem (with weakref) is different: weakrefs are cleared
> > before __del__ is called, so resurrection doesn't affect the whole
> > process. Add a callback to the weakref and you'll see it is getting
> > called.
> >
> > In other words, CPython behaves as expected. Your concern is
> > appreciated, though.
> >
> > Regards
> >
> > Antoine.
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org <javascript:;>
> > https://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
> >
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org <javascript:;>
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>


-- 
--Guido van Rossum (on iPad)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141025/e36204cf/attachment.html>

From Steve.Dower at microsoft.com  Sun Oct 26 02:45:54 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 26 Oct 2014 00:45:54 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dt5AnwPSk0N6xpo_+F-miUQ_E1-V19AokfNUEj+vsuQ+w@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <CAPTjJmr2O-Y0CObP1N5oiCE54_c4AuzF1toZTP64eDATbyZ1_w@mail.gmail.com>
 <CACac1F-pW3DsfGApTt921S7c5+ytioGFo_ExnQ5BoLw8+fLe=w@mail.gmail.com>,
 <CAOYw7dt5AnwPSk0N6xpo_+F-miUQ_E1-V19AokfNUEj+vsuQ+w@mail.gmail.com>
Message-ID: <1414284354729.57145@microsoft.com>

Ray Donnelly wrote:
> On Sat, Oct 25, 2014 at 11:44 PM, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 25 October 2014 23:22, Chris Angelico <rosuav at gmail.com> wrote:
>>> On Sun, Oct 26, 2014 at 9:19 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>>>> My point is that your "Windows build" would not have the same behaviour
>>>> as a MSVC-produced Windows build, and so testing it with it would not
>>>> certify that your code would actually be compatible with genuine
>>>> MSVC builds of CPython, which we will not stop supporting.
>>>>
>>>
>>> So you're saying it's impossible to support two compilers?
>>
>> No, rather that Windows currently only has a single supported compiler
>> (excluding cygwin, which is essentially a different OS). Adding a
>> second compiler doesn't just involve adding support for it - which is
>> all that the people offering mingw patches are doing - but also
>> involves going through the whole CPython ecosystem locating the places
>> where there is an implicit assumption that "all Windows builds use the
>> same compiler" and fixing them. I've already pointed out where this is
>> a question for pip and wheel. Whoever wants to add support for a
>> second compiler needs to be willing to do this part of the job as
>> well.
>>
>> Handwaving arguments that "it's binary compatible" aren't enough. Prove it.
> 
> The msvcrt.dlls that MinGW-w64 depends on are those dating back to
> Windows XP SP3 / XP64. Ironically, the official Windows CPython
> doesn't come with any such crt guarantees and you must ensure that the
> same msvcr??.dll is used for *all* extensions. This puts considerable
> strain on extension developers to use the correct (or any) version of
> Visual Studio to build their extensions for CPython on Windows.

We're well aware of this, and it's a big part of why I'm currently migrating CPython to build with VC14, which will not have the same binary compatibility issues. For VC14, the entire CRT has been cleaned up and mostly hidden behind calls into DLLs, so provided the calling conventions match (which they must or everything would crash very quickly), it should be relatively easy to build compatible extensions with MinGW-w64.

> Also, where are the publicly accessible specifications and other technical
> descriptions that MinGW-w64 would need to implement strong binary
> compatibility with MSVC? As a random example, those for C++ name
> mangling and the PDB file format would be very helpful.

C++ name mangling is always an implementation detail and it changes from version to version. Luckily, CPython is entirely in C, so that doesn't matter. PDBs are another red herring - you can build a loadable PE file without PDBs.

The full source code for the MSVCRT is available with any version of Visual Studio (including the free editions, last time I checked), so feel free to check whatever you need to ensure compatibility. I've suggested to the VC team that they could get in touch with the MinGW projects and offer to help them improve compatibility with MSVC, but unfortunately I don't think anyone will take me up on that. I'm happy to research what I can to answer specific questions, but there's very little that isn't already publicly available other than direct access to the devs.

Cheers,
Steve

>> Paul

From stefankrah at freenet.de  Sun Oct 26 12:37:54 2014
From: stefankrah at freenet.de (Stefan Krah)
Date: Sun, 26 Oct 2014 11:37:54 +0000 (UTC)
Subject: [Python-Dev] Cross compiling Python (for Android)
References: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E0CDA2E@FMSMSX106.amr.corp.intel.com>
Message-ID: <loom.20141026T122746-845@post.gmane.org>

Frank, Matthew I <matthew.i.frank <at> intel.com> writes:
 ?
> 4. Module _decimal is failing to compile.? The problem is that it has
> ?? a header called memory.h.? Android's libc has the problem that
> ?? /usr/include/stdlib.h includes <memory.h>.? But the build system
> ?? puts -I. on the include path before the system dirs (as it should)
> ?? so when compiling _decimal, Modules/_decimal/libmpdec/memory.h gets
> ?? found instead of /usr/include/memory.h.? Shiz has a patch here:
> ??
https://github.com/rave-engine/python3-android/blob/master/mk/python/3.3.5/p\
> ython-3.3.5-android-libmpdec.patch
> ? ?(which renames memory.h -> mpmemory.h) but I don't know
> ?
> ?? a.? Is there a tracker for this yet?? and
> ?? b.? Is Shiz's fix the desired one or should I be looking for
> ?????? another approach?? (Maybe modifying the -I flags for the build
> ?????? of just the build of _decimal or something?)

I think using "memory.h" in an application is standard conforming.
Since _decimal compiles on all other Linux platforms, it may be worth
reporting this to the Android developers and see if they can fix it
(possibly by not including memory.h in stdlib.h).

FWIW, OCaml also has a "memory.h" header.


Stefan Krah


From mal at egenix.com  Sun Oct 26 13:50:20 2014
From: mal at egenix.com (M.-A. Lemburg)
Date: Sun, 26 Oct 2014 13:50:20 +0100
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
In-Reply-To: <nad-95699A.15140225102014@news.gmane.org>
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>	<20141025055716.7e252f85@fsol>
 <m2d29gwzbq.fsf@valheru.db3l.homeip.net>	<m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <nad-95699A.15140225102014@news.gmane.org>
Message-ID: <544CEE0C.4070401@egenix.com>

On 26.10.2014 00:14, Ned Deily wrote:
> In article <m28uk4wxod.fsf at valheru.db3l.homeip.net>,
>  David Bolen <db3l.net at gmail.com> wrote:
> 
>> David Bolen <db3l.net at gmail.com> writes:
>>
>>> which appears to die mid-stream while receiving the manifests.
>>>
>>> So I'm sort of hoping there might be some record server-side as to why
>>> things are falling apart mid-way.
>>
>> Just to follow-up to myself, I get the same same error trying to do a
>> clone from my own personal XP machine rather than the buildbot (which
>> is a VM).  I've had the issue with hg 1.6.2, 2.5.2 and 3.1.2.
>>
>> However, the same clones completely successfully under OSX and Linux.
>>
>> So that's sort of strange.
> 
> Very interesting!  I had been doing some housekeeping on some of my 
> older OS X build systems over the past few days and I've run into the 
> same problem.  In particular, I am seeing this failure on an OS X 10.5.8 
> system (running in a Fusion VM) which I've used for years and from which 
> I have regularly cloned repos from hg.python.org.  I spent some time 
> yesterday trying to isolate it.  I came to the conclusion that it was 
> independent of the version of OpenSSL (identical failures occurred with 
> the system's ancient Apple 0.9.7 as well as a newly-build 1.0.1j) and 
> independent of the version of hg (at least with two data points, current 
> and a year-old version) and seemingly independent of the network 
> connection.  I was not able to reproduce the failure on the host OS X 
> system (10.10) and I didn't have problems a few days earlier with 
> various other OS X releases (10.6.x through 10.9.x) also running in VMs 
> on the same host.  I stumbled across a workaround for the problem as I 
> was experiencing it:  adding --uncompressed to hg clone eliminated 
> failures.  You can get more info on the hg failures by adding 
> --traceback and --debugger to the clone command.  After spending way too 
> much time on the issue, I was not in the mood to spend more time 
> isolating the problem after finding a workaround but if others are also 
> seeing it, it might be worth doing.  Sigh.
> 
>       $ hg --version
>       Mercurial Distributed SCM (version 3.1.2)
>       $ hg clone -U http://hg.python.org/cpython cpython
>       real URL is https://hg.python.org/cpython
>       requesting all changes
>       adding changesets
>       adding manifests
>       transaction abort!
>       rollback completed
>       abort: connection ended unexpectedly
>       $ hg clone --uncompressed -U https://hg.python.org/cpython cpython
>       streaming all changes
>       10404 files to transfer, 248 MB of data
>       transferred 248 MB in 44.4 seconds (5.58 MB/sec)

If compression is causing the problem, perhaps there's an incompatibility
with the use zlib version between the host and your client system.

hg.python.org was recently updated to a new Ubuntu version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Oct 26 2014)
>>> Python Projects, Consulting and Support ...   http://www.egenix.com/
>>> mxODBC.Zope/Plone.Database.Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2014-10-24: Released eGenix pyOpenSSL 0.13.5 ...  http://egenix.com/go63

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From Stefan.Richthofer at gmx.de  Sun Oct 26 08:51:27 2014
From: Stefan.Richthofer at gmx.de (Stefan Richthofer)
Date: Sun, 26 Oct 2014 08:51:27 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <CAP7+vJKJt-j-AzxmZOcrjdHu8xr_UR6uKYVgT-TF2xDskTHgZw@mail.gmail.com>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>,
 <CAP7+vJKJt-j-AzxmZOcrjdHu8xr_UR6uKYVgT-TF2xDskTHgZw@mail.gmail.com>
Message-ID: <trinity-1f2bb575-9dfd-430b-a59c-c11ecce01e22-1414309887334@3capp-gmx-bs10>

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141026/de8639b5/attachment.html>

From kelman at berkeley.edu  Sun Oct 26 14:12:45 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Sun, 26 Oct 2014 06:12:45 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
Message-ID: <E648C5C014E74B459A705C98324E0A4D@TKsamsung>

Thanks all for the responses. Clearly this is a subject about which
people feel strongly, so that's good at least. David Murray's guidance
in particular points to the most likely path to get improvements to
really happen.

Steve Dower:
> Building CPython for Windows is not something that needs solving.

Not in your opinion, but numerous packagers of MinGW-based native or
cross-compiled package sets would love to include Python. The fact
that they currently can't, without many patches, is a problem.

> The culture on Windows is to redistribute binaries, not source,

There are many cultures using Windows. Including open-source ones.

> and both the core team and a number of redistributors have this figured
> out (and it will only become easier with VC14 and Python 3.5).

With MSVC. It doesn't work with MinGW, it likely doesn't work with Clang.
MSVC is not the only compiler on Windows. There are many use cases for
preferring other compilers. Have you read this wiki page for example?
https://github.com/numpy/numpy/wiki/Numerical-software-on-Windows

In my personal experience, having recently gotten Julia to compile using
MSVC for the first time, MSVC as a compiler is highly deficient for many
needs especially in the scientific software community:
- C99 (getting better recently, but still not done)
- AT&T syntax assembly
- C++11 features (also mostly okay now, but not if you're using an older
  MSVC version with Python 2.7, which many people still have to do)
- 128-bit integer intrinsics
- cannot cross-compile from anything that isn't Windows
- build systems foreign relative to shell/makefile systems used by most
  open-source projects, few projects have time to maintain 2 separate build
  systems (cmake helps but takes a lot of initial effort to convert to)
- no free-as-in-beer Fortran compiler available

I have none of these problems when I use MinGW-w64. Hence the desire to
be able to curate an all-MinGW software stack. It's not a matter of open-
source ideology for me, it's brass tacks "can I do the work I need to do."
With MSVC I can't, with MinGW-w64 I can. Not being able to include CPython
in an all-MinGW stack hurts, a lot.

Only cross-compilation and the build system in the above list are relevant
to CPython, but I hope I have convinced you, Paul Moore, etc. that there are
real reasons for some groups of users and developers to prefer MinGW-w64
over MSVC.

> I'd rather see this effort thrown behind compiling extensions,
> including cross compilation.

There are patches awaiting review that improve this as well. Efforts to
improve CPython's build system and the handling of extensions are not
completely independent, in many cases the patches are written by the same
set of MinGW users. One of these sets of patches is not inherently evil,
you understandably have less interest in them but it's still disappointing
to see so little movement on either.

> Having different builds of CPython out there will only fragment the
> community and hurt extension authors far more than it may seem to help.

The community of people developing and using open-source projects, either
CPython or otherwise, is already highly fragmented. Ignoring it makes it
worse. python.org does not have to distribute or endorse MinGW-compiled
builds of CPython. If the build option never gets incorporated, then it
will continue to be reverse-engineered.

Guido van Rossum:
> Here's the crux of the matter. We want compiled extension modules
> distributed via PyPI to work with the binaries distributed from python.org.

Absolutely. I don't think additional options in the build system would
change this.

R. David Murray:
> And, at this point, we would NEED A BUILDBOT.  That is, a machine that
> has whatever tools are required installed such that tests added to the
> test suite to test MinGW support in distutils would run, so we can be
> sure we don't break anything when making other changes.

That's not too hard. I've done this for other projects. AppVeyor works if
your build is short enough, and I've done cross-compilation from Travis
CI for other projects. Or Jenkins, or a Vagrant VM. I don't know PSF's
infrastructure, but I can offer guidance if it would help.

Steve Dower:
> I'm afraid of users having numpy crash because they're using an MSVC
> CPython instead of a mingw CPython. I'm afraid of users not being able
> to use library A and library B at the same time because A requires MSVC
> CPython and B requires mingw CPython. (I can produce more examples if you
> like, but the general concern is having a fragmented community, as I said
> in my previous post.)

A valid fear. Mixing C runtimes can cause problems, I've seen this myself.
Correct me if I'm wrong, but this is nearly as much of an issue if someone
wants to use a different version of MSVC to compile CPython than the version
used to build the official binaries. It requires care, but you can't deny
that there are use cases where people will want and need to do such things.
Is possible fragmentation a good enough reason to resist making it possible
in the build system?

> though I suspect most would like to see some traction achieved on a fork
> first

Those of us who consider this important should probably just do this. Ray,
Roumen, the maintainer of the Arch MinGW packages, myself and others could
look into making an actual fork on Github or Bitbucket where we merge the
various patches and come up with an out-of-the-box MinGW-[cross-]compilable
version of CPython. I'll happily write the spec files to get this building
from Fedora or openSUSE. That would help us test the feasibility from a
centralized repository. Ray, what do you think? Do you know xantares' email
address to ask if he'd be interested in helping or using the result?

Zachary Ware:
> I'm failing to see where that's simpler :)

If it were hypothetically merged instead of out in an external fork, it
could be ./configure --host=x86_64-w64-mingw32 && make to cross-compile
from Linux or Cygwin, or just ./configure && make after installing MSYS2
(which is just about as easy as installing MSVC) on Windows.

Paul Moore:
> If it were possible to cross-compile compatible extensions on Linux,
> projects developed on Linux could supply Windows binaries much more
> easily, which would be a huge benefit to the whole Windows Python
> community.

I want to do exactly this in an automated repeatable way, preferably on
a build service. This seems harder to do when CPython cannot itself be
built and handled as a dependency by that same automated, repeatable
build service. Unless it becomes possible to cross-compile extensions
using the build machine's own version of Python, which might be the right
approach.

> acutely aware of the common pain points for Python users on Windows.
> And they are all about binary extensions, and none at all about
> building Python itself.

I've done a lot of recent work keeping Julia working well on Windows, and
the interoperability we have with Python packages has propagated most of
these pain points to us as well. We have to rely on Conda in order to have
a reliable way of installing, as an example, IPython with the notebook
interface, in order for IJulia to work. This is not an ideal solution as it
requires a great deal of user intervention and manual steps to get up and
running (and it would be far worse without Conda). We are, so far, built
around MinGW-w64 on Windows, for the reasons I listed above. Having cross-
compiled builds of CPython and binary extensions available from the same
build services we already use to install other binary packages (Gtk, Cairo,
Tk, Nettle, HDF5, etc) on Windows would be enormously helpful for us.

There's a real use case. Its size and importance can be debated. For now
I'll take David Murray's post to heart and see where I have time or ability
to help things along.

Sincerely,
Tony
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141026/c58b6bf4/attachment.html>

From mingw.android at gmail.com  Sun Oct 26 15:28:38 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 14:28:38 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
Message-ID: <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>

On Sun, Oct 26, 2014 at 1:12 PM, Tony Kelman <kelman at berkeley.edu> wrote:
> Thanks all for the responses. Clearly this is a subject about which
> people feel strongly, so that's good at least. David Murray's guidance
> in particular points to the most likely path to get improvements to
> really happen.
>
> Steve Dower:
>> Building CPython for Windows is not something that needs solving.
>
> Not in your opinion, but numerous packagers of MinGW-based native or
> cross-compiled package sets would love to include Python. The fact
> that they currently can't, without many patches, is a problem.
>
>> The culture on Windows is to redistribute binaries, not source,
>
> There are many cultures using Windows. Including open-source ones.
>
>> and both the core team and a number of redistributors have this figured
>> out (and it will only become easier with VC14 and Python 3.5).
>
> With MSVC. It doesn't work with MinGW, it likely doesn't work with Clang.
> MSVC is not the only compiler on Windows. There are many use cases for
> preferring other compilers. Have you read this wiki page for example?
> https://github.com/numpy/numpy/wiki/Numerical-software-on-Windows
>
> In my personal experience, having recently gotten Julia to compile using
> MSVC for the first time, MSVC as a compiler is highly deficient for many
> needs especially in the scientific software community:
> - C99 (getting better recently, but still not done)
> - AT&T syntax assembly
> - C++11 features (also mostly okay now, but not if you're using an older
>   MSVC version with Python 2.7, which many people still have to do)
> - 128-bit integer intrinsics
> - cannot cross-compile from anything that isn't Windows
> - build systems foreign relative to shell/makefile systems used by most
>   open-source projects, few projects have time to maintain 2 separate build
>   systems (cmake helps but takes a lot of initial effort to convert to)
> - no free-as-in-beer Fortran compiler available
>
> I have none of these problems when I use MinGW-w64. Hence the desire to
> be able to curate an all-MinGW software stack. It's not a matter of open-
> source ideology for me, it's brass tacks "can I do the work I need to do."
> With MSVC I can't, with MinGW-w64 I can. Not being able to include CPython
> in an all-MinGW stack hurts, a lot.
>
> Only cross-compilation and the build system in the above list are relevant
> to CPython, but I hope I have convinced you, Paul Moore, etc. that there are
> real reasons for some groups of users and developers to prefer MinGW-w64
> over MSVC.
>
>> I'd rather see this effort thrown behind compiling extensions,
>> including cross compilation.
>
> There are patches awaiting review that improve this as well. Efforts to
> improve CPython's build system and the handling of extensions are not
> completely independent, in many cases the patches are written by the same
> set of MinGW users. One of these sets of patches is not inherently evil,
> you understandably have less interest in them but it's still disappointing
> to see so little movement on either.
>
>> Having different builds of CPython out there will only fragment the
>> community and hurt extension authors far more than it may seem to help.
>
> The community of people developing and using open-source projects, either
> CPython or otherwise, is already highly fragmented. Ignoring it makes it
> worse. python.org does not have to distribute or endorse MinGW-compiled
> builds of CPython. If the build option never gets incorporated, then it
> will continue to be reverse-engineered.
>
> Guido van Rossum:
>> Here's the crux of the matter. We want compiled extension modules
>> distributed via PyPI to work with the binaries distributed from
>> python.org.
>
> Absolutely. I don't think additional options in the build system would
> change this.
>
> R. David Murray:
>> And, at this point, we would NEED A BUILDBOT.  That is, a machine that
>> has whatever tools are required installed such that tests added to the
>> test suite to test MinGW support in distutils would run, so we can be
>> sure we don't break anything when making other changes.
>
> That's not too hard. I've done this for other projects. AppVeyor works if
> your build is short enough, and I've done cross-compilation from Travis
> CI for other projects. Or Jenkins, or a Vagrant VM. I don't know PSF's
> infrastructure, but I can offer guidance if it would help.
>
> Steve Dower:
>> I'm afraid of users having numpy crash because they're using an MSVC
>> CPython instead of a mingw CPython. I'm afraid of users not being able
>> to use library A and library B at the same time because A requires MSVC
>> CPython and B requires mingw CPython. (I can produce more examples if you
>> like, but the general concern is having a fragmented community, as I said
>> in my previous post.)
>
> A valid fear. Mixing C runtimes can cause problems, I've seen this myself.
> Correct me if I'm wrong, but this is nearly as much of an issue if someone
> wants to use a different version of MSVC to compile CPython than the version
> used to build the official binaries. It requires care, but you can't deny
> that there are use cases where people will want and need to do such things.
> Is possible fragmentation a good enough reason to resist making it possible
> in the build system?
>
>> though I suspect most would like to see some traction achieved on a fork
>> first
>
> Those of us who consider this important should probably just do this. Ray,
> Roumen, the maintainer of the Arch MinGW packages, myself and others could
> look into making an actual fork on Github or Bitbucket where we merge the
> various patches and come up with an out-of-the-box MinGW-[cross-]compilable
> version of CPython. I'll happily write the spec files to get this building
> from Fedora or openSUSE. That would help us test the feasibility from a
> centralized repository. Ray, what do you think? Do you know xantares' email
> address to ask if he'd be interested in helping or using the result?

I like this idea. To reduce the workload, we should probably pick
Python3 (at least initially)?

I have collaborated with xantares on the ArchLinux AUR
mingw-w64-python2 package (whom I've bcc'ed). In fact, as of now, our
patches are exactly the same except ArchLinux is missing one new
patch. Looking at
https://aur.archlinux.org/packages/mingw-w64-python2/ it seems
xantares handed over maintainer-ship to Dr Shadow. I've left a comment
asking for the new maintainer to email me.

If we pick Python3 instead of 2 then bringing up an ArchLinux AUR
package for that would be my next course of action. Cross-compilation
of mingw-w64-python3 will no doubt need some fixes as I've not done it
for a while.

Ideally, we'd hook this repository up to as complete a CI system as
possible and introduce each patch one at a time so that any and every
breakage or regression on the currently supported systems gets fixed
immediately. Also having reviews from some core Python developers (if
we can get motivated supporters from that group) would be immensely
helpful. My fear is that without such core involvement, the attempt to
upstream the final patch-set would be overwhelming.

>
> Zachary Ware:
>> I'm failing to see where that's simpler :)
>
> If it were hypothetically merged instead of out in an external fork, it
> could be ./configure --host=x86_64-w64-mingw32 && make to cross-compile
> from Linux or Cygwin, or just ./configure && make after installing MSYS2
> (which is just about as easy as installing MSVC) on Windows.
>
> Paul Moore:
>> If it were possible to cross-compile compatible extensions on Linux,
>> projects developed on Linux could supply Windows binaries much more
>> easily, which would be a huge benefit to the whole Windows Python
>> community.
>
> I want to do exactly this in an automated repeatable way, preferably on
> a build service. This seems harder to do when CPython cannot itself be
> built and handled as a dependency by that same automated, repeatable
> build service. Unless it becomes possible to cross-compile extensions
> using the build machine's own version of Python, which might be the right
> approach.
>
>> acutely aware of the common pain points for Python users on Windows.
>> And they are all about binary extensions, and none at all about
>> building Python itself.
>
> I've done a lot of recent work keeping Julia working well on Windows, and
> the interoperability we have with Python packages has propagated most of
> these pain points to us as well. We have to rely on Conda in order to have
> a reliable way of installing, as an example, IPython with the notebook
> interface, in order for IJulia to work. This is not an ideal solution as it
> requires a great deal of user intervention and manual steps to get up and
> running (and it would be far worse without Conda). We are, so far, built
> around MinGW-w64 on Windows, for the reasons I listed above. Having cross-
> compiled builds of CPython and binary extensions available from the same
> build services we already use to install other binary packages (Gtk, Cairo,
> Tk, Nettle, HDF5, etc) on Windows would be enormously helpful for us.
>
> There's a real use case. Its size and importance can be debated. For now
> I'll take David Murray's post to heart and see where I have time or ability
> to help things along.
>
> Sincerely,
> Tony
>

From rdmurray at bitdance.com  Sun Oct 26 16:48:34 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 26 Oct 2014 11:48:34 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
Message-ID: <20141026154835.8D7D1250167@webabinitio.net>

On Sun, 26 Oct 2014 06:12:45 -0700, "Tony Kelman" <kelman at berkeley.edu> wrote:
> Steve Dower:
> > Building CPython for Windows is not something that needs solving.
> 
> Not in your opinion, but numerous packagers of MinGW-based native or
> cross-compiled package sets would love to include Python. The fact
> that they currently can't, without many patches, is a problem.

If this includes (or would likely include) a significant portion of the
Scientific Computing community, I would think that would be a compelling
use case.  We'd need to be educated more about the reasons why this
approach works better than remaining compatible with MSVC CPython so we
could evaluate the risks and reward intelligently.  (I wonder..."things
are going to fragment anyway if you (cpython) don't do anything" might
be an argument, if true...but wouldn't make the consequences any
easier to deal with :(

But as has been discussed, it seems better to focus first on fixing the
issues on which we are all in agreement (building extensions with MinGW).

> R. David Murray:
> > And, at this point, we would NEED A BUILDBOT.  That is, a machine that
> > has whatever tools are required installed such that tests added to the
> > test suite to test MinGW support in distutils would run, so we can be
> > sure we don't break anything when making other changes.
> 
> That's not too hard. I've done this for other projects. AppVeyor works if
> your build is short enough, and I've done cross-compilation from Travis
> CI for other projects. Or Jenkins, or a Vagrant VM. I don't know PSF's
> infrastructure, but I can offer guidance if it would help.

When I say "we need a buildbot", what I mean is that we need someone
willing to donate the resources and the *time and expertise* to setting
up and maintaining something that integrates with our existing buildbot
setup.  You set up a buildbot slave, request an id and password from
Antoine, keep the thing running, and respond in a timely fashion to
requests for help resolving issues that arise on the buildbot (both
buildbot issues and help-me-diagnose-this-failure issues).  After the
initial setup the load isn't generally heavy (I haven't had to do
anything with the OSX buildbot running on the machine in my dinning room
for months and months now, for example).

So your guidance would have to go to someone who was volunteering to
take on this task...there isn't anyone on the existing core team who
would have time to do it (if I'm wrong, I'm sure someone will speak up).
On the other hand, you don't have to be a committer to run a buildbot,
and there *are* people on the core-mentorship list who have expressed
interest in helping out with our automated testing infrastructure,
including (if I understand correctly) adding some level of integration
to other CI systems (which might just be messages to the IRC
channel)[*].  So that could be a fruitful avenue to explore.

--David

[*] This is an area in which I have an interest, but it hasn't gotten
high enough on my todo list yet for me to figure out exactly what the
current state of things is so I can help it along.

From mingw.android at gmail.com  Sun Oct 26 16:43:28 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 15:43:28 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
Message-ID: <CAOYw7dtfX+r3JJj3yyW7Kd5xDMZ+FRQU0POEmC_7ERj+86g8fw@mail.gmail.com>

On Sun, Oct 26, 2014 at 2:28 PM, Ray Donnelly <mingw.android at gmail.com> wrote:
> On Sun, Oct 26, 2014 at 1:12 PM, Tony Kelman <kelman at berkeley.edu> wrote:
>> Thanks all for the responses. Clearly this is a subject about which
>> people feel strongly, so that's good at least. David Murray's guidance
>> in particular points to the most likely path to get improvements to
>> really happen.
>>
>> Steve Dower:
>>> Building CPython for Windows is not something that needs solving.
>>
>> Not in your opinion, but numerous packagers of MinGW-based native or
>> cross-compiled package sets would love to include Python. The fact
>> that they currently can't, without many patches, is a problem.
>>
>>> The culture on Windows is to redistribute binaries, not source,
>>
>> There are many cultures using Windows. Including open-source ones.
>>
>>> and both the core team and a number of redistributors have this figured
>>> out (and it will only become easier with VC14 and Python 3.5).
>>
>> With MSVC. It doesn't work with MinGW, it likely doesn't work with Clang.
>> MSVC is not the only compiler on Windows. There are many use cases for
>> preferring other compilers. Have you read this wiki page for example?
>> https://github.com/numpy/numpy/wiki/Numerical-software-on-Windows
>>
>> In my personal experience, having recently gotten Julia to compile using
>> MSVC for the first time, MSVC as a compiler is highly deficient for many
>> needs especially in the scientific software community:
>> - C99 (getting better recently, but still not done)
>> - AT&T syntax assembly
>> - C++11 features (also mostly okay now, but not if you're using an older
>>   MSVC version with Python 2.7, which many people still have to do)
>> - 128-bit integer intrinsics
>> - cannot cross-compile from anything that isn't Windows
>> - build systems foreign relative to shell/makefile systems used by most
>>   open-source projects, few projects have time to maintain 2 separate build
>>   systems (cmake helps but takes a lot of initial effort to convert to)
>> - no free-as-in-beer Fortran compiler available
>>
>> I have none of these problems when I use MinGW-w64. Hence the desire to
>> be able to curate an all-MinGW software stack. It's not a matter of open-
>> source ideology for me, it's brass tacks "can I do the work I need to do."
>> With MSVC I can't, with MinGW-w64 I can. Not being able to include CPython
>> in an all-MinGW stack hurts, a lot.
>>
>> Only cross-compilation and the build system in the above list are relevant
>> to CPython, but I hope I have convinced you, Paul Moore, etc. that there are
>> real reasons for some groups of users and developers to prefer MinGW-w64
>> over MSVC.
>>
>>> I'd rather see this effort thrown behind compiling extensions,
>>> including cross compilation.
>>
>> There are patches awaiting review that improve this as well. Efforts to
>> improve CPython's build system and the handling of extensions are not
>> completely independent, in many cases the patches are written by the same
>> set of MinGW users. One of these sets of patches is not inherently evil,
>> you understandably have less interest in them but it's still disappointing
>> to see so little movement on either.
>>
>>> Having different builds of CPython out there will only fragment the
>>> community and hurt extension authors far more than it may seem to help.
>>
>> The community of people developing and using open-source projects, either
>> CPython or otherwise, is already highly fragmented. Ignoring it makes it
>> worse. python.org does not have to distribute or endorse MinGW-compiled
>> builds of CPython. If the build option never gets incorporated, then it
>> will continue to be reverse-engineered.
>>
>> Guido van Rossum:
>>> Here's the crux of the matter. We want compiled extension modules
>>> distributed via PyPI to work with the binaries distributed from
>>> python.org.
>>
>> Absolutely. I don't think additional options in the build system would
>> change this.
>>
>> R. David Murray:
>>> And, at this point, we would NEED A BUILDBOT.  That is, a machine that
>>> has whatever tools are required installed such that tests added to the
>>> test suite to test MinGW support in distutils would run, so we can be
>>> sure we don't break anything when making other changes.
>>
>> That's not too hard. I've done this for other projects. AppVeyor works if
>> your build is short enough, and I've done cross-compilation from Travis
>> CI for other projects. Or Jenkins, or a Vagrant VM. I don't know PSF's
>> infrastructure, but I can offer guidance if it would help.
>>
>> Steve Dower:
>>> I'm afraid of users having numpy crash because they're using an MSVC
>>> CPython instead of a mingw CPython. I'm afraid of users not being able
>>> to use library A and library B at the same time because A requires MSVC
>>> CPython and B requires mingw CPython. (I can produce more examples if you
>>> like, but the general concern is having a fragmented community, as I said
>>> in my previous post.)
>>
>> A valid fear. Mixing C runtimes can cause problems, I've seen this myself.
>> Correct me if I'm wrong, but this is nearly as much of an issue if someone
>> wants to use a different version of MSVC to compile CPython than the version
>> used to build the official binaries. It requires care, but you can't deny
>> that there are use cases where people will want and need to do such things.
>> Is possible fragmentation a good enough reason to resist making it possible
>> in the build system?
>>
>>> though I suspect most would like to see some traction achieved on a fork
>>> first
>>
>> Those of us who consider this important should probably just do this. Ray,
>> Roumen, the maintainer of the Arch MinGW packages, myself and others could
>> look into making an actual fork on Github or Bitbucket where we merge the
>> various patches and come up with an out-of-the-box MinGW-[cross-]compilable
>> version of CPython. I'll happily write the spec files to get this building
>> from Fedora or openSUSE. That would help us test the feasibility from a
>> centralized repository. Ray, what do you think? Do you know xantares' email
>> address to ask if he'd be interested in helping or using the result?
>
> I like this idea. To reduce the workload, we should probably pick
> Python3 (at least initially)?
>
> I have collaborated with xantares on the ArchLinux AUR
> mingw-w64-python2 package (whom I've bcc'ed). In fact, as of now, our
> patches are exactly the same except ArchLinux is missing one new
> patch. Looking at
> https://aur.archlinux.org/packages/mingw-w64-python2/ it seems
> xantares handed over maintainer-ship to Dr Shadow. I've left a comment
> asking for the new maintainer to email me.
>
> If we pick Python3 instead of 2 then bringing up an ArchLinux AUR
> package for that would be my next course of action. Cross-compilation
> of mingw-w64-python3 will no doubt need some fixes as I've not done it
> for a while.

It seems that Dr Shadow has already done this:
https://aur.archlinux.org/packages/mingw-w64-python/ and, happily, the
used patches are the exact same ones as used on MSYS2.

>
> Ideally, we'd hook this repository up to as complete a CI system as
> possible and introduce each patch one at a time so that any and every
> breakage or regression on the currently supported systems gets fixed
> immediately. Also having reviews from some core Python developers (if
> we can get motivated supporters from that group) would be immensely
> helpful. My fear is that without such core involvement, the attempt to
> upstream the final patch-set would be overwhelming.
>
>>
>> Zachary Ware:
>>> I'm failing to see where that's simpler :)
>>
>> If it were hypothetically merged instead of out in an external fork, it
>> could be ./configure --host=x86_64-w64-mingw32 && make to cross-compile
>> from Linux or Cygwin, or just ./configure && make after installing MSYS2
>> (which is just about as easy as installing MSVC) on Windows.
>>
>> Paul Moore:
>>> If it were possible to cross-compile compatible extensions on Linux,
>>> projects developed on Linux could supply Windows binaries much more
>>> easily, which would be a huge benefit to the whole Windows Python
>>> community.
>>
>> I want to do exactly this in an automated repeatable way, preferably on
>> a build service. This seems harder to do when CPython cannot itself be
>> built and handled as a dependency by that same automated, repeatable
>> build service. Unless it becomes possible to cross-compile extensions
>> using the build machine's own version of Python, which might be the right
>> approach.
>>
>>> acutely aware of the common pain points for Python users on Windows.
>>> And they are all about binary extensions, and none at all about
>>> building Python itself.
>>
>> I've done a lot of recent work keeping Julia working well on Windows, and
>> the interoperability we have with Python packages has propagated most of
>> these pain points to us as well. We have to rely on Conda in order to have
>> a reliable way of installing, as an example, IPython with the notebook
>> interface, in order for IJulia to work. This is not an ideal solution as it
>> requires a great deal of user intervention and manual steps to get up and
>> running (and it would be far worse without Conda). We are, so far, built
>> around MinGW-w64 on Windows, for the reasons I listed above. Having cross-
>> compiled builds of CPython and binary extensions available from the same
>> build services we already use to install other binary packages (Gtk, Cairo,
>> Tk, Nettle, HDF5, etc) on Windows would be enormously helpful for us.
>>
>> There's a real use case. Its size and importance can be debated. For now
>> I'll take David Murray's post to heart and see where I have time or ability
>> to help things along.
>>
>> Sincerely,
>> Tony
>>

From arigo at tunes.org  Sun Oct 26 18:44:53 2014
From: arigo at tunes.org (Armin Rigo)
Date: Sun, 26 Oct 2014 18:44:53 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
Message-ID: <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>

Hi Stefan,

On 26 October 2014 02:50, Stefan Richthofer <Stefan.Richthofer at gmx.de> wrote:
> It appears weakrefs are only cleared if this is done by gc (where no
> resurrection can happen anyway). If a resurrection-performing-__del__ is
> just called by ref-count-drop-to-0, weakrefs persist -

How do you reach this conclusion?  The following test program seems to
show the opposite, by printing None on Python 2.7.6:

    import weakref
   class X(object):
        def __del__(self):
            print ref()
    x = X()
    ref = weakref.ref(x)
    del x


A bient?t,

Armin.

From kelman at berkeley.edu  Sun Oct 26 18:59:16 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Sun, 26 Oct 2014 10:59:16 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dtfX+r3JJj3yyW7Kd5xDMZ+FRQU0POEmC_7ERj+86g8fw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung><f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com><CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com><CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com><E648C5C014E74B459A705C98324E0A4D@TKsamsung><CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
 <CAOYw7dtfX+r3JJj3yyW7Kd5xDMZ+FRQU0POEmC_7ERj+86g8fw@mail.gmail.com>
Message-ID: <9A0205FD440A4CA1A3D124F3D2BFDE56@TKsamsung>

> If this includes (or would likely include) a significant portion of the
> Scientific Computing community, I would think that would be a compelling
> use case.

I can't speak for any of the scientific computing community besides myself,
but my thoughts: much of the development, as you know, happens on Linux
with GCC (or OSX with clang). But it's important for users across all
platforms to be able to install binaries with a minimum of fuss.
Limitations of MSVC have already led the numpy/scipy community to
investigate building with MinGW-w64. (See several related threads from
April on the numpy-discussion list.) Ensuring compatibility with CPython's
chosen msvcrt has made that work even more difficult for them.

And Julia is not yet a significant portion of anything, but our community
is growing rapidly. See https://github.com/JuliaLang/IJulia.jl/pull/211 -
with respect to IJulia, "Python is just an implementation detail." Even
such a silly thing as automating the execution of the Python installer, to
set up a private only-used-by-IJulia copy, is needlessly difficult to do.
The work on Jupyter will hopefully help this specific situation sooner or
later, but there are other cases where CPython needs to serve as part of
the infrastructure, and the status quo makes that harder to automate.

> We'd need to be educated more about the reasons why this approach works
> better than remaining compatible with MSVC CPython so we could evaluate
> the risks and reward intelligently.

Ideally, we can pursue an approach that would be able to remain compatible
with MSVC CPython. Even if this needs involvement from MinGW-w64 to make
happen, I don't think it's intractable. But I know less about the inner
details of CPython than you do so I could be wrong.

> But as has been discussed, it seems better to focus first on fixing the
> issues on which we are all in agreement (building extensions with MinGW).

Yes. We'll look into how much of the work has already been done on this.

> there *are* people on the core-mentorship list who have expressed
> interest in helping out with our automated testing infrastructure,
> including (if I understand correctly) adding some level of integration
> to other CI systems (which might just be messages to the IRC
> channel)[*].  So that could be a fruitful avenue to explore.

If we pursue a fork (which not everyone will like but might happen anyway)
then we likely would do this type of CI integration along the way as Ray
suggested. So even if it turns out to fail as an endeavor, some good may
come of it.

Sincerely,
Tony


From p.f.moore at gmail.com  Sun Oct 26 23:41:39 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Oct 2014 22:41:39 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
Message-ID: <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>

On 26 October 2014 13:12, Tony Kelman <kelman at berkeley.edu> wrote:
> Only cross-compilation and the build system in the above list are relevant
> to CPython, but I hope I have convinced you, Paul Moore, etc. that there are
> real reasons for some groups of users and developers to prefer MinGW-w64
> over MSVC.

Not really, to be honest. I still don't understand why anyone not
directly involved in CPython development would need to build their own
Python executable on Windows. Can you explain a single specific
situation where installing and using the python.org executable is not
possible (on the assumption that the mingw build is functionally
identical and ABI compatible with the CPython build, the claim being
made here)? Note that "not possible" is different from "I don't want
to" or "it doesn't fit my views about free software" or similar. Also
note that building extensions is different - you have to assume that
building extensions using mingw with a mingw-built CPython is just as
hard as building them with a MSVC-built CPython (otherwise you've made
changes to extension building and you should contribute them
independently so that everyone can benefit, not just those who build
their own Python with mingw!)

> Paul Moore:
>> If it were possible to cross-compile compatible extensions on Linux,
>> projects developed on Linux could supply Windows binaries much more
>> easily, which would be a huge benefit to the whole Windows Python
>> community.
>
> I want to do exactly this in an automated repeatable way, preferably on
> a build service. This seems harder to do when CPython cannot itself be
> built and handled as a dependency by that same automated, repeatable
> build service.

I cannot see why you would need to build Python in order to build
extensions. After all, if your build service is on Linux, it couldn't
run a mingw-built Python anyway. If your build service is a Windows
machine, just install the python.org binaries (which is a simple
download and install, that can be fully automated, but which is a
one-off process anyway).

> Unless it becomes possible to cross-compile extensions
> using the build machine's own version of Python, which might be the right
> approach.

This may be where we are getting confused. I see this as the only
practical way of cross-compiling Windows extensions on Linux, by using
the Linux Python. So being able to cross-compile Python is not
relevant.

On a tangential note, any work on supporting mingw builds and
cross-compilation should probably be done using setuptools, so that it
is external to the CPython code. That way (a) it isn't constrained by
the CPython release schedules and backward compatibility constraints,
and (b) it can be used in older versions of Python (which is pretty
much essential if it's to be useful, TBH).

Paul

From p.f.moore at gmail.com  Sun Oct 26 23:51:54 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Oct 2014 22:51:54 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <9A0205FD440A4CA1A3D124F3D2BFDE56@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
 <CAOYw7dtfX+r3JJj3yyW7Kd5xDMZ+FRQU0POEmC_7ERj+86g8fw@mail.gmail.com>
 <9A0205FD440A4CA1A3D124F3D2BFDE56@TKsamsung>
Message-ID: <CACac1F9VaRMCbvyJPM-RjMMv18qsU8S4JtOANoU4W=yuqrMcMw@mail.gmail.com>

On 26 October 2014 17:59, Tony Kelman <kelman at berkeley.edu> wrote:
> Ensuring compatibility with CPython's
> chosen msvcrt has made that work even more difficult for them.

Ensuring compatibility with CPython's msvcrt is mandatory unless you
want to create a split in the community over which extensions work
with which builds. That's precisely the scenario Steve Dower and
myself (among others) fear, and want to avoid at all cost.

Paul

From p.f.moore at gmail.com  Sun Oct 26 23:58:34 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Oct 2014 22:58:34 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CAOYw7dvTgZj9G8vkZhB9TAcNAQ2xQCxjwLbQwKDa1dT--mFFMw@mail.gmail.com>
Message-ID: <CACac1F-mn4PxdUzd=U_QKnELkmMHcjoJsCo091aPC-K4nEWKvw@mail.gmail.com>

On 26 October 2014 14:28, Ray Donnelly <mingw.android at gmail.com> wrote:
> I like this idea. To reduce the workload, we should probably pick
> Python3 (at least initially)?

Aren't the existing patches on the tracker already for Python 3.5+?
They should be, as that's the only version that's likely to be a
possible target (unless you get someone to agree to allow a change
like this as in scope for Pythhon 2.7, which I've seen no indication
of). Maybe I'm confused here.

Paul

From mingw.android at gmail.com  Mon Oct 27 00:11:58 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Sun, 26 Oct 2014 23:11:58 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
Message-ID: <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>

On Sun, Oct 26, 2014 at 10:41 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 13:12, Tony Kelman <kelman at berkeley.edu> wrote:
>> Only cross-compilation and the build system in the above list are relevant
>> to CPython, but I hope I have convinced you, Paul Moore, etc. that there are
>> real reasons for some groups of users and developers to prefer MinGW-w64
>> over MSVC.
>
> Not really, to be honest. I still don't understand why anyone not
> directly involved in CPython development would need to build their own
> Python executable on Windows. Can you explain a single specific
> situation where installing and using the python.org executable is not
> possible (on the assumption that the mingw build is functionally
> identical and ABI compatible with the CPython build, the claim being
> made here)?

I don't know where this "ABI compatible" thing came into being; I
think Steve Dower eluded to it by stating that we should focus on
enabling MinGW-w64 as an extension-building compiler, using a core
interpreter built with MSVC, and that by limiting the interfaces to
the Windows C calling conventions everything would be OK.
Unfortunately this is not possible. MinGW-w64-built extensions need to
link to msvcrt.dll to do anything useful and you cannot mix two
different msvcr??.dlls in one application. Please see
http://msdn.microsoft.com/en-us/library/abx4dbyh%28v=VS.100%29.aspx
and http://msdn.microsoft.com/en-us/library/ms235460%28v=VS.100%29.aspx
for the details. MinGW-w64 assumes the very old msvcrt.dll files from
Windows XP SP3 and XP64 specifically to avoid this mess. The rest of
your reply assumes that this ABI compatibility is a given so I'll stop
at this point.

> Note that "not possible" is different from "I don't want
> to" or "it doesn't fit my views about free software" or similar. Also
> note that building extensions is different - you have to assume that
> building extensions using mingw with a mingw-built CPython is just as
> hard as building them with a MSVC-built CPython (otherwise you've made
> changes to extension building and you should contribute them
> independently so that everyone can benefit, not just those who build
> their own Python with mingw!)
>
>> Paul Moore:
>>> If it were possible to cross-compile compatible extensions on Linux,
>>> projects developed on Linux could supply Windows binaries much more
>>> easily, which would be a huge benefit to the whole Windows Python
>>> community.
>>
>> I want to do exactly this in an automated repeatable way, preferably on
>> a build service. This seems harder to do when CPython cannot itself be
>> built and handled as a dependency by that same automated, repeatable
>> build service.
>
> I cannot see why you would need to build Python in order to build
> extensions. After all, if your build service is on Linux, it couldn't
> run a mingw-built Python anyway. If your build service is a Windows
> machine, just install the python.org binaries (which is a simple
> download and install, that can be fully automated, but which is a
> one-off process anyway).
>
>> Unless it becomes possible to cross-compile extensions
>> using the build machine's own version of Python, which might be the right
>> approach.
>
> This may be where we are getting confused. I see this as the only
> practical way of cross-compiling Windows extensions on Linux, by using
> the Linux Python. So being able to cross-compile Python is not
> relevant.
>
> On a tangential note, any work on supporting mingw builds and
> cross-compilation should probably be done using setuptools, so that it
> is external to the CPython code. That way (a) it isn't constrained by
> the CPython release schedules and backward compatibility constraints,
> and (b) it can be used in older versions of Python (which is pretty
> much essential if it's to be useful, TBH).
>
> Paul

From kelman at berkeley.edu  Mon Oct 27 00:24:20 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Sun, 26 Oct 2014 16:24:20 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung><f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com><CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com><CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com><E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
Message-ID: <D7B19369FBE14DF4B9A5BC58A4C2098A@TKsamsung>

> Not really, to be honest. I still don't understand why anyone not
> directly involved in CPython development would need to build their own
> Python executable on Windows. Can you explain a single specific
> situation where installing and using the python.org executable is not
> possible

I want, and in many places *need*, an all-MinGW stack. For a great deal
of software that is not Python, I can do this today. I can use build
services, package management, and dependency resolution tools that work
very well together with this all-MinGW software stack. These are problems
that Python has notoriously struggled with on Windows for a long time.
It's not "my views on free software," it's the reality of MSVC being a
near-useless compiler for scientific software. (And I don't see that
changing much.) Do my requirements conflict with many non-scientific
Python users on Windows? Probably. So you're welcome to ignore my
requirements and I'll do my own thing, but I don't think I'm alone.
There's likely no desire from the scientific Python community to branch
off and separate in quite the way I'm willing to do from non-scientific
Python, but it would solve some of their problems (while introducing many
others). I suspect a MinGW-w64-oriented equivalent to Conda would be
attractive to many. That's approximately what I'm aiming for.

There are some ways in which I can use the python.org MSVC executable and
installer. But it is nearly impossible for me to integrate it into the rest
of the tools and stack that I am using; it sticks out like a sore thumb.
Likewise MinGW-compiled CPython may stick out like a sore thumb relative
to the existing way things work with Python on Windows. I'm okay with that,
you probably aren't.

> changes to extension building and you should contribute them
> independently so that everyone can benefit

Noted.

> I cannot see why you would need to build Python in order to build
> extensions.

No, of course they are separate. CPython is one of my dependencies.
Compiled extensions are other dependencies. Software completely unrelated
to Python is yet another set of dependencies. It's not a very coherent
stack if I can't handle all of these dependencies in a uniform way.

> On a tangential note, any work on supporting mingw builds and
> cross-compilation should probably be done using setuptools, so that it
> is external to the CPython code.

Noted.

Sincerely,
Tony


From p.f.moore at gmail.com  Mon Oct 27 00:37:25 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Oct 2014 23:37:25 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <D7B19369FBE14DF4B9A5BC58A4C2098A@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <D7B19369FBE14DF4B9A5BC58A4C2098A@TKsamsung>
Message-ID: <CACac1F96gZ03PvysSfX13B1ePtQ=KTO6CMy0Sr9c4H5NxVxG4g@mail.gmail.com>

On 26 October 2014 23:24, Tony Kelman <kelman at berkeley.edu> wrote:
> I want, and in many places *need*, an all-MinGW stack.

OK, I'm willing to accept that statement. But I don't understand it,
and I don't think you've explained why you *need* your CPython
interpreter to be compiled with mingw (as opposed to a number of other
things you might need around building extensions). You may well "need"
a mingw-compiled CPython because no-one has yet fixed the issues
around using mingw to build extensions for the python.org python
build. But that's my point - I'd rather "they" fixed that issue,
rather than perpetuating your need for a non-standard compiler that
uses extensions no-one else can use.

Paul

From p.f.moore at gmail.com  Mon Oct 27 00:44:06 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 26 Oct 2014 23:44:06 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
Message-ID: <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>

On 26 October 2014 23:11, Ray Donnelly <mingw.android at gmail.com> wrote:
> I don't know where this "ABI compatible" thing came into being;

Simple. If a mingw-built CPython doesn't work with the same extensions
as a MSVC-built CPython, then the community gets fragmented (because
you can only use the extensions built for your stack). Assuming numpy
needs mingw and ultimately only gets built for a mingw-compiled Python
(because the issues building for MSVC-built Python are too hard) and
assuming that nobody wants to make the effort to build pywin32 under
mingw, then what does someone who needs both numpy and pywin32 do?

Avoiding that issue is what I mean by ABI-compatible. (And that's all
I mean by it, nothing more subtle or controversial).

I view it as critical (because availability of binaries is *already*
enough of a problem in the Windows world, without making it worse)
that we avoid this sort of fragmentation. I'm not seeing an
acknowledgement from the mingw side that they agree. That's my
concern. If we both agree, there's nothing to argue about.

Paul

From martin at v.loewis.de  Mon Oct 27 00:52:28 2014
From: martin at v.loewis.de (martin at v.loewis.de)
Date: Mon, 27 Oct 2014 00:52:28 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
Message-ID: <20141027005228.Horde.iBp3Vlgrn9AY2Zm9m-V4KQ2@webmail.df.eu>


Zitat von Tony Kelman <kelman at berkeley.edu>:

> A maintainer has volunteered. Others will help. Can any core developers
> please begin reviewing some of his patches?

Unfortunately, every attempt to review these patches has failed for me,
every time. In the last iteration of an attempt to add mingw64 support,
I had asked contributors to also provide instructions on how to use these
patches, and haven't received any instructions that actually worked.

I'm hesitant to add code that I cannot verify as actually working.

I guess anybody else reviewing these patches ran into similar problems
(I know some other core developers have tried reviewing them as well,
others have stated here that they are unable to review the patches).

Regards,
Martin



From ncoghlan at gmail.com  Mon Oct 27 13:12:16 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Oct 2014 22:12:16 +1000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
Message-ID: <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>

On 27 October 2014 09:44, Paul Moore <p.f.moore at gmail.com> wrote:
> I view it as critical (because availability of binaries is *already*
> enough of a problem in the Windows world, without making it worse)
> that we avoid this sort of fragmentation. I'm not seeing an
> acknowledgement from the mingw side that they agree. That's my
> concern. If we both agree, there's nothing to argue about.

I think there's consensus on this front. From Ray: "MinGW-w64 assumes
the very old msvcrt.dll files from
Windows XP SP3 and XP64 specifically to avoid this mess."

That assumption will allow MinGW-w64 to link with the appropriate
MSVCRT versions for extention building without anything breaking.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Mon Oct 27 13:30:31 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 27 Oct 2014 22:30:31 +1000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F96gZ03PvysSfX13B1ePtQ=KTO6CMy0Sr9c4H5NxVxG4g@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <D7B19369FBE14DF4B9A5BC58A4C2098A@TKsamsung>
 <CACac1F96gZ03PvysSfX13B1ePtQ=KTO6CMy0Sr9c4H5NxVxG4g@mail.gmail.com>
Message-ID: <CADiSq7ehATEjdCpeyk5aWciuYvO-9NNEmnBWOADtPZEuMhfVFQ@mail.gmail.com>

On 27 October 2014 09:37, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 23:24, Tony Kelman <kelman at berkeley.edu> wrote:
>> I want, and in many places *need*, an all-MinGW stack.
>
> OK, I'm willing to accept that statement. But I don't understand it,
> and I don't think you've explained why you *need* your CPython
> interpreter to be compiled with mingw (as opposed to a number of other
> things you might need around building extensions).

I can take a go at an explanation that may make more sense to you.
Consider one of our key requirements for packaging applications for
Fedora: that Fedora builds be *self-hosting*. Given a base Fedora
system, and a source RPM, we need to be able to *build the binary RPM
from source*. (Other Linux distros generally have a similar
requirement)

Relying on opaque binary blobs downloaded from the internet as part of
the build process is not permitted (modulo a few exceptions for
firmware blobs in device drivers).

Now consider that this "automatically rebuild the entire system from
source" model is not unique to Linux - you can use it for any system
where your build process is sufficiently automated, and you have a
need for it. However, the *structure* of those kind of automation
tends to differ wildly between POSIX style tooling (gcc, clang) and
MSVC. If you have an existing build automation system for *nix
targets, then cross-compilation via MinGW is likely going to be your
smoothest path to adding Windows binary support.

At that point, if CPython is one of your dependencies, you're going to
have the choice of allowing the python.org binaries to be pulled in as
opaque pre-built blobs, or else figuring out how to build an ABI
compatible version with MinGW rather than with MSVC. Think of this
more in the case of *embedding* the CPython runtime in a larger
context (e.g. in Tony's case, to make Python software usable with the
Julia runtime), rather than in building a standalone Python
interpreter for general use.

So, for embedding cases, and for incorporation into POSIX-style build
systems using MinGW-w64 for cross-compilation of Windows binaries, it
may make sense to incorporate the patches that allow building with
MinGW-w64 into mainline CPython (if I recall correctly, we supported
building with Intel's C compiler for a long time, even though we never
shipped anything built with it).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Mon Oct 27 14:03:51 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 13:03:51 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CADiSq7ehATEjdCpeyk5aWciuYvO-9NNEmnBWOADtPZEuMhfVFQ@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <D7B19369FBE14DF4B9A5BC58A4C2098A@TKsamsung>
 <CACac1F96gZ03PvysSfX13B1ePtQ=KTO6CMy0Sr9c4H5NxVxG4g@mail.gmail.com>
 <CADiSq7ehATEjdCpeyk5aWciuYvO-9NNEmnBWOADtPZEuMhfVFQ@mail.gmail.com>
Message-ID: <CACac1F8FL2GJP3roy1YceP+o5rS15weYF+Yfx4N72ua5Y7vg9Q@mail.gmail.com>

On 27 October 2014 12:30, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> OK, I'm willing to accept that statement. But I don't understand it,
>> and I don't think you've explained why you *need* your CPython
>> interpreter to be compiled with mingw (as opposed to a number of other
>> things you might need around building extensions).
>
> I can take a go at an explanation that may make more sense to you.
> Consider one of our key requirements for packaging applications for
> Fedora: that Fedora builds be *self-hosting*. Given a base Fedora
> system, and a source RPM, we need to be able to *build the binary RPM
> from source*. (Other Linux distros generally have a similar
> requirement)
>
> Relying on opaque binary blobs downloaded from the internet as part of
> the build process is not permitted (modulo a few exceptions for
> firmware blobs in device drivers).
>
> Now consider that this "automatically rebuild the entire system from
> source" model is not unique to Linux - you can use it for any system
> where your build process is sufficiently automated, and you have a
> need for it. However, the *structure* of those kind of automation
> tends to differ wildly between POSIX style tooling (gcc, clang) and
> MSVC. If you have an existing build automation system for *nix
> targets, then cross-compilation via MinGW is likely going to be your
> smoothest path to adding Windows binary support.
>
> At that point, if CPython is one of your dependencies, you're going to
> have the choice of allowing the python.org binaries to be pulled in as
> opaque pre-built blobs, or else figuring out how to build an ABI
> compatible version with MinGW rather than with MSVC. Think of this
> more in the case of *embedding* the CPython runtime in a larger
> context (e.g. in Tony's case, to make Python software usable with the
> Julia runtime), rather than in building a standalone Python
> interpreter for general use.
>
> So, for embedding cases, and for incorporation into POSIX-style build
> systems using MinGW-w64 for cross-compilation of Windows binaries, it
> may make sense to incorporate the patches that allow building with
> MinGW-w64 into mainline CPython (if I recall correctly, we supported
> building with Intel's C compiler for a long time, even though we never
> shipped anything built with it).

Thanks Nick. That explanation makes sense to me. I was aware of this
sort of scenario, and as I've said before I don't have any objection
per se to making things easier for people with that sort of
requirement. But some of the other arguments in this thread seemed to
imply more than that. Without specifics, though, I concede that I may
be over-interpreting the rhetoric, so that's the part of the debate
I'm stepping back from, to avoid descending into FUD.

Paul

From stefan.richthofer at gmx.de  Mon Oct 27 14:36:31 2014
From: stefan.richthofer at gmx.de (Stefan Richthofer)
Date: Mon, 27 Oct 2014 14:36:31 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
 <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>
Message-ID: <544E4A5F.4000703@gmx.de>

Your test program performs no resurrection of x.

Interestingly, it does not change behavior if you write

class X(object):
     def __del__(self):
         X.x = self
         print ref()

(Thanks for making me aware of this! My test-case was already
initially the more complex one given below)

But if the resurrection occurs indirectly, the weakref persists:
(I refined it to old-style class, because Jython will support new-style
class finalizers only from 2.7. beta 4 onwards, i.e. the test would be
pointless with any current release)

import weakref, time, gc
class ReferentDummy():
     pass

class X():
     def __del__(self):
         X.y = self.z
         print "__del__: "+str(ref())

x = X()
x2 = ReferentDummy()
ref = weakref.ref(x2)
x.z = x2
del x2
del x #Everything is now deleted, isn't it?
gc.collect() #needed in Jython-case
time.sleep(0.2) #wait for Java's async gc to finnish
print ref()
print weakref.getweakrefs(X.y)


---------------CPython output:
__del__: <__main__.ReferentDummy instance at 0x7fd2603e1950>
<__main__.ReferentDummy instance at 0x7fd2603e1950>
[<weakref at 0x7fd2603d2c00; to 'instance' at 0x7fd2603e1950>]

---------------Jython 2.7 beta 3 output:
__del__: None
None
[]

One can surely argue x2 has never been dead, or see it as "it was killed 
along with x and
then resurrected by x". Jython clearly takes the second point of view 
and also clears weakrefs
to x.z, while CPython does not. Yes, these details probably hardly 
matter in practice (however
could cause subtle bugs when porting complex stuff from CPython to 
Jython), but since I try to
bridge it, I have to look into this more carefully.

Best,

Stefan



On 10/26/2014 06:44 PM, Armin Rigo wrote:
> Hi Stefan,
>
> On 26 October 2014 02:50, Stefan Richthofer <Stefan.Richthofer at gmx.de> wrote:
>> It appears weakrefs are only cleared if this is done by gc (where no
>> resurrection can happen anyway). If a resurrection-performing-__del__ is
>> just called by ref-count-drop-to-0, weakrefs persist -
> How do you reach this conclusion?  The following test program seems to
> show the opposite, by printing None on Python 2.7.6:
>
>      import weakref
>     class X(object):
>          def __del__(self):
>              print ref()
>      x = X()
>      ref = weakref.ref(x)
>      del x
>
>
> A bient?t,
>
> Armin.


From solipsis at pitrou.net  Mon Oct 27 15:14:22 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Oct 2014 15:14:22 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
 <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>
 <544E4A5F.4000703@gmx.de>
Message-ID: <20141027151422.3e6e0e48@fsol>

On Mon, 27 Oct 2014 14:36:31 +0100
Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
> Your test program performs no resurrection of x.
> 
> Interestingly, it does not change behavior if you write
> 
> class X(object):
>      def __del__(self):
>          X.x = self
>          print ref()
> 
> (Thanks for making me aware of this! My test-case was already
> initially the more complex one given below)
> 
> But if the resurrection occurs indirectly, the weakref persists:

It's not that resurrection occurs indirectly, it's that the object
pointed to by "x2" always remains alive (first as an instance attribute
of x, second as a class attribute of X *before x is deleted*).

Regards

Antoine.



From p.f.moore at gmail.com  Mon Oct 27 15:18:26 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 14:18:26 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <20141025232439.C6214250D5B@webabinitio.net>
 <20141026013036.4a6be805@fsol>
 <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>
Message-ID: <CACac1F_4=sB0kdzkT28WHhGLjLjp-pZ0=bHdvPxN8wd+n4NDbw@mail.gmail.com>

On 26 October 2014 01:05, Ray Donnelly <mingw.android at gmail.com> wrote:
> Download and run:
> http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20141003.exe/download

Sending this offline because I really don't want to start up another
extended debate, but is there a version of this that I can use that

From p.f.moore at gmail.com  Mon Oct 27 15:19:23 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 14:19:23 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F_4=sB0kdzkT28WHhGLjLjp-pZ0=bHdvPxN8wd+n4NDbw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <20141025232439.C6214250D5B@webabinitio.net>
 <20141026013036.4a6be805@fsol>
 <CAOYw7dsQ_zbL3CmHNDYgvO5q=LqytYfWOEYOT=45b4frYd0_Qg@mail.gmail.com>
 <CACac1F_4=sB0kdzkT28WHhGLjLjp-pZ0=bHdvPxN8wd+n4NDbw@mail.gmail.com>
Message-ID: <CACac1F-00Rs8mPZ3jp-dvEbiKGS-OHE4PhWFBG=vJE1u7bXdag@mail.gmail.com>

Please ignore this. I hit the wrong button.

On 27 October 2014 14:18, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 01:05, Ray Donnelly <mingw.android at gmail.com> wrote:
>> Download and run:
>> http://sourceforge.net/projects/msys2/files/Base/x86_64/msys2-x86_64-20141003.exe/download
>
> Sending this offline because I really don't want to start up another
> extended debate, but is there a version of this that I can use that

From stefan.richthofer at gmx.de  Mon Oct 27 16:20:06 2014
From: stefan.richthofer at gmx.de (Stefan Richthofer)
Date: Mon, 27 Oct 2014 16:20:06 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <20141027151422.3e6e0e48@fsol>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
 <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>
 <544E4A5F.4000703@gmx.de> <20141027151422.3e6e0e48@fsol>
Message-ID: <544E62A6.2070906@gmx.de>

 >It's not that resurrection occurs indirectly, it's that the object
 >pointed to by "x2" always remains alive

Yes, this is right for CPython, more precisely this is about the definition
of the word "resurrection" (in language-independent sense),
which seems not to be unique.

I already pointed out
"One can surely argue x2 has never been dead, or see it as
"it was killed along with x and then resurrected by x"."

In Java and thus in Jython, it is treated as the second one. An equal
program written in Java or Jython would even call the finalizer of x2 (if
it had one) and clear weakrefs before it is available "again" as a class
attribute of X.
So there actually *is* a notion to refer to this scenario as resurrection.
I admit it is arguable and maybe misleading in CPython case and I was
not aware of the whole behavior when I called the topic "resurrection".

What would still be interesting (at least when Jython 3 is born),
is which of the mentioned behaviors occurs if it is
performed by CPython's cyclic gc (consistently the first one I would guess).
As you pointed out, this is only relevant from 3.4 on since in 2.x etc 
it does
not call finalizers in cycles. (Since I mainly work on Jython or Python 
2.7 I
currently have no 3.4 installed to test this instantaneously. I will 
test it someday...)


Best,

Stefan



On 10/27/2014 03:14 PM, Antoine Pitrou wrote:
> On Mon, 27 Oct 2014 14:36:31 +0100
> Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
>> Your test program performs no resurrection of x.
>>
>> Interestingly, it does not change behavior if you write
>>
>> class X(object):
>>       def __del__(self):
>>           X.x = self
>>           print ref()
>>
>> (Thanks for making me aware of this! My test-case was already
>> initially the more complex one given below)
>>
>> But if the resurrection occurs indirectly, the weakref persists:
> It's not that resurrection occurs indirectly, it's that the object
> pointed to by "x2" always remains alive (first as an instance attribute
> of x, second as a class attribute of X *before x is deleted*).
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


From solipsis at pitrou.net  Mon Oct 27 16:31:12 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Oct 2014 16:31:12 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <544E62A6.2070906@gmx.de>
References: <trinity-da3501e1-058a-49b8-838e-0a2d48032915-1414275647252@3capp-gmx-bs31>
 <20141026012258.6deea3aa@fsol>
 <trinity-4ff1784a-4971-4cbb-90b1-cfdc6194787a-1414284639508@3capp-gmx-bs31>
 <CAMSv6X0VFYW59zL9kcYzK5BpKeXOjrh6-HO+PpdyuKd4QRejQA@mail.gmail.com>
 <544E4A5F.4000703@gmx.de> <20141027151422.3e6e0e48@fsol>
 <544E62A6.2070906@gmx.de>
Message-ID: <20141027163112.54c724eb@fsol>

On Mon, 27 Oct 2014 16:20:06 +0100
Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
> 
> I already pointed out
> "One can surely argue x2 has never been dead, or see it as
> "it was killed along with x and then resurrected by x"."
> 
> In Java and thus in Jython, it is treated as the second one.

You mean Jython deletes instance attributes before calling __del__ ?
That would make most __del__ implementations quite useless...
And actually your own example would fail with an AttributeError on
"X.y = self.z".

> What would still be interesting (at least when Jython 3 is born),
> is which of the mentioned behaviors occurs if it is
> performed by CPython's cyclic gc (consistently the first one I would guess).

In which use case exactly? :-) I've lost track a bit, since you've
posted several examples...

Regards

Antoine.

From stefan.richthofer at gmx.de  Mon Oct 27 17:23:23 2014
From: stefan.richthofer at gmx.de (Stefan Richthofer)
Date: Mon, 27 Oct 2014 17:23:23 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <544E70C5.1050508@gmx.de>
References: <544E70C5.1050508@gmx.de>
Message-ID: <544E717B.7060906@gmx.de>


>You mean Jython deletes instance attributes before calling __del__ ?

No. I think the term of "object resurrection" usually does not mean bringing
back a deleted object in the sense that memory was already freed.
I think it rather means that nothing referred to an object, so it was on the
"kill-list" of gc or zero-ref-count macro.
During its finalizer call, the object managed to get some reference
again. GC or zero-ref-count macro checks refcount again after the
finalizer call (doesn't it?) and then refrains from the originally triggered
deletion-task.

Where things get weired is how to treat objects (e.g. x2) that are
reachable via the original
object (e.g. x) only.

x becomes unreachable => x2 is unreachable too

CPython behavior:
free x's weakrefs, call x.__del__ => x2 is reachable again => free
memory of x; don't touch x2 at all

Java/Jython behavior:
free all weakrefs, call all finalizers of unreachable objects, i.e. call
x.__del__, call x2.__del__ (and maybe more)
=> x2 is reachable again => free memory of x; let x2 survive
(x2 even survives at least for another gc-cycle if the finalizer of x or
x2 only created a weak ref)

At least in Java/Jython case I would call x2 to be "resurrected", i.e.
its finalizer was called and weakrefs were cleared. It was on the
death-list and escaped from it. This finally brings the definition of
the word "resurrection" to its limit in language independent sense as
one can argue there was no resurrection of x2 in CPython although it's
one and the same scenario.

>In which use case exactly?
The one with "indirect resurrection".
Would it have CPython behavior as sketched above or Java/Jython
behavior? (I confirmed the sketched behavior only for ref-drop-to-zero
triggered cleanup)


Best,

Stefan


On 10/27/2014 04:31 PM, Antoine Pitrou wrote:
> On Mon, 27 Oct 2014 16:20:06 +0100
> Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
>> I already pointed out
>> "One can surely argue x2 has never been dead, or see it as
>> "it was killed along with x and then resurrected by x"."
>>
>> In Java and thus in Jython, it is treated as the second one.
> You mean Jython deletes instance attributes before calling __del__ ?
> That would make most __del__ implementations quite useless...
> And actually your own example would fail with an AttributeError on
> "X.y = self.z".
>
>> What would still be interesting (at least when Jython 3 is born),
>> is which of the mentioned behaviors occurs if it is
>> performed by CPython's cyclic gc (consistently the first one I would guess).
> In which use case exactly? :-) I've lost track a bit, since you've
> posted several examples...
>
> Regards
>
> Antoine.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


From solipsis at pitrou.net  Mon Oct 27 17:36:19 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Oct 2014 17:36:19 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
References: <544E70C5.1050508@gmx.de>
	<544E717B.7060906@gmx.de>
Message-ID: <20141027173619.26b955af@fsol>

On Mon, 27 Oct 2014 17:23:23 +0100
Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
> 
> >You mean Jython deletes instance attributes before calling __del__ ?
> 
> No. I think the term of "object resurrection" usually does not mean bringing
> back a deleted object in the sense that memory was already freed.
> I think it rather means that nothing referred to an object, so it was on the
> "kill-list" of gc or zero-ref-count macro.

"x2" does *not* have its refcount drop to zero, since it is still
referenced by x. In other words, "x2" can only be on a "kill list"
after "x" has been finalized, which can only be *after* __del__ was
executed.

If Jython does things differently, then certainly its behaviour is
incompatible with the common expectations of Python developers.

Regards

Antoine.



From stefan.richthofer at gmx.de  Mon Oct 27 18:40:24 2014
From: stefan.richthofer at gmx.de (Stefan Richthofer)
Date: Mon, 27 Oct 2014 18:40:24 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <20141027173619.26b955af@fsol>
References: <544E70C5.1050508@gmx.de>	<544E717B.7060906@gmx.de>
 <20141027173619.26b955af@fsol>
Message-ID: <544E8388.1030200@gmx.de>

I already admitted that it is implementation specific whether one would
talk of resurrection, even in one and the same scenario. (Although
I would prefer to agree on an abstract notion of the resurrection term.)

>If Jython does things differently, then certainly its behaviour is
>incompatible with the common expectations of Python developers.

Guido recently pointed out that it is allowed for different Python
implementations to alter details of gc behavior. (And I suppose this
was more a reminder of already common consensus.)

However I agree that some aspects could be improved and I am looking
at it. So far I have all answers I needed. Thanks for the discussion!


-Stefan





On 10/27/2014 05:36 PM, Antoine Pitrou wrote:
> On Mon, 27 Oct 2014 17:23:23 +0100
> Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
>>> You mean Jython deletes instance attributes before calling __del__ ?
>> No. I think the term of "object resurrection" usually does not mean bringing
>> back a deleted object in the sense that memory was already freed.
>> I think it rather means that nothing referred to an object, so it was on the
>> "kill-list" of gc or zero-ref-count macro.
> "x2" does *not* have its refcount drop to zero, since it is still
> referenced by x. In other words, "x2" can only be on a "kill list"
> after "x" has been finalized, which can only be *after* __del__ was
> executed.
>
> If Jython does things differently, then certainly its behaviour is
> incompatible with the common expectations of Python developers.
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de


From p.f.moore at gmail.com  Mon Oct 27 18:48:40 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 17:48:40 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
Message-ID: <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>

On 26 October 2014 23:44, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 23:11, Ray Donnelly <mingw.android at gmail.com> wrote:
>> I don't know where this "ABI compatible" thing came into being;
>
> Simple. If a mingw-built CPython doesn't work with the same extensions
> as a MSVC-built CPython, then the community gets fragmented (because
> you can only use the extensions built for your stack). Assuming numpy
> needs mingw and ultimately only gets built for a mingw-compiled Python
> (because the issues building for MSVC-built Python are too hard) and
> assuming that nobody wants to make the effort to build pywin32 under
> mingw, then what does someone who needs both numpy and pywin32 do?
>
> Avoiding that issue is what I mean by ABI-compatible. (And that's all
> I mean by it, nothing more subtle or controversial).
>
> I view it as critical (because availability of binaries is *already*
> enough of a problem in the Windows world, without making it worse)
> that we avoid this sort of fragmentation. I'm not seeing an
> acknowledgement from the mingw side that they agree. That's my
> concern. If we both agree, there's nothing to argue about.

I have just done some experiments with building CPython extensions
with mingw-w64. Thanks to Ray for helping me set this up.

The bad news is that the support added to the old 32-bit mingw to
support linking to alternative C runtime libraries (specifically
-lmsvcr100) has bitrotted, and no longer functions correctly in
mingw-w64. As a result, not only can mingw-w64 not build extensions
that are compatible with python.org Python, it can't build extensions
that function at all [1]. They link incompatibly to *both* msvcrt and
msvcr100.

This is a bug in mingw-w64. I have reported it to Ray, who's passed it
onto one of the mingw-w64 developers. But as things stand, mingw
builds will definitely produce binary extensions that aren't
compatible with python.org Python.

Paul

[1] Note, that's if you just use --compiler=mingw32 as supported by
distutils. Looking at how the numpy folks build, they seem to hack
their own version of the distutils C compiler classes. I don't know
whether that's just to work around this bug, or whether they do it for
other reasons as well (but I suspect the latter).

From solipsis at pitrou.net  Mon Oct 27 18:54:13 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 27 Oct 2014 18:54:13 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <544E8388.1030200@gmx.de>
References: <544E70C5.1050508@gmx.de> <544E717B.7060906@gmx.de>
 <20141027173619.26b955af@fsol> <544E8388.1030200@gmx.de>
Message-ID: <20141027185413.22b06d66@fsol>

On Mon, 27 Oct 2014 18:40:24 +0100
Stefan Richthofer <stefan.richthofer at gmx.de> wrote:
> >If Jython does things differently, then certainly its behaviour is
> >incompatible with the common expectations of Python developers.
> 
> Guido recently pointed out that it is allowed for different Python
> implementations to alter details of gc behavior.

I'm afraid you misunderstood this whole sub-branch of the discussion.

Regards

Antoine.

From db3l.net at gmail.com  Mon Oct 27 19:29:47 2014
From: db3l.net at gmail.com (David Bolen)
Date: Mon, 27 Oct 2014 14:29:47 -0400
Subject: [Python-Dev] XP buildbot problem cloning from hg.python.org
References: <m2oat0x13a.fsf@valheru.db3l.homeip.net>
 <20141025055716.7e252f85@fsol> <m2d29gwzbq.fsf@valheru.db3l.homeip.net>
 <m28uk4wxod.fsf@valheru.db3l.homeip.net>
 <nad-95699A.15140225102014@news.gmane.org>
 <nad-987DF6.15420025102014@news.gmane.org>
Message-ID: <m2oasxv010.fsf@valheru.db3l.homeip.net>

Ned Deily <nad at acm.org> writes:

> Update: after consulting with Donald on IRC, it appears that the problem 
> was on the python.org end and is now fixed.  David, is it now working 
> again for you?

Sorry for the delay - yes, it appears to be working again for me as
well.  And it looks like clones during the buildbot tests were working
again as of tests yesterday.

-- David


From casevh at gmail.com  Mon Oct 27 19:47:13 2014
From: casevh at gmail.com (Case Van Horsen)
Date: Mon, 27 Oct 2014 11:47:13 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>
Message-ID: <CANerV6n=T2gS0tyJUFsXc20JutRBTbk4FmYiMjywRRHzD8U6xg@mail.gmail.com>

On Mon, Oct 27, 2014 at 10:48 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> The bad news is that the support added to the old 32-bit mingw to
> support linking to alternative C runtime libraries (specifically
> -lmsvcr100) has bitrotted, and no longer functions correctly in
> mingw-w64. As a result, not only can mingw-w64 not build extensions
> that are compatible with python.org Python, it can't build extensions
> that function at all [1]. They link incompatibly to *both* msvcrt and
> msvcr100.
>
> This is a bug in mingw-w64. I have reported it to Ray, who's passed it
> onto one of the mingw-w64 developers. But as things stand, mingw
> builds will definitely produce binary extensions that aren't
> compatible with python.org Python.
>
> Paul
>
> [1] Note, that's if you just use --compiler=mingw32 as supported by
> distutils. Looking at how the numpy folks build, they seem to hack
> their own version of the distutils C compiler classes. I don't know
> whether that's just to work around this bug, or whether they do it for
> other reasons as well (but I suspect the latter).
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/casevh%40gmail.com

I've managed to build gmpy2 (which requires GMP, MPFR, and MPC
libraries) using msys2. I've detailed the steps (hacking) at:

https://code.google.com/p/gmpy/source/browse/trunk/msys2_build.txt

One of the hacks I made addresses the linking bug. The extension
does run with the both the 32-bit and 64-bit versions of CPython 2.7,
3.2, 3.3, and 3.4.

It is possible, just not easy. Anything that makes is easier would
be very helpful.

casevh

From p.f.moore at gmail.com  Mon Oct 27 19:55:07 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 18:55:07 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CANerV6n=T2gS0tyJUFsXc20JutRBTbk4FmYiMjywRRHzD8U6xg@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>
 <CANerV6n=T2gS0tyJUFsXc20JutRBTbk4FmYiMjywRRHzD8U6xg@mail.gmail.com>
Message-ID: <CACac1F8CvgXpgQg6w=O5_pnfAOuTbV=41YHtjAOqEuVUg14Vhw@mail.gmail.com>

On 27 October 2014 18:47, Case Van Horsen <casevh at gmail.com> wrote:
> I've managed to build gmpy2 (which requires GMP, MPFR, and MPC
> libraries) using msys2. I've detailed the steps (hacking) at:
>
> https://code.google.com/p/gmpy/source/browse/trunk/msys2_build.txt

Thanks for this. I don't have the time to read your notes right now,
but I will do so.

> One of the hacks I made addresses the linking bug. The extension
> does run with the both the 32-bit and 64-bit versions of CPython 2.7,
> 3.2, 3.3, and 3.4.

Did you report the linking bug to the mingw-w64 project? They key
thing here is that without gcc -lmsvcrt100 foo.c working (i.e., not
resulting in linking with msvcrt), building Python extensions will
always need hacks to workaround that bug.

> It is possible, just not easy. Anything that makes is easier would
> be very helpful.

With the bug fixed, the steps should be as trivial as:

1. Using python.org Python, with gcc on your PATH.
2. Install any dependencies (e.g., gmp) where gcc can see them.
3. python setup.py build_ext --compiler=mingw32 bdist_wheel

(or whatever setup.py invocation suits you, as long as you set
compiler=mingw32).

Paul

From mingw.android at gmail.com  Mon Oct 27 01:02:05 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Mon, 27 Oct 2014 00:02:05 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141027005228.Horde.iBp3Vlgrn9AY2Zm9m-V4KQ2@webmail.df.eu>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <20141027005228.Horde.iBp3Vlgrn9AY2Zm9m-V4KQ2@webmail.df.eu>
Message-ID: <CAOYw7duopFnVD6PoToBjUsEeUf4go8U+P-y7i6C067JNhGJ6YA@mail.gmail.com>

On Sun, Oct 26, 2014 at 11:52 PM,  <martin at v.loewis.de> wrote:
>
> Zitat von Tony Kelman <kelman at berkeley.edu>:
>
>> A maintainer has volunteered. Others will help. Can any core developers
>> please begin reviewing some of his patches?
>
>
> Unfortunately, every attempt to review these patches has failed for me,
> every time. In the last iteration of an attempt to add mingw64 support,
> I had asked contributors to also provide instructions on how to use these
> patches, and haven't received any instructions that actually worked.
>
> I'm hesitant to add code that I cannot verify as actually working.
>
> I guess anybody else reviewing these patches ran into similar problems
> (I know some other core developers have tried reviewing them as well,
> others have stated here that they are unable to review the patches).
>

https://mail.python.org/pipermail/python-dev/2014-October/136756.html

> Regards,
> Martin
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From njs at pobox.com  Mon Oct 27 20:02:52 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Mon, 27 Oct 2014 19:02:52 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CACac1F84=M74amzvOFoStkf7WcpWR7oYZ-utyMur4wjN0Sthcw@mail.gmail.com>
Message-ID: <CAPJVwBn89WVnfqCm6XmY5vasp38aaVa=5fh_WyOvoH5wenLYhA@mail.gmail.com>

On Mon, Oct 27, 2014 at 5:48 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 26 October 2014 23:44, Paul Moore <p.f.moore at gmail.com> wrote:
>> On 26 October 2014 23:11, Ray Donnelly <mingw.android at gmail.com> wrote:
>>> I don't know where this "ABI compatible" thing came into being;
>>
>> Simple. If a mingw-built CPython doesn't work with the same extensions
>> as a MSVC-built CPython, then the community gets fragmented (because
>> you can only use the extensions built for your stack). Assuming numpy
>> needs mingw and ultimately only gets built for a mingw-compiled Python
>> (because the issues building for MSVC-built Python are too hard) and
>> assuming that nobody wants to make the effort to build pywin32 under
>> mingw, then what does someone who needs both numpy and pywin32 do?
>>
>> Avoiding that issue is what I mean by ABI-compatible. (And that's all
>> I mean by it, nothing more subtle or controversial).
>>
>> I view it as critical (because availability of binaries is *already*
>> enough of a problem in the Windows world, without making it worse)
>> that we avoid this sort of fragmentation. I'm not seeing an
>> acknowledgement from the mingw side that they agree. That's my
>> concern. If we both agree, there's nothing to argue about.
>
> I have just done some experiments with building CPython extensions
> with mingw-w64. Thanks to Ray for helping me set this up.
>
> The bad news is that the support added to the old 32-bit mingw to
> support linking to alternative C runtime libraries (specifically
> -lmsvcr100) has bitrotted, and no longer functions correctly in
> mingw-w64. As a result, not only can mingw-w64 not build extensions
> that are compatible with python.org Python, it can't build extensions
> that function at all [1]. They link incompatibly to *both* msvcrt and
> msvcr100.
>
> This is a bug in mingw-w64. I have reported it to Ray, who's passed it
> onto one of the mingw-w64 developers. But as things stand, mingw
> builds will definitely produce binary extensions that aren't
> compatible with python.org Python.

IIUC, getting mingw-w64 to link against msvcr100 instead of msvcrt
requires a custom mingw-w64 build, because by default mingw-w64's
internal runtime libraries (libgcc etc.) are linked against msvcrt. So
by the time you're choosing compiler switches etc., it's already too
late -- your switches might affect how *your* code is built, but your
code will still be linked against pre-existing runtime libraries that
are linked against msvcrt.

It's possible to hack the mingw-w64 build process to build the runtime
libraries against msvcr100 (or whatever) instead of msvcrt, but this
is still not a panacea -- the different msvcr* libraries are, of
course, incompatible with each other, and IIUC the mingw-w64
developers have never tried to make their libraries work against
anything except msvcrt. For example, mingw-w64's gfortran runtime uses
a symbol that's only available in msvcrt, not msvcr90 or msvcrt100:
  http://sourceforge.net/p/mingw-w64/mailman/message/31768118/

So my impression is that these issues are all fixable, but they will
require real engagement with mingw-w64 upstream.

> [1] Note, that's if you just use --compiler=mingw32 as supported by
> distutils. Looking at how the numpy folks build, they seem to hack
> their own version of the distutils C compiler classes. I don't know
> whether that's just to work around this bug, or whether they do it for
> other reasons as well (but I suspect the latter).

numpy.distutils is a massive pile of hacks to handle all kinds of
weird things including recursive builds, fortran, runtime capability
detection (like autoconf), and every random issue anyone ran into at
some point in the last 10 years and couldn't be bothered filing a
proper upstream bug report. Basically no-one knows what it actually
does -- the source is your only hope :-).

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From greg.ewing at canterbury.ac.nz  Mon Oct 27 21:45:50 2014
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 28 Oct 2014 09:45:50 +1300
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
Message-ID: <544EAEFE.5070603@canterbury.ac.nz>

Nick Coghlan wrote:
> That assumption will allow MinGW-w64 to link with the appropriate
> MSVCRT versions for extention building without anything breaking.

If that works, then the same technique should allow CPython
itself to be built in a VS-compatible way with mingw,
shouldn't it?

Those objecting to a mingw-built python seem to be assuming
that such a thing will necessarily be incompatible with
VS builds, but I don't see why that has to be the case.

-- 
Greg

From p.f.moore at gmail.com  Mon Oct 27 22:11:13 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 21:11:13 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <544EAEFE.5070603@canterbury.ac.nz>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
 <544EAEFE.5070603@canterbury.ac.nz>
Message-ID: <CACac1F-PKKc+jWLXiT7NT8sf8J63xvXTtpXbGHMAA4-Esvu48g@mail.gmail.com>

On 27 October 2014 20:45, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Nick Coghlan wrote:
>>
>> That assumption will allow MinGW-w64 to link with the appropriate
>> MSVCRT versions for extention building without anything breaking.
>
>
> If that works, then the same technique should allow CPython
> itself to be built in a VS-compatible way with mingw,
> shouldn't it?

Yes.

> Those objecting to a mingw-built python seem to be assuming
> that such a thing will necessarily be incompatible with
> VS builds, but I don't see why that has to be the case.

No, we've been trying to establish whether the patches to build with
mingw were intended to produce such a compatible build. It's not
clear, but so far it seems that apparently that is *not* the intent
(and worse, mingw-w64 may not even be able to build viable executables
that link with msvcr100 without some heavy hacking, although that's
still somewhat unclear).

Paul

From Steve.Dower at microsoft.com  Mon Oct 27 21:54:43 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 27 Oct 2014 20:54:43 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <544EAEFE.5070603@canterbury.ac.nz>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
 <544EAEFE.5070603@canterbury.ac.nz>
Message-ID: <713df909b7b243fc93a1eae6e4cd24b2@DM2PR0301MB0734.namprd03.prod.outlook.com>

Greg Ewing wrote:
> Nick Coghlan wrote:
>> That assumption will allow MinGW-w64 to link with the appropriate
>> MSVCRT versions for extention building without anything breaking.
>
> If that works, then the same technique should allow CPython itself to be built
> in a VS-compatible way with mingw, shouldn't it?
>
> Those objecting to a mingw-built python seem to be assuming that such a thing
> will necessarily be incompatible with VS builds, but I don't see why that has to
> be the case.

That's true, and a good point that I missed. However, the main (practical) desire for building CPython with something other than VS seems to be to avoid having to be compatible with VS.

It's entirely possible that having two alternative builds of CPython would force everyone to be more compatible, but I think it's more likely to simply end up being two different worlds. Maybe I'm being unnecessarily cynical :)

Cheers,
Steve

> --
> Greg

From Steve.Dower at microsoft.com  Mon Oct 27 22:19:28 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 27 Oct 2014 21:19:28 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F-PKKc+jWLXiT7NT8sf8J63xvXTtpXbGHMAA4-Esvu48g@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
 <544EAEFE.5070603@canterbury.ac.nz>
 <CACac1F-PKKc+jWLXiT7NT8sf8J63xvXTtpXbGHMAA4-Esvu48g@mail.gmail.com>
Message-ID: <920092d128234aabb555c4fec85203e3@DM2PR0301MB0734.namprd03.prod.outlook.com>

> Paul Moore wrote:
> On 27 October 2014 20:45, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
>> Nick Coghlan wrote:
>>>
>>> That assumption will allow MinGW-w64 to link with the appropriate
>>> MSVCRT versions for extention building without anything breaking.
>>
>>
>> If that works, then the same technique should allow CPython itself to
>> be built in a VS-compatible way with mingw, shouldn't it?
> 
> Yes.
> 
>> Those objecting to a mingw-built python seem to be assuming that such
>> a thing will necessarily be incompatible with VS builds, but I don't
>> see why that has to be the case.
> 
> No, we've been trying to establish whether the patches to build with mingw were
> intended to produce such a compatible build. It's not clear, but so far it seems
> that apparently that is *not* the intent (and worse, mingw-w64 may not even be
> able to build viable executables that link with msvcr100 without some heavy
> hacking, although that's still somewhat unclear).

Unless there is also opposition to moving to VC14, I'd rather see the mingw projects invest in linking to those libraries. I believe they'll have a much easier time of it than worrying about VC10, and the investment will be worth more in the future as the public API of the CRT stops changing.

Unfortunately, I'm not able to help out more than I've already offered (researching answers to specific questions). Largely because I have enough work-outside-work going on, but also because my employer won't like me getting involved with GPL'd software at all.

Cheers,
Steve

> Paul

From mingw.android at gmail.com  Mon Oct 27 22:41:01 2014
From: mingw.android at gmail.com (Ray Donnelly)
Date: Mon, 27 Oct 2014 21:41:01 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <713df909b7b243fc93a1eae6e4cd24b2@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
 <544EAEFE.5070603@canterbury.ac.nz>
 <713df909b7b243fc93a1eae6e4cd24b2@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CAOYw7duw99ML-z7Yi09o2fu5TtxRyDuTXFzAk=KUKPi_jfDpNQ@mail.gmail.com>

On Mon, Oct 27, 2014 at 8:54 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Greg Ewing wrote:
>> Nick Coghlan wrote:
>>> That assumption will allow MinGW-w64 to link with the appropriate
>>> MSVCRT versions for extention building without anything breaking.
>>
>> If that works, then the same technique should allow CPython itself to be built
>> in a VS-compatible way with mingw, shouldn't it?
>>
>> Those objecting to a mingw-built python seem to be assuming that such a thing
>> will necessarily be incompatible with VS builds, but I don't see why that has to
>> be the case.
>
> That's true, and a good point that I missed. However, the main (practical) desire for building CPython with something other than VS seems to be to avoid having to be compatible with VS.

I've no idea where you get that impression from, no one has expressed
anything even approximating that. For me it's to avoid using closed
source software for my hobbyist programming and to help to create a
vibrant Open Source distribution for Windows, because I quite like
Windows; it's got a lot going for it. For others it's to ensure that
everything they care about (CPython with Fortran for example) works
together properly and reliably. I expect that avoiding compatibility
couldn't be further from any of our wishes.

>
> It's entirely possible that having two alternative builds of CPython would force everyone to be more compatible, but I think it's more likely to simply end up being two different worlds. Maybe I'm being unnecessarily cynical :)
>
> Cheers,
> Steve
>
>> --
>> Greg
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/mingw.android%40gmail.com

From tjreedy at udel.edu  Mon Oct 27 23:36:35 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 27 Oct 2014 18:36:35 -0400
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <544E717B.7060906@gmx.de>
References: <544E70C5.1050508@gmx.de> <544E717B.7060906@gmx.de>
Message-ID: <m2mhdp$ebj$1@ger.gmane.org>

On 10/27/2014 12:23 PM, Stefan Richthofer wrote:
>
>> You mean Jython deletes instance attributes before calling __del__ ?
>
> No. I think the term of "object resurrection" usually does not mean
> bringing
> back a deleted object in the sense that memory was already freed.
> I think it rather means that nothing referred to an object, so it was on
> the
> "kill-list" of gc or zero-ref-count macro.

In either case, there is a final reference keeping the object alive, 
like an hospital patient kept alive by a final link with a life-support 
machine.  I think 'resuscitation' might be a better metaphor.

-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Mon Oct 27 23:37:00 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 27 Oct 2014 22:37:00 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <920092d128234aabb555c4fec85203e3@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
 <CAOYw7ds0koCT+RaArpobjSTcNKXVn9utZ8VKaoi31qWBBR+jBw@mail.gmail.com>
 <CACac1F_3twwuuhs+h_uycudSVjctUPWgdXAtLog-e+xQxUMUaw@mail.gmail.com>
 <CADiSq7e=rZx8qukGbYsruYm1MKRNj3pXVA3cBeYu8e9Kg2sJag@mail.gmail.com>
 <544EAEFE.5070603@canterbury.ac.nz>
 <CACac1F-PKKc+jWLXiT7NT8sf8J63xvXTtpXbGHMAA4-Esvu48g@mail.gmail.com>
 <920092d128234aabb555c4fec85203e3@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CACac1F9y7biwssAhbmhy54yiTs44nVbjGV4OLPyRsJqn+GTxUA@mail.gmail.com>

On 27 October 2014 21:19, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> No, we've been trying to establish whether the patches to build with mingw were
>> intended to produce such a compatible build. It's not clear, but so far it seems
>> that apparently that is *not* the intent (and worse, mingw-w64 may not even be
>> able to build viable executables that link with msvcr100 without some heavy
>> hacking, although that's still somewhat unclear).
>
> Unless there is also opposition to moving to VC14, I'd rather see the mingw
> projects invest in linking to those libraries. I believe they'll have a much easier
> time of it than worrying about VC10, and the investment will be worth more in
> the future as the public API of the CRT stops changing.

I think the point is that anything other than msvcrt is extra work,
because using msvcrt is coded into the guts of gcc (which in turn is
because msvcrt is apparently OK to consider as "part of the OS" in GPL
legality terms). So whether it's the vc10 libraries or the vc14 ones
is irrelevant - and mingw ships with the vc10 link library, so it's
easier to discuss the problem in terms of vc10. But yes, vc14 would be
the long term target.

Of course if the vc14 libs were deemed as "shipped with the OS" and/or
were named msvcrt.dll, then that would be different. But I assume
that's not what will happen.

Paul

From jeanpierreda at gmail.com  Mon Oct 27 23:47:08 2014
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Mon, 27 Oct 2014 15:47:08 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <CAP7+vJKK5=XxN04H_6A8REeXdCJaLDBwRvVvvpa_jcHzedRWgw@mail.gmail.com>
 <E648C5C014E74B459A705C98324E0A4D@TKsamsung>
 <CACac1F-1ViS5GKrgiCjvhL_wqSQtq3yTKmHGgY30s2GBv4aAww@mail.gmail.com>
Message-ID: <CABicbJJzHrjLM_6UrB5rfJo2r_2rVUaX8ZutXxeruSFYA6b=yA@mail.gmail.com>

On Sun, Oct 26, 2014 at 3:41 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> Not really, to be honest. I still don't understand why anyone not
> directly involved in CPython development would need to build their own
> Python executable on Windows.

Late Python bugfix releases are source-only, so if you suffer from a
bug and need to get it fixed, you need to build Python from source.

https://www.python.org/download/releases/2.6.9/ has no windows binary
and includes several security fixes.

-- Devin

From Stefan.Richthofer at gmx.de  Tue Oct 28 01:32:53 2014
From: Stefan.Richthofer at gmx.de (Stefan Richthofer)
Date: Tue, 28 Oct 2014 01:32:53 +0100
Subject: [Python-Dev] results of id() and weakref.getweakrefs()
 sometimes break on object resurrection
In-Reply-To: <m2mhdp$ebj$1@ger.gmane.org>
References: <544E70C5.1050508@gmx.de> <544E717B.7060906@gmx.de>,
 <m2mhdp$ebj$1@ger.gmane.org>
Message-ID: <trinity-b5b19a54-054d-47fc-9807-5f96c0c9d3b7-1414456373087@3capp-gmx-bs35>

>I think 'resuscitation' might be a better metaphor.

The term 'resurrection' is not my invention, but well established:
http://en.wikipedia.org/wiki/Object_resurrection

I well understand why Antoine objects to calling it "resurrection" in CPython due to
implementation specific reasons. But in the above article (which I consider rather
detailed) I can't find anything stating that an object's ref-count must drop to zero
at any time in order to call it "resurrected". In contrast, it clarifies that objects
can not only resurrect themselves:
"...which may in turn make that object or another garbage object (reachable from the
object with a finalizer) reachable again"
"If this happens, the referenced object ? which is not necessarily the finalized
object ? is no longer garbage, and cannot be deallocated"

> "x2" does *not* have its refcount drop to zero, since it is still
> referenced by x. In other words, "x2" can only be on a "kill list"
> after "x" has been finalized, which can only be *after* __del__ was
> executed.

x resurrects x2 in the sense that it must "actively" have an action
in its finalizer that establishes a new reference to x2 in non-garbage or
environment memory. Otherwise x as the "final life support link" of x2
would cause x2's ref count *actually* drop to zero in the next step.

I never wanted this to become a discussion about the definition of object
resurrection. I just wanted to understand which details in different
behavior (such as weakref breaking) are okay and which are bugs (as
breaking consistency of id() in Jython).


Regards

-Stefan



> Gesendet: Montag, 27. Oktober 2014 um 23:36 Uhr
> Von: "Terry Reedy" <tjreedy at udel.edu>
> An: python-dev at python.org
> Betreff: Re: [Python-Dev] results of id() and weakref.getweakrefs() sometimes break on object resurrection
>
> On 10/27/2014 12:23 PM, Stefan Richthofer wrote:
> >
> >> You mean Jython deletes instance attributes before calling __del__ ?
> >
> > No. I think the term of "object resurrection" usually does not mean
> > bringing
> > back a deleted object in the sense that memory was already freed.
> > I think it rather means that nothing referred to an object, so it was on
> > the
> > "kill-list" of gc or zero-ref-count macro.
> 
> In either case, there is a final reference keeping the object alive, 
> like an hospital patient kept alive by a final link with a life-support 
> machine.  I think 'resuscitation' might be a better metaphor.
> 
> -- 
> Terry Jan Reedy
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/stefan.richthofer%40gmx.de
>

From stephen at xemacs.org  Tue Oct 28 02:47:29 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 28 Oct 2014 10:47:29 +0900
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141025232439.C6214250D5B@webabinitio.net>
References: <E3EFDC53A34F4888B80D4013E9EAB5B1@TKsamsung>
 <f4bca96dea224d228d5ff07623268541@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAOYw7dukya_9eivwrKYY6FTe4sj8bEmSW25YkpAN-A3-JLmASA@mail.gmail.com>
 <1414270256914.92610@microsoft.com>
 <CAPTjJmrSHGB0LcrZSGzMxm8YRxuJ=tCFK7s=ELjjNm_+mSVGFw@mail.gmail.com>
 <20141025234742.0119e061@fsol>
 <CAPTjJmoxKqbeei4A4D5F56rkV9Gm6cO8iCPdXndXdrna6EiBAA@mail.gmail.com>
 <20141025235945.2a3a7ddd@fsol>
 <CAPTjJmprLdGqz2V4yzQe3vM1Y4Xq6PCnd-6EtpPQOfx+XaQM3A@mail.gmail.com>
 <20141026001944.0373cb92@fsol>
 <20141025232439.C6214250D5B@webabinitio.net>
Message-ID: <87zjch3qz2.fsf@uwakimon.sk.tsukuba.ac.jp>

R. David Murray writes:
 > On Sun, 26 Oct 2014 00:19:44 +0200, Antoine Pitrou <solipsis at pitrou.net> wrote:

 > > My point is that your "Windows build" would not have the same behaviour
 > > as a MSVC-produced Windows build, and so testing it with it would not
 > > certify that your code would actually be compatible with genuine
 > > MSVC builds of CPython, which we will not stop supporting.
 > 
 > While true, I don't think that matters for Chris' point.

[...]

 > If I could use a more linux-like toolchain to build CPython on windows,
 > I would doubtless do much more testing on windows for stuff where I
 > think windows might behave differently (and I might look at more Windows
 > bugs...though frankly there are plenty of bugs for me to look at without
 > looking at Windows bugs).
 > 
 > This is not necessarily a compelling argument for MinGW support.
 > However, it *is* a valid argument, IMO.

Nobody claims that the there are not arguments, even compelling
arguments, for MinGW support (more generally, support for alternative
toolchains).

But there are *also* compelling arguments for *supporting* *both*
those "no need to worry about mixed ABIs" situations and *mixed*
situations.  And that becomes Python Dev's problem if the patches are
added to core Python.  Currently, they're somebody else's problem, and
that's as it should be at this stage.

Python is open source.  Nobody is objecting to "somebody else" doing
this.[1]  The problem here is simply that some "somebody elses" are
trying to throw future work over the wall into python-dev space.
There is nothing wrong with that, either -- that's why there is a
stdlib, for example -- but the python-dev concerns about platform
fragmentation are genuine (even if not applicable to all potential
users of the alternative toolchains), and substantial resources will
be needed to do the testing required to meet python-dev's requirement
that such code be *binary* compatible with other binaries downloaded
for Windows, as well as for maintenance of the code itself.


Footnotes: 
[1]  Some *do* question whether there's a need for anybody to do this,
and that's bogus.  "I just wanna" is good enough reason to do it.  The
issue here is that it's not good enough reason for python-dev to do
the support and maintenance going forward.


From kelman at berkeley.edu  Tue Oct 28 09:10:38 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Tue, 28 Oct 2014 01:10:38 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
Message-ID: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>

Stephen J. Turnbull:
> Python is open source.  Nobody is objecting to "somebody else" doing
> this.[1]  The problem here is simply that some "somebody elses" are
> trying to throw future work over the wall into python-dev space.

If that's how it's seen at this point, then it sounds like the logical
course of action for CPython-with-MinGW is to demonstrate feasibility
in a fork. Get what currently works as a set of 80-something patches
and PKGBUILD instructions into a single repository that is usable by a
wider variety of people, in more distributions, etc. Set up as much CI as
possible so every patch gets tested in as many configurations as we can.
Experiment with extension compatibility and find out what is actually
achievable, with or without needing to modify MinGW-w64 in the process.

There are people, though evidently not much of python-dev, who have a
need and desire to make this happen. It seems python-dev would rather
have us spend zero time proposing changes that allow CPython itself to
be built differently than today, and all of our time on MinGW extensions.
I personally find 3 of the 4 combinations of how one could build CPython
and how one could build extensions interesting and worth looking into for
different reasons (the one that's uninteresting to me is the currently
used all-MSVC method, due to its many limitations I listed earlier).
Ray for example may only care about using MinGW for everything. Python's
a big community with lots of room for different people to work on
different aspects of the same set of problems.

For the combination of MSVC Python and MinGW extensions that most of you
have recommended we focus on, it would be more productive to engage with
setuptools, distutils-sig, and likely numpy as well, instead of python-dev.
My experience lies more in getting troublesome C codebases to build with
MinGW than it does with the messy intricacies and backwards-compatibility
requirements of Python extensions and package management however, so my
ability to contribute on that side of things is more limited. I'll just
note that every project I've ever had a patch for which improved
functionality with a new compiler (whether GCC, MSVC, clang, icc or ifort,
etc.) reacted with review and thanks for the patches, not "why do you want
to do this?" pushback. If potential contributors have a desire to get it
working in the first place, then they will also be invested in helping
keep it working on an ongoing basis.

Sincerely,
Tony


From stephen at xemacs.org  Tue Oct 28 14:45:03 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 28 Oct 2014 22:45:03 +0900
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
Message-ID: <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>

Tony Kelman writes:

 > If potential contributors have a desire to get it working in the
 > first place, then they will also be invested in helping keep it
 > working on an ongoing basis.

Sure -- as long as it works for them, though, such potential
contributors don't necessarily care if it works for anybody else.  My
experience (in other projects) is that allowing that level of
commitment to be sufficient for inclusion in the maintained code base
frequently results in bug reports from third parties who try to use
the new feature and discover it *doesn't* work for them.  The core
maintainers then have a very unpleasant choice: to say "we don't
support that usage", or to deal with the problem on a continuing basis
(because the same issues tend to regress repeatedly as the said
"invested contributors" continue to modify their code on the same
"works for us" basis).

 > If that's how it's seen at this point, then it sounds like the
 > logical course of action for CPython-with-MinGW is to demonstrate
 > feasibility in a fork. Get what currently works as a set of
 > 80-something patches and PKGBUILD instructions into a single
 > repository that is usable by a wider variety of people, in more
 > distributions, etc. Set up as much CI as possible so every patch
 > gets tested in as many configurations as we can.  Experiment with
 > extension compatibility and find out what is actually achievable,
 > with or without needing to modify MinGW-w64 in the process.

Sounds good to me.  You seem to think that's excessive, though:

 > There are people, though evidently not much of python-dev, who have a
 > need and desire to make this happen.

Well, for starters, most of python-dev would rather avoid any contact
whatsoever with Windows.  I think part of the problem is that Windows
developers *of* Python are *very* rare (fingers of one hand rare).
Sure, there are many Windows-based Python developers, and quite of few
of them do a fair amount to contribute to Python on Windows -- but
they do that work in *Python*, not at a level where they deal with the
extension ABI.  Even those who do work on C extensions have so far
been happy to work with the VC build (except for the recurrent issue
of getting one's hands on the toolchain).

So why should python-dev have a desire to do that work?  It should be
evident by now that our belief is that the large majority of Windows
users is well-served by the current model (produce an official binary
and a plethora of compatible extensions, which provides strong
incentive to third parties to ensure that their extensions and
alternative builds of core are also compatible).

I think the repeated query, "why isn't this model good enough for
you?" is being misinterpreted.  I suppose that some who ask that
really want to know, because if you have what they consider a good
reason, they'd be willing to help.  Others are asking but by "you"
they mean the world at large, in particular, "what benefit is there to
the large number of users well-served by the current model?"

 > It seems python-dev would rather have us spend zero time proposing
 > changes that allow CPython itself to be built differently than
 > today, and all of our time on MinGW extensions.

Of course they would.  (Third person because I'm not competent to do
the work anyway.)  It's quite clear that one thing the two sides agree
on is that ensuring that MinGW and VC interpreter and extension builds
can mix and match is quite a bit of work.  They quite naturally don't
want to do that work, and don't see much point in discussing it if the
(apparently) few people who need it aren't going to supply the
resources.

That's quite a reasonable solution, really: *both* communities avoid
spending effort on mix and match, and because it's a fork, it's a
different name, and people won't expect them to mix and match.

 > I personally find 3 of the 4 combinations of how one could build
 > CPython and how one could build extensions interesting and worth
 > looking into for different reasons (the one that's uninteresting to
 > me is the currently used all-MSVC method, due to its many
 > limitations I listed earlier).  Ray for example may only care about
 > using MinGW for everything. Python's a big community with lots of
 > room for different people to work on different aspects of the same
 > set of problems.

There's a reason we call it "core".  One of python-dev's more
important responsibilities is to ensure that the "aspects" work and
play well together.  "Aspects" people tend to deprecate that
responsibility (until somebody else's "aspect" treads on their blue
suede shoes).  That's as it should be, IMO -- but so is python-dev's
reluctance to admit new "aspects" until their impact on core
responsibilities is made clear.


From kelman at berkeley.edu  Tue Oct 28 15:46:19 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Tue, 28 Oct 2014 07:46:19 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <133A95BE786341E892819607833EF8AC@TKsamsung>

Stephen J. Turnbull:
> Sure -- as long as it works for them, though, such potential
> contributors don't necessarily care if it works for anybody else.  My
> experience (in other projects) is that allowing that level of
> commitment to be sufficient for inclusion in the maintained code base
> frequently results in bug reports from third parties who try to use
> the new feature and discover it *doesn't* work for them.

Good point. I definitely care whether patches work for everyone else.
Patches should be done well and accompanied with proper documentation
so new functionality is fully reproducible. If that's what's holding
up review, comments in the bug threads indicating as much would be
helpful. Any fork will also have to be thoroughly documented if it's
to have any chance of working.

> Sounds good to me.  You seem to think that's excessive, though:

No, just hearing the words come out of my mouth they sound a little nuts.
Maybe not, there are after all half a dozen or more incompatible alternate
Python implementations floating around. I think most of them started as
from-scratch rewrites rather than source forks, but maybe that's irrelevant.

> Well, for starters, most of python-dev would rather avoid any contact
> whatsoever with Windows.  I think part of the problem is that Windows
> developers *of* Python are *very* rare (fingers of one hand rare).

In my opinion the MSVC toolchain makes that problem worse, as it's far
harder for unix developers to have any familiarity with how things work.
But you do have involvement and core developers from Microsoft which is
obviously incredibly important. Maybe even mandatory for Python on Windows
to be viable in your eyes.

> Even those who do work on C extensions have so far
> been happy to work with the VC build (except for the recurrent issue
> of getting one's hands on the toolchain).
>
> It should be evident by now that our belief is that the large majority
> of Windows users is well-served by the current model

This is not the case at all in the scientific community. NumPy and SciPy
put in a lot of extra work to come up with something that is compatible
with the MSVC build of CPython because they have to, not because they're
"happy to" jump through the necessary hoops. The situation today with
NumPy AIUI is they already have to build with a custom hacked MinGW
toolchain that only one person knows how to use. Evidently until very
recently they were using a many-years-old 32-bit-only build of GCC 3.x.
Do python-dev and numpy-discussion not talk to one another? I get that
not everyone uses numpy, or Windows, but it sounds like there's very
little understanding, or even awareness, of the issues they face.

> They quite naturally don't want to do that work, and don't see much
> point in discussing it if the (apparently) few people who need it aren't
> going to supply the resources.

I'm going to move the "extensions with MinGW-w64" part of this conversation
over to numpy-discussion, since they have a need for this today and are
already using workarounds and processes that I don't think anyone is fully
satisfied with. I do find this combination interesting, worth working on,
and possible to make work well, but not completely in isolation as it does
not address my embedding use case.

> but so is python-dev's
> reluctance to admit new "aspects" until their impact on core
> responsibilities is made clear.

Okay. I'll table the discussion with python-dev for now then.

-Tony


From p.f.moore at gmail.com  Tue Oct 28 16:24:54 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 28 Oct 2014 15:24:54 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <133A95BE786341E892819607833EF8AC@TKsamsung>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
Message-ID: <CACac1F-CpaXaVd=aMdwmCoouDnycsXXgZrpEqa8R1G5myuu7Wg@mail.gmail.com>

On 28 October 2014 14:46, Tony Kelman <kelman at berkeley.edu> wrote:
> Patches should be done well and accompanied with proper documentation
> so new functionality is fully reproducible. If that's what's holding
> up review, comments in the bug threads indicating as much would be
> helpful.

Typically that tends to be expressed as a terse "I can't get this
working". That's not ideal, but is an indication. When the response is
"but it's easy" (as it sometimes is) communication degenerates quite
fast. There's an example here in this thread - it took me a *long*
time, with help from a couple of people, to work out how to get a
mingw toolchain I could use to try things out. Even though the premise
of the whole discussion was "it's easy to set up a mingw toolchain"...

> In my opinion the MSVC toolchain makes that problem worse, as it's far
> harder for unix developers to have any familiarity with how things work.
> But you do have involvement and core developers from Microsoft which is
> obviously incredibly important. Maybe even mandatory for Python on Windows
> to be viable in your eyes.

One problem I've seen a lot on other projects is that when Unix
developers set up a comfortable, Unix-like environment on Windows
using something like mingw, they frequently aren't aware of crucial
differences between Unix and Windows and tend to write software that
even though it's Windows-native, *feels* Unixy to Windows users (don't
ask me to be more specific :-)). That's always going to happen, and
the people with Windows experience have to take up the slack by
keeping an eye out for such things, but setting the bar marginally
higher, to "you have to at least be willing to download and use a
native Windows compiler" can at least remind said Unix developers that
they are in a different environment.

That's *not* a criticism of anyone in the Python community, btw.
Typically the experience of Windows users is very well respected by
python core and package developers. But I tend to think that's partly
*because* we didn't take the "Unix-like toolchain" approach by
default.

> This is not the case at all in the scientific community. NumPy and SciPy
> put in a lot of extra work to come up with something that is compatible
> with the MSVC build of CPython because they have to, not because they're
> "happy to" jump through the necessary hoops. The situation today with
> NumPy AIUI is they already have to build with a custom hacked MinGW
> toolchain that only one person knows how to use. Evidently until very
> recently they were using a many-years-old 32-bit-only build of GCC 3.x.
> Do python-dev and numpy-discussion not talk to one another? I get that
> not everyone uses numpy, or Windows, but it sounds like there's very
> little understanding, or even awareness, of the issues they face.

Sadly, no. The numpy developers have always been a pretty much
separate world. We're seeing a bit more interaction these days, but
it's very limited and far from the level that's needed. The fault (if
that's the right word) probably lies on both sides. It's certainly not
purely the responsibility of the core team to build communication
links. As this thread has demonstrated, python-dev (and distutils-sig,
which is another place that desparately needs more numpy interaction)
is not the most welcoming of places for someone who is 100% focused on
the scientific stack, but conversely the scientific types do tend to
do their own thing, and sometimes when they do surface, they can dive
in with little appreciation of the wider picture. But we can get
along, and we can make progress (albeit not always as fast as either
party might like).

If all this thread has done is raise awareness of each others'
concerns, it's still been a win IMO.

> I'm going to move the "extensions with MinGW-w64" part of this conversation
> over to numpy-discussion, since they have a need for this today and are
> already using workarounds and processes that I don't think anyone is fully
> satisfied with. I do find this combination interesting, worth working on,
> and possible to make work well, but not completely in isolation as it does
> not address my embedding use case.

Please keep distutils-sig in the loop as well. I can't promise we'll
be able to help, but we should at least make sure we're not hindering
you, and we can make you aware of when your solutions won't work
outside your area of interest. Now the door is open, let's not close
it again.

Paul

From victor.stinner at gmail.com  Tue Oct 28 22:07:45 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 28 Oct 2014 22:07:45 +0100
Subject: [Python-Dev] PEP 475
Message-ID: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>

Hi,

At the end of August, I sent the PEP 475 which I wrote with
Charles-Fran?ois Natali:

   https://mail.python.org/pipermail/python-dev/2014-August/136077.html
   https://mail.python.org/pipermail/python-dev/2014-September/136101.html

Antoine Pitrou wrote " I'm +1 on the whole PEP" and R. David Murray
wrote "Personally, I really want Python to handle EINTR for me".

What's the next step? Who wants to handle this PEP? Guido? Antoine?


I will try to answer to questions if previous answers were not enough.

Antoine Pitrou wrote:
>> On Unix, the ``asyncio`` module uses the wakeup file descriptor to
>> wake up its event loop.
> How about Windows?

I modified signal.set_wakeup_fd() in Python 3.5 to support a socket on
Windows. So it becomes possible to support signals with
signal.set_wakeup_fd() on Windows (for SelectorEventLoop and
ProactorEventLoop):

   https://code.google.com/p/tulip/issues/detail?id=191

Antoine Pitrou wrote:
>> Some signals are not interesting and should not interrupt the the
>> application.  There are two options to only interrupt an application
>> on some signals:
>>
>> * Raise an exception in the signal handler, like ``KeyboardInterrupt`` for
>>   ``SIGINT``
>> * Use a I/O multiplexing function like ``select()`` with the Python
>>   signal "wakeup" file descriptor: see the function
>>   ``signal.set_wakeup_fd()``.
>
> This section looks a bit incomplete. Some calls such as os.read() or
> os.write() will (should) return a partial result when interrupted and
> they already handled >0 bytes. Perhaps other functions have a similar
> behaviour?

In Python 3.4, os.read() is dummy: it only calls the C function read() once.

With the PEP 475, read() is only called again on EINTR if the signal
handler did not raise an exception. When read() returns EINTR, there
is "partial read", the read did not start yet.

So I don't understand what should be added to the PEP. There is no
specific change.


Matthew Woodcraft wrote:
> In any case I think PEP 475 should be explaining what is going to happen
> to signal.siginterrupt(). Will setting flag=True be supported? If so,
> will doing so change the behaviour of those parts of the stdlib which
> have already been modified to retry after EINTR?

In Python 3.4, signal.signal() calls siginterrupt(signum, True):
syscalls raise InterruptedError when interrupted by a signal. Calling
explicitly signal.siginterrupt(signum, True) doesn't change anything.

In Python 3.4, signal.siginterrupt(signum, False) reduces the cases
when InterruptedError is raised, but they are still cases when
InterruptedError is raised. The exact behaviour probably depends on
the operating system or even the version of the operating system. It's
better to not rely on siginterrupt(False) to write portable code in
Python 3.4.

With the PEP, signal.siginterrupt(signum, False) is still supported.
The PEP doesn't change the behaviour when the syscall is directly
restarted by the C library. If the function returns EINTR, the
interrupted syscall is retried if the signal handler didn't raise an
exception.

The main problem of siginterrupt(False) is that the Python signal
handler is *not* called if the C library directly retries the
interrupted syscall.

Note: if signals are blocked, the C signal handlers are not called, so
the PEP doesn't change the behaviour neither.

Victor wrote:
> I think that the stdlib should not handle InterruptedError exception
> anymore in the Python code, to simplify the code.

I modified the PEP to mention that:
https://hg.python.org/peps/rev/627fefe0394f

Victor

From victor.stinner at gmail.com  Tue Oct 28 22:13:55 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 28 Oct 2014 22:13:55 +0100
Subject: [Python-Dev] PEP 475
In-Reply-To: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
References: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
Message-ID: <CAMpsgway+pn6qNG3rrhn3tbuBFQi5p6CcnKDWtcAEUK5sZAW3w@mail.gmail.com>

Oh, I forgot the link to the PEP:
http://legacy.python.org/dev/peps/pep-0475/

Victor

From v+python at g.nevcal.com  Tue Oct 28 22:21:21 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 28 Oct 2014 14:21:21 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <545008D1.7050504@g.nevcal.com>

On 10/28/2014 6:45 AM, Stephen J. Turnbull wrote:
> because it's a fork, it's a different name
I think this is an important point, and first brought to this discussion 
here. A fork is _not_ called Python, but something else... but if it is 
kept extremely compatible and up-to-date in the hopes of reintegration 
once it proves that it solves a problem, and that there are sufficient 
users, then such hopes seem reasonable.

And targeting the new "future compatible" MSVCRT sounds like the right 
approach, although it won't solve today's problems today, but it may 
solve tomorrow's problems for a long time into the future... if the 
MinGW people can be convinced to support that new MSVCRT as well.

In addition to all the components that are enabled by MinGW 
(particularly Fortran based extensions), one must remember that the 
current Windows Python also has extensions that interface to MSVC 
libraries that have never been ported to MinGW or Linux, and may never 
be. So an incompatible MinGW-built fork will lose some of those 
extensions... they may not be important to the folks that need MinGW, 
but that would be where & why the community would be split if the MinGW 
fork is not compatible with (some version of MSVC).  Of course, the 
current MSVC-based community is _already_ having issues with 
incompatible versions of MSVC (not limiting that community to Python, 
but broader users of MSVC)... enough problems that even M$ has noticed 
that their incompatibilities are problematical, and are attempting to 
address... not just for Python, but for many other systems and libraries 
as well. So gathering around and supporting their attempts to achieve 
that, by using their new system early, when feedback can still have a 
chance of improving that new "future compatible" system, will benefit 
everyone... but that time is limited, from what Steve Dower reports of 
the schedule... hoping to be ready for Python 3.5.

So now is an excellent time for this discussion to be happening, and if 
some work can be done to fork, pull the patches together, and tweak them 
to work with the new MSVC, in the Python 3.5 timeframe, you can have a 
phenomenal result, even if it is still a fork at that time.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141028/0fd35eb2/attachment.html>

From guido at python.org  Tue Oct 28 22:35:58 2014
From: guido at python.org (Guido van Rossum)
Date: Tue, 28 Oct 2014 14:35:58 -0700
Subject: [Python-Dev] PEP 475
In-Reply-To: <CAMpsgway+pn6qNG3rrhn3tbuBFQi5p6CcnKDWtcAEUK5sZAW3w@mail.gmail.com>
References: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
 <CAMpsgway+pn6qNG3rrhn3tbuBFQi5p6CcnKDWtcAEUK5sZAW3w@mail.gmail.com>
Message-ID: <CAP7+vJKNuCY9CVuVPxTkY8o=YKqLHeixBXHDQLQeqt=D5zHT_Q@mail.gmail.com>

I would like this to happen, but I'm afraid of breakage, and I don't have
time. I would be okay if Antoine agrees to be the PEP-BDFL.

On Tue, Oct 28, 2014 at 2:13 PM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Oh, I forgot the link to the PEP:
> http://legacy.python.org/dev/peps/pep-0475/
>
> Victor
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141028/9d8a0695/attachment.html>

From solipsis at pitrou.net  Tue Oct 28 22:36:22 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 28 Oct 2014 22:36:22 +0100
Subject: [Python-Dev] PEP 475
References: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
Message-ID: <20141028223622.1af60e1c@fsol>

On Tue, 28 Oct 2014 22:07:45 +0100
Victor Stinner <victor.stinner at gmail.com> wrote:
> Hi,
> 
> At the end of August, I sent the PEP 475 which I wrote with
> Charles-Fran?ois Natali:
> 
>    https://mail.python.org/pipermail/python-dev/2014-August/136077.html
>    https://mail.python.org/pipermail/python-dev/2014-September/136101.html
> 
> Antoine Pitrou wrote " I'm +1 on the whole PEP" and R. David Murray
> wrote "Personally, I really want Python to handle EINTR for me".
> 
> What's the next step? Who wants to handle this PEP? Guido? Antoine?

Is there an implementation somewhere?
(I can handle the PEP if Guido doesn't want to and other people agree)

Regards

Antoine.



From victor.stinner at gmail.com  Wed Oct 29 00:05:57 2014
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 29 Oct 2014 00:05:57 +0100
Subject: [Python-Dev] PEP 475
In-Reply-To: <20141028223622.1af60e1c@fsol>
References: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
 <20141028223622.1af60e1c@fsol>
Message-ID: <CAMpsgwYyj8Exxy5JWWWcFz-dpa8yPO=4GPUnFFgZ_vU7bHT_-A@mail.gmail.com>

2014-10-28 22:36 GMT+01:00 Antoine Pitrou <solipsis at pitrou.net>:
> Is there an implementation somewhere?

There is no implementation yet. This time, I chose to focus on the PEP
before working on an implementation :-)

We can work on the implementation if it helps discuss the PEP. I
created a repository 3 months ago, but it has no commit yet:
https://hg.python.org/features/eintr/

The issue contains a first patch:
http://bugs.python.org/issue18885

Antoine wrote:
> (I can handle the PEP if Guido doesn't want to and other people agree)

Guido just wrote:
> I would be okay if Antoine agrees to be the PEP-BDFL.

Victor

From stephen at xemacs.org  Wed Oct 29 04:59:13 2014
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 29 Oct 2014 12:59:13 +0900
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <133A95BE786341E892819607833EF8AC@TKsamsung>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
Message-ID: <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>

Tony Kelman writes:

 > No, just hearing the words come out of my mouth they sound a little
 > nuts.  Maybe not, there are after all half a dozen or more
 > incompatible alternate Python implementations floating around. I
 > think most of them started as from-scratch rewrites rather than
 > source forks, but maybe that's irrelevant.

Well, they have different names and they clearly are not intended to
be ABI compatible, so noone expects that.  OTOH, there clearly is an
expectation among many (and not just in the Windows world, cf. all of
the distros that provide whole stacks of everything for each version
of Python) that downloaded packages will just work without
incompatibility.

 > > Well, for starters, most of python-dev would rather avoid any contact
 > > whatsoever with Windows.  I think part of the problem is that Windows
 > > developers *of* Python are *very* rare (fingers of one hand rare).
 > 
 > In my opinion the MSVC toolchain makes that problem worse, as it's far
 > harder for unix developers to have any familiarity with how things
 > work.

I've used Cygwin, I've used MinGW, and I've used VC.  Sure, the former
two are GCC-based so I have a lot of muscle memory for command-line
switches.  But that's not very important; the pain of using Windows is
what drives me away from all of them.

 > But you do have involvement and core developers from Microsoft
 > which is obviously incredibly important. Maybe even mandatory for
 > Python on Windows to be viable in your eyes.

No, I don't think that's true.  What I think *is* true is that most
developers on Windows do have access to Microsoft tools, so we do need
to provide compatibility with them.  As you say, the VC toolchain is
not all things to all men, but what's visible to python-dev makes it
more important than Cygwin or MinGW.  See Paul Moore's post about
communications between the scientific Python community and python-dev
for what I mean by "visible".

 > > It should be evident by now that our belief is that the large
 > > majority of Windows users is well-served by the current model
 > 
 > This is not the case at all in the scientific community. NumPy and
 > SciPy put in a lot of extra work to come up with something that is
 > compatible with the MSVC build of CPython because they have to, not
 > because they're "happy to" jump through the necessary hoops.

Agreed.  This is well-known to python-dev, and AFAICS it *is* a
concern for us.  However, as Paul points out, a bridge needs to be
built.  Your posts have been a contribution to building that bridge,
for sure, but more work on the bridge is needed.

 > Do python-dev and numpy-discussion not talk to one another?

Exactly the issue here.  To resolve this, we need to talk more.
Unfortunately, I'm not one to help build the bridge as I haven't
developed on Windows at all since about 2003.

 > I'm going to move the "extensions with MinGW-w64" part of this
 > conversation over to numpy-discussion,

As far as I can tell, that's a good idea right now.  They have the
need, they have the expertise, both of which are somewhat lacking
here.

 > Okay. I'll table the discussion with python-dev for now then.

I hope you'll be able to come pick it back up at some point.  You
might want to interact with Steve Dower off-list, as he's spearheading
the effort to move the official builds to the "stable ABI" version of
MSVC.  Once that's in place, the MinGW guys will have a stationary
target which is up to date to shoot at.




From solipsis at pitrou.net  Wed Oct 29 09:38:16 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Oct 2014 09:38:16 +0100
Subject: [Python-Dev] PEP 475
In-Reply-To: <CAMpsgwYyj8Exxy5JWWWcFz-dpa8yPO=4GPUnFFgZ_vU7bHT_-A@mail.gmail.com>
References: <CAMpsgwa1wOTtp==FzqgW0qO+PZ=xoJbmzb4i7xcyfEyW_DFrsw@mail.gmail.com>
 <20141028223622.1af60e1c@fsol>
 <CAMpsgwYyj8Exxy5JWWWcFz-dpa8yPO=4GPUnFFgZ_vU7bHT_-A@mail.gmail.com>
Message-ID: <20141029093816.104e0f01@fsol>

On Wed, 29 Oct 2014 00:05:57 +0100
Victor Stinner <victor.stinner at gmail.com> wrote:
> 2014-10-28 22:36 GMT+01:00 Antoine Pitrou <solipsis at pitrou.net>:
> > Is there an implementation somewhere?
> 
> There is no implementation yet. This time, I chose to focus on the PEP
> before working on an implementation :-)
> 
> We can work on the implementation if it helps discuss the PEP. I
> created a repository 3 months ago, but it has no commit yet:
> https://hg.python.org/features/eintr/

An implementation would help assess whether the change breaks existing
code or not (in the stdlib; I am not asking you to test third-party
packages :-)).

Regards

Antoine.

From Steve.Dower at microsoft.com  Wed Oct 29 05:17:33 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 29 Oct 2014 04:17:33 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>,
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <44c16522add546288666932fd9f2289c@DM2PR0301MB0734.namprd03.prod.outlook.com>

"You might want to interact with Steve Dower off-list"

FWIW, I'm happy to talk specifics off list, and have already been involved in a number of discussions with the numpy and Scipy guys wrt figuring out specific technical challenges or clarifying non obvious parts of dealing with Windows. (As far as coding goes, practically all my spare time is taken up already, which is why I haven't contributed directly to those projects, much as I'd like to.)

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Stephen J. Turnbull<mailto:stephen at xemacs.org>
Sent: ?10/?28/?2014 20:59
To: Tony Kelman<mailto:kelman at berkeley.edu>
Cc: python-dev at python.org<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Status of C compilers for Python on Windows

Tony Kelman writes:

 > No, just hearing the words come out of my mouth they sound a little
 > nuts.  Maybe not, there are after all half a dozen or more
 > incompatible alternate Python implementations floating around. I
 > think most of them started as from-scratch rewrites rather than
 > source forks, but maybe that's irrelevant.

Well, they have different names and they clearly are not intended to
be ABI compatible, so noone expects that.  OTOH, there clearly is an
expectation among many (and not just in the Windows world, cf. all of
the distros that provide whole stacks of everything for each version
of Python) that downloaded packages will just work without
incompatibility.

 > > Well, for starters, most of python-dev would rather avoid any contact
 > > whatsoever with Windows.  I think part of the problem is that Windows
 > > developers *of* Python are *very* rare (fingers of one hand rare).
 >
 > In my opinion the MSVC toolchain makes that problem worse, as it's far
 > harder for unix developers to have any familiarity with how things
 > work.

I've used Cygwin, I've used MinGW, and I've used VC.  Sure, the former
two are GCC-based so I have a lot of muscle memory for command-line
switches.  But that's not very important; the pain of using Windows is
what drives me away from all of them.

 > But you do have involvement and core developers from Microsoft
 > which is obviously incredibly important. Maybe even mandatory for
 > Python on Windows to be viable in your eyes.

No, I don't think that's true.  What I think *is* true is that most
developers on Windows do have access to Microsoft tools, so we do need
to provide compatibility with them.  As you say, the VC toolchain is
not all things to all men, but what's visible to python-dev makes it
more important than Cygwin or MinGW.  See Paul Moore's post about
communications between the scientific Python community and python-dev
for what I mean by "visible".

 > > It should be evident by now that our belief is that the large
 > > majority of Windows users is well-served by the current model
 >
 > This is not the case at all in the scientific community. NumPy and
 > SciPy put in a lot of extra work to come up with something that is
 > compatible with the MSVC build of CPython because they have to, not
 > because they're "happy to" jump through the necessary hoops.

Agreed.  This is well-known to python-dev, and AFAICS it *is* a
concern for us.  However, as Paul points out, a bridge needs to be
built.  Your posts have been a contribution to building that bridge,
for sure, but more work on the bridge is needed.

 > Do python-dev and numpy-discussion not talk to one another?

Exactly the issue here.  To resolve this, we need to talk more.
Unfortunately, I'm not one to help build the bridge as I haven't
developed on Windows at all since about 2003.

 > I'm going to move the "extensions with MinGW-w64" part of this
 > conversation over to numpy-discussion,

As far as I can tell, that's a good idea right now.  They have the
need, they have the expertise, both of which are somewhat lacking
here.

 > Okay. I'll table the discussion with python-dev for now then.

I hope you'll be able to come pick it back up at some point.  You
might want to interact with Steve Dower off-list, as he's spearheading
the effort to move the official builds to the "stable ABI" version of
MSVC.  Once that's in place, the MinGW guys will have a stationary
target which is up to date to shoot at.



_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/2a432673/attachment.html>

From tseaver at palladion.com  Wed Oct 29 15:22:14 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 29 Oct 2014 10:22:14 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <m2qt6n$nlk$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/28/2014 11:59 PM, Stephen J. Turnbull wrote:

> most developers on Windows do have access to Microsoft tool

I assume you mean python-dev folks who work on Windows:  it is certainly
not true for the vast majority of develoeprs who use Python on Windows,
who don't have the toolchain build their own C extensions.


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlRQ+BYACgkQ+gerLs4ltQ7L8QCeJ4NYs+//39O0dUUNqG1lXy1Z
7GMAniUCjmodCfKAVBF/yeOv3GrR03Fm
=n4bm
-----END PGP SIGNATURE-----


From rdmurray at bitdance.com  Wed Oct 29 15:31:50 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 29 Oct 2014 10:31:50 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m2qt6n$nlk$1@ger.gmane.org>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
Message-ID: <20141029143151.2E9C2250DAC@webabinitio.net>

On Wed, 29 Oct 2014 10:22:14 -0400, Tres Seaver <tseaver at palladion.com> wrote:
> On 10/28/2014 11:59 PM, Stephen J. Turnbull wrote:
> 
> > most developers on Windows do have access to Microsoft tool
> 
> I assume you mean python-dev folks who work on Windows:  it is certainly
> not true for the vast majority of develoeprs who use Python on Windows,
> who don't have the toolchain build their own C extensions.

I'm pretty sure he meant "most people who develop software for Windows",
even though that's not how he phrased it.  But this does not include, as
you point out, people who develop Python software that *also* works on
Windows.

If you are writing code targeted for Windows, I think you are very
likely to have an MSDN subscription of some sort if your package includes C
code.  I'm sure it's not 100%, though.

--David

From solipsis at pitrou.net  Wed Oct 29 15:46:14 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Oct 2014 15:46:14 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
Message-ID: <20141029154614.5de0c6ba@fsol>

On Wed, 29 Oct 2014 10:31:50 -0400
"R. David Murray" <rdmurray at bitdance.com> wrote:

> On Wed, 29 Oct 2014 10:22:14 -0400, Tres Seaver <tseaver at palladion.com> wrote:
> > On 10/28/2014 11:59 PM, Stephen J. Turnbull wrote:
> > 
> > > most developers on Windows do have access to Microsoft tool
> > 
> > I assume you mean python-dev folks who work on Windows:  it is certainly
> > not true for the vast majority of develoeprs who use Python on Windows,
> > who don't have the toolchain build their own C extensions.
> 
> I'm pretty sure he meant "most people who develop software for Windows",
> even though that's not how he phrased it.  But this does not include, as
> you point out, people who develop Python software that *also* works on
> Windows.
> 
> If you are writing code targeted for Windows, I think you are very
> likely to have an MSDN subscription of some sort if your package includes C
> code.  I'm sure it's not 100%, though.

You can use Express editions of Visual Studio.

Regards

Antoine.



From ncoghlan at gmail.com  Wed Oct 29 15:55:15 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 30 Oct 2014 00:55:15 +1000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141029154614.5de0c6ba@fsol>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
Message-ID: <CADiSq7fRUXPs9UriaRn+yRJ44VP=wgrou3D_6LbvCqAeCqvKrA@mail.gmail.com>

On 30 October 2014 00:46, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Wed, 29 Oct 2014 10:31:50 -0400
> "R. David Murray" <rdmurray at bitdance.com> wrote:
>
>> On Wed, 29 Oct 2014 10:22:14 -0400, Tres Seaver <tseaver at palladion.com> wrote:
>> > On 10/28/2014 11:59 PM, Stephen J. Turnbull wrote:
>> >
>> > > most developers on Windows do have access to Microsoft tool
>> >
>> > I assume you mean python-dev folks who work on Windows:  it is certainly
>> > not true for the vast majority of develoeprs who use Python on Windows,
>> > who don't have the toolchain build their own C extensions.
>>
>> I'm pretty sure he meant "most people who develop software for Windows",
>> even though that's not how he phrased it.  But this does not include, as
>> you point out, people who develop Python software that *also* works on
>> Windows.
>>
>> If you are writing code targeted for Windows, I think you are very
>> likely to have an MSDN subscription of some sort if your package includes C
>> code.  I'm sure it's not 100%, though.
>
> You can use Express editions of Visual Studio.

And the PSF can help out if folks working on Python software for
Windows need an MSDN subscription. The objections general aren't about
the availability (or lack thereof) of Windows build tools (especially
now the Python 2.7 compiler availability issue has been addressed via
http://www.microsoft.com/en-us/download/details.aspx?id=44266), it's
more about:

* wanting to build for Windows within the scope of an existing POSIX
based build automation system
* folks that prefer to use only open source software themselves
wanting to make their projects available to Windows users

"Python as first language" is still a relatively novel phenomenon, and
the existing distribution infrastructure certainly wasn't originally
designed for that use case.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tseaver at palladion.com  Wed Oct 29 16:07:53 2014
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 29 Oct 2014 11:07:53 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141029143151.2E9C2250DAC@webabinitio.net>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
Message-ID: <m2qvsb$94a$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 10/29/2014 10:31 AM, R. David Murray wrote:

> If you are writing code targeted for Windows, I think you are very 
> likely to have an MSDN subscription of some sort if your package
> includes C code.  I'm sure it's not 100%, though.

My experience with distributing distributions-with-extensions indicates
that the vast majority of Windows developers who are "downstream" users
for those distributions *cannot* build them from source:  if there is no
MSI / bdist_win (maybe now wheel), they won't use the project.

(Note that "having an MSDN subscription" is not the same as "knowing how
to configure which compiler such that it can bulid extensions against an
installed Python binary").


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlRRAskACgkQ+gerLs4ltQ7ILQCfbFCmgZqH+mZa28bQwjNuZruK
6BcAoLG/fxhi4LBkAgZoXNaxq6gi+Pbx
=8OvV
-----END PGP SIGNATURE-----


From ncoghlan at gmail.com  Wed Oct 29 16:09:45 2014
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 30 Oct 2014 01:09:45 +1000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <133A95BE786341E892819607833EF8AC@TKsamsung>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
Message-ID: <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>

(Paul Moore already covered most of this, but I'll go into a bit more
detail in a couple of areas)

On 29 October 2014 00:46, Tony Kelman <kelman at berkeley.edu> wrote:
> Stephen J. Turnbull:
>> It should be evident by now that our belief is that the large majority
>> of Windows users is well-served by the current model
>
>
> This is not the case at all in the scientific community. NumPy and SciPy
> put in a lot of extra work to come up with something that is compatible
> with the MSVC build of CPython because they have to, not because they're
> "happy to" jump through the necessary hoops.

Lots of folks are happy with POSIX emulation layers on Windows, as
they're OK with "basically works" rather than "works like any other
native application". "Basically works" isn't sufficient for many
Python-on-Windows use cases though, so the core ABI is a platform
native one, rather than a POSIX emulation.

This makes Python fit in more cleanly with other Windows applications,
but makes it harder to write Python applications that span both POSIX
and Windows. The degree to which a language runtime abstracts away the
details of the underlying operating system is a complex trade-off,
from C (where the C/C++ ABI is a core part of what *defines* an
operating system), through Python (which lets the developer largely
choose whether to use high level abstractions or lower level platform
specific APIs) to the JVM and CLR (which both largely abstract away
many of the details of the underlying operating system, aside from
carefully contained "native" code snippets).

This approach *does* fragment the community a bit, into at least
"Python-on-POSIX-only", "Python-on-Windows-only",
"platform-independent-Python" and "Python-on-POSIX-and-Windows" (where
the latter code has to deal directly with the differences between
Windows and POSIX because it needs the low level platform specific
APIs).

I personally consider that trade-off worth it, as it gives Python a
scope of applicability that more isolated runtimes lack, as well as
providing access to sophisticated platform level capabilities far more
cheaply than would be possible with a higher level of default
isolation.

> The situation today with
> NumPy AIUI is they already have to build with a custom hacked MinGW
> toolchain that only one person knows how to use. Evidently until very
> recently they were using a many-years-old 32-bit-only build of GCC 3.x.
> Do python-dev and numpy-discussion not talk to one another? I get that
> not everyone uses numpy, or Windows, but it sounds like there's very
> little understanding, or even awareness, of the issues they face.

There have been major ongoing communication problems stemming from the
wildly different assumptions that go into "programming as a tool for
building applications" (what python-dev and distutils-sig mostly do)
and "programming as a tool for thinking" (what scientists and data
analysts mostly do). Adding FORTRAN dependencies to the mix (which
*none* of the platform vendors really support properly) then makes the
whole problem of dealing with scientific tools even more alien to most
professional developers.

Those differing assumptions then meant that the two groups end up
frequently talk past each other because the worldviews are too
different. While that's starting to improve of late (as the above
distinction becomes better recognised), there's still a long way to
go.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Wed Oct 29 16:21:08 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Oct 2014 16:21:08 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <m2qvsb$94a$1@ger.gmane.org>
Message-ID: <20141029162108.43db29f1@fsol>

On Wed, 29 Oct 2014 11:07:53 -0400
Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 10/29/2014 10:31 AM, R. David Murray wrote:
> 
> > If you are writing code targeted for Windows, I think you are very 
> > likely to have an MSDN subscription of some sort if your package
> > includes C code.  I'm sure it's not 100%, though.
> 
> My experience with distributing distributions-with-extensions indicates
> that the vast majority of Windows developers who are "downstream" users
> for those distributions *cannot* build them from source:  if there is no
> MSI / bdist_win (maybe now wheel), they won't use the project.
> 
> (Note that "having an MSDN subscription" is not the same as "knowing how
> to configure which compiler such that it can bulid extensions against an
> installed Python binary").

I don't think you have to configure anything. Just install the right
Visual Studio version and it's done.

Regards

Antoine.



From solipsis at pitrou.net  Wed Oct 29 16:25:15 2014
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 29 Oct 2014 16:25:15 +0100
Subject: [Python-Dev] Status of C compilers for Python on Windows
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
Message-ID: <20141029162515.17cd3afb@fsol>

On Thu, 30 Oct 2014 01:09:45 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Lots of folks are happy with POSIX emulation layers on Windows, as
> they're OK with "basically works" rather than "works like any other
> native application". "Basically works" isn't sufficient for many
> Python-on-Windows use cases though, so the core ABI is a platform
> native one, rather than a POSIX emulation.
> 
> This makes Python fit in more cleanly with other Windows applications,
> but makes it harder to write Python applications that span both POSIX
> and Windows.

I don't really understanding why that's the case. Only the
building and packaging may be more difficult, and that assumes you're
familiar with mingw32. But mingw32, AFAIK, doesn't make the Windows
runtime magically POSIX-compatible (Cygwin does, to some extent).

Regards

Antoine.



From njs at pobox.com  Wed Oct 29 16:31:17 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 29 Oct 2014 15:31:17 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141029154614.5de0c6ba@fsol>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
Message-ID: <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>

On 29 Oct 2014 14:47, "Antoine Pitrou" <solipsis at pitrou.net> wrote:
>
> On Wed, 29 Oct 2014 10:31:50 -0400
> "R. David Murray" <rdmurray at bitdance.com> wrote:
>
> > On Wed, 29 Oct 2014 10:22:14 -0400, Tres Seaver <tseaver at palladion.com>
wrote:
> > > On 10/28/2014 11:59 PM, Stephen J. Turnbull wrote:
> > >
> > > > most developers on Windows do have access to Microsoft tool
> > >
> > > I assume you mean python-dev folks who work on Windows:  it is
certainly
> > > not true for the vast majority of develoeprs who use Python on
Windows,
> > > who don't have the toolchain build their own C extensions.
> >
> > I'm pretty sure he meant "most people who develop software for Windows",
> > even though that's not how he phrased it.  But this does not include, as
> > you point out, people who develop Python software that *also* works on
> > Windows.
> >
> > If you are writing code targeted for Windows, I think you are very
> > likely to have an MSDN subscription of some sort if your package
includes C
> > code.  I'm sure it's not 100%, though.
>
> You can use Express editions of Visual Studio.

IIUC, the express edition compilers are 32-bit only, and what you actually
want are the "SDK compilers":
https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows

These are freely downloadable by anyone, no msdn subscription required, but
only if you know where to find them!

AFAICT the main obstacle to using MSVC to build python extensions (assuming
it can handle your code at all) is not anything technical, but rather that
there's no clear and correct tutorial on how to do it, and lots of
confusion and misinformation circulating.

-n
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/1c64d124/attachment-0001.html>

From rdmurray at bitdance.com  Wed Oct 29 17:19:02 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 29 Oct 2014 12:19:02 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
Message-ID: <20141029161902.ECB1B25005B@webabinitio.net>

On Thu, 30 Oct 2014 01:09:45 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> (Paul Moore already covered most of this, but I'll go into a bit more
> detail in a couple of areas)
> 
> On 29 October 2014 00:46, Tony Kelman <kelman at berkeley.edu> wrote:
> > Stephen J. Turnbull:
> >> It should be evident by now that our belief is that the large majority
> >> of Windows users is well-served by the current model
> >
> >
> > This is not the case at all in the scientific community. NumPy and SciPy
> > put in a lot of extra work to come up with something that is compatible
> > with the MSVC build of CPython because they have to, not because they're
> > "happy to" jump through the necessary hoops.
> 
> Lots of folks are happy with POSIX emulation layers on Windows, as
> they're OK with "basically works" rather than "works like any other
> native application". "Basically works" isn't sufficient for many
> Python-on-Windows use cases though, so the core ABI is a platform
> native one, rather than a POSIX emulation.

Since some of the context here is scientific use of Python, it might be
a useful bit of perspective to know that, while there are doubtless many
scientists using windows and using the windows native interfaces
happily, the Software Carpentry bootcamps that aim to give scientists
the basic framework for making better use of computers and software and
programming have as one foundation the bash shell, taught on Windows via
git-bash.  That is, the common toolset being taught to scientists (by
Software Carpentry) is the posix one, even on Windows.

--David

From Steve.Dower at microsoft.com  Wed Oct 29 16:37:48 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 29 Oct 2014 15:37:48 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141029162108.43db29f1@fsol>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <m2qvsb$94a$1@ger.gmane.org>
 <20141029162108.43db29f1@fsol>
Message-ID: <578f8e0c798746b3becce31f2a0abff2@DM2PR0301MB0734.namprd03.prod.outlook.com>

Antoine Pitrou wrote:
> On Wed, 29 Oct 2014 11:07:53 -0400
> Tres Seaver <tseaver at palladion.com> wrote:
>> -----BEGIN PGP SIGNED MESSAGE-----
>> Hash: SHA1
>> 
>> On 10/29/2014 10:31 AM, R. David Murray wrote:
>> 
>> > If you are writing code targeted for Windows, I think you are very 
>> > likely to have an MSDN subscription of some sort if your package 
>> > includes C code.  I'm sure it's not 100%, though.
>> 
>> My experience with distributing distributions-with-extensions 
>> indicates that the vast majority of Windows developers who are 
>> "downstream" users for those distributions *cannot* build them from 
>> source:  if there is no MSI / bdist_win (maybe now wheel), they won't use the project.
>>
>> (Note that "having an MSDN subscription" is not the same as "knowing 
>> how to configure which compiler such that it can bulid extensions 
>> against an installed Python binary").
> 
> I don't think you have to configure anything. Just install the right Visual Studio version and it's done.

The tricky part here is the *right* Visual Studio version. As many have noted, there were bugs/missing components in some of the older Express editions (like the 64-bit compiler integration), and even the compiler pack we released recently doesn't actually help building Python itself, though it's good for extensions. In general, VS 2008 Professional and VS 2010 Professional are easiest to set up if you can get them, the C++ Express editions are fairly easy to set up, and the Windows SDK is difficult but possible (and won't currently build CPython either, as the build depends on VS - I'm also fixing this for 3.5).

My ideal target (Utopia refined to be achievable) is for python-dev to handle the Python binaries themselves (already done) and to make them easy to bundle with applications/etc (I'm working on this for 3.5), and for PyPI to include pre-built wheels for Windows where necessary.

Note that I don't require package developers to build their own wheels, though I think that they are generally the right people to be doing it. It would be nice to create a culture of delegation, so that "someone" could be a trusted builder for a range of packages (for example, I'd love it if Continuum/Enthought/similar could provide their builds of numpy/scipy as wheels and those projects would be willing to have them be the official PyPI downloads). This is roughly how the python.org installers are handled, and while it may cause some lag between source and binary releases, I think the release cycles are slow enough that most users would only notice that "pip install numpy" now works.

Cheers,
Steve

> Regards
> 
> Antoine.

From cournape at gmail.com  Wed Oct 29 18:17:43 2014
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 29 Oct 2014 17:17:43 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141029162515.17cd3afb@fsol>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
 <20141029162515.17cd3afb@fsol>
Message-ID: <CAGY4rcXPHrwS6Dk3cj2cRGz5P6grFMct64wM69c_e2k+3QGgKQ@mail.gmail.com>

On Wed, Oct 29, 2014 at 3:25 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Thu, 30 Oct 2014 01:09:45 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > Lots of folks are happy with POSIX emulation layers on Windows, as
> > they're OK with "basically works" rather than "works like any other
> > native application". "Basically works" isn't sufficient for many
> > Python-on-Windows use cases though, so the core ABI is a platform
> > native one, rather than a POSIX emulation.
> >
> > This makes Python fit in more cleanly with other Windows applications,
> > but makes it harder to write Python applications that span both POSIX
> > and Windows.
>
> I don't really understanding why that's the case. Only the
> building and packaging may be more difficult, and that assumes you're
> familiar with mingw32. But mingw32, AFAIK, doesn't make the Windows
> runtime magically POSIX-compatible (Cygwin does, to some extent).
>

mingw32 is a more compliant C compiler (VS2008 does not implement much from
C89), and it does implement quite a few things not implemented in the C
runtime, especially for math.

But TBH, those are not compelling cases to build python itself on mingw,
only to better support C extensions with mingw.

David


> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/cournape%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/d8863486/attachment.html>

From cournape at gmail.com  Wed Oct 29 18:19:08 2014
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 29 Oct 2014 17:19:08 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAGY4rcXPHrwS6Dk3cj2cRGz5P6grFMct64wM69c_e2k+3QGgKQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
 <20141029162515.17cd3afb@fsol>
 <CAGY4rcXPHrwS6Dk3cj2cRGz5P6grFMct64wM69c_e2k+3QGgKQ@mail.gmail.com>
Message-ID: <CAGY4rcVv95-K7QQQ-9yDUOMcZ0o=t9Uq9M3Qf6TX15diLk+tTg@mail.gmail.com>

On Wed, Oct 29, 2014 at 5:17 PM, David Cournapeau <cournape at gmail.com>
wrote:

>
>
> On Wed, Oct 29, 2014 at 3:25 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
>
>> On Thu, 30 Oct 2014 01:09:45 +1000
>> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> >
>> > Lots of folks are happy with POSIX emulation layers on Windows, as
>> > they're OK with "basically works" rather than "works like any other
>> > native application". "Basically works" isn't sufficient for many
>> > Python-on-Windows use cases though, so the core ABI is a platform
>> > native one, rather than a POSIX emulation.
>> >
>> > This makes Python fit in more cleanly with other Windows applications,
>> > but makes it harder to write Python applications that span both POSIX
>> > and Windows.
>>
>> I don't really understanding why that's the case. Only the
>> building and packaging may be more difficult, and that assumes you're
>> familiar with mingw32. But mingw32, AFAIK, doesn't make the Windows
>> runtime magically POSIX-compatible (Cygwin does, to some extent).
>>
>
> mingw32 is a more compliant C compiler (VS2008 does not implement much
> from C89)
>

That should read much C99, of course, otherwise VS 2008 would have been a
completely useless C compiler !

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/527b0367/attachment.html>

From donald at stufft.io  Wed Oct 29 18:20:27 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 29 Oct 2014 13:20:27 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <578f8e0c798746b3becce31f2a0abff2@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <m2qvsb$94a$1@ger.gmane.org>
 <20141029162108.43db29f1@fsol>
 <578f8e0c798746b3becce31f2a0abff2@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <FC7A7770-A6AA-49C5-B62E-82AA1EC7D276@stufft.io>


> On Oct 29, 2014, at 11:37 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> 
> Antoine Pitrou wrote:
>> On Wed, 29 Oct 2014 11:07:53 -0400
>> Tres Seaver <tseaver at palladion.com> wrote:
>>> -----BEGIN PGP SIGNED MESSAGE-----
>>> Hash: SHA1
>>> 
>>> On 10/29/2014 10:31 AM, R. David Murray wrote:
>>> 
>>>> If you are writing code targeted for Windows, I think you are very 
>>>> likely to have an MSDN subscription of some sort if your package 
>>>> includes C code.  I'm sure it's not 100%, though.
>>> 
>>> My experience with distributing distributions-with-extensions 
>>> indicates that the vast majority of Windows developers who are 
>>> "downstream" users for those distributions *cannot* build them from 
>>> source:  if there is no MSI / bdist_win (maybe now wheel), they won't use the project.
>>> 
>>> (Note that "having an MSDN subscription" is not the same as "knowing 
>>> how to configure which compiler such that it can bulid extensions 
>>> against an installed Python binary").
>> 
>> I don't think you have to configure anything. Just install the right Visual Studio version and it's done.
> 
> The tricky part here is the *right* Visual Studio version. As many have noted, there were bugs/missing components in some of the older Express editions (like the 64-bit compiler integration), and even the compiler pack we released recently doesn't actually help building Python itself, though it's good for extensions. In general, VS 2008 Professional and VS 2010 Professional are easiest to set up if you can get them, the C++ Express editions are fairly easy to set up, and the Windows SDK is difficult but possible (and won't currently build CPython either, as the build depends on VS - I'm also fixing this for 3.5).
> 
> My ideal target (Utopia refined to be achievable) is for python-dev to handle the Python binaries themselves (already done) and to make them easy to bundle with applications/etc (I'm working on this for 3.5), and for PyPI to include pre-built wheels for Windows where necessary.
> 
> Note that I don't require package developers to build their own wheels, though I think that they are generally the right people to be doing it. It would be nice to create a culture of delegation, so that "someone" could be a trusted builder for a range of packages (for example, I'd love it if Continuum/Enthought/similar could provide their builds of numpy/scipy as wheels and those projects would be willing to have them be the official PyPI downloads). This is roughly how the python.org installers are handled, and while it may cause some lag between source and binary releases, I think the release cycles are slow enough that most users would only notice that "pip install numpy" now works.

For the record, a long term ?down the road? kind of thing I want to do is have PyPI have a build farm which can build Wheels. So that people upload a source distribution to PyPI and it kicks off Wheel builds on the various platforms automatically.

What this looks like is a complete unknown, all I have is the general idea.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From v+python at g.nevcal.com  Wed Oct 29 20:34:59 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 29 Oct 2014 12:34:59 -0700
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
Message-ID: <54514163.4000607@g.nevcal.com>

New package manager from M$... article here 
<http://www.neowin.net/news/windows-10-oneget-a-linux-style-package-management-framework>.

It seems doubtful that M$ will eliminate .msi (their obscure, hard to 
configure and use, installation format), so it seems doubtful that the 
addition of OneGet will _force_ any changes to Python packaging.

However, it does open the question in my mind about whether there will 
be any _benefits_ of OneGet that would inspire helpful, useful changes 
to Python packaging. They speak of "trusted repositories", and the like, 
and it sounds like a the various *nix package managers (apt-get, et 
alia), but perhaps allowing multiple repositories rather than just a 
single source vendor repository (I'm actually not sure if *nix package 
managers allow multiple repositories or not, but from the way people 
talk about them, it always sounds like a "distribution" also provides "a 
repository" of additional packages).

"trusted repositories" sounds more like Perl's CPAN.

One of the links 
<http://blogs.technet.com/b/windowsserver/archive/2014/04/03/windows-management-framework-v5-preview.aspx> 
contains this quote: "This first version of OneGet installs and searches 
software from Chocolatey repositories.  Support of additional 
repositories will come in subsequent versions."

I have no clue what a Chocolatey repository is (yet, will Google), but 
unknown others will come, it says... whether it is possible to write a 
"repository plugin" such that Perl's CPAN or Python's PyPI or other 
preexisting repositories can be accessed is not clear.

The relationship between PowerShell and OneGet is not clear either... is 
OneGet written in PowerShell, or is PowerShell just one way to invoke 
OneGet, or???

Just a heads up.






-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/6ae2df6c/attachment.html>

From donald at stufft.io  Wed Oct 29 20:44:25 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 29 Oct 2014 15:44:25 -0400
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <54514163.4000607@g.nevcal.com>
References: <54514163.4000607@g.nevcal.com>
Message-ID: <D891267E-94D2-468F-B773-CD781412B9CD@stufft.io>


> On Oct 29, 2014, at 3:34 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> 
> New package manager from M$... article here <http://www.neowin.net/news/windows-10-oneget-a-linux-style-package-management-framework>.
> 
> It seems doubtful that M$ will eliminate .msi (their obscure, hard to configure and use, installation format), so it seems doubtful that the addition of OneGet will _force_ any changes to Python packaging.
> 
> However, it does open the question in my mind about whether there will be any _benefits_ of OneGet that would inspire helpful, useful changes to Python packaging. They speak of "trusted repositories", and the like, and it sounds like a the various *nix package managers (apt-get, et alia), but perhaps allowing multiple repositories rather than just a single source vendor repository (I'm actually not sure if *nix package managers allow multiple repositories or not, but from the way people talk about them, it always sounds like a "distribution" also provides "a repository" of additional packages).
> 
> "trusted repositories" sounds more like Perl's CPAN.
> 
> One of the links <http://blogs.technet.com/b/windowsserver/archive/2014/04/03/windows-management-framework-v5-preview.aspx> contains this quote: "This first version of OneGet installs and searches software from Chocolatey repositories.  Support of additional repositories will come in subsequent versions."
> 
> I have no clue what a Chocolatey repository is (yet, will Google), but unknown others will come, it says... whether it is possible to write a "repository plugin" such that Perl's CPAN or Python's PyPI or other preexisting repositories can be accessed is not clear.
> 
> The relationship between PowerShell and OneGet is not clear either... is OneGet written in PowerShell, or is PowerShell just one way to invoke OneGet, or???
> 
> Just a heads up.
> 

It appears to be a package manager manager. Chocolatey is one of the third party package managers available on Windows.

I also just learned that OneGet is apparently OSS and developed on github (https://github.com/OneGet/oneget <https://github.com/OneGet/oneget>).

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/a161bdeb/attachment.html>

From p.f.moore at gmail.com  Wed Oct 29 21:05:42 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 20:05:42 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
Message-ID: <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>

On 29 October 2014 15:31, Nathaniel Smith <njs at pobox.com> wrote:
>> You can use Express editions of Visual Studio.
>
> IIUC, the express edition compilers are 32-bit only, and what you actually
> want are the "SDK compilers":
> https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
>
> These are freely downloadable by anyone, no msdn subscription required, but
> only if you know where to find them!
>
> AFAICT the main obstacle to using MSVC to build python extensions (assuming
> it can handle your code at all) is not anything technical, but rather that
> there's no clear and correct tutorial on how to do it, and lots of confusion
> and misinformation circulating.

Would it help if I wrote a document explaining how to set up the MS
compilers (free and paid for) to allow building of Python extensions?

There are a few provisos:

1. A lot of it will be pretty trivial: "If you have Vistal Studio
(full edition), install it. Done."
2. It will be out of date very fast as the situation for Python 3.5+
will be trivial across the board.
3. I don't have anywhere particularly authoritative to host it (maybe
the Python Wiki?) and it could easily get lost in the huge swamp of
variously outdated, over-complicated, or otherwise alternative
documents available. Ideally I'd like someone to suggest an "official"
location I could use.

I don't want to do this if it won't be useful, as it'll take me a bit
of effort to confirm the process for the only non-trivial scenario
(64-bit Python 3.3/3.4 with free tools). But if people think it would
help, that's fine, I volunteer.

Paul

PS Even if I don't get positive feedback, I may just say "to heck with
it" and do it anyway, because it *is* so trivial :-) I just won't
promise.

From kelman at berkeley.edu  Wed Oct 29 20:23:19 2014
From: kelman at berkeley.edu (Tony Kelman)
Date: Wed, 29 Oct 2014 12:23:19 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung><87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp><133A95BE786341E892819607833EF8AC@TKsamsung>
 <CADiSq7dhACKfr9rrL3dpnhNp4KEranWEA_7t1rBF64LgwrABcQ@mail.gmail.com>
Message-ID: <001BB3093DE24394A43BA4599E9D1DAD@TKsamsung>

Stephen J. Turnbull:
> the pain of using Windows is what drives me away from all of them.

Enough that you are not able to make the software you write usable on
Windows? I see your point and agree with it - I don't even like Windows
much at all, but supporting it is important for plenty of reasons.

Steve Dower:
> FWIW, I'm happy to talk specifics off list

Off-list, on-list, either way. I've read the blog post about the CRT
refactoring 
http://blogs.msdn.com/b/vcblog/archive/2014/06/10/the-great-crt-refactoring.aspx
but have not yet experimented with the early CTP versions of Visual Studio
"14." Is there any plan to promote these runtimes to installed-by-default
components of the operating system in future service packs or new releases
of Windows? The fact that the redistributables are a separate download and
not always there by default means MinGW developers are not quite so
optimistic about them suddently solving all problems. In the embedded-
Python-for-IJulia pull request on github for example, it turns out the
Python msi installer does not include the necessary runtimes and wouldn't
work by itself on a brand new computer.

It would also help if there were a preview version of just the runtime
libraries available. More far-fetched and not likely to happen would be a
smaller "command line tools for Visual Studio" type package, like Apple
provides for Xcode, with only the compiler, tools, and libraries needed
to build C/C++ libraries or Python extensions but not the whole IDE.

Nick Coghlan:
> * wanting to build for Windows within the scope of an existing POSIX
> based build automation system
> * folks that prefer to use only open source software themselves
> wanting to make their projects available to Windows users

Don't forget: * MSVC is not capable of compiling the code I need to use.
That's the crux of the scientific-software problem. If it were possible to
use MSVC for everything, I'd probably suck it up and spend my time writing
CMake build systems for every project I wanted to get to work on Windows.
Most software I work with is a library first, Python extension second, if
at all.

Nick Coghlan:
> Lots of folks are happy with POSIX emulation layers on Windows, as
> they're OK with "basically works" rather than "works like any other
> native application". "Basically works" isn't sufficient for many
> Python-on-Windows use cases though, so the core ABI is a platform
> native one, rather than a POSIX emulation.

To clarify here: MinGW does not use a POSIX layer at all. Cygwin does,
MSYS does (it's based on Cygwin with relatively minor changes), but when
we're discussing the MinGW.org or MinGW-w64 compilers, there is no POSIX.
There is a GCC runtime library, which can be linked statically if you
choose, though it's not always a good idea to do so. Cygwin or MSYS provide
a POSIX build environment where you can run bash, make, etc, but with MinGW
you are not targeting or depending on that build environment with the
compiled binaries.

The distinction in runtime libraries is that MinGW links to msvcrt.dll,
which is from an old version of Visual Studio but is installed by default
on every version of Windows since XP. The python.org installers are
compiled with a particular more recent version of Visual Studio, so
they (and hence all extensions used with them) depend on the specific
corresponding msvcr##.dll.

Most MinGW developers and users would rather avoid the dll hell and issues
stemming from choosing a particular MSVC runtime and having to stick with
it for everything. NumPy was able to coax a custom MinGW-w64 build into
linking to CPython's designated runtime and statically link all the GCC
libraries (these are 2 separate issues however). I do think some sensitivity
to the requirements of the existing Windows CPython/PyPI environment from
the MinGW-w64 side would be valuable. If NumPy's needs can be met with
fewer modifications and without having to rebuild a custom GCC from source,
I think that would be beneficial for everyone. We'll see how many of the
MinGW-w64 developers I can get to agree with me on that. Any changes that
may need to be upstreamed to GCC will have their own set of challenges.

> Adding FORTRAN dependencies to the mix (which
> *none* of the platform vendors really support properly)

I'm not sure what you mean by platform vendors? Are you talking Conda,
Enthought, etc? To the best of my knowledge, they're using the Intel
toolchain for Fortran, at least on Windows and probably elsewhere. So
I suspect you'll find a libifcoremd.dll somewhere in those distributions.
That imposes some cost requirements on reproducing their processes that
are a bit high for average users to meet.

Donald Stufft:
> So that people upload a source distribution to PyPI and it kicks off
> Wheel builds on the various platforms automatically.
>
> What this looks like is a complete unknown, all I have is the general
> idea.

I do something similar today for C, C++, and Fortran libraries using
the MinGW-w64 cross-compiler. It looks like this
https://build.opensuse.org/project/show/windows:mingw:win64
I write a spec file and upload source, and get back dll's and exe's -
compressed in RPM's in this case, along with the RPM dependency metadata.
For OSX I would look at what Homebrew does with their "bottle"
infrastructure. A build farm with Vagrant (or similar) VM's could even
be made to do the same basic thing on Windows with MSVC, at least for
binaries that MSVC is capable of compiling.

-Tony


From donald at stufft.io  Wed Oct 29 21:26:30 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 29 Oct 2014 16:26:30 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
Message-ID: <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>

This sounds like something good for packaging.python.org


> On Oct 29, 2014, at 4:05 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 29 October 2014 15:31, Nathaniel Smith <njs at pobox.com> wrote:
>>> You can use Express editions of Visual Studio.
>> 
>> IIUC, the express edition compilers are 32-bit only, and what you actually
>> want are the "SDK compilers":
>> https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
>> 
>> These are freely downloadable by anyone, no msdn subscription required, but
>> only if you know where to find them!
>> 
>> AFAICT the main obstacle to using MSVC to build python extensions (assuming
>> it can handle your code at all) is not anything technical, but rather that
>> there's no clear and correct tutorial on how to do it, and lots of confusion
>> and misinformation circulating.
> 
> Would it help if I wrote a document explaining how to set up the MS
> compilers (free and paid for) to allow building of Python extensions?
> 
> There are a few provisos:
> 
> 1. A lot of it will be pretty trivial: "If you have Vistal Studio
> (full edition), install it. Done."
> 2. It will be out of date very fast as the situation for Python 3.5+
> will be trivial across the board.
> 3. I don't have anywhere particularly authoritative to host it (maybe
> the Python Wiki?) and it could easily get lost in the huge swamp of
> variously outdated, over-complicated, or otherwise alternative
> documents available. Ideally I'd like someone to suggest an "official"
> location I could use.
> 
> I don't want to do this if it won't be useful, as it'll take me a bit
> of effort to confirm the process for the only non-trivial scenario
> (64-bit Python 3.3/3.4 with free tools). But if people think it would
> help, that's fine, I volunteer.
> 
> Paul
> 
> PS Even if I don't get positive feedback, I may just say "to heck with
> it" and do it anyway, because it *is* so trivial :-) I just won't
> promise.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/donald%40stufft.io

From p.f.moore at gmail.com  Wed Oct 29 23:09:56 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 22:09:56 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
Message-ID: <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>

On 29 October 2014 20:26, Donald Stufft <donald at stufft.io> wrote:
> This sounds like something good for packaging.python.org

Yeah, I wondered about that. I'll work up a patch for that. But the
more I think about it, it really is trivial:

- For non-free MSVC, install the appropriate version, and everything just works.
- For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
package and everything just works as long as you're using setuptools.
- For 32 bit Python 3.2-3.4, install Visual Studio Express and
everything just works.
- For 64 bit Python 3.2-3.4, install the SDK, set some environment
variables, and everything just works.
- For Python 3.5+, install the current Visual Studion Express and
everything just works.

(I propose to deem Python 2.7 without setuptools as "unsupported")

The only things I can see that need expansion are:

1. The precise versions to use (trivial to add, I'm just to lazy to
check right now).
2. Where to get VS 2010 Express (as it's no longer supported or
available from MS).
3. How to set up the SDK environment for 64-bit Python 3.2-3.4.

Before I do a write-up, I want to set up clean VMs with these
configurations, so that I can confirm the details. But I don't expect
any problems, as I've done them all before.

Paul.

From donald at stufft.io  Wed Oct 29 23:11:22 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 29 Oct 2014 18:11:22 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
Message-ID: <17A7394E-11A2-43DC-BB8D-011A4EC532B7@stufft.io>


> On Oct 29, 2014, at 6:09 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
> On 29 October 2014 20:26, Donald Stufft <donald at stufft.io> wrote:
>> This sounds like something good for packaging.python.org
> 
> Yeah, I wondered about that. I'll work up a patch for that. But the
> more I think about it, it really is trivial:
> 
> - For non-free MSVC, install the appropriate version, and everything just works.
> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
> package and everything just works as long as you're using setuptools.
> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
> everything just works.
> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
> variables, and everything just works.
> - For Python 3.5+, install the current Visual Studion Express and
> everything just works.
> 
> (I propose to deem Python 2.7 without setuptools as "unsupported")
> 
> The only things I can see that need expansion are:
> 
> 1. The precise versions to use (trivial to add, I'm just to lazy to
> check right now).
> 2. Where to get VS 2010 Express (as it's no longer supported or
> available from MS).
> 3. How to set up the SDK environment for 64-bit Python 3.2-3.4.
> 
> Before I do a write-up, I want to set up clean VMs with these
> configurations, so that I can confirm the details. But I don't expect
> any problems, as I've done them all before.
> 
> Paul.

I think it?d be good even if considered trivial. I know I certainly have no
idea which pieces I needed to do that.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From ethan at stoneleaf.us  Wed Oct 29 23:19:29 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 29 Oct 2014 15:19:29 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
Message-ID: <545167F1.1030502@stoneleaf.us>

On 10/29/2014 03:09 PM, Paul Moore wrote:
> On 29 October 2014 20:26, Donald Stufft <donald at stufft.io> wrote:
>> This sounds like something good for packaging.python.org
>
> Yeah, I wondered about that. I'll work up a patch for that. But the
> more I think about it, it really is trivial:

I am reminded of an interview question I was once asked which was prefaced with: "Here's an easy one..."

My reply was, if you know the answer, it's easy!


> - For non-free MSVC, install the appropriate version, and everything just works.
> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
> package and everything just works as long as you're using setuptools.
> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
> everything just works.
> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
> variables, and everything just works.
> - For Python 3.5+, install the current Visual Studion Express and
> everything just works.

I would suggest
   - where to get these packages
   - where to get any dependencies
   - any options to [not] specify during install
   - what environment variables to what value
   - where one should be at when one starts the compile process

and, of course, a gotchas section for uncommon but frustrating things that might go wrong.

And thanks for doing this!

--
~Ethan~

From p.f.moore at gmail.com  Wed Oct 29 23:46:45 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 22:46:45 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <545167F1.1030502@stoneleaf.us>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
Message-ID: <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>

On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>> Yeah, I wondered about that. I'll work up a patch for that. But the
>> more I think about it, it really is trivial:
>
> I am reminded of an interview question I was once asked which was prefaced
> with: "Here's an easy one..."
>
> My reply was, if you know the answer, it's easy!

Yeah, I know what you mean. My take on this is that I agree it's not
easy if you don't know and can't get access to the information, but if
you can, there's very little to it.

>> - For non-free MSVC, install the appropriate version, and everything just
>> works.
>> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
>> package and everything just works as long as you're using setuptools.
>> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
>> everything just works.
>> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
>> variables, and everything just works.
>> - For Python 3.5+, install the current Visual Studion Express and
>> everything just works.
>
>
> I would suggest
>   - where to get these packages

Conceded. Working out how to point people at stuff on MSDN is a
challenge, things seem to move around. Maybe Steve Dower could help
here with canonical URLs for some of them (IIRC, he provided one for
the VS compilers for Python 2.7 package when it was announced).

For the paid versions, I'm going to assume that anyone who paid for a
compiler and doesn't know where their copy is, probably can't be
helped ;-)

And of course there's the awkward problem that VS 2010 Express is
unsupported and therefore no longer available from *any* official
location. I can't solve that, sadly.

>   - where to get any dependencies

There are none. I could state that explicitly, I guess.

>   - any options to [not] specify during install

I'll have to go through the installs again just to be sure, but I'm
pretty certain "Select the default for everything" is correct.

>   - what environment variables to what value

None, except for the SDK which I did say I needed to test out and
cover in more detail.

>   - where one should be at when one starts the compile process

I don't understand this. It's just "pip wheel foo" to build a wheel
for foo (which will be downloaded), or "pip wheel ." or  "python
setup.py bdist_wheel" as you prefer for a local package. That's
standard distutils/setuptools/pip extension building. I don't propose
to cover that, just how to *set up* the environment.

With the sole exception of the SDK case, once installed, everything
just works everywhere, nothing to set up, no special environment to
configure. Start up a new cmd or powershell console (or use the one
you're already in) and go. Maybe Unix users expect more complexity
because it's not that simple on Unix? But I thought it was - install
the appropriate OS packages and that's it?

> and, of course, a gotchas section for uncommon but frustrating things that
> might go wrong.

Hmm, I see your point here, but I'm not sure what I might include. You
*can* get in a mess [1] but generally I don't as I'm an experienced
Windows user. I also don't want to offer to debug and fix everyone's
problems in getting things set up, so offering to collect and document
"common issues" that I've seen is impractical. Maybe a section
entitled "Common Issues and Their Solutions", with some placeholder
text saying "If you have any issues installing any of the compiler
packages mentioned, please document what went wrong and how to fix it,
and submit a PR!" would do?

> And thanks for doing this!

No problem!

Paul

[1] I once spent a *long* time fighting failed installs of the Windows
SDK. Turns out it was because I was trying to install a 32-bit SDK on
a 64-bit machine (doh!), and the installer really doesn't like that
:-( About all I could say to document that is "Read the instructions
properly" and "I'm sorry that the MS installers fail really badly when
faced with relatively obvious idiot-user errors, but I can't do
anything about it :-("

From ethan at stoneleaf.us  Thu Oct 30 00:02:26 2014
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 29 Oct 2014 16:02:26 -0700
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
Message-ID: <54517202.2010109@stoneleaf.us>

On 10/29/2014 03:46 PM, Paul Moore wrote:
> On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>>
>>    - where one should be at when one starts the compile process
>
> I don't understand this. It's just "pip wheel foo" to build a wheel
> for foo (which will be downloaded), or "pip wheel ." or  "python
> setup.py bdist_wheel" as you prefer for a local package.

Hmmm...  That looks like it's for installing/compiling somebody else's package.  Is that last command sufficient to 
prepare one's own wheel for uploading to PyPI, or there something else to do?

--
~Ethan~

From donald at stufft.io  Thu Oct 30 00:05:34 2014
From: donald at stufft.io (Donald Stufft)
Date: Wed, 29 Oct 2014 19:05:34 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <54517202.2010109@stoneleaf.us>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
 <54517202.2010109@stoneleaf.us>
Message-ID: <CDD6A7F5-D035-4093-84D2-428EABA8FCC3@stufft.io>


> On Oct 29, 2014, at 7:02 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> 
> On 10/29/2014 03:46 PM, Paul Moore wrote:
>> On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>>> 
>>>   - where one should be at when one starts the compile process
>> 
>> I don't understand this. It's just "pip wheel foo" to build a wheel
>> for foo (which will be downloaded), or "pip wheel ." or  "python
>> setup.py bdist_wheel" as you prefer for a local package.
> 
> Hmmm...  That looks like it's for installing/compiling somebody else's package.  Is that last command sufficient to prepare one's own wheel for uploading to PyPI, or there something else to do?
> 

Generally for uploading to PyPI you do ``python setup.py bdist_wheel``, though I don?t think there?d be any bad thing if you used pip wheel.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From p.f.moore at gmail.com  Thu Oct 30 00:14:41 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 23:14:41 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <54517202.2010109@stoneleaf.us>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
 <54517202.2010109@stoneleaf.us>
Message-ID: <CACac1F_0XmwEiEGHF89yhesyPCW6jc0ENuUegvuavRasVipgxA@mail.gmail.com>

On 29 October 2014 23:02, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 10/29/2014 03:46 PM, Paul Moore wrote:
>>
>> On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>>>
>>>
>>>    - where one should be at when one starts the compile process
>>
>>
>> I don't understand this. It's just "pip wheel foo" to build a wheel
>> for foo (which will be downloaded), or "pip wheel ." or  "python
>> setup.py bdist_wheel" as you prefer for a local package.
>
>
> Hmmm...  That looks like it's for installing/compiling somebody else's
> package.  Is that last command sufficient to prepare one's own wheel for
> uploading to PyPI, or there something else to do?

Oh, I see what you're thinking.

I explicitly *don't* want to get into general packaging issues - they
are covered elsewhere. The point here is that if you have the right
compiler set up, you can read any generic packaging instructions and
it doesn't matter whether there's C code involved. That's it.

But yes, "python setup.py bdist_wheel" is how you build a wheel for
upload. That's true whether your project includes C extensions or not.

I'm also not expecting to explain to people how to build any
dependencies (for example, PyYAML depends on the C compiler being able
to find libyaml). That *is* harder on Windows than on Linux, because
there's no "system location" equivalent to /usr/local/include and
/usr/local/lib. But again, I consider this to be outside the scope of
the document, because it's specific to the particular project you're
building.

Paul

From njs at pobox.com  Thu Oct 30 00:22:37 2014
From: njs at pobox.com (Nathaniel Smith)
Date: Wed, 29 Oct 2014 23:22:37 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
Message-ID: <CAPJVwBnS9MyoUa-WXTC8PC8OuwvYbja73z7d_QJJpek=DrQJ2g@mail.gmail.com>

On Wed, Oct 29, 2014 at 10:46 PM, Paul Moore <p.f.moore at gmail.com> wrote:
> On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>>> Yeah, I wondered about that. I'll work up a patch for that. But the
>>> more I think about it, it really is trivial:
>>
>> I am reminded of an interview question I was once asked which was prefaced
>> with: "Here's an easy one..."
>>
>> My reply was, if you know the answer, it's easy!
>
> Yeah, I know what you mean. My take on this is that I agree it's not
> easy if you don't know and can't get access to the information, but if
> you can, there's very little to it.

That's great, but yeah. In case it helps as a data point, I consider
myself a reasonably technical guy, probably more than your average
python package author -- 15 years of free software hacking, designed a
pretty popular X remote display protocol, helped invent DVCS, have
written patches to gcc and the kernel, numpy co-maintainer, etc. etc.

For the last ~12 months I've had an underlined and circled todo item
saying "make wheels for https://pypi.python.org/pypi/zs/", and every
time I've sat down and spent a few hours trying I've ended up utterly
defeated with a headache. Distutils is spaghetti, and Windows is
spaghetti (esp. if you've never used it for more than checking email
on a friend's laptop), and the toolchain setup is spaghetti, and then
suddenly people are talking about vcvarsall.bats and I don't know what
all. I honestly would totally benefit from one of those
talk-your-grandparents-through-it tutorials ("here's a screenshot of
the dialogue box, and I've drawn an arrow pointing at the 'ok' button.
You should press the 'ok' button.")

>>> - For non-free MSVC, install the appropriate version, and everything just
>>> works.
>>> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
>>> package and everything just works as long as you're using setuptools.
>>> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
>>> everything just works.
>>> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
>>> variables, and everything just works.

I think the SDK covers both 32 and 64 bit? If so then leaving visual
studio out of things entirely is probably simpler.

I'm also unsure how you even get both 32- and 64-bit versions of
python to coexist on the same system.

-n

-- 
Nathaniel J. Smith
Postdoctoral researcher - Informatics - University of Edinburgh
http://vorpus.org

From p.f.moore at gmail.com  Thu Oct 30 00:45:54 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 23:45:54 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CAPJVwBnS9MyoUa-WXTC8PC8OuwvYbja73z7d_QJJpek=DrQJ2g@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
 <CAPJVwBnS9MyoUa-WXTC8PC8OuwvYbja73z7d_QJJpek=DrQJ2g@mail.gmail.com>
Message-ID: <CACac1F-_-ePgDAN0pBnPM3XnzBFsVOUNpO51PeUDue4OajhUUQ@mail.gmail.com>

On 29 October 2014 23:22, Nathaniel Smith <njs at pobox.com> wrote:
>> Yeah, I know what you mean. My take on this is that I agree it's not
>> easy if you don't know and can't get access to the information, but if
>> you can, there's very little to it.
>
> That's great, but yeah. In case it helps as a data point, I consider
> myself a reasonably technical guy, probably more than your average
> python package author -- 15 years of free software hacking, designed a
> pretty popular X remote display protocol, helped invent DVCS, have
> written patches to gcc and the kernel, numpy co-maintainer, etc. etc.

Yeah, that counts :-)

> For the last ~12 months I've had an underlined and circled todo item
> saying "make wheels for https://pypi.python.org/pypi/zs/", and every
> time I've sat down and spent a few hours trying I've ended up utterly
> defeated with a headache. Distutils is spaghetti, and Windows is
> spaghetti (esp. if you've never used it for more than checking email
> on a friend's laptop), and the toolchain setup is spaghetti, and then
> suddenly people are talking about vcvarsall.bats and I don't know what
> all. I honestly would totally benefit from one of those
> talk-your-grandparents-through-it tutorials ("here's a screenshot of
> the dialogue box, and I've drawn an arrow pointing at the 'ok' button.
> You should press the 'ok' button.")

OK. That needs to be fixed. How about this. I'll work with you to get
the wheel build process set up in such a way that you can maintain it,
and we'll document what you needed to know in order to do it. Then I
can put that documentation into the packaging user guide for other
people to work with. It can wait till you have the time, but when you
do, shout and I'll help.

I know nothing about zs, but I just tried building it. MSVC 2010
doesn't support "static inline" (used twice in zs\pycrc-crc64xz.h) but
when I removed the "inline" from that it built fine for me. So it
should be a good example, as there are no particular complexities to
deal with.

>>>> - For non-free MSVC, install the appropriate version, and everything just
>>>> works.
>>>> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
>>>> package and everything just works as long as you're using setuptools.
>>>> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
>>>> everything just works.
>>>> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
>>>> variables, and everything just works.
>
> I think the SDK covers both 32 and 64 bit? If so then leaving visual
> studio out of things entirely is probably simpler.

The SDK is the most complex environment to set up, so avoiding it
unless you need it is a good thing. But maybe it's worth explaining
that if you need it anyway, this is how you set it up to cover some of
the other cases as well.

> I'm also unsure how you even get both 32- and 64-bit versions of
> python to coexist on the same system.

Just install them into different directories. The default is the same
for 32 and 64 bit, so you need to override the default for (at least)
one of them, but that's all.

Paul

From p.f.moore at gmail.com  Thu Oct 30 00:57:29 2014
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 29 Oct 2014 23:57:29 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <d0f46ff3e8d441279d12993864ac796a@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp>
 <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net>
 <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
 <d0f46ff3e8d441279d12993864ac796a@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CACac1F_3CyKmYta=LgSEjxu9Wd2o12wQpCscEopUzw_byeHGrw@mail.gmail.com>

On 29 October 2014 23:49, Steve Dower <Steve.Dower at microsoft.com> wrote:
> "For the paid versions, I'm going to assume that anyone who paid for a
> compiler and doesn't know where their copy is, probably can't be
> helped ;-)"
>
> You could link to visualstudio.com for the trial versions, and maybe to a
> page/post about the PSF's grants process if such a page exists.

That doesn't help for VS 2010 and VS 2008, which are the relevant ones
right now, unfortunately. In thew brave new world of Python 3.5+ that
might well do, though. (Although the feedback from Unix users is that
they really want a direct link to the exact edition they need - i.e.,
a permalink to VC++ Express Edition <latest> for Python 3.5+. I'm not
sure how well the MS website structure does that, though.

> "And of course there's the awkward problem that VS 2010 Express is
> unsupported and therefore no longer available from *any* official
> location. I can't solve that, sadly"
>
> These are still at
> http://www.visualstudio.com/en-us/downloads/download-visual-studio-vs#DownloadFamilies_4,
> which is the main download page. Hopefully they don't go away before 3.5,
> but I have no control over that unfortunately.

Wow! I didn't know that. That's great, solves the biggest issue I had.

> The link I posted for 2.7 was aka.ms/vcpython27, which is a redirect that I
> own and can update if necessary.

Cool. As long as it's stable and reliable, it's exactly what I want.

Many thanks,
Paul

From Steve.Dower at microsoft.com  Thu Oct 30 00:49:32 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 29 Oct 2014 23:49:32 +0000
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <735B6073-0F87-4F63-9629-392636E3B57F@stufft.io>
 <CACac1F8KLFKBLejwPSAykqcz7QoCDON3wUJCV6=94VxpY-8u1g@mail.gmail.com>
 <545167F1.1030502@stoneleaf.us>,
 <CACac1F8WznFTai+xTHEQk8T=y+Oa4j8ZyTeT9ivTp-He8wLyjQ@mail.gmail.com>
Message-ID: <d0f46ff3e8d441279d12993864ac796a@DM2PR0301MB0734.namprd03.prod.outlook.com>

"For the paid versions, I'm going to assume that anyone who paid for a
compiler and doesn't know where their copy is, probably can't be
helped ;-)"

You could link to visualstudio.com for the trial versions, and maybe to a page/post about the PSF's grants process if such a page exists.

"And of course there's the awkward problem that VS 2010 Express is
unsupported and therefore no longer available from *any* official
location. I can't solve that, sadly"

These are still at http://www.visualstudio.com/en-us/downloads/download-visual-studio-vs#DownloadFamilies_4, which is the main download page. Hopefully they don't go away before 3.5, but I have no control over that unfortunately.

The link I posted for 2.7 was aka.ms/vcpython27, which is a redirect that I own and can update if necessary.

Cheers,
Steve

Top-posted from my Windows Phone
________________________________
From: Paul Moore<mailto:p.f.moore at gmail.com>
Sent: ?10/?29/?2014 15:48
To: Ethan Furman<mailto:ethan at stoneleaf.us>
Cc: Python Dev<mailto:python-dev at python.org>
Subject: Re: [Python-Dev] Status of C compilers for Python on Windows

On 29 October 2014 22:19, Ethan Furman <ethan at stoneleaf.us> wrote:
>> Yeah, I wondered about that. I'll work up a patch for that. But the
>> more I think about it, it really is trivial:
>
> I am reminded of an interview question I was once asked which was prefaced
> with: "Here's an easy one..."
>
> My reply was, if you know the answer, it's easy!

Yeah, I know what you mean. My take on this is that I agree it's not
easy if you don't know and can't get access to the information, but if
you can, there's very little to it.

>> - For non-free MSVC, install the appropriate version, and everything just
>> works.
>> - For Python 2.7 (32 or 64 bit), install the compiler for Python 2.7
>> package and everything just works as long as you're using setuptools.
>> - For 32 bit Python 3.2-3.4, install Visual Studio Express and
>> everything just works.
>> - For 64 bit Python 3.2-3.4, install the SDK, set some environment
>> variables, and everything just works.
>> - For Python 3.5+, install the current Visual Studion Express and
>> everything just works.
>
>
> I would suggest
>   - where to get these packages

Conceded. Working out how to point people at stuff on MSDN is a
challenge, things seem to move around. Maybe Steve Dower could help
here with canonical URLs for some of them (IIRC, he provided one for
the VS compilers for Python 2.7 package when it was announced).

For the paid versions, I'm going to assume that anyone who paid for a
compiler and doesn't know where their copy is, probably can't be
helped ;-)

And of course there's the awkward problem that VS 2010 Express is
unsupported and therefore no longer available from *any* official
location. I can't solve that, sadly.

>   - where to get any dependencies

There are none. I could state that explicitly, I guess.

>   - any options to [not] specify during install

I'll have to go through the installs again just to be sure, but I'm
pretty certain "Select the default for everything" is correct.

>   - what environment variables to what value

None, except for the SDK which I did say I needed to test out and
cover in more detail.

>   - where one should be at when one starts the compile process

I don't understand this. It's just "pip wheel foo" to build a wheel
for foo (which will be downloaded), or "pip wheel ." or  "python
setup.py bdist_wheel" as you prefer for a local package. That's
standard distutils/setuptools/pip extension building. I don't propose
to cover that, just how to *set up* the environment.

With the sole exception of the SDK case, once installed, everything
just works everywhere, nothing to set up, no special environment to
configure. Start up a new cmd or powershell console (or use the one
you're already in) and go. Maybe Unix users expect more complexity
because it's not that simple on Unix? But I thought it was - install
the appropriate OS packages and that's it?

> and, of course, a gotchas section for uncommon but frustrating things that
> might go wrong.

Hmm, I see your point here, but I'm not sure what I might include. You
*can* get in a mess [1] but generally I don't as I'm an experienced
Windows user. I also don't want to offer to debug and fix everyone's
problems in getting things set up, so offering to collect and document
"common issues" that I've seen is impractical. Maybe a section
entitled "Common Issues and Their Solutions", with some placeholder
text saying "If you have any issues installing any of the compiler
packages mentioned, please document what went wrong and how to fix it,
and submit a PR!" would do?

> And thanks for doing this!

No problem!

Paul

[1] I once spent a *long* time fighting failed installs of the Windows
SDK. Turns out it was because I was trying to install a 32-bit SDK on
a 64-bit machine (doh!), and the installer really doesn't like that
:-( About all I could say to document that is "Read the instructions
properly" and "I'm sorry that the MS installers fail really badly when
faced with relatively obvious idiot-user errors, but I can't do
anything about it :-("
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
https://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: https://mail.python.org/mailman/options/python-dev/steve.dower%40microsoft.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141029/3e8c28b4/attachment.html>

From tjreedy at udel.edu  Thu Oct 30 04:33:06 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Oct 2014 23:33:06 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
Message-ID: <m2sbhp$hq7$1@ger.gmane.org>

On 10/29/2014 4:05 PM, Paul Moore wrote:
> On 29 October 2014 15:31, Nathaniel Smith <njs at pobox.com> wrote:
>>> You can use Express editions of Visual Studio.
>>
>> IIUC, the express edition compilers are 32-bit only, and what you actually
>> want are the "SDK compilers":
>> https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
>>
>> These are freely downloadable by anyone, no msdn subscription required, but
>> only if you know where to find them!
>>
>> AFAICT the main obstacle to using MSVC to build python extensions (assuming
>> it can handle your code at all) is not anything technical, but rather that
>> there's no clear and correct tutorial on how to do it, and lots of confusion
>> and misinformation circulating.
>
> Would it help if I wrote a document explaining how to set up the MS
> compilers (free and paid for) to allow building of Python extensions?

> There are a few provisos:
>
> 1. A lot of it will be pretty trivial: "If you have Vistal Studio
> (full edition), install it. Done."
> 2. It will be out of date very fast as the situation for Python 3.5+
> will be trivial across the board.
> 3. I don't have anywhere particularly authoritative to host it (maybe
> the Python Wiki?) and it could easily get lost in the huge swamp of
> variously outdated, over-complicated, or otherwise alternative
> documents available. Ideally I'd like someone to suggest an "official"
> location I could use.

There is already a section in the devguide for building Python itself, 
with mostly the same info, except it may not be up to date.

> I don't want to do this if it won't be useful, as it'll take me a bit
> of effort to confirm the process for the only non-trivial scenario
> (64-bit Python 3.3/3.4 with free tools). But if people think it would
> help, that's fine, I volunteer.

The devguide needs to be kept up to date.  If you open a tracker issue, 
put me as nosy to review and commit.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Oct 30 04:41:46 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 29 Oct 2014 23:41:46 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <578f8e0c798746b3becce31f2a0abff2@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <m2qvsb$94a$1@ger.gmane.org>
 <20141029162108.43db29f1@fsol>
 <578f8e0c798746b3becce31f2a0abff2@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <m2sc21$39c$1@ger.gmane.org>

On 10/29/2014 11:37 AM, Steve Dower wrote:

> My ideal target (Utopia refined to be achievable) is for python-dev
> to handle the Python binaries themselves (already done) and to make
> them easy to bundle with applications/etc (I'm working on this for
> 3.5), and for PyPI to include pre-built wheels for Windows where
> necessary.
>
> Note that I don't require package developers to build their own
> wheels, though I think that they are generally the right people to be
> doing it. It would be nice to create a culture of delegation, so that
> "someone" could be a trusted builder for a range of packages

Christoph Gohlke, Laboratory for Fluorescence Dynamics, University of 
California, Irvine now provides 32 and 64 bit builds for 2.6/7, 3.2/3/4 
(10 total) for about 300 'scientific open-source extension packages' an 
his site.
http://www.lfd.uci.edu/~gohlke/pythonlibs/

Perhaps the PyPI folks could discuss incorporating wheel versions on 
PyPI (I have no idea what is involved, but I presume it could be antomated.)

-- 
Terry Jan Reedy


From rosuav at gmail.com  Thu Oct 30 08:52:27 2014
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 30 Oct 2014 18:52:27 +1100
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <54514163.4000607@g.nevcal.com>
References: <54514163.4000607@g.nevcal.com>
Message-ID: <CAPTjJmovRuOGoeEc+k17Nu_hxDs4iWS-x510279g4nSiSdBCXg@mail.gmail.com>

On Thu, Oct 30, 2014 at 6:34 AM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> (I'm actually not sure if *nix package managers allow multiple repositories
> or not, but from the way people talk about them, it always sounds like a
> "distribution" also provides "a repository" of additional packages).

I'm fairly sure they all do. Certainly with apt (the Debian package
manager), it's common to add additional repositories; for instance,
PostgreSQL can be obtained either from the default Debian repos or
from Postgres's own hosting (which usually has more versions
available). A distribution will always provide a repository, and there
are plenty of distros that provide only a small repo and chain to
upstream for most packages - for instance AntiX has its own, and then
pulls in debian.org and a few others. Adding a local-network repo is
fairly straight-forward.

Most likely, OneGet won't replace pip/PyPI, any more than apt or yum
does; but it may be worth having Python itself available that way.
That might simply mean having someone package up Python and put it on
an appropriate server, or maybe python.org could end up hosting a
repo. I've no idea what "trusted" will mean; in the case of apt, any
sysadmin can deem any repo to be trusted (by importing its key), but
this might be more along the lines of "only curated packages" or
something.

To what extent will Windows 10 users expect all their software to come
via OneGet? That's the question.

ChrisA

From rdmurray at bitdance.com  Thu Oct 30 13:59:23 2014
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 30 Oct 2014 08:59:23 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <m2sbhp$hq7$1@ger.gmane.org>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <m2sbhp$hq7$1@ger.gmane.org>
Message-ID: <20141030125924.5A55C250048@webabinitio.net>

On Wed, 29 Oct 2014 23:33:06 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 10/29/2014 4:05 PM, Paul Moore wrote:
> > On 29 October 2014 15:31, Nathaniel Smith <njs at pobox.com> wrote:
> >>> You can use Express editions of Visual Studio.
> >>
> >> IIUC, the express edition compilers are 32-bit only, and what you actually
> >> want are the "SDK compilers":
> >> https://github.com/cython/cython/wiki/64BitCythonExtensionsOnWindows
> >>
> >> These are freely downloadable by anyone, no msdn subscription required, but
> >> only if you know where to find them!
> >>
> >> AFAICT the main obstacle to using MSVC to build python extensions (assuming
> >> it can handle your code at all) is not anything technical, but rather that
> >> there's no clear and correct tutorial on how to do it, and lots of confusion
> >> and misinformation circulating.
> >
> > Would it help if I wrote a document explaining how to set up the MS
> > compilers (free and paid for) to allow building of Python extensions?
> 
> > There are a few provisos:
> >
> > 1. A lot of it will be pretty trivial: "If you have Vistal Studio
> > (full edition), install it. Done."
> > 2. It will be out of date very fast as the situation for Python 3.5+
> > will be trivial across the board.
> > 3. I don't have anywhere particularly authoritative to host it (maybe
> > the Python Wiki?) and it could easily get lost in the huge swamp of
> > variously outdated, over-complicated, or otherwise alternative
> > documents available. Ideally I'd like someone to suggest an "official"
> > location I could use.
> 
> There is already a section in the devguide for building Python itself, 
> with mostly the same info, except it may not be up to date.
> 
> > I don't want to do this if it won't be useful, as it'll take me a bit
> > of effort to confirm the process for the only non-trivial scenario
> > (64-bit Python 3.3/3.4 with free tools). But if people think it would
> > help, that's fine, I volunteer.
> 
> The devguide needs to be kept up to date.  If you open a tracker issue, 
> put me as nosy to review and commit.

The devguide is about building python itself.  Paul is talking about
building extensions.  That should go in the packaging docs.  I don't
*think* Paul is volunteering to explain how to build python itself,
that's pretty much our job :)

--David

From martin at v.loewis.de  Thu Oct 30 15:26:01 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Oct 2014 15:26:01 +0100
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <54514163.4000607@g.nevcal.com>
References: <54514163.4000607@g.nevcal.com>
Message-ID: <54524A79.9020101@v.loewis.de>

Am 29.10.14 20:34, schrieb Glenn Linderman:
> New package manager from M$... article here
> <http://www.neowin.net/news/windows-10-oneget-a-linux-style-package-management-framework>.

I've looked at it, but only by reading its code, not trying it out.
Some notes.

First, what is Chocolatey? It's a PowerShell library, and a package
infrastructure. A Chocolatey package is a PowerShell script that
installs a piece of software on the system. The software itself comes
in a different format. There are installer helpers for exe, msi and msu,
Python packages (running setup.py), Ruby packages, etc. There appears to
be a notion of dependencies also.

OneGet is similar; it is also a PowerShell library. It has the notion
of "providers", which are classes that can make a package installed.
There is the ARP provider (Add-Remove-Programs), which can report that
a package is already installed. There is the MSI provider, which can
install an MSI file that is already on disk. And there is the web
provider which can download a file over http. They claim to include
a re-implementation of Chocolatey (meaning that Chocolatey packages
can be installed), however, ISTM that this only supports exe and msi
packages.

I haven't seen anything resembling dependencies in OneGet.

Regards,
Martin




From martin at v.loewis.de  Thu Oct 30 15:30:55 2014
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 30 Oct 2014 15:30:55 +0100
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <CAPTjJmovRuOGoeEc+k17Nu_hxDs4iWS-x510279g4nSiSdBCXg@mail.gmail.com>
References: <54514163.4000607@g.nevcal.com>
 <CAPTjJmovRuOGoeEc+k17Nu_hxDs4iWS-x510279g4nSiSdBCXg@mail.gmail.com>
Message-ID: <54524B9F.9030108@v.loewis.de>

> Most likely, OneGet won't replace pip/PyPI, any more than apt or yum
> does; but it may be worth having Python itself available that way.
> That might simply mean having someone package up Python and put it on
> an appropriate server, or maybe python.org could end up hosting a
> repo. 

Python is already available in Chocolatey:

https://chocolatey.org/packages/python

Given that OneGet intends to support Chocolatey as a repository,
it might just work already. All python.org would have to guarantee
is stable URLs for the Python msi files.

Regards,
Martin


From Steve.Dower at microsoft.com  Thu Oct 30 16:39:59 2014
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Thu, 30 Oct 2014 15:39:59 +0000
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <54524B9F.9030108@v.loewis.de>
References: <54514163.4000607@g.nevcal.com>
 <CAPTjJmovRuOGoeEc+k17Nu_hxDs4iWS-x510279g4nSiSdBCXg@mail.gmail.com>
 <54524B9F.9030108@v.loewis.de>
Message-ID: <04a573ddde7a45788013c91f8e07d2f1@DM2PR0301MB0734.namprd03.prod.outlook.com>

>> Most likely, OneGet won't replace pip/PyPI, any more than apt or yum
>> does; but it may be worth having Python itself available that way.
>> That might simply mean having someone package up Python and put it on
>> an appropriate server, or maybe python.org could end up hosting a
>> repo.
> 
> Python is already available in Chocolatey:
> 
> https://chocolatey.org/packages/python
> 
> Given that OneGet intends to support Chocolatey as a repository, it might just
> work already. All python.org would have to guarantee is stable URLs for the
> Python msi files.

I'd like it if we guarantee stable URLs anyway. The 3.5 installer will have a single flag to switch between building a full bundle (~23MB) or a tiny downloader (<500KB), but the latter option is only viable if the URLs are stable. I can also include options in the downloader for 32/64-bit versions and debug symbols, which would really reduce the choices for a user (and yes - the entire installer is still automatable and I'll write up docs to go with it for sysadmin scenarios).

(I believe that OneGet does support dependencies, since one of the intended uses is setting up dev environments. If it gets backported, I'll see about setting up a Python build environment in there, unless someone else does it first.)

Cheers,
Steve

> Regards,
> Martin

From tjreedy at udel.edu  Thu Oct 30 18:46:23 2014
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 30 Oct 2014 13:46:23 -0400
Subject: [Python-Dev] Status of C compilers for Python on Windows
In-Reply-To: <20141030125924.5A55C250048@webabinitio.net>
References: <8F79064A3F9B468A8E8927BA6AA6BC60@TKsamsung>
 <87bnow48bk.fsf@uwakimon.sk.tsukuba.ac.jp>
 <133A95BE786341E892819607833EF8AC@TKsamsung>
 <8761f34jce.fsf@uwakimon.sk.tsukuba.ac.jp> <m2qt6n$nlk$1@ger.gmane.org>
 <20141029143151.2E9C2250DAC@webabinitio.net> <20141029154614.5de0c6ba@fsol>
 <CAPJVwBkdt8yN1JKs4n9TNccK63OZX1KHaAr_J0_79f-Ogx7bPA@mail.gmail.com>
 <CACac1F9z1m7cEqpeLREmwxc+fPdd0SSePpF8+StOzBck-G-rdA@mail.gmail.com>
 <m2sbhp$hq7$1@ger.gmane.org> <20141030125924.5A55C250048@webabinitio.net>
Message-ID: <m2tthn$oq3$1@ger.gmane.org>

On 10/30/2014 8:59 AM, R. David Murray wrote:
> On Wed, 29 Oct 2014 23:33:06 -0400, Terry Reedy <tjreedy at udel.edu> wrote:

>> The devguide needs to be kept up to date.  If you open a tracker issue,
>> put me as nosy to review and commit.
>
> The devguide is about building python itself.  Paul is talking about
> building extensions.  That should go in the packaging docs.  I don't
> *think* Paul is volunteering to explain how to build python itself,
> that's pretty much our job :)

I was thinking that building Python and extensions used the same tools, 
but then I read that the new compiler for 2.7 was extensions only and 
ditto for something else.  If VC Express 2010 is in a new location, that 
*should* be recorded in the devguide.

-- 
Terry Jan Reedy


From v+python at g.nevcal.com  Thu Oct 30 18:48:28 2014
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Thu, 30 Oct 2014 10:48:28 -0700
Subject: [Python-Dev] Impact of Windows PowerShell OneGet ?
In-Reply-To: <54524B9F.9030108@v.loewis.de>
References: <54514163.4000607@g.nevcal.com>
 <CAPTjJmovRuOGoeEc+k17Nu_hxDs4iWS-x510279g4nSiSdBCXg@mail.gmail.com>
 <54524B9F.9030108@v.loewis.de>
Message-ID: <545279EC.9010103@g.nevcal.com>

On 10/30/2014 7:30 AM, "Martin v. L?wis" wrote:
>> Most likely, OneGet won't replace pip/PyPI, any more than apt or yum
>> does; but it may be worth having Python itself available that way.
>> That might simply mean having someone package up Python and put it on
>> an appropriate server, or maybe python.org could end up hosting a
>> repo.
> Python is already available in Chocolatey:
>
> https://chocolatey.org/packages/python
>
> Given that OneGet intends to support Chocolatey as a repository,
> it might just work already. All python.org would have to guarantee
> is stable URLs for the Python msi files.
It might. Thanks for that information.

Poking around the link, I discover a weirdness: the claim that the 
package to install 32-bit Python on 64-bit systems is different than 
installing the 32-bit Python on 32-bit systems. While the instructions 
are explicit on what to do inside the chocolatey environment for all 3 
cases (the third being 64-bit install on 64-bit systems), I'm baffled as 
to why there is a difference, because there isn't when downloading 
32-bit Python from python.org...

And there is a weird reference to chocolatey's -x86 parameter, and the 
explanation seems to be that chocolatey has provision for installing 
32-bit or 64-bit packages on 64-bit systems, but  that the way Python is 
included in chocolatey, that provision can't be used.  Sounds very 
strange, like whoever set this up either didn't understand Python, or 
didn't understand chocolatey, or there is some limitation to the 
chocolatey implementation of 32-bit vs 64-bit packages.

Maybe if I understand chocolatey, this wouldn't seem so weird.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141030/f4c21b0d/attachment.html>

From status at bugs.python.org  Fri Oct 31 18:08:15 2014
From: status at bugs.python.org (Python tracker)
Date: Fri, 31 Oct 2014 18:08:15 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20141031170815.7B70456140@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-10-24 - 2014-10-31)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4626 (+20)
  closed 29911 (+34)
  total  34537 (+54)

Open issues with patches: 2160 


Issues opened (37)
==================

#9351: argparse set_defaults on subcommands should override top level
http://bugs.python.org/issue9351  reopened by r.david.murray

#22716: Add reference to the object missing an attribute to AttributeE
http://bugs.python.org/issue22716  reopened by flying sheep

#22722: inheritable pipes are unwieldy without os.pipe2
http://bugs.python.org/issue22722  opened by bukzor

#22724: byte-compile fails for cross-builds
http://bugs.python.org/issue22724  opened by Benedikt.Morbach

#22725: improve documentation for enumerate() (built-in function)
http://bugs.python.org/issue22725  opened by vy0123

#22726: Idle: add help to config dialogs
http://bugs.python.org/issue22726  opened by terry.reedy

#22729: `wait` and `as_completed` depend on private api
http://bugs.python.org/issue22729  opened by bwhmather

#22731: test_capi test fails because of mismatched newlines
http://bugs.python.org/issue22731  opened by steve.dower

#22732: ctypes tests don't set correct restype for intptr_t functions
http://bugs.python.org/issue22732  opened by steve.dower

#22733: MSVC ffi_prep_args doesn't handle 64-bit arguments properly
http://bugs.python.org/issue22733  opened by steve.dower

#22734: marshal needs a lower stack depth for debug builds on Windows
http://bugs.python.org/issue22734  opened by steve.dower

#22735: Fix various crashes exposed through mro() customization
http://bugs.python.org/issue22735  opened by abusalimov

#22737: Provide a rejected execution model and implementations for fut
http://bugs.python.org/issue22737  opened by Joshua.Harlow

#22738: improve  'python -h' documentation for '-c'
http://bugs.python.org/issue22738  opened by vy0123

#22739: "There is no disk in the drive" error
http://bugs.python.org/issue22739  opened by Lachlan.Kingsford

#22742: IDLE shows traceback when printing non-BMP character
http://bugs.python.org/issue22742  opened by belopolsky

#22743: Specify supported XML version
http://bugs.python.org/issue22743  opened by Friedrich.Spee.von.Langenfeld

#22744: os.mkdir on Windows silently strips trailing blanks from direc
http://bugs.python.org/issue22744  opened by tegavu

#22746: cgitb html: wrong encoding for utf-8
http://bugs.python.org/issue22746  opened by wrohdewald

#22747: Interpreter fails in initialize on systems where HAVE_LANGINFO
http://bugs.python.org/issue22747  opened by WanderingLogic

#22750: xmlapp.py display bug when validate XML by DTD
http://bugs.python.org/issue22750  opened by Spider06

#22751: Fix test___all__ warning about modified environment
http://bugs.python.org/issue22751  opened by Michael.Cetrulo

#22752: incorrect time.timezone value
http://bugs.python.org/issue22752  opened by errx

#22753: urllib2 localnet Changed test to lookup IP-address of localhos
http://bugs.python.org/issue22753  opened by hakan

#22755: contextlib.closing documentation should use a new example
http://bugs.python.org/issue22755  opened by mjpieters

#22757: TclStackFree: incorrect freePtr. Call out of sequence?
http://bugs.python.org/issue22757  opened by Charleston

#22758: Regression in Python 3.2 cookie parsing
http://bugs.python.org/issue22758  opened by Tim.Graham

#22761: Catching StopIteraion inside list comprehension
http://bugs.python.org/issue22761  opened by tomirendo

#22763: load_tests chaining into discover from non-discover entry poin
http://bugs.python.org/issue22763  opened by rbcollins

#22764: object lifetime fragility in unittest tests
http://bugs.python.org/issue22764  opened by rbcollins

#22765: Fixes for test_gdb (first frame address, entry values)
http://bugs.python.org/issue22765  opened by bkabrda

#22766: collections.Counter's in-place operators should return NotImpl
http://bugs.python.org/issue22766  opened by Joshua.Chin

#22768: Add a way to get the peer certificate of a SSL Transport
http://bugs.python.org/issue22768  opened by mathieui

#22769: Tttk tag_has() throws TypeError when called without item
http://bugs.python.org/issue22769  opened by ddurrett

#22770: test_ttk_guionly and test_tk can cause Tk segfaults on OS X wh
http://bugs.python.org/issue22770  opened by ned.deily

#22773: Export Readline version and expect ANSI sequence for version <
http://bugs.python.org/issue22773  opened by David.Edelsohn

#22775: SimpleCookie not picklable with HIGHEST_PROTOCOL
http://bugs.python.org/issue22775  opened by Tim.Graham



Most recent 15 issues with no replies (15)
==========================================

#22769: Tttk tag_has() throws TypeError when called without item
http://bugs.python.org/issue22769

#22765: Fixes for test_gdb (first frame address, entry values)
http://bugs.python.org/issue22765

#22751: Fix test___all__ warning about modified environment
http://bugs.python.org/issue22751

#22750: xmlapp.py display bug when validate XML by DTD
http://bugs.python.org/issue22750

#22743: Specify supported XML version
http://bugs.python.org/issue22743

#22742: IDLE shows traceback when printing non-BMP character
http://bugs.python.org/issue22742

#22737: Provide a rejected execution model and implementations for fut
http://bugs.python.org/issue22737

#22733: MSVC ffi_prep_args doesn't handle 64-bit arguments properly
http://bugs.python.org/issue22733

#22729: `wait` and `as_completed` depend on private api
http://bugs.python.org/issue22729

#22726: Idle: add help to config dialogs
http://bugs.python.org/issue22726

#22714: target of 'import statement' entry in general index for 'i' is
http://bugs.python.org/issue22714

#22706: Idle extension configuration and key bindings
http://bugs.python.org/issue22706

#22704: Review extension enable options
http://bugs.python.org/issue22704

#22703: Idle Code Context: separate changing current and future editor
http://bugs.python.org/issue22703

#22700: email's header_value_parser missing defect report for 'abc at xyz
http://bugs.python.org/issue22700



Most recent 15 issues waiting for review (15)
=============================================

#22775: SimpleCookie not picklable with HIGHEST_PROTOCOL
http://bugs.python.org/issue22775

#22773: Export Readline version and expect ANSI sequence for version <
http://bugs.python.org/issue22773

#22770: test_ttk_guionly and test_tk can cause Tk segfaults on OS X wh
http://bugs.python.org/issue22770

#22768: Add a way to get the peer certificate of a SSL Transport
http://bugs.python.org/issue22768

#22766: collections.Counter's in-place operators should return NotImpl
http://bugs.python.org/issue22766

#22765: Fixes for test_gdb (first frame address, entry values)
http://bugs.python.org/issue22765

#22764: object lifetime fragility in unittest tests
http://bugs.python.org/issue22764

#22758: Regression in Python 3.2 cookie parsing
http://bugs.python.org/issue22758

#22753: urllib2 localnet Changed test to lookup IP-address of localhos
http://bugs.python.org/issue22753

#22751: Fix test___all__ warning about modified environment
http://bugs.python.org/issue22751

#22747: Interpreter fails in initialize on systems where HAVE_LANGINFO
http://bugs.python.org/issue22747

#22746: cgitb html: wrong encoding for utf-8
http://bugs.python.org/issue22746

#22735: Fix various crashes exposed through mro() customization
http://bugs.python.org/issue22735

#22734: marshal needs a lower stack depth for debug builds on Windows
http://bugs.python.org/issue22734

#22733: MSVC ffi_prep_args doesn't handle 64-bit arguments properly
http://bugs.python.org/issue22733



Top 10 most discussed issues (10)
=================================

#22725: improve documentation for enumerate() (built-in function)
http://bugs.python.org/issue22725  15 msgs

#22153: Documentation of TestCase.runTest is incorrect and confusing
http://bugs.python.org/issue22153  11 msgs

#17896: Move Windows external libs from <src>\..\ to <src>\externals
http://bugs.python.org/issue17896   9 msgs

#22731: test_capi test fails because of mismatched newlines
http://bugs.python.org/issue22731   9 msgs

#22746: cgitb html: wrong encoding for utf-8
http://bugs.python.org/issue22746   8 msgs

#22764: object lifetime fragility in unittest tests
http://bugs.python.org/issue22764   8 msgs

#10548: Error in setUp not reported as expectedFailure (unittest)
http://bugs.python.org/issue10548   7 msgs

#22672: float arguments in scientific notation not supported by argpar
http://bugs.python.org/issue22672   7 msgs

#22719: os.path.isfile & os.path.exists bug in while loop
http://bugs.python.org/issue22719   7 msgs

#22758: Regression in Python 3.2 cookie parsing
http://bugs.python.org/issue22758   7 msgs



Issues closed (34)
==================

#7559: TestLoader.loadTestsFromName swallows import errors
http://bugs.python.org/issue7559  closed by python-dev

#8876: distutils should not assume that hardlinks will work
http://bugs.python.org/issue8876  closed by pitrou

#17381: IGNORECASE breaks unicode literal range matching
http://bugs.python.org/issue17381  closed by serhiy.storchaka

#18216: gettext doesn't check MO versions
http://bugs.python.org/issue18216  closed by pitrou

#22173: Update lib2to3.tests and test_lib2to3 to use test discovery
http://bugs.python.org/issue22173  closed by python-dev

#22177: Incorrect version reported after downgrade
http://bugs.python.org/issue22177  closed by ezio.melotti

#22196: namedtuple documentation could/should mention the new Enum typ
http://bugs.python.org/issue22196  closed by ezio.melotti

#22217: Reprs for zipfile classes
http://bugs.python.org/issue22217  closed by serhiy.storchaka

#22237: sorted() docs should state that the sort is stable
http://bugs.python.org/issue22237  closed by ezio.melotti

#22249: Possibly incorrect example is given for socket.getaddrinfo()
http://bugs.python.org/issue22249  closed by python-dev

#22305: Documentation on deepcopy's problems is misleading
http://bugs.python.org/issue22305  closed by georg.brandl

#22539: Table formatting errors in pydoc
http://bugs.python.org/issue22539  closed by georg.brandl

#22596: support.transient_internet() doesn't catch connection refused 
http://bugs.python.org/issue22596  closed by berker.peksag

#22613: Several minor doc issues
http://bugs.python.org/issue22613  closed by georg.brandl

#22695: open() declared deprecated in python 3 docs
http://bugs.python.org/issue22695  closed by ??????????????.??????????????

#22723: visited-link styling is not accessible
http://bugs.python.org/issue22723  closed by berker.peksag

#22727: Improve benchmarks precision
http://bugs.python.org/issue22727  closed by pitrou

#22728: Deprecate spurious benchmarks
http://bugs.python.org/issue22728  closed by pitrou

#22730: ensurepip should work with pythonw.exe
http://bugs.python.org/issue22730  closed by steve.dower

#22736: tutorial links at top, book recommendations at bottom of modul
http://bugs.python.org/issue22736  closed by python-dev

#22740: Cache error
http://bugs.python.org/issue22740  closed by georg.brandl

#22741: suggestion for improving wording on len(s) (built-in function)
http://bugs.python.org/issue22741  closed by r.david.murray

#22745: cgitb with Py3: TypeError: 'str' does not support the buffer i
http://bugs.python.org/issue22745  closed by wrohdewald

#22748: Porting Extension Modules to Python 3 documentation mention ab
http://bugs.python.org/issue22748  closed by python-dev

#22749: remove obsolete remark in time.clock() docs
http://bugs.python.org/issue22749  closed by python-dev

#22754: Implicit String Literal Concatenation Is Evil
http://bugs.python.org/issue22754  closed by r.david.murray

#22756: testAssertEqualSingleLine gives poor errors
http://bugs.python.org/issue22756  closed by python-dev

#22759: pathlib: Path.exists broken
http://bugs.python.org/issue22759  closed by pitrou

#22760: re.sub does only first 16 replacements if re.S is used
http://bugs.python.org/issue22760  closed by georg.brandl

#22762: PyObject_Call called with an exception set while displaying a 
http://bugs.python.org/issue22762  closed by haypo

#22767: `separators` argument to json.dumps() behaves unexpectedly acr
http://bugs.python.org/issue22767  closed by r.david.murray

#22771: shutil.make_archive() doesn't use its "verbose" argument
http://bugs.python.org/issue22771  closed by python-dev

#22772: doc error in __ifloordiv__ and __itruediv__
http://bugs.python.org/issue22772  closed by python-dev

#22774: Weird S.rstrip() result
http://bugs.python.org/issue22774  closed by pitrou