From rymg19 at gmail.com  Thu Jan  1 01:32:52 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 31 Dec 2014 18:32:52 -0600
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
Message-ID: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>

Not sure if this is something to post here...but...

[image: Inline image 1]

-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141231/f6eb3a97/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: python_23.png
Type: image/png
Size: 79711 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141231/f6eb3a97/attachment-0001.png>

From benjamin at python.org  Thu Jan  1 02:24:27 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 31 Dec 2014 20:24:27 -0500
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
In-Reply-To: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
References: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
Message-ID: <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>



On Wed, Dec 31, 2014, at 19:32, Ryan Gonzalez wrote:
> Not sure if this is something to post here...but...
> 
> [image: Inline image 1]

That must be some sort of inconsistency in Google's index, since the
actual page's title is correct.

From rosuav at gmail.com  Thu Jan  1 04:03:12 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 1 Jan 2015 14:03:12 +1100
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
In-Reply-To: <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>
References: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
 <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>
Message-ID: <CAPTjJmo59o299P0j4HWv7yhmith=ia=nANB=Opebg-=9CB6xPA@mail.gmail.com>

On Thu, Jan 1, 2015 at 12:24 PM, Benjamin Peterson <benjamin at python.org> wrote:
> On Wed, Dec 31, 2014, at 19:32, Ryan Gonzalez wrote:
>> Not sure if this is something to post here...but...
>>
>> [image: Inline image 1]
>
> That must be some sort of inconsistency in Google's index, since the
> actual page's title is correct.

This issue has persisted for a while. I first noticed it in early
December (though it may have been there longer), and ignored it on the
assumption that it was a transitional state. I don't know how long a
transitional state can last, though.

ChrisA

From vincent at vincentdavis.net  Thu Jan  1 05:26:38 2015
From: vincent at vincentdavis.net (Vincent Davis)
Date: Wed, 31 Dec 2014 21:26:38 -0700
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
In-Reply-To: <CAPTjJmo59o299P0j4HWv7yhmith=ia=nANB=Opebg-=9CB6xPA@mail.gmail.com>
References: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
 <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>
 <CAPTjJmo59o299P0j4HWv7yhmith=ia=nANB=Opebg-=9CB6xPA@mail.gmail.com>
Message-ID: <CALyJZZUvxnVGXKOZxqoqMEevigcm0J+Mut9HisezJJevhDiueA@mail.gmail.com>

About a year ago I posted about this or related issue, Here is the link
<https://mail.python.org/pipermail/python-dev/2014-January/132093.html>.
This was the old website (I think, but that may not appliy to the docs) but
I think the issue is still probably related to the robot.txt file.

Vincent Davis
720-301-3003

On Wed, Dec 31, 2014 at 8:03 PM, Chris Angelico <rosuav at gmail.com> wrote:

> On Thu, Jan 1, 2015 at 12:24 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
> > On Wed, Dec 31, 2014, at 19:32, Ryan Gonzalez wrote:
> >> Not sure if this is something to post here...but...
> >>
> >> [image: Inline image 1]
> >
> > That must be some sort of inconsistency in Google's index, since the
> > actual page's title is correct.
>
> This issue has persisted for a while. I first noticed it in early
> December (though it may have been there longer), and ignored it on the
> assumption that it was a transitional state. I don't know how long a
> transitional state can last, though.
>
> ChrisA
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/vincent%40vincentdavis.net
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20141231/07faf650/attachment.html>

From fijall at gmail.com  Thu Jan  1 14:52:58 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Thu, 1 Jan 2015 15:52:58 +0200
Subject: [Python-Dev] More compact dictionaries with faster iteration
In-Reply-To: <54A3F65A.1060406@gmail.com>
References: <9BD2AD6A-125D-4A34-B6BF-A99B167554B6@gmail.com>
 <54A3F65A.1060406@gmail.com>
Message-ID: <CAK5idxS0+MqKB4Z4h=sA7CjDbBx3h5ujRCi+hk4CeFjgdW7Kqw@mail.gmail.com>

On Wed, Dec 31, 2014 at 3:12 PM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 10.12.12 03:44, Raymond Hettinger wrote:
>>
>> The current memory layout for dictionaries is
>> unnecessarily inefficient.  It has a sparse table of
>> 24-byte entries containing the hash value, key pointer,
>> and value pointer.
>>
>> Instead, the 24-byte entries should be stored in a
>> dense table referenced by a sparse table of indices.
>
>
> FYI PHP 7 will use this technique [1]. In conjunction with other
> optimizations this will decrease memory consumption of PHP hashtables up to
> 4 times.

"up to 4 times" is a bit of a stretch, given that most of their
savings come from:

* saving on the keeping of ordering
* other optimizations in zval

None of it applies to python

PHP does not implement differing sizes of ints in key dict, which
makes memory saving php-only (if we did the same thing as PHP, we
would save more or less nothing, depending on how greedy you are with
the list overallocation)

We implemented the same strategy in PyPy as of last year, testing it
to become a default "dict" and "OrderedDict" for PyPy in the next
release.

Cheers,
fijal

PS. I wonder who came up with the idea first, PHP or rhettinger and
who implemented it first (I'm pretty sure it was used in hippy before
it was used in Zend PHP)

From wes.turner at gmail.com  Thu Jan  1 08:49:52 2015
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 1 Jan 2015 01:49:52 -0600
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
In-Reply-To: <CALyJZZUvxnVGXKOZxqoqMEevigcm0J+Mut9HisezJJevhDiueA@mail.gmail.com>
References: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
 <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>
 <CAPTjJmo59o299P0j4HWv7yhmith=ia=nANB=Opebg-=9CB6xPA@mail.gmail.com>
 <CALyJZZUvxnVGXKOZxqoqMEevigcm0J+Mut9HisezJJevhDiueA@mail.gmail.com>
Message-ID: <CACfEFw8gjSJoQjAydEPxXzL6it33BMWzd+zaEqwBc68YqMesOw@mail.gmail.com>

Might one or more sitemap.xml files be useful here, if theoretically
unnecessary?

* https://github.com/vladris/tinkerer-contrib/blob/master/sitemap/sitemap.py
* https://en.wikipedia.org/wiki/Sitemaps

On Wed, Dec 31, 2014 at 10:26 PM, Vincent Davis <vincent at vincentdavis.net>
wrote:

> About a year ago I posted about this or related issue, Here is the link
> <https://mail.python.org/pipermail/python-dev/2014-January/132093.html>.
> This was the old website (I think, but that may not appliy to the docs) but
> I think the issue is still probably related to the robot.txt file.
>
> Vincent Davis
> 720-301-3003
>
> On Wed, Dec 31, 2014 at 8:03 PM, Chris Angelico <rosuav at gmail.com> wrote:
>
>> On Thu, Jan 1, 2015 at 12:24 PM, Benjamin Peterson <benjamin at python.org>
>> wrote:
>> > On Wed, Dec 31, 2014, at 19:32, Ryan Gonzalez wrote:
>> >> Not sure if this is something to post here...but...
>> >>
>> >> [image: Inline image 1]
>> >
>> > That must be some sort of inconsistency in Google's index, since the
>> > actual page's title is correct.
>>
>> This issue has persisted for a while. I first noticed it in early
>> December (though it may have been there longer), and ignored it on the
>> assumption that it was a transitional state. I don't know how long a
>> transitional state can last, though.
>>
>> ChrisA
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/vincent%40vincentdavis.net
>>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150101/330cd13f/attachment.html>

From phd at phdru.name  Thu Jan  1 17:15:51 2015
From: phd at phdru.name (Oleg Broytman)
Date: Thu, 1 Jan 2015 17:15:51 +0100
Subject: [Python-Dev] Google search labels Python 2.7 docs as Python 3.4
In-Reply-To: <CACfEFw8gjSJoQjAydEPxXzL6it33BMWzd+zaEqwBc68YqMesOw@mail.gmail.com>
References: <CAO41-mM7tKV5M_xs3yLREXo7aPVf1OaEAihUm8eVJJvEd+6zFQ@mail.gmail.com>
 <1420075467.2690500.208461389.1E405790@webmail.messagingengine.com>
 <CAPTjJmo59o299P0j4HWv7yhmith=ia=nANB=Opebg-=9CB6xPA@mail.gmail.com>
 <CALyJZZUvxnVGXKOZxqoqMEevigcm0J+Mut9HisezJJevhDiueA@mail.gmail.com>
 <CACfEFw8gjSJoQjAydEPxXzL6it33BMWzd+zaEqwBc68YqMesOw@mail.gmail.com>
Message-ID: <20150101161551.GA6628@phdru.name>

On Thu, Jan 01, 2015 at 01:49:52AM -0600, Wes Turner <wes.turner at gmail.com> wrote:
> Might one or more sitemap.xml files be useful here, if theoretically
> unnecessary?
> 
> * https://github.com/vladris/tinkerer-contrib/blob/master/sitemap/sitemap.py
> * https://en.wikipedia.org/wiki/Sitemaps

   I'd rather recommend http://goog-sitemapgen.sourceforge.net/

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From status at bugs.python.org  Fri Jan  2 18:07:45 2015
From: status at bugs.python.org (Python tracker)
Date: Fri,  2 Jan 2015 18:07:45 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20150102170745.D23FE561F4@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2014-12-26 - 2015-01-02)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4706 (+15)
  closed 30207 (+21)
  total  34913 (+36)

Open issues with patches: 2202 


Issues opened (27)
==================

#23116: Python Tutorial 4.7.1: Improve ask_ok() to cover more input va
http://bugs.python.org/issue23116  opened by amyluo

#23117: Properly codesign Mac python 2.7.9.pkg so it can work thru OS 
http://bugs.python.org/issue23117  opened by James.Wahlman

#23119: Remove unicode specialization from set objects
http://bugs.python.org/issue23119  opened by rhettinger

#23120: installation order of 32bit and 64bit python seems to matter
http://bugs.python.org/issue23120  opened by peter at psantoro.net

#23121: pip.exe breaks if python 2.7.9 is installed under c:\Program F
http://bugs.python.org/issue23121  opened by joshuaellinger

#23123: Only READ support for Decimal in json
http://bugs.python.org/issue23123  opened by anders.rundgren.net at gmail.com

#23126: Add Python hook function to replace NameError
http://bugs.python.org/issue23126  opened by Rosuav

#23127: socket.setsockopt() is still broken for multicast TTL and Loop
http://bugs.python.org/issue23127  opened by tamentis

#23128: Key presses are doubled in Tkinter dialog invoked from window 
http://bugs.python.org/issue23128  opened by pew

#23129: sqlite3 COMMIT nested in SELECT returns unexpected results
http://bugs.python.org/issue23129  opened by jamercee

#23132: Faster total_ordering
http://bugs.python.org/issue23132  opened by serhiy.storchaka

#23133: Pickling of ipaddress classes
http://bugs.python.org/issue23133  opened by serhiy.storchaka

#23136: BUG in how _strptime() handles week 0
http://bugs.python.org/issue23136  opened by jamercee

#23137: Python 2.7.9 test_gdb fails on CentOS 7
http://bugs.python.org/issue23137  opened by vlee

#23138: cookiejar parses cookie value as int with empty name-value pai
http://bugs.python.org/issue23138  opened by chfoo

#23139: syntax diagram after EBNF description?
http://bugs.python.org/issue23139  opened by Friedrich.Spee.von.Langenfeld

#23140: InvalidStateError on asyncio subprocess task cancelled
http://bugs.python.org/issue23140  opened by xdegaye

#23142: Integration of unittest.FunctionTestCase with automatic discov
http://bugs.python.org/issue23142  opened by vadmium

#23143: Remove some conditional code in _ssl.c
http://bugs.python.org/issue23143  opened by pitrou

#23144: html.parser.HTMLParser: setting 'convert_charrefs = True' lead
http://bugs.python.org/issue23144  opened by xkjq

#23145: regrtest: log test loader errors
http://bugs.python.org/issue23145  opened by haypo

#23146: Incosistency in pathlib between / and \
http://bugs.python.org/issue23146  opened by serhiy.storchaka

#23147: Possible error in _header_value_parser.py
http://bugs.python.org/issue23147  opened by serhiy.storchaka

#23148: Missing the charset parameter in as_encoded_word()
http://bugs.python.org/issue23148  opened by serhiy.storchaka

#23149: Typo in PEP-0008 - "this PEP do not"
http://bugs.python.org/issue23149  opened by jonrsharpe

#23150: urllib parse incorrect handing of params
http://bugs.python.org/issue23150  opened by julian.reschke at gmx.de

#23151: _loggerClass is initialized twice
http://bugs.python.org/issue23151  opened by serhiy.storchaka



Most recent 15 issues with no replies (15)
==========================================

#23150: urllib parse incorrect handing of params
http://bugs.python.org/issue23150

#23149: Typo in PEP-0008 - "this PEP do not"
http://bugs.python.org/issue23149

#23148: Missing the charset parameter in as_encoded_word()
http://bugs.python.org/issue23148

#23147: Possible error in _header_value_parser.py
http://bugs.python.org/issue23147

#23146: Incosistency in pathlib between / and \
http://bugs.python.org/issue23146

#23143: Remove some conditional code in _ssl.c
http://bugs.python.org/issue23143

#23139: syntax diagram after EBNF description?
http://bugs.python.org/issue23139

#23138: cookiejar parses cookie value as int with empty name-value pai
http://bugs.python.org/issue23138

#23137: Python 2.7.9 test_gdb fails on CentOS 7
http://bugs.python.org/issue23137

#23133: Pickling of ipaddress classes
http://bugs.python.org/issue23133

#23126: Add Python hook function to replace NameError
http://bugs.python.org/issue23126

#23116: Python Tutorial 4.7.1: Improve ask_ok() to cover more input va
http://bugs.python.org/issue23116

#23106: Remove smalltable from set objects
http://bugs.python.org/issue23106

#23097: unittest can unnecessarily modify sys.path (and with the wrong
http://bugs.python.org/issue23097

#23095: asyncio: race condition in IocpProactor.wait_for_handle()
http://bugs.python.org/issue23095



Most recent 15 issues waiting for review (15)
=============================================

#23149: Typo in PEP-0008 - "this PEP do not"
http://bugs.python.org/issue23149

#23145: regrtest: log test loader errors
http://bugs.python.org/issue23145

#23136: BUG in how _strptime() handles week 0
http://bugs.python.org/issue23136

#23133: Pickling of ipaddress classes
http://bugs.python.org/issue23133

#23132: Faster total_ordering
http://bugs.python.org/issue23132

#23126: Add Python hook function to replace NameError
http://bugs.python.org/issue23126

#23119: Remove unicode specialization from set objects
http://bugs.python.org/issue23119

#23106: Remove smalltable from set objects
http://bugs.python.org/issue23106

#23103: Reduce memory usage for ipaddress object instances
http://bugs.python.org/issue23103

#23102: distutils: isinstance checks fail with setuptools-monkeypatche
http://bugs.python.org/issue23102

#23099: BytesIO and StringIO values unavailable when closed
http://bugs.python.org/issue23099

#23098: mknod devices can be >32 bits
http://bugs.python.org/issue23098

#23094: Unpickler failing with PicklingError at frame end on readline 
http://bugs.python.org/issue23094

#23089: Update libffi config files
http://bugs.python.org/issue23089

#23088: Document that PyUnicode_AsUTF8() returns a null-terminated str
http://bugs.python.org/issue23088



Top 10 most discussed issues (10)
=================================

#23123: Only READ support for Decimal in json
http://bugs.python.org/issue23123  23 msgs

#23136: BUG in how _strptime() handles week 0
http://bugs.python.org/issue23136  11 msgs

#22560: Add loop-agnostic SSL implementation to asyncio
http://bugs.python.org/issue22560   9 msgs

#23103: Reduce memory usage for ipaddress object instances
http://bugs.python.org/issue23103   6 msgs

#23121: pip.exe breaks if python 2.7.9 is installed under c:\Program F
http://bugs.python.org/issue23121   6 msgs

#10388: spwd returning different value depending on privileges
http://bugs.python.org/issue10388   5 msgs

#23129: sqlite3 COMMIT nested in SELECT returns unexpected results
http://bugs.python.org/issue23129   5 msgs

#13248: deprecated in 3.2/3.3, should be removed in 3.5 or ???
http://bugs.python.org/issue13248   4 msgs

#23102: distutils: isinstance checks fail with setuptools-monkeypatche
http://bugs.python.org/issue23102   4 msgs

#23127: socket.setsockopt() is still broken for multicast TTL and Loop
http://bugs.python.org/issue23127   4 msgs



Issues closed (20)
==================

#13655: Python SSL stack doesn't have a default CA Store
http://bugs.python.org/issue13655  closed by benjamin.peterson

#15025: httplib and http.client are missing response messages for defi
http://bugs.python.org/issue15025  closed by berker.peksag

#17056: Support Visual Studio 2012
http://bugs.python.org/issue17056  closed by steve.dower

#18648: FP Howto and the PEP 8 lambda guildline
http://bugs.python.org/issue18648  closed by akuchling

#19776: Provide expanduser() on Path objects
http://bugs.python.org/issue19776  closed by pitrou

#20981: ssl doesn't build anymore with OpenSSL 0.9.7 or older: X509_ch
http://bugs.python.org/issue20981  closed by pitrou

#22926: asyncio: raise an exception when called from the wrong thread
http://bugs.python.org/issue22926  closed by haypo

#23107: Tighten-up search loops in sets
http://bugs.python.org/issue23107  closed by rhettinger

#23111: ftplib.FTP_TLS's default constructor does not work with TLSv1.
http://bugs.python.org/issue23111  closed by python-dev

#23114: "dist must be a Distribution instance" check fails with setupt
http://bugs.python.org/issue23114  closed by eric.araujo

#23115: Backport #22585 -- getentropy for urandom to Python 2.7
http://bugs.python.org/issue23115  closed by python-dev

#23118: Properly codesign Mac python 2.7.9.pkg so it can work thru OS 
http://bugs.python.org/issue23118  closed by SilentGhost

#23122: python at 93442 breaks build if system Python is 2.6
http://bugs.python.org/issue23122  closed by Marcel.Hellkamp

#23124: Incorrect Integer saving
http://bugs.python.org/issue23124  closed by ezio.melotti

#23125: Link to the nose project obsolete
http://bugs.python.org/issue23125  closed by berker.peksag

#23130: Tools/scripts/ftpmirror.py allows overwriting arbitrary files 
http://bugs.python.org/issue23130  closed by python-dev

#23131: saving to file add inexistance characters
http://bugs.python.org/issue23131  closed by benjamin.peterson

#23134: datetime.strptime at the turn of the year
http://bugs.python.org/issue23134  closed by belopolsky

#23135: import searchpaths as arguments
http://bugs.python.org/issue23135  closed by c2621566

#23141: Command line long filename not passed correctly
http://bugs.python.org/issue23141  closed by eric.smith

From arigo at tunes.org  Sat Jan  3 17:39:10 2015
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 3 Jan 2015 17:39:10 +0100
Subject: [Python-Dev] More compact dictionaries with faster iteration
In-Reply-To: <CAK5idxS0+MqKB4Z4h=sA7CjDbBx3h5ujRCi+hk4CeFjgdW7Kqw@mail.gmail.com>
References: <9BD2AD6A-125D-4A34-B6BF-A99B167554B6@gmail.com>
 <54A3F65A.1060406@gmail.com>
 <CAK5idxS0+MqKB4Z4h=sA7CjDbBx3h5ujRCi+hk4CeFjgdW7Kqw@mail.gmail.com>
Message-ID: <CAMSv6X1M1Qi081xm-ijejYRdnP-kRuLGwG-eZJWXepu3m1nUjw@mail.gmail.com>

Hi all,

On 1 January 2015 at 14:52, Maciej Fijalkowski <fijall at gmail.com> wrote:
> PS. I wonder who came up with the idea first, PHP or rhettinger and
> who implemented it first (I'm pretty sure it was used in hippy before
> it was used in Zend PHP)

We'd need to look more in detail to that question, but a quick look
made me find this Java code from 2012:

https://code.google.com/r/wassermanlouis-guava/source/browse/guava/src/com/google/common/collect/CompactHashMap.java?name=refs/remotes/gcode-clone/compact-maps

which implements almost exactly the original idea of Raymond.  (It has
a twist because Java doesn't support arrays of (int, int, Object,
Object), and instead encodes it as one array of long and one array of
Objects.  It also uses a chain of buckets instead of open addressing.)


A bient?t,

Armin.

From Steve.Dower at microsoft.com  Sun Jan  4 00:34:05 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 3 Jan 2015 23:34:05 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
Message-ID: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>

I've put together a short post showing where I've been taking the Windows installer for Python 3.5, since I know there are interested people on this list who will have valuable feedback.

http://stevedower.id.au/blog/the-python-3-5-installer/

Nothing is merged in yet and everything can still change, so I'm keen to hear whatever feedback people have. I've tried to make improvements fairly for first-time users through to sysadmins, but if I've missed something big I'd like to hear about it before we get too close to alpha 1.

Cheers,
Steve 

From donald at stufft.io  Sun Jan  4 01:13:40 2015
From: donald at stufft.io (Donald Stufft)
Date: Sat, 3 Jan 2015 19:13:40 -0500
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <7DF5403E-B37A-41CD-BFD8-B6F52624CDB5@stufft.io>


> On Jan 3, 2015, at 6:34 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> 
> I've put together a short post showing where I've been taking the Windows installer for Python 3.5, since I know there are interested people on this list who will have valuable feedback.
> 
> http://stevedower.id.au/blog/the-python-3-5-installer/
> 
> Nothing is merged in yet and everything can still change, so I'm keen to hear whatever feedback people have. I've tried to make improvements fairly for first-time users through to sysadmins, but if I've missed something big I'd like to hear about it before we get too close to alpha 1.
> 

Does signing the .exe?s that ship with Python effect pip at all? I?m guessing not since pip.exe is actually a generic .exe that we use for any project and is shipped inside of the .whl files but I figured i?d ask.

Depending on the answer above, does it make sense to sign the generic .exe (does that even work if we rename it?) which will get used for anything that uses entry points on Windows?

Is there any plan to backport this to 2.7 (presumably after some experience is had with it in 3.5)?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From solipsis at pitrou.net  Sun Jan  4 01:16:06 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 4 Jan 2015 01:16:06 +0100
Subject: [Python-Dev] New Windows installer for Python 3.5
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <20150104011606.2a148a41@fsol>

On Sat, 3 Jan 2015 23:34:05 +0000
Steve Dower <Steve.Dower at microsoft.com> wrote:

> I've put together a short post showing where I've been taking the Windows installer for Python 3.5, since I know there are interested people on this list who will have valuable feedback.
> 
> http://stevedower.id.au/blog/the-python-3-5-installer/

I don't use Windows much, but this looks really great.

One comment: I would find it a bit confusing if the default install
path changes when using the customized install. OTOH, maybe you can't
choose another default there.

Regards

Antoine.



From rosuav at gmail.com  Sun Jan  4 01:22:06 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Sun, 4 Jan 2015 11:22:06 +1100
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CAPTjJmravh7THbNPiAtsNuefJeSLiK6BFMATabSHPqYmu6nSkA@mail.gmail.com>

On Sun, Jan 4, 2015 at 10:34 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> http://stevedower.id.au/blog/the-python-3-5-installer/

You talk of installing by default into Program Files, and having a
separate per-user installation mode. How do these two installation
targets interact? Suppose someone installs 3.5 globally, then installs
3.6 for self only? Or installs 3.5.1 for self only? I would normally
expect a per-user installation to trump a global one, but this could
make for a lovely dep-hell on a system that's used by one person, who
then isn't sure what was installed as admin and what wasn't.

(Also: I'm assuming that pip would require admin privs if Python is in
Program Files, and won't require admins if it's per-user, right? Same
as the Python installer itself?)

ChrisA

From v+python at g.nevcal.com  Sun Jan  4 01:48:49 2015
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sat, 03 Jan 2015 16:48:49 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <20150104011606.2a148a41@fsol>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <20150104011606.2a148a41@fsol>
Message-ID: <54A88DF1.3020001@g.nevcal.com>

General comments:

I was a bit concerned when Steve first posted his plans for the Windows 
installer and making a web installer forcing re-downloads for every 
install, and expressed those privately. I'm no longer concerned, this 
outlined scheme is good, but I have some suggestions to make it great :)

With these other options available, if the web installer can do the 
/layout, especially from a checkbox, I'd almost be tempted to agree that 
the 20MB installers wouldn't be needed.

But here's another idea:  automatically keep all the .msi and .cab files 
used for the first installation of Python with it in the directory from 
which it runs (naming convention... prefix them all with 
python-3.5.0a1.<stuff>.(msi|cab)  It is very likely that a reinstall 
will use the same components (if more are needed on a later install, add 
them to the directory also).  And a good naming convention makes it 
obvious what to delete when done with the installer.

And a related idea: on the first install page, have a check box 
"download all installation components", that would do that, even if they 
are not used, and even if either of the one-click installs are chosen.

And a related idea: for custom installs, record the choices made in a 
metadata file in that same directory, and after the first install, 
subsequent installs could have a 3rd single-click install: same custom 
install as last time.  This would be kept in the directory with the 
installer, so could be applied to a zillion machines, and an install 
option /ditto would allow those choices from the command line. That way, 
the administrator could use the friendly interface to install the first 
machine, making the appropriate choices, and then just run 
"python-3.5.0a1.exe /ditto" on all the other zillion-1 machines, without 
needing to learn any other obscure command line parameters. I don't care 
how you spell /ditto, as long as there is documentation.

Regarding /layout, to me, that name has no mnemonic meaning of "download 
all these installation components and save them". Documentation could 
provide that, of course, but choosing a name like /download might be 
nicer.  Saving to the same directory as the installer lives in seems 
easier than needing to specify a directory... the documentation can warn 
that users of the option should put the web installer in the desired, 
shared or private, installation directory prior to running the option...

On 1/3/2015 4:16 PM, Antoine Pitrou wrote:
> One comment: I would find it a bit confusing if the default install
> path changes when using the customized install. OTOH, maybe you can't
> choose another default there.

This could be cured by defaulting to either the new or old install 
location, or to a blank box, and having a couple buttons to "set install 
location to C:\Python35" or "set install location to "C:\Program 
Files....", as well as the browse button and the option to type into the 
box.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150103/03e971cd/attachment.html>

From thomas.f.hahn2 at gmail.com  Sun Jan  4 04:06:07 2015
From: thomas.f.hahn2 at gmail.com (thomas hahn)
Date: Sat, 3 Jan 2015 21:06:07 -0600
Subject: [Python-Dev] Help with finding tutors for Python, Linux, R, Perl,
 Octave, MATLAB and/or Cytoscape for yeast microarray analysis,
 next generation sequencing and constructing gene interaction networks
Message-ID: <CAKJ67ZV7o_iKP+rYnmU+vEs4Ky0sLMQ58XH8SBG=w6mLo2L5vg@mail.gmail.com>

Help with finding tutors for Python, Linux, R, Perl, Octave, MATLAB and/or
Cytoscape for yeast microarray analysis, next generation sequencing and
constructing gene interaction networks



Hi



I am a visually impaired bioinformatics graduate student using microarray
data for my master?s thesis aimed at deciphering the mechanism by which the
yeast wild type can suppress the rise of free reactive oxygen species (ROS)
induced by caloric restriction (CR) but the Atg15 and Erg6 knockout mutant
cannot.



Since my remaining vision is very limited I need very high magnification.
But that makes my visual field very small.  Therefore I need somebody to
teach me how to use these programming environments, especially for
microarray analysis, next generation sequencing and constructing gene and
pathway interaction networks.  This is very difficult for me to figure out
without assistance because Zoomtext, my magnification and text to speech
software, on which I am depending because I am almost blind, has problems
reading out aloud many programming related websites to me.  And even those
websites it can read, it can only read sequentially from left to right and
then from top to bottom.  Unfortunately, this way of acquiring, finding,
selecting and processing new information and answering questions is too
tiresome, exhausting, ineffective and especially way too time consuming for
graduating with a PhD in bioinformatics before my funding runs out despite
being severely limited by my visual disability.  I would also need help
with writing a good literature review and applying the described techniques
to my own yeast Affimetrix microarray dataset because I cannot see well
enough to find all relevant publications on my own.



Some examples for specific tasks I urgently need help with are:



1.    Analyzing and comparing the three publically available microarray
datasets that can be accessed at:

A.    http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE41860

B.    http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE38635

C.   http://www.ncbi.nlm.nih.gov/geo/query/acc.cgi?acc=GSE9217



2.    Learning how to use the Affymetrics microarray analysis software for
the Yeast 2 chip, which can be found at
http://www.affymetrix.com/support/technical/libraryfilesmain.affx



3.    For Cytoscape I need somebody, who can teach me how to execute the
tutorials at the following links because due to my very limited vision
field I cannot see tutorial and program interface simultaneously.

A.
http://opentutorials.cgl.ucsf.edu/index.php/Tutorial:Introduction_to_Cytoscape_3.1-part2#Importing_and_Exploring_Your_Data

B.
http://opentutorials.cgl.ucsf.edu/index.php/Tutorial:Filtering_and_Editing_in_Cytoscape_3

C.
http://cytoscape.org/manual/Cytoscape2_8Manual.html#Import%20Fixed-Format%20Network%20Files

D.   http://wiki.cytoscape.org/Cytoscape_User_Manual/Network_Formats



4.    Learning how to use the TopGo R package to perform statistical
analysis on GO enrichments.



Since I am legally blind the rehab agency is giving me money to pay tutors
for this purpose.  Could you please help me getting in touch regarding this
with anybody, who could potentially be interested in teaching me one on one
thus saving me time for acquiring new information and skills, which I need
to finish my thesis on time, so that I can remain eligible for funding to
continue in my bioinformatics PhD program despite being almost blind?  The
tutoring can be done remotely via TeamViewer 5 and Skype.  Hence, it does
not matter where my tutors are physically located.  Currently I have tutors
in Croatia and UK.  But since they both work full time jobs while working
on their PhD dissertation they only have very limited time to teach me
online.  Could you therefore please forward this request for help to
anybody, who could potentially be interested or, who could connect me to
somebody, who might be, because my graduation and career depend on it?  Who
else would you recommend me to contact regarding this?  Where else could I
post this because I am in urgent need for help?



Could you please contact me directly via email at Thomas.F.Hahn2 at gmail.com
and/or Skype at tfh002 because my text to speech software has problems to
read out this website aloud to me?

I thank you very much in advance for your thoughts, ideas, suggestions,
recommendations, time, help, efforts and support.

With very warm regards,



*Thomas Hahn*

1)    *Graduate student in the Joint Bioinformatics Program at the
University of Arkansas at Little Rock (UALR) and the University of Arkansas
Medical Sciences (UAMS) &*

2)    *Research & Industry Advocate, Founder and Board Member of RADISH
MEDICAL SOLUTIONS, INC. (**http://www.radishmedical.com/thomas-hahn/*
<http://www.radishmedical.com/thomas-hahn/>*) *



*Primary email: **Thomas.F.Hahn2 at gmail.com* <Thomas.F.Hahn2 at gmail.com>

*Cell phone: 318 243 3940*

*Office phone: 501 682 1440*

*Office location: EIT 535*

*Skype ID: tfh002*

*Virtual Google Voice phone to reach me while logged into my email (i.e. *
*Thomas.F.Hahn2 at gmail.com* <Thomas.F.Hahn2 at gmail.com>*), even when having
no cell phone reception, e.g. in big massive buildings:  *(501) 301-4890
<%28501%29%20301-4890>



*Web links: *



1)      https://ualr.academia.edu/ThomasHahn

2)      https://www.linkedin.com/pub/thomas-hahn/42/b29/42

3)      http://facebook.com/Thomas.F.Hahn
<https://www.facebook.com/Thomas.F.Hahn>

4)      https://twitter.com/Thomas_F_Hahn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150103/2357dbb7/attachment-0001.html>

From senthil at uthcode.com  Sun Jan  4 05:05:52 2015
From: senthil at uthcode.com (Senthil Kumaran)
Date: Sat, 3 Jan 2015 20:05:52 -0800
Subject: [Python-Dev] Help with finding tutors for Python, Linux, R, Perl,
 Octave, MATLAB and/or Cytoscape for yeast microarray analysis,
 next generation sequencing and constructing gene interaction networks
In-Reply-To: <CAKJ67ZV7o_iKP+rYnmU+vEs4Ky0sLMQ58XH8SBG=w6mLo2L5vg@mail.gmail.com>
References: <CAKJ67ZV7o_iKP+rYnmU+vEs4Ky0sLMQ58XH8SBG=w6mLo2L5vg@mail.gmail.com>
Message-ID: <BE0F6E767A2F4E7DB5825999656F4EA9@uthcode.com>



On Saturday, January 3, 2015 at 7:06 PM, thomas hahn wrote:

> Help with finding tutors for Python, Linux, R, Perl, Octave, MATLAB and/or Cytoscape for yeast microarray analysis, next generation sequencing and constructing gene interaction networks 

This is an inappropriate list to seek out for those folks. python-dev is dedicated to development of python language itself.

Please ask python-users at python.org


-- 
Senthil Kumaran

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150103/d55f7d2f/attachment.html>

From p.f.moore at gmail.com  Sun Jan  4 13:01:59 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Sun, 4 Jan 2015 12:01:59 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>

On 3 January 2015 at 23:34, Steve Dower <Steve.Dower at microsoft.com> wrote:
> I've put together a short post showing where I've been taking the Windows installer for Python 3.5, since I know there are interested people on this list who will have valuable feedback.
>
> http://stevedower.id.au/blog/the-python-3-5-installer/
>
> Nothing is merged in yet and everything can still change, so I'm keen to hear whatever feedback people have. I've tried to make improvements fairly for first-time users through to sysadmins, but if I've missed something big I'd like to hear about it before we get too close to alpha 1.

Overall, this looks good. One question - will it be possible to
install both 32-bit and 64-bit Python on the same machine? Currently,
you need a custom install to do this (as the default directory doesn't
include the architecture) and IIRC there's some oddness around install
order. It would be nice if installing both versions were a supported
option, both for the "default" install and in custom installs.

Also, what happens now with setting PATH? Is Python (and the scripts
directory) added to PATH by default? If so, what happens when you
install 2 versions of Python?

In case it's not clear, I'm thinking of the impact on build machines,
which often have multiple versions, in both 32- and 64-bit forms,
installed simultaneously (but can also be used as a "normal"
development machine, and for that purpose will want a selected Python
version as the default one.

Also, how does the launcher py.exe fit into the picture? Is it still
installed into the Windows directory? What about for a user install?
Are Python scripts associated with the launcher, and if so, how does
it pick up the version you selected as default?

(Sorry, that was more than one question :-))

Paul

From ncoghlan at gmail.com  Sun Jan  4 15:13:20 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 5 Jan 2015 00:13:20 +1000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
Message-ID: <CADiSq7fuwqDNNKX2AQZEYB657dLkyzofYyx=dAnZgJYezsQYhw@mail.gmail.com>

On 4 January 2015 at 22:01, Paul Moore <p.f.moore at gmail.com> wrote:
> On 3 January 2015 at 23:34, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Also, how does the launcher py.exe fit into the picture? Is it still
> installed into the Windows directory? What about for a user install?
> Are Python scripts associated with the launcher, and if so, how does
> it pick up the version you selected as default?
>
> (Sorry, that was more than one question :-))

The proposed installer changes look like a great improvement to me,
but I think Paul's questions suggest that it will be useful to capture
the many decisions involved in the redesign as a PEP.

That serves a couple of useful purposes:

1. It helps guide the design itself, and ensure that the various usage
scenarios (like build machines) are suitably covered.
2. When Python 3.5 is released, we can point readers of the What's New
document to the PEP as a reference for the detailed decision making
process that went into the changes that were made.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ericfahlgren at gmail.com  Sun Jan  4 15:23:42 2015
From: ericfahlgren at gmail.com (Eric Fahlgren)
Date: Sun, 4 Jan 2015 06:23:42 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
Message-ID: <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>

I believe that the new default location addresses your first question: the
64-bit will install in c:/Program Files/Pythonxx and the 32-bit in
c:/Program Files (x86)/Pythonxx (at least this has been my standard
practice for several years).

Eric
On Jan 4, 2015 7:03 AM, "Paul Moore" <p.f.moore at gmail.com> wrote:

> On 3 January 2015 at 23:34, Steve Dower <Steve.Dower at microsoft.com> wrote:
> > I've put together a short post showing where I've been taking the
> Windows installer for Python 3.5, since I know there are interested people
> on this list who will have valuable feedback.
> >
> > http://stevedower.id.au/blog/the-python-3-5-installer/
> >
> > Nothing is merged in yet and everything can still change, so I'm keen to
> hear whatever feedback people have. I've tried to make improvements fairly
> for first-time users through to sysadmins, but if I've missed something big
> I'd like to hear about it before we get too close to alpha 1.
>
> Overall, this looks good. One question - will it be possible to
> install both 32-bit and 64-bit Python on the same machine? Currently,
> you need a custom install to do this (as the default directory doesn't
> include the architecture) and IIRC there's some oddness around install
> order. It would be nice if installing both versions were a supported
> option, both for the "default" install and in custom installs.
>
> Also, what happens now with setting PATH? Is Python (and the scripts
> directory) added to PATH by default? If so, what happens when you
> install 2 versions of Python?
>
> In case it's not clear, I'm thinking of the impact on build machines,
> which often have multiple versions, in both 32- and 64-bit forms,
> installed simultaneously (but can also be used as a "normal"
> development machine, and for that purpose will want a selected Python
> version as the default one.
>
> Also, how does the launcher py.exe fit into the picture? Is it still
> installed into the Windows directory? What about for a user install?
> Are Python scripts associated with the launcher, and if so, how does
> it pick up the version you selected as default?
>
> (Sorry, that was more than one question :-))
>
> Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/ericfahlgren%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150104/d34d4d8b/attachment.html>

From Steve.Dower at microsoft.com  Sun Jan  4 23:56:10 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sun, 4 Jan 2015 22:56:10 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>,
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
Message-ID: <1420412176002.36681@microsoft.com>

Thanks for all the comments. I've responded all at once, since I don't think any of the points raised are going to immediately turn into intense discussion. Feel free to trim the rest of the post if you want to respond to just one of the points.

Donald Stufft wrote:
> Does signing the .exe?s that ship with Python effect pip at all? I?m guessing
> not since pip.exe is actually a generic .exe that we use for any project and is
> shipped inside of the .whl files but I figured i?d ask.
>
> Depending on the answer above, does it make sense to sign the generic .exe (does
> that even work if we rename it?) which will get used for anything that uses
> entry points on Windows?

Actually, I don't think the generic .exe can be signed, since embedding the script within the file will invalidate the certificate. It's better not to have the certificate at all in this case. If we were still using the separate pip-script.py file then we could sign it.

> Is there any plan to backport this to 2.7 (presumably after some experience is
> had with it in 3.5)?

No. I'm very much in the "2.7 is finished" camp. Hitting rebuild every now and then for security updates is the extent of my interest (and in large part that's because if nobody is producing builds then I'll probably end up having to produce the builds to run on Azure anyway).



Antione Pitrou wrote:
> One comment: I would find it a bit confusing if the default install path
> changes when using the customized install. OTOH, maybe you can't choose
> another default there.

It's a tough one to choose, honestly. I really don't want separate "customize" buttons for per-user and all-user installs (too many up front decisions, especially for people who really should just choose the first option). My reasoning for having "C:\Python35" as the default there is to simplify the "just give me the old style install" case, but that may turn out to be a bad reason. I'm not sure yet.



ChrisA wrote:
> You talk of installing by default into Program Files, and having a
> separate per-user installation mode. How do these two installation
> targets interact? Suppose someone installs 3.5 globally, then installs
> 3.6 for self only? Or installs 3.5.1 for self only? I would normally
> expect a per-user installation to trump a global one, but this could
> make for a lovely dep-hell on a system that's used by one person, who
> then isn't sure what was installed as admin and what wasn't.

Yeah, it gets a little messy, especially if you keep adding all the Python versions to PATH (Windows will always put per-user PATH entries at the end). The py.exe launcher handles this best, and the system version will always be found first.

As for versions, 3.5 vs 3.6 should be distinguished explicitly ("py -3.5" vs "py -3.6") and 3.5.0 vs 3.5.1 can only be installed at the same time if one is system-wide and the other is per-user. In this case, the per-user one will be used by py.exe, even if it is older than the system-wide one.

Compared to the current situation, the per-user and all-user installs do not interfere as much.

> (Also: I'm assuming that pip would require admin privs if Python is in
> Program Files, and won't require admins if it's per-user, right? Same
> as the Python installer itself?)

Correct. The pip guys are already looking into handling installs in this case since it matters on *nix, and having read-only Python installs on Windows will only make it more similar. It seems likely in this case that the user site-packages folder will be used if your user can't write to the shared site-packages. If you can write to it (either per-user install or running pip as admin) then you'll install to that folder.

I am fully expecting to find the most issues in this area once we change the install location, and I don't think we can fix all these issues without actually releasing the change. So it's a bit of a risk, but one I feel is worth it. Of course, if the community and release manager disagree, I'll happily back it out and we can focus on testing it directly for a bit before inflicting it on our users.



Glenn Linderman wrote:
> But here's another idea: automatically keep all the .msi and .cab files used for
> the first installation of Python with it in the directory from which it runs
> (naming convention... prefix them all with python-3.5.0a1.<stuff>.(msi|cab) It
> is very likely that a reinstall will use the same components (if more are needed
> on a later install, add them to the directory also). And a good naming
> convention makes it obvious what to delete when done with the installer.

Unfortunately, I'm fairly restricted on naming convention. For some reason Windows Installer requires CAB files to be 8.3 names, where some of the 8 are reserved, so the names are pretty meaningless.

However, Windows does automatically cache all the files (typically in C:\ProgramData\Package Cache) so that repairs and uninstalls don't need to redownload anything, and this cache is automatically managed to preserve disk space. If you add new features later then you won't have the files in the cache, which is about the only way that this doesn't already fit your suggestion. I think it's fairly unlikely that people will be surprised by this, though when the first complaints come it about it I'll happily reconsider.
 
> And a related idea: on the first install page, have a check box "download all
> installation components", that would do that, even if they are not used, and
> even if either of the one-click installs are chosen.

Interesting idea, but I'm worried that people would think they need to click the box simply because it is there. I've seen the same thing happen in many other places, and I think this functionality is advanced enough that the people who need it know that they need it and will look for it. I could, however, add a help link to the first page of the installer, once there's somewhere for it to go.

> And a related idea: for custom installs, record the choices made in a metadata
> file in that same directory, and after the first install, subsequent installs
> could have a 3rd single-click install: same custom install as last time. This
> would be kept in the directory with the installer, so could be applied to a
> zillion machines, and an install option /ditto would allow those choices from
> the command line. That way, the administrator could use the friendly interface
> to install the first machine, making the appropriate choices, and then just run
> "python-3.5.0a1.exe /ditto" on all the other zillion-1 machines, without needing
> to learn any other obscure command line parameters. I don't care how you spell
> /ditto, as long as there is documentation.

This is really interesting, and certainly possible. I'll keep it in mind as something to look into after alpha. (What may be easier is a message at the end showing the command line to use that will perform the same installation silently... hmm...)

> Regarding /layout, to me, that name has no mnemonic meaning of "download all
> these installation components and save them". Documentation could provide that,
> of course, but choosing a name like /download might be nicer. Saving to the same
> directory as the installer lives in seems easier than needing to specify a
> directory... the documentation can warn that users of the option should put the
> web installer in the desired, shared or private, installation directory prior to
> running the option...

It's the standard option used by Burn (the install engine I'm using) - for example, the Visual Studio web installers will also respond to /layout - and it's probably from some internal Microsoft terminology. However, it's easy enough for me to add an alias.

The installer also responds to "/?" and "/help" (IIRC, it may only be the first) to display all of its options, and I expect this will be the first thing people guess to try.


Paul Moore wrote:
> Overall, this looks good. One question - will it be possible to
> install both 32-bit and 64-bit Python on the same machine? Currently,
> you need a custom install to do this (as the default directory doesn't
> include the architecture) and IIRC there's some oddness around install
> order. It would be nice if installing both versions were a supported
> option, both for the "default" install and in custom installs.

Yes, the default install directories on the first page are different for 32-bit and 64-bit (either "Program Files [(x86)]" or "Python35[-32]"). The default value on the customize page is currently always C:\Python35, and so you'll need to change it if you're doing custom installs, but that's easy to add the "-32" by default. (I used a -32 suffix because it matches the py.exe option.)

> Also, what happens now with setting PATH? Is Python (and the scripts
> directory) added to PATH by default? If so, what happens when you
> install 2 versions of Python?

Yes, and in general the later installed version will win and system-wide installs always beat per-user installs. As I mentioned above, using py.exe with a parameter or shebang line is the most reliable way to get the version you want.

> In case it's not clear, I'm thinking of the impact on build machines,
> which often have multiple versions, in both 32- and 64-bit forms,
> installed simultaneously (but can also be used as a "normal"
> development machine, and for that purpose will want a selected Python
> version as the default one.

You should see my dev machines :) Most have 2.5, 2.6, 2.7 32-bit and 64-bit, 3.0, 3.1, 3.2, 3.3 32-bit and 64-bit, 3.4 32-bit and 64-bit, IronPython and often PyPy, Anaconda or Canopy. So I'm being fairly selfish when I try and make the multiple-versions scenario work better, but it will benefit everyone.

> Also, how does the launcher py.exe fit into the picture? Is it still
> installed into the Windows directory? What about for a user install?
> Are Python scripts associated with the launcher, and if so, how does
> it pick up the version you selected as default?

py.exe is more important than ever. It's still installed into the Windows directory for all-user installs, and into the Python directory for just-for-me. It's installed in a way that will make upgrades more reliable (so if you install 3.6 and then 3.5, you'll keep the newer launcher) and all the file associations go straight to the launcher.

The default Python for the launcher seems to be 2.7 if it's there and the latest version if not (though I could be wrong). Shebang lines are the best way to choose a specific version.

Incidentally, whoever came up with the py.exe launcher deserves a medal. It's made dealing with multiple versions amazingly easy.



Nick Coghlan wrote:
> The proposed installer changes look like a great improvement to me,
> but I think Paul's questions suggest that it will be useful to capture
> the many decisions involved in the redesign as a PEP.

I was afraid someone would suggest this :)

I certainly intend to document it thoroughly. One of my overarching goals is to make it really easy for the next person to take over producing builds, or for someone else to produce a signed release build if I'm unavailable.

My intent was just to drop a text file in Tools/msi and some of the user-facing docs (like command-line options, best practices and rationale) into the main docs. If there's added value in a PEP, then I can do that too, though I would prefer the documentation for this to be more living than most PEPs are.


Cheers,
Steve

From rosuav at gmail.com  Mon Jan  5 01:19:37 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 5 Jan 2015 11:19:37 +1100
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420412176002.36681@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
Message-ID: <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>

On Mon, Jan 5, 2015 at 9:56 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> (Windows will always put per-user PATH entries at the end). The py.exe launcher handles this best, and the system version will always be found first.
>
> As for versions, 3.5 vs 3.6 should be distinguished explicitly ("py -3.5" vs "py -3.6") and 3.5.0 vs 3.5.1 can only be installed at the same time if one is system-wide and the other is per-user. In this case, the per-user one will be used by py.exe, even if it is older than the system-wide one.
>

Wait, what?</anna>

If I'm reading this correctly, PATH is set such that an all-users
version trumps a self-only version, but when you use the py.exe
launcher, it's the other way around? That seems extremely confusing.
But if that's not a deliberate design decision and isn't enforced by
external code, would it be possible to simply invert that for PATH,
and have a per-user installation always be found ahead of a
system-wide one? That would make reasonable sense, if it can be done.

> Incidentally, whoever came up with the py.exe launcher deserves a medal. It's made dealing with multiple versions amazingly easy.

https://www.python.org/dev/peps/pep-0397/ - and I agree :)

ChrisA

From v+python at g.nevcal.com  Mon Jan  5 02:00:37 2015
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 04 Jan 2015 17:00:37 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420412176002.36681@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>,
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
Message-ID: <54A9E235.8030204@g.nevcal.com>

On 1/4/2015 2:56 PM, Steve Dower wrote:
> Unfortunately, I'm fairly restricted on naming convention. For some reason Windows Installer requires CAB files to be 8.3 names, where some of the 8 are reserved, so the names are pretty meaningless.
Least common denominator, I'm sure. But a pretty stupid limitation at 
this point in time. It would convince me to use a different installer 
technology, along with the many obscure and arcane features of .msi and 
.cab files that already convinced me of that years ago.

So then I'd suggest storing all the files in a directory named 
"python-3.5.0a1-install-files" unless that would also have to be limited 
to 8.3 :(

> However, Windows does automatically cache all the files (typically in C:\ProgramData\Package Cache) so that repairs and uninstalls don't need to redownload anything, and this cache is automatically managed to preserve disk space. If you add new features later then you won't have the files in the cache, which is about the only way that this doesn't already fit your suggestion. I think it's fairly unlikely that people will be surprised by this, though when the first complaints come it about it I'll happily reconsider.
Handy, I was unaware of this, being turned off long ago by the stupid 
features of .cab and .msi, so the good features have escaped me. If 
there is a way to convince it to download the rest of the files based on 
the checkbox, that would be handier. Checkbox could be worded "download 
install components even for features not installed" which should keep 
people from checking it unnecessarily.  You mentioned a help link... 
couldn't the help text be included (text compresses great!) and popped 
up in a window?

>> >And a related idea: for custom installs, record the choices made in a metadata
>> >file in that same directory, and after the first install, subsequent installs
>> >could have a 3rd single-click install: same custom install as last time. This
>> >would be kept in the directory with the installer, so could be applied to a
>> >zillion machines, and an install option /ditto would allow those choices from
>> >the command line. That way, the administrator could use the friendly interface
>> >to install the first machine, making the appropriate choices, and then just run
>> >"python-3.5.0a1.exe /ditto" on all the other zillion-1 machines, without needing
>> >to learn any other obscure command line parameters. I don't care how you spell
>> >/ditto, as long as there is documentation.
> This is really interesting, and certainly possible. I'll keep it in mind as something to look into after alpha. (What may be easier is a message at the end showing the command line to use that will perform the same installation silently... hmm...)
>
If you can display the command line (which would be interesting), then 
you can save it in a file, and invoke it from the 3rd single-click 
install button! Or if you can't, at least make sure the command can be 
copied to the clipboard, and the administrator could paste it to 
python-3.4.0a3.ditto.cmd :)

> Incidentally, whoever came up with the py.exe launcher deserves a medal. It's made dealing with multiple versions amazingly easy.

Totally agree!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150104/502b14b2/attachment.html>

From rosuav at gmail.com  Mon Jan  5 02:35:21 2015
From: rosuav at gmail.com (Chris Angelico)
Date: Mon, 5 Jan 2015 12:35:21 +1100
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420420846274.86975@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
Message-ID: <CAPTjJmrCJ9O1Swmdr4On+cdojY2xMmaFVbkzsXPTO6uQAUK8_w@mail.gmail.com>

On Mon, Jan 5, 2015 at 12:20 PM, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Unfortunately, Windows enforces the PATH ordering. It constructs the PATH from two registry keys, one is the system-wide value (that requires administrative privileges to modify) and it is followed by the user's value (that does not require administrative privileges). This is probably for security reasons and can't be changed.
>
> PATH also suffers from including the most-recently installed Python version first, rather than the most recent version. Basically, py.exe gives the behaviour we want and PATH simply can't do it.
>
> I think this means the best way to make multiple versions work properly is to rename python.exe to python3.5.exe, then install the launcher as python.exe and python3.exe (with some logic to look at its own name) so it can resolve the right version. Maybe we can even extend the launcher to resolve launchers in Scripts (pip.exe, etc.) and have consistent rules there too?
>
> Or perhaps this is more trouble than it's worth, and we should let "py.exe" have a fixed set of rules and let PATH work as people have come to expect. Thoughts?

Great. :( Well, if there's no changing the PATH behaviour, so be it...
the question is, should py.exe match that, or do the sane thing?
Neither option is ideal.

ChrisA

From Steve.Dower at microsoft.com  Mon Jan  5 02:20:41 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 5 Jan 2015 01:20:41 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>,
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
Message-ID: <1420420846274.86975@microsoft.com>

Chris Angelico wrote:
> On Mon, Jan 5, 2015 at 9:56 AM, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> (Windows will always put per-user PATH entries at the end). The py.exe
>> launcher handles this best, and the system version will always be found first.
>>
>> As for versions, 3.5 vs 3.6 should be distinguished explicitly ("py -3.5" vs
>> "py -3.6") and 3.5.0 vs 3.5.1 can only be installed at the same time if one is
>> system-wide and the other is per-user. In this case, the per-user one will be
>> used by py.exe, even if it is older than the system-wide one.
>>
> 
> Wait, what?</anna>
> 
> If I'm reading this correctly, PATH is set such that an all-users
> version trumps a self-only version, but when you use the py.exe
> launcher, it's the other way around? That seems extremely confusing.
> But if that's not a deliberate design decision and isn't enforced by
> external code, would it be possible to simply invert that for PATH,
> and have a per-user installation always be found ahead of a
> system-wide one? That would make reasonable sense, if it can be done.

Unfortunately, Windows enforces the PATH ordering. It constructs the PATH from two registry keys, one is the system-wide value (that requires administrative privileges to modify) and it is followed by the user's value (that does not require administrative privileges). This is probably for security reasons and can't be changed.

PATH also suffers from including the most-recently installed Python version first, rather than the most recent version. Basically, py.exe gives the behaviour we want and PATH simply can't do it.

I think this means the best way to make multiple versions work properly is to rename python.exe to python3.5.exe, then install the launcher as python.exe and python3.exe (with some logic to look at its own name) so it can resolve the right version. Maybe we can even extend the launcher to resolve launchers in Scripts (pip.exe, etc.) and have consistent rules there too?

Or perhaps this is more trouble than it's worth, and we should let "py.exe" have a fixed set of rules and let PATH work as people have come to expect. Thoughts?

Cheers,
Steve

From p.f.moore at gmail.com  Mon Jan  5 09:13:41 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Jan 2015 08:13:41 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420412176002.36681@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
Message-ID: <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>

On 4 January 2015 at 22:56, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> Also, what happens now with setting PATH? Is Python (and the scripts
>> directory) added to PATH by default? If so, what happens when you
>> install 2 versions of Python?
>
> Yes, and in general the later installed version will win and system-wide installs always beat per-user installs. As I mentioned above, using py.exe with a parameter or shebang line is the most reliable way to get the version you want.

Hmm, that's unfortunate. Normally I leave "Add to PATH" off for all
but one installation. I certainly don''t want all versions on there -
I've had too many issues with running something like py.test and
finding it's running Python 3.3 because I didn't install it into
Python 3.4.

I'm pretty sure we'll get bug reports "I installed Python 3.5.1 but
I'm still getting Python 3.5" from the per-user behaviour. And saying
"don't do that" isn't a fix, nor is blaming Microsoft, really.
Unfortunately, I don't see a good solution here.

>> Also, how does the launcher py.exe fit into the picture? Is it still
>> installed into the Windows directory? What about for a user install?
>> Are Python scripts associated with the launcher, and if so, how does
>> it pick up the version you selected as default?
>
> py.exe is more important than ever. It's still installed into the Windows directory for all-user installs, and into the Python directory for just-for-me. It's installed in a way that will make upgrades more reliable (so if you install 3.6 and then 3.5, you'll keep the newer launcher) and all the file associations go straight to the launcher.

The biggest problem here is that py.exe doesn't help in the slightest
with script wrappers like pip.exe, virtualenv.exe, py.test.exe,
ipython.exe ... I've actually drifted away from using py.exe because
of this. Having just the interpreter special cased isn't really good
enough. (I know there's py -m pip, but it's not easy to get people to
use this...)

I think it's really important to only have one Python on your PATH, so
just dumping everything there by default, particularly if user
installs get pushed to the end and don't have precedence, is a bad
experience.

> The default Python for the launcher seems to be 2.7 if it's there and the latest version if not (though I could be wrong). Shebang lines are the best way to choose a specific version.

Yes, that's a very bad default IMO. It was probably reasonable at the
time, but it should be fixed now - at least to a simple "the latest
version". Getting a user who's just installed Python 3.5 (with a
system Python 2.7 present) to edit an ini file in C:\Windows\System32
just to use his new version is *not* a good user experience.

Paul

From p.f.moore at gmail.com  Mon Jan  5 09:17:21 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Jan 2015 08:17:21 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420420846274.86975@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
Message-ID: <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>

On 5 January 2015 at 01:20, Steve Dower <Steve.Dower at microsoft.com> wrote:
> I think this means the best way to make multiple versions work properly is to rename python.exe to python3.5.exe, then install the launcher as python.exe and python3.exe (with some logic to look at its own name) so it can resolve the right version. Maybe we can even extend the launcher to resolve launchers in Scripts (pip.exe, etc.) and have consistent rules there too?

This was a big debate over on distutils-sig. I advocated "py -m pip"
for portable use, but got basically nowhere. People want "pip" to
work, for compatibility with Unix (and hence ease of documentation).
If we can't get PATH sorted out, we need to do something about this,
IMO. I don't know what (believe me, I tried to find a solution)
unfortunately.

Paul

From donald at stufft.io  Mon Jan  5 09:40:13 2015
From: donald at stufft.io (Donald Stufft)
Date: Mon, 5 Jan 2015 03:40:13 -0500
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
Message-ID: <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>


> On Jan 5, 2015, at 3:13 AM, Paul Moore <p.f.moore at gmail.com> wrote:
> 
>> py.exe is more important than ever. It's still installed into the Windows directory for all-user installs, and into the Python directory for just-for-me. It's installed in a way that will make upgrades more reliable (so if you install 3.6 and then 3.5, you'll keep the newer launcher) and all the file associations go straight to the launcher.
> 
> The biggest problem here is that py.exe doesn't help in the slightest
> with script wrappers like pip.exe, virtualenv.exe, py.test.exe,
> ipython.exe ... I've actually drifted away from using py.exe because
> of this. Having just the interpreter special cased isn't really good
> enough. (I know there's py -m pip, but it's not easy to get people to
> use this...)

Won?t script wrappers use whatever version they were installs against? Or do
you mean the ?installed? version might be 3.5 for ``pip.exe`` even though
there?s a 3.5.1 for ``pip.exe`` on the PATH?

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From breamoreboy at yahoo.co.uk  Mon Jan  5 09:49:12 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 05 Jan 2015 08:49:12 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420412176002.36681@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>,
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
Message-ID: <m8dj6b$fgi$1@ger.gmane.org>

On 04/01/2015 22:56, Steve Dower wrote:
>
> Paul Moore wrote:
>> Overall, this looks good. One question - will it be possible to
>> install both 32-bit and 64-bit Python on the same machine? Currently,
>> you need a custom install to do this (as the default directory doesn't
>> include the architecture) and IIRC there's some oddness around install
>> order. It would be nice if installing both versions were a supported
>> option, both for the "default" install and in custom installs.
>
> Yes, the default install directories on the first page are different for 32-bit and 64-bit (either "Program Files [(x86)]" or "Python35[-32]"). The default value on the customize page is currently always C:\Python35, and so you'll need to change it if you're doing custom installs, but that's easy to add the "-32" by default. (I used a -32 suffix because it matches the py.exe option.)
>
>> Also, what happens now with setting PATH? Is Python (and the scripts
>> directory) added to PATH by default? If so, what happens when you
>> install 2 versions of Python?
>
> Yes, and in general the later installed version will win and system-wide installs always beat per-user installs. As I mentioned above, using py.exe with a parameter or shebang line is the most reliable way to get the version you want.
>
>> In case it's not clear, I'm thinking of the impact on build machines,
>> which often have multiple versions, in both 32- and 64-bit forms,
>> installed simultaneously (but can also be used as a "normal"
>> development machine, and for that purpose will want a selected Python
>> version as the default one.
>
> You should see my dev machines :) Most have 2.5, 2.6, 2.7 32-bit and 64-bit, 3.0, 3.1, 3.2, 3.3 32-bit and 64-bit, 3.4 32-bit and 64-bit, IronPython and often PyPy, Anaconda or Canopy. So I'm being fairly selfish when I try and make the multiple-versions scenario work better, but it will benefit everyone.
>
>> Also, how does the launcher py.exe fit into the picture? Is it still
>> installed into the Windows directory? What about for a user install?
>> Are Python scripts associated with the launcher, and if so, how does
>> it pick up the version you selected as default?
>
> py.exe is more important than ever. It's still installed into the Windows directory for all-user installs, and into the Python directory for just-for-me. It's installed in a way that will make upgrades more reliable (so if you install 3.6 and then 3.5, you'll keep the newer launcher) and all the file associations go straight to the launcher.
>
> The default Python for the launcher seems to be 2.7 if it's there and the latest version if not (though I could be wrong). Shebang lines are the best way to choose a specific version.
>
> Incidentally, whoever came up with the py.exe launcher deserves a medal. It's made dealing with multiple versions amazingly easy.
>

If I'm reading this correctly it means that py.exe gets picked up from 
PATH so it could be 32 or 64 bit.  Does this mean that the launcher 
could be or needs enhancing so 32 or 64 bit can be selected?  I'm not 
sure if anything can be done about pyw.exe, perhaps you (plural) can 
throw some light on this for me.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From p.f.moore at gmail.com  Mon Jan  5 10:09:51 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Jan 2015 09:09:51 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
Message-ID: <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>

On 5 January 2015 at 08:40, Donald Stufft <donald at stufft.io> wrote:
>> The biggest problem here is that py.exe doesn't help in the slightest
>> with script wrappers like pip.exe, virtualenv.exe, py.test.exe,
>> ipython.exe ... I've actually drifted away from using py.exe because
>> of this. Having just the interpreter special cased isn't really good
>> enough. (I know there's py -m pip, but it's not easy to get people to
>> use this...)
>
> Won?t script wrappers use whatever version they were installs against? Or do
> you mean the ?installed? version might be 3.5 for ``pip.exe`` even though
> there?s a 3.5.1 for ``pip.exe`` on the PATH?

What I mean is that if you have Python 3.4 and Python 3.5 installed,
and pip.exe is in the Scripts directory of each, then which pip.exe
you get (and hence which Python you install into) if you just type
"pip install xxx" depends on your PATH. Steve is in essence saying
that it's not possible to sanely manage PATH as part of the new
installer, but that py.exe makes that unnecessary. My point is that
while py handles the interpreter, it doesn't handle things like pip
(unless we change the standard usage instructions to say "py -m pip"
is the supported approach on Windows).

Paul

From ethan at stoneleaf.us  Mon Jan  5 19:04:37 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 05 Jan 2015 10:04:37 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <1420412176002.36681@microsoft.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>,
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
Message-ID: <54AAD235.1000205@stoneleaf.us>

On 01/04/2015 02:56 PM, Steve Dower wrote:

> ChrisA wrote:
>> You talk of installing by default into Program Files, and having a
>> separate per-user installation mode. How do these two installation
>> targets interact? Suppose someone installs 3.5 globally, then installs
>> 3.6 for self only? Or installs 3.5.1 for self only? I would normally
>> expect a per-user installation to trump a global one, but this could
>> make for a lovely dep-hell on a system that's used by one person, who
>> then isn't sure what was installed as admin and what wasn't.
> 
> Yeah, it gets a little messy, especially if you keep adding all the
> Python versions to PATH (Windows will always put per-user PATH entries
> at the end). The py.exe launcher handles this best, and the system
> version will always be found first.

py.exe should *not* follow this behavior.  User installs should be searched first, then system installs -- otherwise,
what's the point in having a user-install?  Filling up the gobs of disk-space we now have available?

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150105/91fc2a42/attachment.sig>

From Steve.Dower at microsoft.com  Mon Jan  5 18:09:12 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 5 Jan 2015 17:09:12 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
 <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
Message-ID: <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>


Mark Lawrence wrote:
> If I'm reading this correctly it means that py.exe gets picked up from PATH so
> it could be 32 or 64 bit. Does this mean that the launcher could be or needs
> enhancing so 32 or 64 bit can be selected? I'm not sure if anything can be done
> about pyw.exe, perhaps you (plural) can throw some light on this for me.

We only ever install a 32-bit launcher (including with the current installer, though it doesn't quite work properly), so this isn't really an issue. To explicitly request a 32-bit interpreter you can pass "-3.5-32" - without the "-32" suffix you'll get 64-bit if it's there and 32-bit otherwise.

Paul Moore wrote:
> Steve is in essence saying that it's not possible to sanely manage PATH as part
> of the new installer, but that py.exe makes that unnecessary.

It's actually not possible to sanely manage PATH from any installer - it really needs to be handled by a user directly (though I can't ever bring myself to tell anyone to use the UI for it). The old installers were less susceptible because they didn't support per-user installs, but you'd still get the "last install Python wins" behaviour.

> My point is that while py handles the interpreter, it doesn't handle things like
> pip (unless we change the standard usage instructions to say "py -m pip" is the
> supported approach on Windows).

Or we make pip3.5.exe launch against 3.5 and pip.exe and pip3.exe behave like the launcher, regardless of where they run from.

For example, say I have Python 3.5 and Python 3.6 installed and PATH=C:\Python35;C:\Python35\Scripts;C:\Python36;C:\Python36\Scripts;... (yes, those are the 'wrong' way around). If I just type "pip" it will run C:\Python35\Scripts\pip.exe, which may or may not be what I expect.

If pip behaved like the launcher, it would find the latest installed version (3.6) and launch pip3.6.exe, which would always run against C:\Python36\python.exe. (How does it do this? No idea yet. Technical details...)

This might also mean you could write "pip -3.5 install ..." to run against a specific version, but that doesn't have to be supported if it's likely to break scripts by stealing an argument. Since we'll install "pip", "pip3" and "pip3.5", the options are already there.

Would we need "pip3.5-32" as well? This could get out of hand pretty quickly. Perhaps the launcher could include "usepy -3.5" to update the user's PATH for the current session? Would that get any more use? I don't know, but it feels unlikely. There have been multiple requests for the Add to PATH option to be enabled by default in the installer, but frankly it concerns me too much to do so. There are too many effects that most users won't expect.

Cheers,
Steve

From Steve.Dower at microsoft.com  Mon Jan  5 19:14:27 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 5 Jan 2015 18:14:27 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <54AAD235.1000205@stoneleaf.us>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>,
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com> <54AAD235.1000205@stoneleaf.us>
Message-ID: <DM2PR0301MB0734FD7CB18519CBDC8ABA98F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>

Ethan Furman:
> On 01/04/2015 02:56 PM, Steve Dower wrote:
> 
>> ChrisA wrote:
>>> You talk of installing by default into Program Files, and having a 
>>> separate per-user installation mode. How do these two installation 
>>> targets interact? Suppose someone installs 3.5 globally, then 
>>> installs
>>> 3.6 for self only? Or installs 3.5.1 for self only? I would normally 
>>> expect a per-user installation to trump a global one, but this could 
>>> make for a lovely dep-hell on a system that's used by one person, who 
>>> then isn't sure what was installed as admin and what wasn't.
>> 
>> Yeah, it gets a little messy, especially if you keep adding all the 
>> Python versions to PATH (Windows will always put per-user PATH entries 
>> at the end). The py.exe launcher handles this best, and the system 
>> version will always be found first.
> 
> py.exe should *not* follow this behavior. User installs should be searched
> first, then system installs -- otherwise, what's the point in having a
> user-install? Filling up the gobs of disk-space we now have available?

Typo on my part. The py.exe launcher will always find the *user* install first. Searching PATH will always find the system version first.

From breamoreboy at yahoo.co.uk  Mon Jan  5 22:30:13 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 05 Jan 2015 21:30:13 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
 <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
 <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <m8evpa$9hi$1@ger.gmane.org>

On 05/01/2015 17:09, Steve Dower wrote:
> Paul Moore wrote:
>> Steve is in essence saying that it's not possible to sanely manage PATH as part
>> of the new installer, but that py.exe makes that unnecessary.
>
> It's actually not possible to sanely manage PATH from any installer - it really needs to be handled by a user directly (though I can't ever bring myself to tell anyone to use the UI for it). The old installers were less susceptible because they didn't support per-user installs, but you'd still get the "last install Python wins" behaviour.
>

Something that's help keep me slightly sane is the Rapid Environment 
Editor http://www.rapidee.com/en/about.  I'm sure there are plenty of 
other choices but it does what I need.

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From p.f.moore at gmail.com  Mon Jan  5 22:30:14 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Jan 2015 21:30:14 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
 <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
 <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CACac1F97ETucrdOWk0LHCotj1+5DaF7ckpskiMHZ89Pzbu2gZg@mail.gmail.com>

tl;dr We should have "Add this Python to PATH" as a user choice on the
initial installer screen, applicable to whichever install type the
user chooses. Details as to why are below.

On 5 January 2015 at 17:09, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Paul Moore wrote:
>> Steve is in essence saying that it's not possible to sanely manage PATH as part
>> of the new installer, but that py.exe makes that unnecessary.
>
> It's actually not possible to sanely manage PATH from any installer - it really needs to be handled by a user directly (though I can't ever bring myself to tell anyone to use the UI for it). The old installers were less susceptible because they didn't support per-user installs, but you'd still get the "last install Python wins" behaviour.

Agreed. But the current behaviour has "Add Python to PATH" as an
explicit option for the user to choose (I can't recall whether the
default is still not to do so, or if it was changed in 3.4, but that's
a minor issue). People (the minority) who install multiple Pythons
should (IMO) only set this flag on for one Python at most - and I
don't feel obliged to include any special behaviour for anyone who
doesn't follow that rule.

The new installers, with their "One-Click" installs, don't give that
choice. So they are left having to juggle the multiple path entry
problem, without enough input from the user on what he/she wants.

For myself, I'll probably have to use the custom install to get the
control I want. But that's suboptimal, as I probably *want* to use the
new locations, and having to override the "Custom" default of
C:\Python35 to specify "Program Files" is a nuisance. And I bet I'll
install a 32-bit Python in "C:\Program Files" rather than "C:\Program
Files (x86)" at least once, and probably more than once...

Maybe all that's needed is to have an extra checkbox on the first page
saying "Add this Python to PATH" (doesn't matter whether it's on or
off by default). We can warn if it's set on when there's another
Python already on PATH, and warn more vigorously (possibly even an
error) if the other Python will take precedence over the one we're
installing. But I don't think you can avoid giving the user the choice
of whether to put Python on PATH. There are too many bad corner cases
to let the installer guess.[1]

>> My point is that while py handles the interpreter, it doesn't handle things like
>> pip (unless we change the standard usage instructions to say "py -m pip" is the
>> supported approach on Windows).
>
> Or we make pip3.5.exe launch against 3.5 and pip.exe and pip3.exe behave like the launcher, regardless of where they run from.

That would be a change to pip (the wrapper generation code for wheels)
and setuptools (the equivalent code for sdist installs). The launcher
behaviour is more complex to handle, but could be done (at least in
the case of wheels - that uses the distil wrapper code which doesn't
work like the launcher but Vinay could comment on whether that would
be something that could be added). I'm not against this, it's been
mentioned in the past for pip but no-one has had the time or
inclination to work on it.

The alternative is to special-case pip and I'm strongly -1 on that -
this is a general issue for all wrappers.

> For example, say I have Python 3.5 and Python 3.6 installed and PATH=C:\Python35;C:\Python35\Scripts;C:\Python36;C:\Python36\Scripts;... (yes, those are the 'wrong' way around). If I just type "pip" it will run C:\Python35\Scripts\pip.exe, which may or may not be what I expect.

As I say, I don't really care much about supporting this case - my
view is that the user should have only added *one* of the Python
installations to PATH.

> If pip behaved like the launcher, it would find the latest installed version (3.6) and launch pip3.6.exe, which would always run against C:\Python36\python.exe. (How does it do this? No idea yet. Technical details...)

One hard problem - if pip is only installed in Python 3.5, it would
not be able to import pip from the Python 3.6 interpreter because it's
simply not present. I'm not sure this is soluble.

> This might also mean you could write "pip -3.5 install ..." to run against a specific version, but that doesn't have to be supported if it's likely to break scripts by stealing an argument. Since we'll install "pip", "pip3" and "pip3.5", the options are already there.
>
> Would we need "pip3.5-32" as well? This could get out of hand pretty quickly. Perhaps the launcher could include "usepy -3.5" to update the user's PATH for the current session? Would that get any more use? I don't know, but it feels unlikely. There have been multiple requests for the Add to PATH option to be enabled by default in the installer, but frankly it concerns me too much to do so. There are too many effects that most users won't expect.

If we assume there's never more than one Python installation on PATH
(and make it easy for users to stick to this) then most of these
problems go away. The worst is the user install that fails to override
a system install[2] and I don't know much we can do about that except
raise a warning at install time and suggest not adding it to PATH as
it won't work as expected.

Virtualenv has this problem, too, as activating a venv *always* leaves
the venv and the system Python on PATH. But activation is per-process
and can always put the venv at the start, so the bad problems are
avoided.

Paul

[1] Not least if someone who uses Python 3.6 routinely has a need to
quickly install 3.5 to check a bug report - having to do a full custom
install just to not get bitten by the "last install wins" rule is a
lousy user experience.
[2] The other problem case is a user who's had 3.5 installed on PATH
who later installs 3.6 when it comes out and wants to switch to that.
If the user is upgrading, the uninstall of 3.5 will remove it from
PATH so all is fine. Only if the user wants to keep both versions is
there an issue, and in that case I'd be happy enough to assume they
could use the system PATH editor (or do a "change" on the 3.5 install
to switch off the "Add to PATH" option - I assume the new installers
allow that?)

From Steve.Dower at microsoft.com  Mon Jan  5 22:47:53 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 5 Jan 2015 21:47:53 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F97ETucrdOWk0LHCotj1+5DaF7ckpskiMHZ89Pzbu2gZg@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
 <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
 <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F97ETucrdOWk0LHCotj1+5DaF7ckpskiMHZ89Pzbu2gZg@mail.gmail.com>
Message-ID: <DM2PR0301MB0734D3F58FA99A9D9A3FC111F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>

Paul Moore wrote:
> tl;dr We should have "Add this Python to PATH" as a user choice on the initial
> installer screen, applicable to whichever install type the user chooses. Details
> as to why are below.

I agree. I'll work this up before alpha 1. (FWIW, it defaults to unselected.)

Displaying a warning about other Pythons already being on the path at this point is trickier, but I can probably figure something out.

> For myself, I'll probably have to use the custom install to get the control I
> want. But that's suboptimal, as I probably *want* to use the new locations, and
> having to override the "Custom" default of C:\Python35 to specify "Program
> Files" is a nuisance. And I bet I'll install a 32-bit Python in "C:\Program
> Files" rather than "C:\Program Files (x86)" at least once, and probably more
> than once...

Inclined to agree, and it is possible to make the default on the customise page switch with the Install as Administrator checkbox until it's been manually edited. My main concern is people who want it the old way, but they may not actually exist. Perhaps making the old way trickier is the best way to flush them out sooner...

Cheers,
Steve

From p.f.moore at gmail.com  Mon Jan  5 23:13:43 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 5 Jan 2015 22:13:43 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734D3F58FA99A9D9A3FC111F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CACac1F9m8_dPWR0pXXitd1SVx_toXYsryVc4H6arHWXk2wwGMg@mail.gmail.com>
 <B2610670-A654-4968-89C6-87527BB61A90@stufft.io>
 <CACac1F8KmdDgXaNwiPVjw_qq_-Eos5DNr0yY4uRS1FJ7xi6jNw@mail.gmail.com>
 <DM2PR0301MB0734981DFFEC190D928CAB46F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F97ETucrdOWk0LHCotj1+5DaF7ckpskiMHZ89Pzbu2gZg@mail.gmail.com>
 <DM2PR0301MB0734D3F58FA99A9D9A3FC111F5580@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CACac1F-uje4RD=MnOv-qud8U5tf=g39YCQtbttUwhoQDqLxRZA@mail.gmail.com>

On 5 January 2015 at 21:47, Steve Dower <Steve.Dower at microsoft.com> wrote:
> Paul Moore wrote:
>> tl;dr We should have "Add this Python to PATH" as a user choice on the initial
>> installer screen, applicable to whichever install type the user chooses. Details
>> as to why are below.
>
> I agree. I'll work this up before alpha 1. (FWIW, it defaults to unselected.)

Cool. Would you make the new installer keep "Add to PATH" as
unselected by default? I got the impression that the "One Click"
installs would add to PATH, making it reasonable to have the checkbox
selected by default.

> Displaying a warning about other Pythons already being on the path at this point is trickier, but I can probably figure something out.

No big deal if you can't. It's a "nice to have" but not essential.

>> For myself, I'll probably have to use the custom install to get the control I
>> want. But that's suboptimal, as I probably *want* to use the new locations, and
>> having to override the "Custom" default of C:\Python35 to specify "Program
>> Files" is a nuisance. And I bet I'll install a 32-bit Python in "C:\Program
>> Files" rather than "C:\Program Files (x86)" at least once, and probably more
>> than once...
>
> Inclined to agree, and it is possible to make the default on the customise page switch with the Install as Administrator checkbox until it's been manually edited. My main concern is people who want it the old way, but they may not actually exist. Perhaps making the old way trickier is the best way to flush them out sooner...

Having said this, if I can choose "Add to PATH" without needing a
custom install, then probably the only reason I'd be likely to want a
custom install would be to change the path, so the value of the
default is not such a big deal for me personally.

Paul

From jsbfox at gmail.com  Tue Jan  6 08:52:43 2015
From: jsbfox at gmail.com (Dmitry Kazakov)
Date: Tue, 6 Jan 2015 10:52:43 +0300
Subject: [Python-Dev] Issue 22619 at bugs.python.org
Message-ID: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>

Greetings.

I'm sorry if I'm too insistent, but it's not truly rewarding to
constantly improve a patch that no one appears to need. Again, I
understand people are busy working and/or reviewing critical patches,
but 2 months of inactivity is not right. Yes, I posted a message
yesterday, but no one seemed to be bothered. In any case, I'll respect
your decision about this patch and will never ask for a review of this
patch again.

Regards, Dmitry.

From victor.stinner at gmail.com  Tue Jan  6 13:34:46 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 6 Jan 2015 13:34:46 +0100
Subject: [Python-Dev] Issue 22619 at bugs.python.org
In-Reply-To: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>
References: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>
Message-ID: <CAMpsgwaQhuLAnyH4fodyEokZz7Gf0YdspeHi2q1bRYidQqSD5A@mail.gmail.com>

http://bugs.python.org/issue22619
"Possible implementation of negative limit for traceback functions"

I see that Serhiy Storchaka reviewed a patch.

Victor

From victor.stinner at gmail.com  Tue Jan  6 13:39:13 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 6 Jan 2015 13:39:13 +0100
Subject: [Python-Dev] python 2.7.9 regression in argparse?
In-Reply-To: <CAPkN8xKSr0LadFk-A7YJiMX5Vk_A29nAa_c4DtC_J41y+LCceQ@mail.gmail.com>
References: <CAPkN8xKSr0LadFk-A7YJiMX5Vk_A29nAa_c4DtC_J41y+LCceQ@mail.gmail.com>
Message-ID: <CAMpsgwZfaOFHF1VSLDLcLHqwHaX0P2vqfwFw26mym6Bd3LG=eA@mail.gmail.com>

More context:

2014-12-19 12:43 GMT+01:00 anatoly techtonik <techtonik at gmail.com>:
> https://github.com/nickstenning/honcho/pull/121

The link mentions the following changeset:
---
changeset:   93122:1a3143752db2
branch:      2.7
parent:      93112:927cca0b9337
user:        R David Murray <rdmurray at bitdance.com>
date:        Fri Oct 17 20:07:08 2014 -0400
files:       Lib/argparse.py Lib/test/test_argparse.py Misc/NEWS
description:
#9351: set_defaults on subparser is no longer ignored if set on parent.

Before, if a default was set on the parent parser, any default for that
variable set via set_defaults on a subparser would be ignored.  Now
the subparser set_defaults is honored.

Patch by Jyrki Pullianinen.


diff -r 927cca0b9337 -r 1a3143752db2 Lib/argparse.py
--- a/Lib/argparse.py   Fri Oct 17 16:20:15 2014 -0500
+++ b/Lib/argparse.py   Fri Oct 17 20:07:08 2014 -0400
@@ -1089,7 +1089,14 @@ class _SubParsersAction(Action):
         # parse all the remaining options into the namespace
         # store any unrecognized options on the object, so that the top
         # level parser can decide what to do with them
-        namespace, arg_strings = parser.parse_known_args(arg_strings,
namespace)
+
+        # In case this subparser defines new defaults, we parse them
+        # in a new namespace object and then update the original
+        # namespace for the relevant parts.
+        subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
+        for key, value in vars(subnamespace).items():
+            setattr(namespace, key, value)
+
         if arg_strings:
             vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
             getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
---

Which is related to http://bugs.python.org/issue9351

Maybe argparse just became more strict? I don't understand the issue.

Victor

From stefanrin at gmail.com  Tue Jan  6 13:48:47 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Tue, 6 Jan 2015 13:48:47 +0100
Subject: [Python-Dev] Issue 22619 at bugs.python.org
In-Reply-To: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>
References: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>
Message-ID: <CAAxjCEyfy_cmVLXiUApJ7g_qDgO4+9yTTQsmfjObYOOAmAtiaw@mail.gmail.com>

On Tue, Jan 6, 2015 at 8:52 AM, Dmitry Kazakov <jsbfox at gmail.com> wrote:
> Greetings.
>
> I'm sorry if I'm too insistent, but it's not truly rewarding to
> constantly improve a patch that no one appears to need. Again, I
> understand people are busy working and/or reviewing critical patches,
> but 2 months of inactivity is not right. Yes, I posted a message
> yesterday, but no one seemed to be bothered. In any case, I'll respect
> your decision about this patch and will never ask for a review of this
> patch again.

The later patches seem to miss the Mercurial header that would allow
the integrated review functionality on bugs.python.org to kick in (I
presume) and thus make it much easier to review.

From ncoghlan at gmail.com  Tue Jan  6 14:56:30 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 6 Jan 2015 23:56:30 +1000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
 <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>
Message-ID: <CADiSq7cnx8qELYOX8OKwuMcsK45-UxH2kiOovMcYrfWcX6p+ww@mail.gmail.com>

On 5 January 2015 at 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
> On 5 January 2015 at 01:20, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> I think this means the best way to make multiple versions work properly is to rename python.exe to python3.5.exe, then install the launcher as python.exe and python3.exe (with some logic to look at its own name) so it can resolve the right version. Maybe we can even extend the launcher to resolve launchers in Scripts (pip.exe, etc.) and have consistent rules there too?
>
> This was a big debate over on distutils-sig. I advocated "py -m pip"
> for portable use, but got basically nowhere. People want "pip" to
> work, for compatibility with Unix (and hence ease of documentation).

"pip" is problematic on Linux as well (due to the pip/pip3 split at
the system level). Hence this section in the stdlib docs:
https://docs.python.org/3/installing/#work-with-multiple-versions-of-python-installed-in-parallel

> If we can't get PATH sorted out, we need to do something about this,
> IMO. I don't know what (believe me, I tried to find a solution)
> unfortunately.

I personally believe we should aim to make the "Windows" section in
the above link universal (i.e. working across *nix, Windows, and
virtual environments). That's the only one that can feasibly be made
universal - all the other alternatives are already known to be
incompatible with particular scenarios.

It's a fair bit of work to make it happen though, so it will likely
remain a "wish list" item unless someone gets particularly inspired to
do something about it :)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From p.f.moore at gmail.com  Tue Jan  6 15:15:06 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 6 Jan 2015 14:15:06 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CADiSq7cnx8qELYOX8OKwuMcsK45-UxH2kiOovMcYrfWcX6p+ww@mail.gmail.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
 <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>
 <CADiSq7cnx8qELYOX8OKwuMcsK45-UxH2kiOovMcYrfWcX6p+ww@mail.gmail.com>
Message-ID: <CACac1F9JUPvZAC2d2Utf=DwLOeV9_+_GZ+7QHAqWfGnPttXV5w@mail.gmail.com>

On 6 January 2015 at 13:56, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> If we can't get PATH sorted out, we need to do something about this,
>> IMO. I don't know what (believe me, I tried to find a solution)
>> unfortunately.
>
> I personally believe we should aim to make the "Windows" section in
> the above link universal (i.e. working across *nix, Windows, and
> virtual environments). That's the only one that can feasibly be made
> universal - all the other alternatives are already known to be
> incompatible with particular scenarios.

Making it the universally recommended approach is one thing. Getting
people to actually work that way is another thing entirely :-(

Also, the "Windows recommendation" (currently) doesn't work properly
with virtualenv - there's no way to get py.exe to use "the currently
active virtualenv" (i.e., the default version from PATH). It should be
possible to fix this - the difficulty is likely to be designing a
viable interface.

As a first cut, I think py.exe should default to (in the following order):

1. The first version of Python on PATH that "looks like a virtualenv"
(need to consider both venv and virtualenv styles).
2. The default version of Python set in the ini file.
3. The latest version of Python installed on the machine (no longer
preferring Python 2 over Python 3!)

A #! line in a script being run will override all of these, of course,
as will explicit command line flags.

Does this seem reasonable? I could work on a patch if it does.

Paul

From solipsis at pitrou.net  Tue Jan  6 15:12:21 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 6 Jan 2015 15:12:21 +0100
Subject: [Python-Dev] New Windows installer for Python 3.5
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
 <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>
 <CADiSq7cnx8qELYOX8OKwuMcsK45-UxH2kiOovMcYrfWcX6p+ww@mail.gmail.com>
Message-ID: <20150106151221.4f807b6a@fsol>

On Tue, 6 Jan 2015 23:56:30 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 5 January 2015 at 18:17, Paul Moore <p.f.moore at gmail.com> wrote:
> > On 5 January 2015 at 01:20, Steve Dower <Steve.Dower at microsoft.com> wrote:
> >> I think this means the best way to make multiple versions work properly is to rename python.exe to python3.5.exe, then install the launcher as python.exe and python3.exe (with some logic to look at its own name) so it can resolve the right version. Maybe we can even extend the launcher to resolve launchers in Scripts (pip.exe, etc.) and have consistent rules there too?
> >
> > This was a big debate over on distutils-sig. I advocated "py -m pip"
> > for portable use, but got basically nowhere. People want "pip" to
> > work, for compatibility with Unix (and hence ease of documentation).
> 
> "pip" is problematic on Linux as well (due to the pip/pip3 split at
> the system level). Hence this section in the stdlib docs:
> https://docs.python.org/3/installing/#work-with-multiple-versions-of-python-installed-in-parallel

Why doesn't it recommend "pip3.4", etc. instead? Am I missing something
about it?

Thanks

Antoine.



From jsbfox at gmail.com  Tue Jan  6 14:27:13 2015
From: jsbfox at gmail.com (Mason Hill)
Date: Tue, 6 Jan 2015 08:27:13 -0500
Subject: [Python-Dev] Issue 22619 at bugs.python.org
In-Reply-To: <CAAxjCEyfy_cmVLXiUApJ7g_qDgO4+9yTTQsmfjObYOOAmAtiaw@mail.gmail.com>
References: <CAMPw9HSe3rPPgw2FOWxdKciLiaxajnVvwSKQf+C-v-r_HdoHOQ@mail.gmail.com>
 <CAAxjCEyfy_cmVLXiUApJ7g_qDgO4+9yTTQsmfjObYOOAmAtiaw@mail.gmail.com>
Message-ID: <CAMPw9HSvH1LPgBfOB-raPPu3X-M=TCpK6GZSnzJgU49tHT3nzg@mail.gmail.com>

Thank you for pointing this out. That's what I meant when I asked to
"say what's wrong with it " :-) I fixed the latest patch and uploaded
it.

On Tue, Jan 6, 2015 at 7:48 AM, Stefan Ring <stefanrin at gmail.com> wrote:
> On Tue, Jan 6, 2015 at 8:52 AM, Dmitry Kazakov <jsbfox at gmail.com> wrote:
>> Greetings.
>>
>> I'm sorry if I'm too insistent, but it's not truly rewarding to
>> constantly improve a patch that no one appears to need. Again, I
>> understand people are busy working and/or reviewing critical patches,
>> but 2 months of inactivity is not right. Yes, I posted a message
>> yesterday, but no one seemed to be bothered. In any case, I'll respect
>> your decision about this patch and will never ask for a review of this
>> patch again.
>
> The later patches seem to miss the Mercurial header that would allow
> the integrated review functionality on bugs.python.org to kick in (I
> presume) and thus make it much easier to review.

From ncoghlan at gmail.com  Tue Jan  6 16:12:18 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 7 Jan 2015 01:12:18 +1000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <20150106151221.4f807b6a@fsol>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CACac1F_g7vnFxqRZsyr4ezhFMssVhOwh1PoRY1nem5upYioKww@mail.gmail.com>
 <CAP2Qz+XfVTMM19Tag01Re1HRj-h3HgGYy1zZmtvnnOQMRbFLtg@mail.gmail.com>
 <1420412176002.36681@microsoft.com>
 <CAPTjJmod55fUTUtXpgq5xYRLCMeyuDvKg0-p2vRoQ2MXrzZ82w@mail.gmail.com>
 <1420420846274.86975@microsoft.com>
 <CACac1F8Bv0renVvsGwBijkCJYFdcUqCFaTNRyLCwaLC4ZY_bFg@mail.gmail.com>
 <CADiSq7cnx8qELYOX8OKwuMcsK45-UxH2kiOovMcYrfWcX6p+ww@mail.gmail.com>
 <20150106151221.4f807b6a@fsol>
Message-ID: <CADiSq7eqm0uAjNcF4DsJQMNu0-D8WBpYLehc0NPAPwAoY7EiXw@mail.gmail.com>

On 7 January 2015 at 00:12, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Tue, 6 Jan 2015 23:56:30 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
>> "pip" is problematic on Linux as well (due to the pip/pip3 split at
>> the system level). Hence this section in the stdlib docs:
>> https://docs.python.org/3/installing/#work-with-multiple-versions-of-python-installed-in-parallel
>
> Why doesn't it recommend "pip3.4", etc. instead? Am I missing something
> about it?

Those assume a pip-binary-per-Python-implementation model which isn't
a valid assumption once Jython, PyPy, etc come into consideration.
Indirecting through the Python implementation name helps ensure
compatibility with a wider range of scenarios.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tjreedy at udel.edu  Tue Jan  6 21:25:59 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 06 Jan 2015 15:25:59 -0500
Subject: [Python-Dev] python 2.7.9 regression in argparse?
In-Reply-To: <CAMpsgwZfaOFHF1VSLDLcLHqwHaX0P2vqfwFw26mym6Bd3LG=eA@mail.gmail.com>
References: <CAPkN8xKSr0LadFk-A7YJiMX5Vk_A29nAa_c4DtC_J41y+LCceQ@mail.gmail.com>
 <CAMpsgwZfaOFHF1VSLDLcLHqwHaX0P2vqfwFw26mym6Bd3LG=eA@mail.gmail.com>
Message-ID: <m8hgd2$7l8$1@ger.gmane.org>

On 1/6/2015 7:39 AM, Victor Stinner wrote:
> More context:
>
> 2014-12-19 12:43 GMT+01:00 anatoly techtonik <techtonik at gmail.com>:
>> https://github.com/nickstenning/honcho/pull/121
>
> The link mentions the following changeset:
> ---
> changeset:   93122:1a3143752db2
> branch:      2.7
> parent:      93112:927cca0b9337
> user:        R David Murray <rdmurray at bitdance.com>
> date:        Fri Oct 17 20:07:08 2014 -0400
> files:       Lib/argparse.py Lib/test/test_argparse.py Misc/NEWS
> description:
> #9351: set_defaults on subparser is no longer ignored if set on parent.
>
> Before, if a default was set on the parent parser, any default for that
> variable set via set_defaults on a subparser would be ignored.  Now
> the subparser set_defaults is honored.
>
> Patch by Jyrki Pullianinen.
>
>
> diff -r 927cca0b9337 -r 1a3143752db2 Lib/argparse.py
> --- a/Lib/argparse.py   Fri Oct 17 16:20:15 2014 -0500
> +++ b/Lib/argparse.py   Fri Oct 17 20:07:08 2014 -0400
> @@ -1089,7 +1089,14 @@ class _SubParsersAction(Action):
>           # parse all the remaining options into the namespace
>           # store any unrecognized options on the object, so that the top
>           # level parser can decide what to do with them
> -        namespace, arg_strings = parser.parse_known_args(arg_strings,
> namespace)
> +
> +        # In case this subparser defines new defaults, we parse them
> +        # in a new namespace object and then update the original
> +        # namespace for the relevant parts.
> +        subnamespace, arg_strings = parser.parse_known_args(arg_strings, None)
> +        for key, value in vars(subnamespace).items():
> +            setattr(namespace, key, value)
> +
>           if arg_strings:
>               vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
>               getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).extend(arg_strings)
> ---
>
> Which is related to http://bugs.python.org/issue9351
>
> Maybe argparse just became more strict? I don't understand the issue.

Steven Bethard, the argparse maintainer, defined the old behavior of 
ignoring subparser defaults (where there are also top level defaults) as 
a bug "counter to what people probably expect".  If the old behavior had 
been documented, changing it in a bug-fix release would have been a 
mistake.  But looking at the patch, the doc seems to have been silent on 
the issue.

This is not the first time someone considered a 'bug fix' to be a 
'regression', which it might be from their viewpoint.  The last comment 
on the github thread suggests that an easy fix was found.

-- 
Terry Jan Reedy


From guido at python.org  Tue Jan  6 21:41:55 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Jan 2015 12:41:55 -0800
Subject: [Python-Dev] python 2.7.9 regression in argparse?
In-Reply-To: <m8hgd2$7l8$1@ger.gmane.org>
References: <CAPkN8xKSr0LadFk-A7YJiMX5Vk_A29nAa_c4DtC_J41y+LCceQ@mail.gmail.com>
 <CAMpsgwZfaOFHF1VSLDLcLHqwHaX0P2vqfwFw26mym6Bd3LG=eA@mail.gmail.com>
 <m8hgd2$7l8$1@ger.gmane.org>
Message-ID: <CAP7+vJLjvD9HWHfac7UxW69fuX9nwcxDYZ7NXFQ7aoO1_+tShw@mail.gmail.com>

There's an obligatory XKCD reference for this: http://xkcd.com/1172/

On Tue, Jan 6, 2015 at 12:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 1/6/2015 7:39 AM, Victor Stinner wrote:
>
>> More context:
>>
>> 2014-12-19 12:43 GMT+01:00 anatoly techtonik <techtonik at gmail.com>:
>>
>>> https://github.com/nickstenning/honcho/pull/121
>>>
>>
>> The link mentions the following changeset:
>> ---
>> changeset:   93122:1a3143752db2
>> branch:      2.7
>> parent:      93112:927cca0b9337
>> user:        R David Murray <rdmurray at bitdance.com>
>> date:        Fri Oct 17 20:07:08 2014 -0400
>> files:       Lib/argparse.py Lib/test/test_argparse.py Misc/NEWS
>> description:
>> #9351: set_defaults on subparser is no longer ignored if set on parent.
>>
>> Before, if a default was set on the parent parser, any default for that
>> variable set via set_defaults on a subparser would be ignored.  Now
>> the subparser set_defaults is honored.
>>
>> Patch by Jyrki Pullianinen.
>>
>>
>> diff -r 927cca0b9337 -r 1a3143752db2 Lib/argparse.py
>> --- a/Lib/argparse.py   Fri Oct 17 16:20:15 2014 -0500
>> +++ b/Lib/argparse.py   Fri Oct 17 20:07:08 2014 -0400
>> @@ -1089,7 +1089,14 @@ class _SubParsersAction(Action):
>>           # parse all the remaining options into the namespace
>>           # store any unrecognized options on the object, so that the top
>>           # level parser can decide what to do with them
>> -        namespace, arg_strings = parser.parse_known_args(arg_strings,
>> namespace)
>> +
>> +        # In case this subparser defines new defaults, we parse them
>> +        # in a new namespace object and then update the original
>> +        # namespace for the relevant parts.
>> +        subnamespace, arg_strings = parser.parse_known_args(arg_strings,
>> None)
>> +        for key, value in vars(subnamespace).items():
>> +            setattr(namespace, key, value)
>> +
>>           if arg_strings:
>>               vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
>>               getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).
>> extend(arg_strings)
>> ---
>>
>> Which is related to http://bugs.python.org/issue9351
>>
>> Maybe argparse just became more strict? I don't understand the issue.
>>
>
> Steven Bethard, the argparse maintainer, defined the old behavior of
> ignoring subparser defaults (where there are also top level defaults) as a
> bug "counter to what people probably expect".  If the old behavior had been
> documented, changing it in a bug-fix release would have been a mistake.
> But looking at the patch, the doc seems to have been silent on the issue.
>
> This is not the first time someone considered a 'bug fix' to be a
> 'regression', which it might be from their viewpoint.  The last comment on
> the github thread suggests that an easy fix was found.
>
> --
> Terry Jan Reedy
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150106/a7f5ff53/attachment-0001.html>

From rymg19 at gmail.com  Tue Jan  6 23:42:46 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Tue, 6 Jan 2015 16:42:46 -0600
Subject: [Python-Dev] python 2.7.9 regression in argparse?
In-Reply-To: <CAP7+vJLjvD9HWHfac7UxW69fuX9nwcxDYZ7NXFQ7aoO1_+tShw@mail.gmail.com>
References: <CAPkN8xKSr0LadFk-A7YJiMX5Vk_A29nAa_c4DtC_J41y+LCceQ@mail.gmail.com>
 <CAMpsgwZfaOFHF1VSLDLcLHqwHaX0P2vqfwFw26mym6Bd3LG=eA@mail.gmail.com>
 <m8hgd2$7l8$1@ger.gmane.org>
 <CAP7+vJLjvD9HWHfac7UxW69fuX9nwcxDYZ7NXFQ7aoO1_+tShw@mail.gmail.com>
Message-ID: <CAO41-mN7xHmHekfhSf3Cr1goHYZCOySq+YUtL7GdZxetJ511FA@mail.gmail.com>

I thought of this exact comment when I read the "bug fix considered a
regression."

On Tue, Jan 6, 2015 at 2:41 PM, Guido van Rossum <guido at python.org> wrote:

> There's an obligatory XKCD reference for this: http://xkcd.com/1172/
>
> On Tue, Jan 6, 2015 at 12:25 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>
>> On 1/6/2015 7:39 AM, Victor Stinner wrote:
>>
>>> More context:
>>>
>>> 2014-12-19 12:43 GMT+01:00 anatoly techtonik <techtonik at gmail.com>:
>>>
>>>> https://github.com/nickstenning/honcho/pull/121
>>>>
>>>
>>> The link mentions the following changeset:
>>> ---
>>> changeset:   93122:1a3143752db2
>>> branch:      2.7
>>> parent:      93112:927cca0b9337
>>> user:        R David Murray <rdmurray at bitdance.com>
>>> date:        Fri Oct 17 20:07:08 2014 -0400
>>> files:       Lib/argparse.py Lib/test/test_argparse.py Misc/NEWS
>>> description:
>>> #9351: set_defaults on subparser is no longer ignored if set on parent.
>>>
>>> Before, if a default was set on the parent parser, any default for that
>>> variable set via set_defaults on a subparser would be ignored.  Now
>>> the subparser set_defaults is honored.
>>>
>>> Patch by Jyrki Pullianinen.
>>>
>>>
>>> diff -r 927cca0b9337 -r 1a3143752db2 Lib/argparse.py
>>> --- a/Lib/argparse.py   Fri Oct 17 16:20:15 2014 -0500
>>> +++ b/Lib/argparse.py   Fri Oct 17 20:07:08 2014 -0400
>>> @@ -1089,7 +1089,14 @@ class _SubParsersAction(Action):
>>>           # parse all the remaining options into the namespace
>>>           # store any unrecognized options on the object, so that the top
>>>           # level parser can decide what to do with them
>>> -        namespace, arg_strings = parser.parse_known_args(arg_strings,
>>> namespace)
>>> +
>>> +        # In case this subparser defines new defaults, we parse them
>>> +        # in a new namespace object and then update the original
>>> +        # namespace for the relevant parts.
>>> +        subnamespace, arg_strings = parser.parse_known_args(arg_strings,
>>> None)
>>> +        for key, value in vars(subnamespace).items():
>>> +            setattr(namespace, key, value)
>>> +
>>>           if arg_strings:
>>>               vars(namespace).setdefault(_UNRECOGNIZED_ARGS_ATTR, [])
>>>               getattr(namespace, _UNRECOGNIZED_ARGS_ATTR).
>>> extend(arg_strings)
>>> ---
>>>
>>> Which is related to http://bugs.python.org/issue9351
>>>
>>> Maybe argparse just became more strict? I don't understand the issue.
>>>
>>
>> Steven Bethard, the argparse maintainer, defined the old behavior of
>> ignoring subparser defaults (where there are also top level defaults) as a
>> bug "counter to what people probably expect".  If the old behavior had been
>> documented, changing it in a bug-fix release would have been a mistake.
>> But looking at the patch, the doc seems to have been silent on the issue.
>>
>> This is not the first time someone considered a 'bug fix' to be a
>> 'regression', which it might be from their viewpoint.  The last comment on
>> the github thread suggests that an easy fix was found.
>>
>> --
>> Terry Jan Reedy
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
>> guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>
>


-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150106/739f469b/attachment.html>

From guido at python.org  Wed Jan  7 04:48:18 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 6 Jan 2015 19:48:18 -0800
Subject: [Python-Dev] Any volunteers to implement PEP 479?
Message-ID: <CAP7+vJJW630SKE6jw+yLm6R1wzEzj3b=ObNFWFm0a=pUOkmJnA@mail.gmail.com>

There's a proof of concept patch in http://bugs.python.org/issue22906, but
it doesn't have the __future__ import and probably gets other details wrong.

Reference:
PEP 479 -- Change StopIteration handling inside generators
<https://www.python.org/dev/peps/pep-0479/>

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150106/ae98a98a/attachment.html>

From ethan at stoneleaf.us  Wed Jan  7 20:07:15 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 07 Jan 2015 11:07:15 -0800
Subject: [Python-Dev] Cpython code and ...
Message-ID: <54AD83E3.7040804@stoneleaf.us>

I found this:

  PyObject *
  PyBytes_FromFormat(const char *format, ...)
  {

Can someone enlighten me on what the '...' means?

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150107/c26160c3/attachment.sig>

From demianbrecht at gmail.com  Wed Jan  7 20:10:33 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 07 Jan 2015 11:10:33 -0800
Subject: [Python-Dev] Cpython code and ...
In-Reply-To: <54AD83E3.7040804@stoneleaf.us>
References: <54AD83E3.7040804@stoneleaf.us>
Message-ID: <54AD84A9.9030509@gmail.com>

It denotes a variadic function:
http://www.gnu.org/software/libc/manual/html_node/Variadic-Functions.html.

On 2015-01-07 11:07 AM, Ethan Furman wrote:
> I found this:
> 
>   PyObject *
>   PyBytes_FromFormat(const char *format, ...)
>   {
> 
> Can someone enlighten me on what the '...' means?
> 
> --
> ~Ethan~
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/demianbrecht%40gmail.com
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150107/e63f7798/attachment.sig>

From ethan at stoneleaf.us  Wed Jan  7 20:46:33 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 07 Jan 2015 11:46:33 -0800
Subject: [Python-Dev] Cpython code and ...
In-Reply-To: <54AD84A9.9030509@gmail.com>
References: <54AD83E3.7040804@stoneleaf.us> <54AD84A9.9030509@gmail.com>
Message-ID: <54AD8D19.5050905@stoneleaf.us>

On 01/07/2015 11:10 AM, Demian Brecht wrote:
> On 2015-01-07 11:07 AM, Ethan Furman wrote:
>>
>> I found this:
>>
>>   PyObject *
>>   PyBytes_FromFormat(const char *format, ...)
>>   {
>>
>> Can someone enlighten me on what the '...' means?
>
> It denotes a variadic function:
>   http://www.gnu.org/software/libc/manual/html_node/Variadic-Functions.html.

Ah, thanks!

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150107/e88bf974/attachment.sig>

From ncoghlan at gmail.com  Thu Jan  8 07:59:49 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 8 Jan 2015 16:59:49 +1000
Subject: [Python-Dev] My thinking about the development process
In-Reply-To: <CAP1=2W6ayQdmGHOqfz8=KMVggjn1aXwEOe-aZQaFCA3VPGofPw@mail.gmail.com>
References: <CAP1=2W6ayQdmGHOqfz8=KMVggjn1aXwEOe-aZQaFCA3VPGofPw@mail.gmail.com>
Message-ID: <CADiSq7fO3FkgQcfvzhsXouv+Uy09+XQy_ZhxTadLkAKwYgP9ew@mail.gmail.com>

On 6 December 2014 at 06:04, Brett Cannon <bcannon at gmail.com> wrote:
> # Next steps
> I'm thinking first draft PEPs by February 1 to know who's all-in (8 weeks
> away), all details worked out in final PEPs and whatever is required to
> prove to me it will work by the PyCon language summit (4 months away). I
> make a decision by May 1, and
> then implementation aims to be done by the time 3.5.0 is cut so we can
> switch over shortly thereafter (9 months away). Sound like a reasonable
> timeline?

I've now updated PEP 474 to cover my current proposal for the support
repositories, as well as some of the preparatory work that is already
being undertaken: https://www.python.org/dev/peps/pep-0474/

By the end of the month, I'll also aim to have an updated version of
PEP 462 published that considers how the forge.python.org service
could potentially be extended to handle CPython itself, rather than
attempting to build those flows directly into the existing Roundup and
Rietveld based approach.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From tymoteusz.jankowski at gmail.com  Thu Jan  8 10:26:28 2015
From: tymoteusz.jankowski at gmail.com (Tymoteusz Jankowski)
Date: Thu, 08 Jan 2015 09:26:28 +0000
Subject: [Python-Dev] My thinking about the development process
References: <CAP1=2W6ayQdmGHOqfz8=KMVggjn1aXwEOe-aZQaFCA3VPGofPw@mail.gmail.com>
 <CADiSq7fO3FkgQcfvzhsXouv+Uy09+XQy_ZhxTadLkAKwYgP9ew@mail.gmail.com>
Message-ID: <CAJnKBfkOpVVDuqyt+kS083JQaADtVWkRBcZ1kGTPh8Ee9w4iSQ@mail.gmail.com>

I did a pull-request with current progress:
https://github.com/python/psf-salt/pull/25
Any feedback is appreciated.
Btw: Donald is very patient and helpful. :)


On Thu Jan 08 2015 at 8:00:59 AM Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 6 December 2014 at 06:04, Brett Cannon <bcannon at gmail.com> wrote:
> > # Next steps
> > I'm thinking first draft PEPs by February 1 to know who's all-in (8 weeks
> > away), all details worked out in final PEPs and whatever is required to
> > prove to me it will work by the PyCon language summit (4 months away). I
> > make a decision by May 1, and
> > then implementation aims to be done by the time 3.5.0 is cut so we can
> > switch over shortly thereafter (9 months away). Sound like a reasonable
> > timeline?
>
> I've now updated PEP 474 to cover my current proposal for the support
> repositories, as well as some of the preparatory work that is already
> being undertaken: https://www.python.org/dev/peps/pep-0474/
>
> By the end of the month, I'll also aim to have an updated version of
> PEP 462 published that considers how the forge.python.org service
> could potentially be extended to handle CPython itself, rather than
> attempting to build those flows directly into the existing Roundup and
> Rietveld based approach.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> tymoteusz.jankowski%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150108/8c35c4d3/attachment.html>

From donald at stufft.io  Thu Jan  8 15:54:32 2015
From: donald at stufft.io (Donald Stufft)
Date: Thu, 8 Jan 2015 09:54:32 -0500
Subject: [Python-Dev] My thinking about the development process
In-Reply-To: <CAJnKBfkOpVVDuqyt+kS083JQaADtVWkRBcZ1kGTPh8Ee9w4iSQ@mail.gmail.com>
References: <CAP1=2W6ayQdmGHOqfz8=KMVggjn1aXwEOe-aZQaFCA3VPGofPw@mail.gmail.com>
 <CADiSq7fO3FkgQcfvzhsXouv+Uy09+XQy_ZhxTadLkAKwYgP9ew@mail.gmail.com>
 <CAJnKBfkOpVVDuqyt+kS083JQaADtVWkRBcZ1kGTPh8Ee9w4iSQ@mail.gmail.com>
Message-ID: <AA619C02-0A9A-4458-8645-49B9A00CD394@stufft.io>


> On Jan 8, 2015, at 4:26 AM, Tymoteusz Jankowski <tymoteusz.jankowski at gmail.com> wrote:
> 
> I did a pull-request with current progress: https://github.com/python/psf-salt/pull/25 <https://github.com/python/psf-salt/pull/25>
> Any feedback is appreciated.
> Btw: Donald is very patient and helpful. :)

Ah oops, I forgot to review that. *goes to do so now*.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150108/0f64595a/attachment.html>

From wes.turner at gmail.com  Thu Jan  8 17:15:58 2015
From: wes.turner at gmail.com (Wes Turner)
Date: Thu, 8 Jan 2015 10:15:58 -0600
Subject: [Python-Dev] My thinking about the development process
In-Reply-To: <CADiSq7fO3FkgQcfvzhsXouv+Uy09+XQy_ZhxTadLkAKwYgP9ew@mail.gmail.com>
References: <CAP1=2W6ayQdmGHOqfz8=KMVggjn1aXwEOe-aZQaFCA3VPGofPw@mail.gmail.com>
 <CADiSq7fO3FkgQcfvzhsXouv+Uy09+XQy_ZhxTadLkAKwYgP9ew@mail.gmail.com>
Message-ID: <CACfEFw8smnLnb51JgWQPxN4xJTcyOiLaEJ+=sNuHZ9YAZddj_g@mail.gmail.com>

On Thu, Jan 8, 2015 at 12:59 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 6 December 2014 at 06:04, Brett Cannon <bcannon at gmail.com> wrote:
> > # Next steps
> > I'm thinking first draft PEPs by February 1 to know who's all-in (8 weeks
> > away), all details worked out in final PEPs and whatever is required to
> > prove to me it will work by the PyCon language summit (4 months away). I
> > make a decision by May 1, and
> > then implementation aims to be done by the time 3.5.0 is cut so we can
> > switch over shortly thereafter (9 months away). Sound like a reasonable
> > timeline?
>
> I've now updated PEP 474 to cover my current proposal for the support
> repositories, as well as some of the preparatory work that is already
> being undertaken: https://www.python.org/dev/peps/pep-0474/
>
> By the end of the month, I'll also aim to have an updated version of
> PEP 462 published that considers how the forge.python.org service
> could potentially be extended to handle CPython itself, rather than
> attempting to build those flows directly into the existing Roundup and
> Rietveld based approach.
>

There could be JSON[-LD] schema to describe resources with attributes:

https://github.com/westurner/wiki/wiki/ideas#open-source-mailing-list-extractor

>
> There could be configurable per-list link heuristics:
>
>    - http[s]
>
>
>    - Issue: https://bugs.python.org/issue(d+)
>
>
>    - Src: https://hg.python.org/<repo>/<path>
>
>
>    - Src: https://github.com/<org>/<project>/<path>
>
>
>    - Src: https://bitbucket.org/<org>/<project>/<path>
>
>
>    - Patch/Attachment: http[s]://bugs.python.org/(file[d]+)/
>    <filename(.diff)>
>
>
>    - Doc: https://docs.python.org/<ver>/<path>
>
>
>    - Wiki: https://wiki.python.org/moin/<path>
>
>
>    - Homepage: https://www.python.org/<path>
>
>
>    - PyPI pkg: https://pypi.python.org/pypi/<path>
>
>
>    - Warehouse pkg: https://warehouse.python.org/project/<path>
>
>
>    - Wikipedia: https://[lang].wikipedia.org/wiki/<page> -->
>    (dbpedia:<page>)
>
>
>    - Build:
>    http://buildbot.python.org/all/builders/AMD64%20Ubuntu%20LTS%203.4/builds/771
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150108/10caf237/attachment.html>

From python at discontinuity.net  Fri Jan  9 00:21:06 2015
From: python at discontinuity.net (Davin Potts)
Date: Thu, 8 Jan 2015 17:21:06 -0600
Subject: [Python-Dev] contributing to multiprocessing
Message-ID: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>

Hi all --

I am interested in making some serious ongoing contributions around
multiprocessing.

My inspiration, first and foremost, comes from the current documentation
for multiprocessing.  There is great material there but I believe it is
being presented in a way that hinders adoption and understanding.  I've
taken some initial baby-steps to propose specific changes:
http://bugs.python.org/issue22952
http://bugs.python.org/issue23100

The first, issue22952, can reasonably be tackled with a patch like I've
submitted.  Continuing with patches for issue23100 can also be made to
work.  I realize that reviewing such patches takes non-trivial time from
volunteers yet I'm interested in submitting a series of patches to
hopefully make the documentation for multiprocessing much more consistent
with other module docs and much more accessible to end users.  I don't want
to simply create more work for other volunteers -- I'd like to volunteer to
reduce / share some of their work as well.

Beyond the documentation, there is currently a backlog of 186 issues
mentioning multiprocessing, some with patches on offer, some without.  I'd
like to volunteer my time reviewing and triaging these issues.  Hopefully
you can already get a sense of my voice on issues from what I wrote in
those two issues above.

Rather than me simply walking through that backlog, offering comments or
encouragement here and there on issues, it makes more sense for me to ask:
what is the right way for me to proceed?  What is the next step towards me
helping triage issues?  Is there a bridge-keeper with at least three, no
more than five questions for me?


Thanks,

Davin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150108/60f40fb5/attachment.html>

From ethan at stoneleaf.us  Fri Jan  9 02:08:07 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Thu, 08 Jan 2015 17:08:07 -0800
Subject: [Python-Dev] contributing to multiprocessing
In-Reply-To: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>
References: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>
Message-ID: <54AF29F7.1080300@stoneleaf.us>

On 01/08/2015 03:21 PM, Davin Potts wrote:
> 
> I am interested in making some serious ongoing contributions around multiprocessing.

Great!

> Rather than me simply walking through that backlog, offering comments or encouragement here and there on issues, it
> makes more sense for me to ask:  what is the right way for me to proceed?  What is the next step towards me helping
> triage issues?  Is there a bridge-keeper with at least three, no more than five questions for me?

I would suggest having at least one, if not two or three, current core-devs ready and willing to quickly review your
work (I believe Raymond Hettinger may be one); then, go ahead and triage, improve and/or submit patches, and make
comments.  Once you've got a few of these under your belt, with favorable reviews and your patches committed, you may
get stuck with commit privileges of your own.  ;)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150108/2cc6771d/attachment.sig>

From rdmurray at bitdance.com  Fri Jan  9 04:19:44 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 08 Jan 2015 22:19:44 -0500
Subject: [Python-Dev] contributing to multiprocessing
In-Reply-To: <54AF29F7.1080300@stoneleaf.us>
References: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>
 <54AF29F7.1080300@stoneleaf.us>
Message-ID: <20150109031944.E58C9251179@webabinitio.net>

On Thu, 08 Jan 2015 17:08:07 -0800, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 01/08/2015 03:21 PM, Davin Potts wrote:
> > 
> > I am interested in making some serious ongoing contributions around multiprocessing.
> 
> Great!
> 
> > Rather than me simply walking through that backlog, offering comments or encouragement here and there on issues, it
> > makes more sense for me to ask:  what is the right way for me to proceed?  What is the next step towards me helping
> > triage issues?  Is there a bridge-keeper with at least three, no more than five questions for me?
> 
> I would suggest having at least one, if not two or three, current core-devs ready and willing to quickly review your
> work (I believe Raymond Hettinger may be one); then, go ahead and triage, improve and/or submit patches, and make
> comments.  Once you've got a few of these under your belt, with favorable reviews and your patches committed, you may
> get stuck with commit privileges of your own.  ;)

Indeed, the best way to proceed, regardless of any other issues, is in
fact to review, triage, comment on, and improve the issues you are
interested in.  Get the patches commit ready, and then get a current
core dev to do a commit review.

Oddly, the doc issues may be more problematic than the code issues.
Fixing bugs in docs isn't difficult to get done, but restructuring
documentation sometimes gets bogged down in differing opinions.  (I
haven't myself looked at your proposals since I don't use
multiprocessing, so I don't know how radical the proposed changes are).

--David

From python at discontinuity.net  Fri Jan  9 05:20:32 2015
From: python at discontinuity.net (Davin Potts)
Date: Thu, 8 Jan 2015 22:20:32 -0600
Subject: [Python-Dev] contributing to multiprocessing
In-Reply-To: <20150109031944.E58C9251179@webabinitio.net>
References: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>
 <54AF29F7.1080300@stoneleaf.us> <20150109031944.E58C9251179@webabinitio.net>
Message-ID: <960745F2-B835-499E-B8F0-4A8E6B1133B0@discontinuity.net>

Thanks!  That sounds like a nice, clear path forward.

Regarding the doc issues being a bit more problematic to work through, I thoroughly understand.



Davin


On Jan 8, 2015, at 21:19, R. David Murray <rdmurray at bitdance.com> wrote:

> On Thu, 08 Jan 2015 17:08:07 -0800, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 01/08/2015 03:21 PM, Davin Potts wrote:
>>> 
>>> I am interested in making some serious ongoing contributions around multiprocessing.
>> 
>> Great!
>> 
>>> Rather than me simply walking through that backlog, offering comments or encouragement here and there on issues, it
>>> makes more sense for me to ask:  what is the right way for me to proceed?  What is the next step towards me helping
>>> triage issues?  Is there a bridge-keeper with at least three, no more than five questions for me?
>> 
>> I would suggest having at least one, if not two or three, current core-devs ready and willing to quickly review your
>> work (I believe Raymond Hettinger may be one); then, go ahead and triage, improve and/or submit patches, and make
>> comments.  Once you've got a few of these under your belt, with favorable reviews and your patches committed, you may
>> get stuck with commit privileges of your own.  ;)
> 
> Indeed, the best way to proceed, regardless of any other issues, is in
> fact to review, triage, comment on, and improve the issues you are
> interested in.  Get the patches commit ready, and then get a current
> core dev to do a commit review.
> 
> Oddly, the doc issues may be more problematic than the code issues.
> Fixing bugs in docs isn't difficult to get done, but restructuring
> documentation sometimes gets bogged down in differing opinions.  (I
> haven't myself looked at your proposals since I don't use
> multiprocessing, so I don't know how radical the proposed changes are).
> 
> --David
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/python%2Bpython_dev%40discontinuity.net


From mbec at gmto.org  Fri Jan  9 05:56:56 2015
From: mbec at gmto.org (matthieu bec)
Date: Thu, 08 Jan 2015 20:56:56 -0800
Subject: [Python-Dev] datetime nanosecond support (ctd?)
In-Reply-To: <54933D75.7020409@gmail.com>
References: <5487E8DD.5010806@gmail.com>	<CADiSq7ekEQ6CMLS2u-jkuNeq4=zXn8RtmxP1aEDJiOyChhjRkw@mail.gmail.com>	<548890EB.4070002@gmail.com>	<87ppbqhivm.fsf@uwakimon.sk.tsukuba.ac.jp>	<5489DB3E.7020005@gmail.com>	<CANc-5Uy48ku-uMsJJNavZcU78muLyMzOt11DNYkRa=-c+6epkg@mail.gmail.com>	<CAP7+vJ+4rvFAkAKN80Rd9PcYHX49VSsaGbLd4vQS8BE9Ry_eHA@mail.gmail.com>	<20141211202356.07462c01@fsol>	<CANc-5UxZHkA9GnWSzw_Yup7i7xu0TBSqZtSNuYK1ZanM0RMLLw@mail.gmail.com>	<20141211204652.40a0c807@fsol>	<5490679C.9050308@gmto.org>	<CAP7h-xZfXf8ATiy5B_pPgqC+DxNDHBMJa817DziKm0b+LpQ1wA@mail.gmail.com>	<5490C020.6030405@gmto.org>	<5490C0C8.3080409@gmto.org>	<5490D145.5050409@gmto.org>	<5492415A.5010103@gmail.com>
 <CALFfu7B5XUvqDVP+MCSS5NQTdq4k3DjiEk78gstvwY0Ms9J2gw@mail.gmail.com>
 <54933D75.7020409@gmail.com>
Message-ID: <54AF5F98.9030109@gmto.org>

FWIW issue23084 was withdrawn as it appears to be touching and PEP 
sized, and/or anyways nanosec is defacto just an int.
another patch went to http://bugs.python.org/issue15443 seemingly under 
the radar, I'm not arguing the quality of the patch, but bringing it up 
here in case with the holidays season it didn't get noticed. I wont 
bring much anything more so you may rest otherwise. Happy new year!



On 12/18/2014 12:47 PM, mdcb808 wrote:
> done - http://bugs.python.org/issue23084
>
> On 12/17/14 8:20 PM, Eric Snow wrote:
>> On Wed, Dec 17, 2014 at 7:52 PM, Matthieu Bec <mdcb808 at gmail.com> wrote:
>>>
>>>
>>> Attached patch defines a new type struct_timespec for the time
>>> module. A new
>>> capsule exports the type along with to/from converters - opening a
>>> bridge
>>> for C, and for example the datetime module.
>>
>> I'd recommend opening a new issue in the bug tracker (bugs.python.org)
>> and attach the patch there.  Attaching it to an email is a good way
>> for it to get lost and forgotten. :)
>>
>> -eric
>>


-- 
Matthieu Bec                GMTO Corp
cell : +1 626 425 7923      251 S Lake Ave, Suite 300
phone: +1 626 204 0527      Pasadena, CA 91101

From ncoghlan at gmail.com  Fri Jan  9 08:20:13 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 9 Jan 2015 17:20:13 +1000
Subject: [Python-Dev] contributing to multiprocessing
In-Reply-To: <960745F2-B835-499E-B8F0-4A8E6B1133B0@discontinuity.net>
References: <CABHLM6vQ9Pwe=kMSgDjc48EN82zcRPC5RydfHa+w_iSkOYunDQ@mail.gmail.com>
 <54AF29F7.1080300@stoneleaf.us>
 <20150109031944.E58C9251179@webabinitio.net>
 <960745F2-B835-499E-B8F0-4A8E6B1133B0@discontinuity.net>
Message-ID: <CADiSq7d9=XBhFRdtfizEqNTMuBiWCaae-U7kw9CiDCHnWDUK=w@mail.gmail.com>

On 9 January 2015 at 14:20, Davin Potts <python at discontinuity.net> wrote:
> Thanks!  That sounds like a nice, clear path forward.
>
> Regarding the doc issues being a bit more problematic to work through, I thoroughly understand.

In the case of changes to the multiprocessing docs, accepting larger
restructures would mainly be the domain of Richard Oudkerk (sbt) as
the lead maintainer for the module.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From status at bugs.python.org  Fri Jan  9 18:07:51 2015
From: status at bugs.python.org (Python tracker)
Date: Fri,  9 Jan 2015 18:07:51 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20150109170751.120A85640D@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2015-01-02 - 2015-01-09)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4726 (+20)
  closed 30244 (+37)
  total  34970 (+57)

Open issues with patches: 2214 


Issues opened (44)
==================

#14099: ZipFile.open() should not reopen the underlying file
http://bugs.python.org/issue14099  reopened by serhiy.storchaka

#14853: test_file.py depends on sys.stdin being unseekable
http://bugs.python.org/issue14853  reopened by haypo

#16192: ctypes - documentation example
http://bugs.python.org/issue16192  reopened by berker.peksag

#19776: Provide expanduser() on Path objects
http://bugs.python.org/issue19776  reopened by haypo

#22919: Update PCBuild for VS 2015
http://bugs.python.org/issue22919  reopened by haypo

#23152: fstat64 required on Windows
http://bugs.python.org/issue23152  opened by steve.dower

#23155: unittest: object has no attribute '_removed_tests'
http://bugs.python.org/issue23155  opened by wiz

#23156: Update tix install information in tkinter tix chapter of doc
http://bugs.python.org/issue23156  opened by terry.reedy

#23159: argparse: Provide equivalent of optparse.OptionParser.{option_
http://bugs.python.org/issue23159  opened by emcd

#23160: Respect the environment variable SVNROOT in external-common.ba
http://bugs.python.org/issue23160  opened by anselm.kruis

#23162: collections.abc sequences don't check identity before equality
http://bugs.python.org/issue23162  opened by Devin Jeanpierre

#23163: pdb docs need to contain a statement on threads/multithreaded 
http://bugs.python.org/issue23163  opened by krichter

#23164: "pydoc filter" documentation restrictive
http://bugs.python.org/issue23164  opened by lebigot

#23166: urllib2 ignores opener configuration under certain circumstanc
http://bugs.python.org/issue23166  opened by crazyjurich

#23169: Reflect that PreReq and BuildPreReq are deprecated in the late
http://bugs.python.org/issue23169  opened by radeksimko

#23171: csv.writer.writerow() does not accept generator (must be coerc
http://bugs.python.org/issue23171  opened by jdufresne

#23173: asyncio: kill the subprocess if the creation failed
http://bugs.python.org/issue23173  opened by haypo

#23174: shelve.open fails with error "anydbm.error: db type could not 
http://bugs.python.org/issue23174  opened by krichter

#23176: socket.recvfrom(0) waits for data
http://bugs.python.org/issue23176  opened by Sworddragon

#23177: test_ssl: failures on OpenBSD with LibreSSL
http://bugs.python.org/issue23177  opened by haypo

#23180: Rename IDLE's "Windows" menu item to "Window"
http://bugs.python.org/issue23180  opened by Al.Sweigart

#23181: Unicode "code point" should be two words in documentation
http://bugs.python.org/issue23181  opened by Al.Sweigart

#23182: Update grammar tests to use new style for annotated function d
http://bugs.python.org/issue23182  opened by IanLee1521

#23183: timeit CLI best of 3: undocumented output format
http://bugs.python.org/issue23183  opened by hachat

#23184: Unused imports, variables, file in IDLE
http://bugs.python.org/issue23184  opened by Al.Sweigart

#23185: add inf and nan to math module
http://bugs.python.org/issue23185  opened by ethan.furman

#23187: Segmentation fault, possibly asyncio related
http://bugs.python.org/issue23187  opened by karamanolev

#23188: Exception chaining should trigger for non-normalised exception
http://bugs.python.org/issue23188  opened by ncoghlan

#23189: Set docstrings to empty string when optimizing with -OO.
http://bugs.python.org/issue23189  opened by jvs

#23191: fnmatch regex cache use is not threadsafe
http://bugs.python.org/issue23191  opened by mschmitzer

#23192: Generator return value ignored in lambda function
http://bugs.python.org/issue23192  opened by Rosuav

#23193: Please support "numeric_owner" in tarfile
http://bugs.python.org/issue23193  opened by mvo

#23195: Sorting with locale (strxfrm) does not work properly with Pyth
http://bugs.python.org/issue23195  opened by pnugues

#23197: asyncio: check if a future is cancelled before calling set_res
http://bugs.python.org/issue23197  opened by haypo

#23198: asyncio: refactor StreamReader
http://bugs.python.org/issue23198  opened by haypo

#23199: libpython27.a in amd64 release is 32-bit
http://bugs.python.org/issue23199  opened by zwelch

#23200: Clarify max_length and flush() for zlib decompression
http://bugs.python.org/issue23200  opened by vadmium

#23201: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0
http://bugs.python.org/issue23201  opened by Devin Jeanpierre

#23202: pyvenv does not fail like documented when a venv already exist
http://bugs.python.org/issue23202  opened by The Compiler

#23203: Aliasing import of sub-{module,package} from the package raise
http://bugs.python.org/issue23203  opened by franck

#23205: Unit test needed for IDLE's GrepDialog.py's findfiles()
http://bugs.python.org/issue23205  opened by Al.Sweigart

#23206: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps(
http://bugs.python.org/issue23206  opened by naoki

#23207: logging.basicConfig does not validate keyword arguments
http://bugs.python.org/issue23207  opened by The Compiler

#23208: asyncio: add BaseEventLoop._current_handle
http://bugs.python.org/issue23208  opened by haypo



Most recent 15 issues with no replies (15)
==========================================

#23207: logging.basicConfig does not validate keyword arguments
http://bugs.python.org/issue23207

#23205: Unit test needed for IDLE's GrepDialog.py's findfiles()
http://bugs.python.org/issue23205

#23202: pyvenv does not fail like documented when a venv already exist
http://bugs.python.org/issue23202

#23199: libpython27.a in amd64 release is 32-bit
http://bugs.python.org/issue23199

#23183: timeit CLI best of 3: undocumented output format
http://bugs.python.org/issue23183

#23182: Update grammar tests to use new style for annotated function d
http://bugs.python.org/issue23182

#23181: Unicode "code point" should be two words in documentation
http://bugs.python.org/issue23181

#23173: asyncio: kill the subprocess if the creation failed
http://bugs.python.org/issue23173

#23169: Reflect that PreReq and BuildPreReq are deprecated in the late
http://bugs.python.org/issue23169

#23164: "pydoc filter" documentation restrictive
http://bugs.python.org/issue23164

#23163: pdb docs need to contain a statement on threads/multithreaded 
http://bugs.python.org/issue23163

#23159: argparse: Provide equivalent of optparse.OptionParser.{option_
http://bugs.python.org/issue23159

#23148: Missing the charset parameter in as_encoded_word()
http://bugs.python.org/issue23148

#23147: Possible error in _header_value_parser.py
http://bugs.python.org/issue23147

#23146: Incosistency in pathlib between / and \
http://bugs.python.org/issue23146



Most recent 15 issues waiting for review (15)
=============================================

#23208: asyncio: add BaseEventLoop._current_handle
http://bugs.python.org/issue23208

#23206: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps(
http://bugs.python.org/issue23206

#23205: Unit test needed for IDLE's GrepDialog.py's findfiles()
http://bugs.python.org/issue23205

#23200: Clarify max_length and flush() for zlib decompression
http://bugs.python.org/issue23200

#23198: asyncio: refactor StreamReader
http://bugs.python.org/issue23198

#23197: asyncio: check if a future is cancelled before calling set_res
http://bugs.python.org/issue23197

#23193: Please support "numeric_owner" in tarfile
http://bugs.python.org/issue23193

#23191: fnmatch regex cache use is not threadsafe
http://bugs.python.org/issue23191

#23185: add inf and nan to math module
http://bugs.python.org/issue23185

#23184: Unused imports, variables, file in IDLE
http://bugs.python.org/issue23184

#23182: Update grammar tests to use new style for annotated function d
http://bugs.python.org/issue23182

#23181: Unicode "code point" should be two words in documentation
http://bugs.python.org/issue23181

#23180: Rename IDLE's "Windows" menu item to "Window"
http://bugs.python.org/issue23180

#23173: asyncio: kill the subprocess if the creation failed
http://bugs.python.org/issue23173

#23171: csv.writer.writerow() does not accept generator (must be coerc
http://bugs.python.org/issue23171



Top 10 most discussed issues (10)
=================================

#23185: add inf and nan to math module
http://bugs.python.org/issue23185  19 msgs

#23086: Add start and stop parameters to the Sequence.index() ABC mixi
http://bugs.python.org/issue23086  13 msgs

#23201: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0
http://bugs.python.org/issue23201  10 msgs

#23152: fstat64 required on Windows
http://bugs.python.org/issue23152   8 msgs

#23187: Segmentation fault, possibly asyncio related
http://bugs.python.org/issue23187   8 msgs

#22906: PEP 479: Change StopIteration handling inside generators
http://bugs.python.org/issue22906   7 msgs

#22919: Update PCBuild for VS 2015
http://bugs.python.org/issue22919   7 msgs

#23162: collections.abc sequences don't check identity before equality
http://bugs.python.org/issue23162   7 msgs

#23174: shelve.open fails with error "anydbm.error: db type could not 
http://bugs.python.org/issue23174   7 msgs

#23177: test_ssl: failures on OpenBSD with LibreSSL
http://bugs.python.org/issue23177   7 msgs



Issues closed (39)
==================

#3180: Interrupts are lost during readline PyOS_InputHook processing
http://bugs.python.org/issue3180  closed by haypo

#14076: sqlite3 module ignores placeholders in CREATE TRIGGER code
http://bugs.python.org/issue14076  closed by ghaering

#18644: Got ResourceWarning: unclosed file when using test function fr
http://bugs.python.org/issue18644  closed by berker.peksag

#19548: 'codecs' module docs improvements
http://bugs.python.org/issue19548  closed by ncoghlan

#20487: Odd words in unittest.mock document.
http://bugs.python.org/issue20487  closed by berker.peksag

#20606: Operator Documentation Example doesn't work
http://bugs.python.org/issue20606  closed by benjamin.peterson

#21356: Support LibreSSL (instead of OpenSSL): make RAND_egd optional
http://bugs.python.org/issue21356  closed by haypo

#21980: Implement `logging.LogRecord.__repr__`
http://bugs.python.org/issue21980  closed by python-dev

#22038: Implement atomic operations on non-x86 platforms
http://bugs.python.org/issue22038  closed by haypo

#22428: asyncio: KeyboardInterrupt inside a coroutine causes Attribute
http://bugs.python.org/issue22428  closed by haypo

#22638: ssl module: the SSLv3 protocol is vulnerable ("POODLE" attack)
http://bugs.python.org/issue22638  closed by pitrou

#23014: Don't have importlib.abc.Loader.create_module() be optional
http://bugs.python.org/issue23014  closed by brett.cannon

#23119: Remove unicode specialization from set objects
http://bugs.python.org/issue23119  closed by rhettinger

#23127: socket.setsockopt() is still broken for multicast TTL and Loop
http://bugs.python.org/issue23127  closed by benjamin.peterson

#23132: Faster total_ordering
http://bugs.python.org/issue23132  closed by rhettinger

#23140: InvalidStateError on asyncio subprocess task cancelled
http://bugs.python.org/issue23140  closed by haypo

#23143: Remove some conditional code in _ssl.c
http://bugs.python.org/issue23143  closed by pitrou

#23145: regrtest: log test loader errors
http://bugs.python.org/issue23145  closed by haypo

#23149: Typo in PEP-0008 - "this PEP do not"
http://bugs.python.org/issue23149  closed by python-dev

#23151: _loggerClass is initialized twice
http://bugs.python.org/issue23151  closed by python-dev

#23153: Clarify Boolean Clause Results
http://bugs.python.org/issue23153  closed by r.david.murray

#23154: MSVC 2013 Express needlessly rebuilds code
http://bugs.python.org/issue23154  closed by zach.ware

#23157: Lib/test/time_hashlib.py doesn't work
http://bugs.python.org/issue23157  closed by gregory.p.smith

#23158: IDLE's help.txt "corrent" typo
http://bugs.python.org/issue23158  closed by terry.reedy

#23161: collections.abc.MutableSet missing methods
http://bugs.python.org/issue23161  closed by rhettinger

#23165: Heap overwrite in Python/fileutils.c:_Py_char2wchar() on 32 bi
http://bugs.python.org/issue23165  closed by python-dev

#23167: Marshal docs say format version is 3; actual format version is
http://bugs.python.org/issue23167  closed by python-dev

#23168: test_file2k.py depends on sys.stdin being unseekable
http://bugs.python.org/issue23168  closed by haypo

#23170: test_uuid test_ifconfig_getnode fails with Temporary failure i
http://bugs.python.org/issue23170  closed by r.david.murray

#23172: Bad file descriptor error occurs if random library is imported
http://bugs.python.org/issue23172  closed by haypo

#23175: logging.exception doesn't accept custom exc_info
http://bugs.python.org/issue23175  closed by vinay.sajip

#23178: csv.reader does not handle BOM
http://bugs.python.org/issue23178  closed by r.david.murray

#23179: New function proposal: string.from_iterable(iterable [,map_fun
http://bugs.python.org/issue23179  closed by r.david.murray

#23186: expose the client's list of ciphers during the handshake
http://bugs.python.org/issue23186  closed by python-dev

#23190: OpenSSL fails building with build.bat
http://bugs.python.org/issue23190  closed by pitrou

#23194: Antigravity prints osascript errors in OS X Yosemite
http://bugs.python.org/issue23194  closed by ned.deily

#23196: Greek letters not sorted properly
http://bugs.python.org/issue23196  closed by r.david.murray

#23204: list.index and rest of list methods disagree if a value is in 
http://bugs.python.org/issue23204  closed by rhettinger

#1687125: cannot catch KeyboardInterrupt when using curses getkey()
http://bugs.python.org/issue1687125  closed by haypo

From anthoff at berkeley.edu  Fri Jan  9 21:34:00 2015
From: anthoff at berkeley.edu (David Anthoff)
Date: Fri, 9 Jan 2015 12:34:00 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
Message-ID: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>

> Nothing is merged in yet and everything can still change, so I'm keen to
hear whatever
> feedback people have. I've tried to make improvements fairly for
first-time users
> through to sysadmins, but if I've missed something big I'd like to hear
about it before
> we get too close to alpha 1.

It would be great if the new installer supported silent, portable installs.
What I have in mind is a silent installation that drops all the necessary
files for a working python into a folder, but does not put ANY file anywhere
else and does not register anything anywhere on the system. So no PATH
modification, no registering of this install as one of the available python
interpreters, no uninstall entry or anything like that. By "all the
necessary files" I mean that it drops things like the MSVC runtime dlls etc
into that folder into which I'm installing python, but again doesn't try to
install things like that system or even user wide.

I'll give you my very specific use-case, but I think this might be useful
for others as well: we are trying to build an installer for some other
product X that internally requires a python instance to work. This python
instance should NOT be visible to anything other than product X, i.e. no
user should ever start it directly or anything like that, it really is an
implementation detail of product X. If we had an official python installer
that supported silent, portable installs, I could just call that python
installer inside the setup program for X, and it would drop a fully working
python installation into a sub-directory of the install directory of product
X. And we would be happy :)

The old MSI installer sort of had something like that with the MSI
administrative install option. But it never really worked because the
administrative install didn't drop the MSVC runtime dlls anywhere, as far as
I could tell. At some point I hacked around that by modifying the MSI file
in my setup program to also drop the MSVC runtime dlls, but that was VERY
hacky... 

Thanks,
David

From v+python at g.nevcal.com  Fri Jan  9 22:52:05 2015
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Fri, 09 Jan 2015 13:52:05 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
Message-ID: <54B04D85.4040909@g.nevcal.com>

On 1/9/2015 12:34 PM, David Anthoff wrote:
>> Nothing is merged in yet and everything can still change, so I'm keen to
> hear whatever
>> feedback people have. I've tried to make improvements fairly for
> first-time users
>> through to sysadmins, but if I've missed something big I'd like to hear
> about it before
>> we get too close to alpha 1.
> It would be great if the new installer supported silent, portable installs.
> What I have in mind is a silent installation that drops all the necessary
> files for a working python into a folder, but does not put ANY file anywhere
> else and does not register anything anywhere on the system. So no PATH
> modification, no registering of this install as one of the available python
> interpreters, no uninstall entry or anything like that. By "all the
> necessary files" I mean that it drops things like the MSVC runtime dlls etc
> into that folder into which I'm installing python, but again doesn't try to
> install things like that system or even user wide.
>
> I'll give you my very specific use-case, but I think this might be useful
> for others as well: we are trying to build an installer for some other
> product X that internally requires a python instance to work. This python
> instance should NOT be visible to anything other than product X, i.e. no
> user should ever start it directly or anything like that, it really is an
> implementation detail of product X. If we had an official python installer
> that supported silent, portable installs, I could just call that python
> installer inside the setup program for X, and it would drop a fully working
> python installation into a sub-directory of the install directory of product
> X. And we would be happy :)
>
> The old MSI installer sort of had something like that with the MSI
> administrative install option. But it never really worked because the
> administrative install didn't drop the MSVC runtime dlls anywhere, as far as
> I could tell. At some point I hacked around that by modifying the MSI file
> in my setup program to also drop the MSVC runtime dlls, but that was VERY
> hacky...
>
> Thanks,
> David
Sounds like a nice feature.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150109/daeaceee/attachment.html>

From Steve.Dower at microsoft.com  Sat Jan 10 01:09:48 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Sat, 10 Jan 2015 00:09:48 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
Message-ID: <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>

David Anthoff wrote:
> It would be great if the new installer supported silent, portable installs.
> What I have in mind is a silent installation that drops all the necessary files
> for a working python into a folder, but does not put ANY file anywhere else and
> does not register anything anywhere on the system. So no PATH modification, no
> registering of this install as one of the available python interpreters, no
> uninstall entry or anything like that. By "all the necessary files" I mean that
> it drops things like the MSVC runtime dlls etc into that folder into which I'm
> installing python, but again doesn't try to install things like that system or
> even user wide.
>
> I'll give you my very specific use-case, but I think this might be useful for
> others as well: we are trying to build an installer for some other product X
> that internally requires a python instance to work. This python instance should
> NOT be visible to anything other than product X, i.e. no user should ever start
> it directly or anything like that, it really is an implementation detail of
> product X. If we had an official python installer that supported silent,
> portable installs, I could just call that python installer inside the setup
> program for X, and it would drop a fully working python installation into a
> sub-directory of the install directory of product X. And we would be happy :)

I'll look into this, but it probably isn't going to work as part of the installer. I have previously looked into being able to install arbitrary side-by-side copies of Python, but that's near impossible as well. Windows Installer doesn't really let you just copy files - it isn't part of its intended functionality. It isn't too difficult to build custom MSIs with certain parts of Python (such as the DLLs and the standard library, but no docs, headers or EXEs) in a way that won't conflict with other installs, but you're still using an MSI here which is not necessarily ideal.

The easiest way is still going to be to install a copy of Python without administrative privileges, which *currently* will drop all the files you want into the main install path (wherever you customise that to be), and then copy them directly into your installer, but that may change depending on the redistribution requirements of the CRT. There are still limitations within Python itself that would be nice to fix, such as looking at the registry too soon when it could resolve directories relative to its own location, but these are independent from the installer. If you don't need python.exe, then it should be fine as long as you put pythonXY.dll alongside the executable loading it (otherwise a system version may take priority).

As you can see, this is a fairly deep hole with lots of caveats :)

We could release a ZIP file containing all the Python files. The only reason I hesitate on this is that it could cause significant confusion for someone who doesn't really understand the implications, while people like yourself who have thought about this are also capable of finding workarounds and don't really need the ZIP file apart from convenience. Making some of the fixes to make python.exe more portable would relieve my concerns here.

> The old MSI installer sort of had something like that with the MSI
> administrative install option. But it never really worked because the
> administrative install didn't drop the MSVC runtime dlls anywhere, as far as I
> could tell. At some point I hacked around that by modifying the MSI file in my
> setup program to also drop the MSVC runtime dlls, but that was VERY hacky...

I'm a little surprised that worked at all for what you were trying to do. You'd be better off installing it once and then copying the files yourself. 

But overall, this is the sort of thing I do want to enable. I firmly believe that Python from python.org is for *developers*, and those who just want to run a Python application should be able to get a complete package. This is very different from the *nix approach, but it makes far more sense to Windows users. A good example is TortoiseHg, which bundles everything so that users never even know they have Python 2.7 or Mercurial installed on their machine. Making it easier for people to bundle Python into their own applications is a good thing, as far as I'm concerned.

Cheers,
Steve

> Thanks,
> David

From anthoff at berkeley.edu  Sat Jan 10 02:01:49 2015
From: anthoff at berkeley.edu (David Anthoff)
Date: Fri, 9 Jan 2015 17:01:49 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>

> I'll look into this, but it probably isn't going to work as part of the
installer. I
> have previously looked into being able to install arbitrary side-by-side
copies
> of Python, but that's near impossible as well. Windows Installer doesn't
really
> let you just copy files - it isn't part of its intended functionality. It
isn't too
> difficult to build custom MSIs with certain parts of Python (such as the
DLLs
> and the standard library, but no docs, headers or EXEs) in a way that
won't
> conflict with other installs, but you're still using an MSI here which is
not
> necessarily ideal.

Are administrative MSI installs an option, though? They don't register
anything but just drop files, right? But see my comments below about a zip
drop, which would be a much, much nicer option in my opinion. 

> We could release a ZIP file containing all the Python files.

That would be absolutely FANTASTIC and would solve all problems around this
topic. In theory we could handle this ourselves on our end, but this is for
a small open source project and we are really hesitant to take on another
software packaging job. Doing this right just for our product would be a
fair amount of work, we would have to do this with every new Python release,
we might mix things up etc., and really we are more interested in our piece
of software than packaging Python ;) I think it would be a much nicer model
if there was a zip to download from python.org. There is another issue that
keeps us from hosting our own files: we would have to figure out licensing
issues, both related to python and to the msvcr*.dll and msvcp*.dll. I would
feel much more comfortable if we didn't distribute python or other binaries,
but just downloaded stuff from python.org as part of the setup process.

> The only reason I
> hesitate on this is that it could cause significant confusion for someone
who
> doesn't really understand the implications, while people like yourself who
> have thought about this are also capable of finding workarounds and don't
> really need the ZIP file apart from convenience. 

I was not clear in my previous email. I have NOT found a way to work around
this. I have tried various hacks, but none really works. I got pretty far,
but none really worked in all cases in a robust way. So I would certainly
welcome a downloadable zip file a great deal. Is there maybe a compromise
for now to have such a zip on the server, but not advertise it widely, and
maybe put an "experimental"/"beta" moniker into the filename?

I assume you would include the MS VC runtime files msvcr100.dll and
msvcp100.dll in such a zip file?

Is there any chance this might even be done (as an experimental version) for
Python 3.4?

> Making some of the fixes to
> make python.exe more portable would relieve my concerns here.

I see that. For our cases things seem to work, but I agree, it would be good
if python.exe would try hard to work in a xcopy mode.

> > The old MSI installer sort of had something like that with the MSI
> > administrative install option. But it never really worked because the
> > administrative install didn't drop the MSVC runtime dlls anywhere, as
> > far as I could tell. At some point I hacked around that by modifying
> > the MSI file in my setup program to also drop the MSVC runtime dlls, but
> that was VERY hacky...
> 
> I'm a little surprised that worked at all for what you were trying to do.
You'd
> be better off installing it once and then copying the files yourself.

I got it to drop the msvcr100.dll, but haven't managed to get the
msvcp100.dll out via an administrative install. The whole approach is a
mess, I would MUCH prefer a zip file.
 
> But overall, this is the sort of thing I do want to enable. I firmly
believe that
> Python from python.org is for *developers*, and those who just want to run
> a Python application should be able to get a complete package. This is
very
> different from the *nix approach, but it makes far more sense to Windows
> users. A good example is TortoiseHg, which bundles everything so that
users
> never even know they have Python 2.7 or Mercurial installed on their
> machine. Making it easier for people to bundle Python into their own
> applications is a good thing, as far as I'm concerned.

Yes, those are good examples. Right now doing this in the way these guys do
is too much work for our small project... Anything that makes this easier
would be appreciated.

Thanks! And the new installer looks great in general.

Best,
David

From ethan at stoneleaf.us  Sat Jan 10 02:29:35 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 09 Jan 2015 17:29:35 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
Message-ID: <54B0807F.4090303@stoneleaf.us>

On 01/09/2015 05:01 PM, David Anthoff wrote:
> On 01/09/2015 04:09 PM, Steve Dower wrote:

>> The only reason I hesitate on this is that it could cause significant
>> confusion for someone who doesn't really understand the implications,
>> while people like yourself who have thought about this are also capable
>> of finding workarounds and don't really need the ZIP file apart from
>> convenience.
> 
> I was not clear in my previous email. I have NOT found a way to work around
> this.

> Is there any chance this might even be done (as an experimental version) for
> Python 3.4?

Couldn't this zip file be advertised in the embedded section of python.org?  Or with a "for embedding in Windows apps"
tag?  (or whatever the proper verbiage is)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150109/87afe9f8/attachment-0001.sig>

From ncoghlan at gmail.com  Sat Jan 10 12:28:04 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 10 Jan 2015 21:28:04 +1000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <54B0807F.4090303@stoneleaf.us>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
 <54B0807F.4090303@stoneleaf.us>
Message-ID: <CADiSq7cdcNxGuo1gEQLB53HxOPV0CMz-ibAahZJUKE-nGAhUxg@mail.gmail.com>

On 10 January 2015 at 11:29, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 01/09/2015 05:01 PM, David Anthoff wrote:
>> On 01/09/2015 04:09 PM, Steve Dower wrote:
>
>>> The only reason I hesitate on this is that it could cause significant
>>> confusion for someone who doesn't really understand the implications,
>>> while people like yourself who have thought about this are also capable
>>> of finding workarounds and don't really need the ZIP file apart from
>>> convenience.
>>
>> I was not clear in my previous email. I have NOT found a way to work around
>> this.
>
>> Is there any chance this might even be done (as an experimental version) for
>> Python 3.4?
>
> Couldn't this zip file be advertised in the embedded section of python.org?  Or with a "for embedding in Windows apps"
> tag?  (or whatever the proper verbiage is)

For the time being, things like PyInstaller, PyRun, Portable Python,
etc are going to offer a better solution than anything we provide in
the standard installers.

As Steve says, addressing the situation properly requires eliminating
various current assumptions in the interpreter startup sequence that
are really only valid in a normal installation context, and if you
start pulling on *that* particular thread you may find yourself
writing a proposal like PEP 432 as your brain starts to hurt and you
consider ways you may be able to make the pain stop :)

By bypassing the normal interpreter entirely, and instead creating
your own custom binary that embeds the CPython runtime, you get a lot
more flexibility and control in terms of what happens during the
startup sequence. It's not complete control (hence the need for PEP
432, or something like it), but it's still a lot more control than if
you try to use the python.org binaries directly.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From Steve.Dower at microsoft.com  Mon Jan 12 18:26:43 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 12 Jan 2015 17:26:43 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
Message-ID: <DM2PR0301MB07347AF6388CAE84E3C7BB97F5430@DM2PR0301MB0734.namprd03.prod.outlook.com>

David Anthoff wrote:
>> I'll look into this, but it probably isn't going to work as part of
>> the installer. I have previously looked into being able to install arbitrary
>> side-by-side copies of Python, but that's near impossible as well.
>> Windows Installer doesn't really let you just copy files - it isn't
>> part of its intended functionality. It isn't too difficult to build custom
>> MSIs with certain parts of Python (such as the DLLs and the standard
>> library, but no docs, headers or EXEs) in a way that won't conflict with
>> other installs, but you're still using an MSI here which is not
>> necessarily ideal.
> 
> Are administrative MSI installs an option, though? They don't register anything
> but just drop files, right? But see my comments below about a zip drop, which
> would be a much, much nicer option in my opinion.

Not to my knowledge. An administrative install puts the files in a shared location and allows users to run much faster installs that will then refer to that shared location rather than copying the files locally. As I said, MSI doesn't support plain file drops (often called "xcopy install" - you use that term later, but I'm not sure how well known it is).

>> We could release a ZIP file containing all the Python files.
> 
> That would be absolutely FANTASTIC and would solve all problems around this
> topic. In theory we could handle this ourselves on our end, but this is for a
> small open source project and we are really hesitant to take on another software
> packaging job. Doing this right just for our product would be a fair amount of
> work, we would have to do this with every new Python release, we might mix
> things up etc., and really we are more interested in our piece of software than
> packaging Python ;) I think it would be a much nicer model if there was a zip to
> download from python.org.

Apart from the bit below, this would be identical to you installing Python once on your own machine and zipping up the files yourself. For the (considerably) less than 1% of Python users who want to do this, I don't think it's a big ask.

> There is another issue that keeps us from hosting our
> own files: we would have to figure out licensing issues, both related to python
> and to the msvcr*.dll and msvcp*.dll. I would feel much more comfortable if we
> didn't distribute python or other binaries, but just downloaded stuff from
> python.org as part of the setup process.

IANAL, but I've dealt with enough licensing issues in my day job that this makes me even more concerned about making a ZIP file available. If you're planning to do this, putting up a ZIP file could potentially expose Python (and presumably the PSF indirectly) to the liability that you should be taking on as a redistributor, if it's that important to your product. Without an actual legal opinion, I'm now -1 on making a ZIP download available for this purpose.

>> The only reason I hesitate on this is that it could cause significant
>> confusion for someone who doesn't really understand the implications,
>> while people like yourself who have thought about this are also capable
>> of finding workarounds and don't really need the ZIP file apart from
>> convenience.
> 
> I was not clear in my previous email. I have NOT found a way to work around
> this. I have tried various hacks, but none really works. I got pretty far, but
> none really worked in all cases in a robust way. So I would certainly welcome a
> downloadable zip file a great deal. Is there maybe a compromise for now to have
> such a zip on the server, but not advertise it widely, and maybe put an
> "experimental"/"beta" moniker into the filename?

I gave you a workaround. Install Python just for yourself and zip up the install directory.

> I assume you would include the MS VC runtime files msvcr100.dll and msvcp100.dll
> in such a zip file?

Either that or redistribute them using the tools provided by Microsoft (there are redistributable merge modules and executable installers available). The advantage of the latter approach is that your users will automatically get the latest versions and security fixes.

> Is there any chance this might even be done (as an experimental version) for
> Python 3.4?

You'd have to ask Martin, so probably not. But the workaround I gave you will work for 3.4.

>> Making some of the fixes to
>> make python.exe more portable would relieve my concerns here.
> 
> I see that. For our cases things seem to work, but I agree, it would be good if
> python.exe would try hard to work in a xcopy mode.
> 
>>> The old MSI installer sort of had something like that with the MSI
>>> administrative install option. But it never really worked because
>>> the administrative install didn't drop the MSVC runtime dlls
>>> anywhere, as far as I could tell. At some point I hacked around that
>>> by modifying the MSI file in my setup program to also drop the MSVC
>>> runtime dlls, but
>>> that was VERY hacky...
>>
>> I'm a little surprised that worked at all for what you were trying to do.
>> You'd be better off installing it once and then copying the files yourself.
> 
> I got it to drop the msvcr100.dll, but haven't managed to get the msvcp100.dll
> out via an administrative install. The whole approach is a mess, I would MUCH
> prefer a zip file.
>
>> But overall, this is the sort of thing I do want to enable. I firmly
>> believe that Python from python.org is for *developers*, and those who
>> just want to run a Python application should be able to get a complete
>> package. This is very different from the *nix approach, but it makes far
>> more sense to Windows users. A good example is TortoiseHg, which bundles
>> everything so that users never even know they have Python 2.7 or
>> Mercurial installed on their machine. Making it easier for people to
>> bundle Python into their own applications is a good thing, as far as I'm
>> concerned.
> 
> Yes, those are good examples. Right now doing this in the way these guys do is
> too much work for our small project... Anything that makes this easier would be
> appreciated.

I don't see how. All they've done is literally copy a Python installation into their install directory. Yes, they have their own launcher executables (py2exe generated, it looks like) and have precompiled the standard library and put it in a ZIP file, but you don't even need to go that far. Without knowing anything about your project I can't give specific suggestions, but simply dropping a Python installation in is not that difficult (and until the issues that Nick referred to are fixed, will have the same problems as TortoiseHg presumably has).

> Thanks! And the new installer looks great in general.

Thanks.

Cheers,
Steve

From rdmurray at bitdance.com  Mon Jan 12 18:48:59 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 12 Jan 2015 12:48:59 -0500
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB07347AF6388CAE84E3C7BB97F5430@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
 <DM2PR0301MB07347AF6388CAE84E3C7BB97F5430@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <20150112174859.99148250F98@webabinitio.net>

On Mon, 12 Jan 2015 17:26:43 +0000, Steve Dower <Steve.Dower at microsoft.com> wrote:
> David Anthoff wrote:
> > Yes, those are good examples. Right now doing this in the way these guys do is
> > too much work for our small project... Anything that makes this easier would be
> > appreciated.
> 
> I don't see how. All they've done is literally copy a Python
> installation into their install directory. Yes, they have their own
> launcher executables (py2exe generated, it looks like) and have
> precompiled the standard library and put it in a ZIP file, but you
> don't even need to go that far. Without knowing anything about your
> project I can't give specific suggestions, but simply dropping a
> Python installation in is not that difficult (and until the issues
> that Nick referred to are fixed, will have the same problems as
> TortoiseHg presumably has).

That's what py2exe *does*.  It does all the python-integration and
launcher-building work for you.  I use it for a small project myself
(proprietary), which I build an installer for using Inno Setup.  Works
very well, supports python3.

--David

From chris.barker at noaa.gov  Mon Jan 12 18:48:15 2015
From: chris.barker at noaa.gov (Chris Barker)
Date: Mon, 12 Jan 2015 09:48:15 -0800
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CADiSq7cdcNxGuo1gEQLB53HxOPV0CMz-ibAahZJUKE-nGAhUxg@mail.gmail.com>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
 <54B0807F.4090303@stoneleaf.us>
 <CADiSq7cdcNxGuo1gEQLB53HxOPV0CMz-ibAahZJUKE-nGAhUxg@mail.gmail.com>
Message-ID: <CALGmxE+u6LRj=2WJsr6FJ0Bf7hiQO_mGuNuXCbchbS8oducksQ@mail.gmail.com>

On Sat, Jan 10, 2015 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> For the time being, things like PyInstaller, PyRun, Portable Python,
> etc are going to offer a better solution than anything we provide in
> the standard installers.
>

See also Anaconda and Enthought Canopy. I think miniconda, for instance,
may give you just what you need, if you don't want to go the
py2exe/PyInstaller approach (though you probably do want to go that way, as
far as I can tell from your description of your use-case.

I'm inclined to think that this does not belong as part of the standard
installer.

-Chris

-- 

Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150112/ef0c23b3/attachment.html>

From Steve.Dower at microsoft.com  Mon Jan 12 21:02:58 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Mon, 12 Jan 2015 20:02:58 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <CALGmxE+u6LRj=2WJsr6FJ0Bf7hiQO_mGuNuXCbchbS8oducksQ@mail.gmail.com>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
 <54B0807F.4090303@stoneleaf.us>
 <CADiSq7cdcNxGuo1gEQLB53HxOPV0CMz-ibAahZJUKE-nGAhUxg@mail.gmail.com>
 <CALGmxE+u6LRj=2WJsr6FJ0Bf7hiQO_mGuNuXCbchbS8oducksQ@mail.gmail.com>
Message-ID: <DM2PR0301MB0734FA9464A37199595DFA3FF5430@DM2PR0301MB0734.namprd03.prod.outlook.com>

> On Sat, Jan 10, 2015 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> For the time being, things like PyInstaller, PyRun, Portable Python,
>> etc are going to offer a better solution than anything we provide in
>> the standard installers.
> 
> See also Anaconda and Enthought Canopy. I think miniconda, for instance, may
> give you just what you need, if you don't want to go the py2exe/PyInstaller
> approach (though you probably do want to go that way, as far as I can tell from
> your description of your use-case.

Actually, both Anaconda and Canopy suffer from the initialization process issues. You can't really use virtualenv with either of them unless they set the registry PythonPath value (since they won't find the libraries they need to launch), and if you install both then they'll fight over the registry key and you'll get Canopy launching with Anaconda's path or vice-versa.

> I'm inclined to think that this does not belong as part of the standard
> installer.

The problems are inherent to the standard python.exe, and are likely part of the standard pythonXY.dll. Nick's scared of fixing it, so I'm absolutely petrified :)

Cheers,
Steve

> -Chris

From martin at v.loewis.de  Tue Jan 13 14:39:28 2015
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 13 Jan 2015 14:39:28 +0100
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <54B52010.3080109@v.loewis.de>

Am 04.01.15 00:34, schrieb Steve Dower:
> so I'm keen to hear whatever feedback people have. 

One issue that I always wanted to address is patch releases. There are
two aspects I dislike about the current implementation

a) an upgrade install first uninstalls the previous installation
(removing all files), and then reinstalls all "new" files. In many patch
releases, a lot of .py files remain unmodified, so it should
speed up the upgrade if they would not need to be replaced.
b) Installer supports patch files (.msp); I always wanted to provide
them in the hope that this would reduce the download size. IIUC, it
would require stable component IDs for components to be upgraded, which
I could not manage to provide.

So, do you have any plans on dealing with a or b?

Other issues:
- what MSI version do you require, and what is the minimum
  Windows version supporting that MSI version?
- Since you are going to install into Program Files by default,
  I think the library should be precompiled by default - there
  is little chance that a regular user can save .pyc files when
  running Python. It might be possible to even include the pyc
  files in the distribution, if we can arrange to somehow support
  relative paths in co_filename.

Regards,
Martin


From Steve.Dower at microsoft.com  Tue Jan 13 17:06:40 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 13 Jan 2015 16:06:40 +0000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <54B52010.3080109@v.loewis.de>
References: <DM2PR0301MB0734BCDB3A013EE260CAEF7DF55A0@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <54B52010.3080109@v.loewis.de>
Message-ID: <DM2PR0301MB07349878D2D574C6066435C4F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>

Martin v. L?wis wrote:
> Am 04.01.15 00:34, schrieb Steve Dower:
>> so I'm keen to hear whatever feedback people have.
> 
> One issue that I always wanted to address is patch releases. There are two
> aspects I dislike about the current implementation
> 
> a) an upgrade install first uninstalls the previous installation (removing all
>    files), and then reinstalls all "new" files. In many patch releases, a lot of
>    .py files remain unmodified, so it should speed up the upgrade if they would not
>    need to be replaced.
> b) Installer supports patch files (.msp); I always wanted to provide them in the
>    hope that this would reduce the download size. IIUC, it would require stable
>    component IDs for components to be upgraded, which I could not manage to
>    provide.
> 
> So, do you have any plans on dealing with a or b?

So the stable component IDs is dealt with - WiX will auto-generate them based on the relative install path of each file and a user-provided ID (to distinguish between the 32-bit and 64-bit installs, for example. This ID is currently generated from the hash of a user provided URI, which is also used to generate upgrade codes and a few fixed component IDs).

Installing patches is something we can defer until 3.5.1, but I am interested in looking into it. My main concern is that it may hurt fresh installs (for example, 3.5.5 is actually 3.5.0.msi+.1.msp+.2.msp+.3.msp+.4.msp+.5.msp) and I'm not sure that true slipstreams are supported/recommended because it will affect future patches (different product codes, IIRC). So fresh installs of later versions may have a significantly increased download size.

Because of the split into multiple MSIs, it's also possible to independently version some parts of it. I suspect this will only apply to the py.exe launcher, but if that is unchanged between 3.5.0 and 3.5.1 then we can leave its version at 3.5.0 and it won't be redownloaded/installed. Not a huge saving, but it's a possibility. I expect most MSIs will change in some way between versions, so an MSP is the only good way to improve upgrades (the main benefit of the MSI split here is we will always install the latest 32-bit launcher, and if you install 3.5 32-bit and 64-bit and eventually 3.6 32-bit and 64-bit you'll only have one launcher installed).

Another possible problem is that MSI uses file version information to determine whether to update files. .py files don't have version information, so MSI to MSI updates are probably always going to replace everything - another reason why MSPs are the only good choice here.

I'll chat to some of the guys who work on the Visual Studio installer, since it uses the same technology and is far more complex than Python's one. They may have some suggestions about how to approach this.

> Other issues:
> - what MSI version do you require, and what is the minimum
>   Windows version supporting that MSI version?

I need to double-check the support still, but the aim is to support back to Windows Vista, so I think that's Windows Installer 3.0. Of more concern is the installer EXE, which is going to require a minimum level C runtime that may not be available before we've installed...

> - Since you are going to install into Program Files by default,
>   I think the library should be precompiled by default - there
>   is little chance that a regular user can save .pyc files when
>   running Python. It might be possible to even include the pyc
>   files in the distribution, if we can arrange to somehow support
>   relative paths in co_filename.

Agreed. The library will be precompiled by default if you choose the All Users button on the front page, and you can control this independently through the customize option (to precompile Just for Me or install without precompiling).

> Regards,
> Martin

Cheers,
Steve

From victor.stinner at gmail.com  Tue Jan 13 23:04:22 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 13 Jan 2015 23:04:22 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
Message-ID: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>

Hi,

To compile Python on Windows, there are a few information in the
Developer Guide:
https://docs.python.org/devguide/setup.html#windows-compiling

Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
http://bugs.python.org/issue22919#msg233637

I found PCbuild\readme.txt which is not mentionned in the devguide :-/
https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
(at least not on the Windows section of the setup page)

I found some clues to build OpenSSL to be able to build the Python ssl
module, but I still have issues.

Is there a more complete documentation?

I found how to install svn.exe, perl.exe and nasm.exe, but not how to
install nmake.exe. I don't know the command to build OpenSSL.

I don't care of building OpenSSL, my goal is only to build the Python
ssl module. Is there a way to install a development version of OpenSSL
(.lib files if I remember correctly) from an installer/binary?

My draft notes:

+Compile CPython on Windows
+==========================
+
+To build the Python ssl extension:
+
+Need:
+
+* Visual Studio 2010 SP1 or newer
+* CPython source code (default branch: 3.5)
+* perl binary: ActivePerl
+* svn binary, ex: SilkSVN
+* nasm and nmake binaries: compile NASM (install the binary doesn't
install nmake)
+
+Read PCbuild/readme.txt.
+
+* Build Python (in debug mode)
+* Type: PCbuild\get_externals.bat
+* Type: PCbuild\win32\python_d.exe PCbuild\prepare_ssl.py
externals\openssl-1.0.1j

Victor

From mal at egenix.com  Tue Jan 13 23:09:01 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Jan 2015 23:09:01 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
Message-ID: <54B5977D.6010609@egenix.com>

On 13.01.2015 23:04, Victor Stinner wrote:
> Hi,
> 
> To compile Python on Windows, there are a few information in the
> Developer Guide:
> https://docs.python.org/devguide/setup.html#windows-compiling
> 
> Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
> http://bugs.python.org/issue22919#msg233637
> 
> I found PCbuild\readme.txt which is not mentionned in the devguide :-/
> https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
> (at least not on the Windows section of the setup page)
> 
> I found some clues to build OpenSSL to be able to build the Python ssl
> module, but I still have issues.
> 
> Is there a more complete documentation?
> 
> I found how to install svn.exe, perl.exe and nasm.exe, but not how to
> install nmake.exe. I don't know the command to build OpenSSL.
> 
> I don't care of building OpenSSL, my goal is only to build the Python
> ssl module. Is there a way to install a development version of OpenSSL
> (.lib files if I remember correctly) from an installer/binary?

If you just need the compiled OpenSSL DLLs and .libs, you
can use the ones we ship with the source package of egenix-pyopenssl:

http://www.egenix.com/products/python/pyOpenSSL/#Download

The files are in openssl-win32 / openssl-win64.

Those dirs also have batch files which we use to build the
OpenSSL libs on Windows.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...        http://egenix.com/go69

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From brian at python.org  Tue Jan 13 23:10:58 2015
From: brian at python.org (Brian Curtin)
Date: Tue, 13 Jan 2015 16:10:58 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
Message-ID: <CAD+XWwot+74mvDoazAcqmVGUiARf8RY=5hqkdShn87gO+4Tk7A@mail.gmail.com>

On Tue, Jan 13, 2015 at 4:04 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> To compile Python on Windows, there are a few information in the
> Developer Guide:
> https://docs.python.org/devguide/setup.html#windows-compiling
>
> Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
> http://bugs.python.org/issue22919#msg233637
>
> I found PCbuild\readme.txt which is not mentionned in the devguide :-/
> https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
> (at least not on the Windows section of the setup page)
>
> I found some clues to build OpenSSL to be able to build the Python ssl
> module, but I still have issues.
>
> Is there a more complete documentation?
>
> I found how to install svn.exe, perl.exe and nasm.exe, but not how to
> install nmake.exe. I don't know the command to build OpenSSL.

For nmake, are you running this in a regular Command Prompt or in the
Visual Studio Command Prompt? The latter sets the right environment to
point you to some tools that VS installs, including nmake.

From zachary.ware+pydev at gmail.com  Tue Jan 13 23:15:08 2015
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Tue, 13 Jan 2015 16:15:08 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
Message-ID: <CAKJDb-Pw55wCQQt6Zi5xdAQi0pR6szRZ0PWs0hzDF=aMSVuqDg@mail.gmail.com>

On Tue, Jan 13, 2015 at 4:04 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> To compile Python on Windows, there are a few information in the
> Developer Guide:
> https://docs.python.org/devguide/setup.html#windows-compiling
>
> Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
> http://bugs.python.org/issue22919#msg233637
>
> I found PCbuild\readme.txt which is not mentionned in the devguide :-/
> https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
> (at least not on the Windows section of the setup page)

The first line of the section you linked to is "The readme included in
the solution has more details, especially on what additional software
is required to build which parts of Python.", and 'readme' is a link
to the readme on h.p.o. :)

> I found some clues to build OpenSSL to be able to build the Python ssl
> module, but I still have issues.
>
> Is there a more complete documentation?

The readme *should* be fairly comprehensive as I rewrote it last year,
but it may still be lacking since I wrote it after doing regular
builds for a year or two.  In particular, the Quick Start guide at the
very beginning of the readme should be enough to get you going.

"""
Quick Start Guide
-----------------

1.  Install Microsoft Visual Studio 2015, any edition.
2.  Install Subversion, and make sure 'svn.exe' is on your PATH.
3.  Run "build.bat -e" to build Python in 32-bit Release configuration.
4.  (Optional, but recommended) Run the test suite with "rt.bat -q".
"""

That should be enough to build Python, OpenSSL, Tcl/Tk/Tix, and all
other external projects.  One thing that might be a gotcha there, if
you're using a Command Prompt window that you opened before installing
SVN, you'll need to either adjust PATH manually or open a new Command
Prompt to get the PATH changes.

> I found how to install svn.exe, perl.exe and nasm.exe, but not how to
> install nmake.exe. I don't know the command to build OpenSSL.

Perl is not necessary if you're using sources from svn.python.org
(which "build.bat -e" will do).  You also no longer need to install
NASM.  nmake.exe is part of Visual Studio (or the Windows SDK,
whichever).

> I don't care of building OpenSSL, my goal is only to build the Python
> ssl module. Is there a way to install a development version of OpenSSL
> (.lib files if I remember correctly) from an installer/binary?
>
> My draft notes:
>
> +Compile CPython on Windows
> +==========================
> +
> +To build the Python ssl extension:
> +
> +Need:
> +
> +* Visual Studio 2010 SP1 or newer
> +* CPython source code (default branch: 3.5)
> +* perl binary: ActivePerl
> +* svn binary, ex: SilkSVN
> +* nasm and nmake binaries: compile NASM (install the binary doesn't
> install nmake)
> +
> +Read PCbuild/readme.txt.
> +
> +* Build Python (in debug mode)
> +* Type: PCbuild\get_externals.bat
> +* Type: PCbuild\win32\python_d.exe PCbuild\prepare_ssl.py
> externals\openssl-1.0.1j

You don't need to use prepare_ssl.py if you use get_externals.bat.
The 'ssl' section of the readme should cover that (if it can be
clarified, please point out the problems to me!).

-- 
Zach

From Steve.Dower at microsoft.com  Tue Jan 13 23:18:40 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 13 Jan 2015 22:18:40 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
Message-ID: <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>

Victor Stinner wrote:
> To compile Python on Windows, there are a few information in the Developer
> Guide:
> https://docs.python.org/devguide/setup.html#windows-compiling

I'm sorry, I've been slack and haven't updated this guide with the new changes. That said, there aren't that many changes to what is already in the guide - mostly new information that I'd like to add. I'll try and get something written up this week.

> Python 3.5 now requires Visual Studio 2010 *SP1*, or newer Visual Studio:
> http://bugs.python.org/issue22919#msg233637

Technically, Python 3.5 requires Visual Studio 2015, as the decision was made to move to the newer compiler for this release (and make whatever schedule changes were necessary to ensure everything will be available - it was on python-dev, though I don't have links handy).

It will still open and build with Visual Studio 2010 SP1 or later, however, buildbots should move to VS 2015 as they can, since that will be the official toolset. (I'm not pushing too hard until 2015 RC is released, since that's when Microsoft start supporting upgrades to newer versions. We do have one buildbot that's already using VS 2015 Preview though.)

> I found PCbuild\readme.txt which is not mentionned in the devguide :-/
> https://hg.python.org/cpython/file/56f717235c45/PCbuild/readme.txt
> (at least not on the Windows section of the setup page)

That should help, since I've updated that file. Though it's a little ahead of its time still, so a lot of the instructions do not yet have to be followed strictly, and I consider being able to build with VS 2010 SP1 a feature that we want to keep supporting for now while we transition.

> I found some clues to build OpenSSL to be able to build the Python ssl module,
> but I still have issues.
> 
> Is there a more complete documentation?
> 
> I found how to install svn.exe, perl.exe and nasm.exe, but not how to install
> nmake.exe. I don't know the command to build OpenSSL.

You don't require perl, you don't have to install nasm or build OpenSSL. Calling PCBuild/get_externals.bat will get nasm and the OpenSSL sources, and PCBuild/build.bat will rebuild OpenSSL as necessary.

nmake is part of Visual C++, so it should be available once you have your path configured (PCBuild/env.bat).

So a complete guide to check out and build Python on Windows should be:

1. Install SVN and put it on your path
2. Run PCBuild\get_externals.bat (this is the only step that requires SVN)
3. Install Visual Studio 2010 SP1 or later
4. Open PCBuild\pcbuild.sln in Visual Studio *or* run PCBuild\build.bat from the command line.

> I don't care of building OpenSSL, my goal is only to build the Python ssl
> module. Is there a way to install a development version of OpenSSL (.lib files
> if I remember correctly) from an installer/binary?

If you can find ssleay.lib and libeay.lib somewhere, they'll probably work. If not, those projects are in PCBuild and will be built automatically (if necessary) when you build _ssl.vcxproj or _hashlib.vcxproj.

Cheers,
Steve

> My draft notes:
> 
> +Compile CPython on Windows
> +==========================
> +
> +To build the Python ssl extension:
> +
> +Need:
> +
> +* Visual Studio 2010 SP1 or newer
> +* CPython source code (default branch: 3.5)
> +* perl binary: ActivePerl
> +* svn binary, ex: SilkSVN
> +* nasm and nmake binaries: compile NASM (install the binary doesn't
> install nmake)
> +
> +Read PCbuild/readme.txt.
> +
> +* Build Python (in debug mode)
> +* Type: PCbuild\get_externals.bat
> +* Type: PCbuild\win32\python_d.exe PCbuild\prepare_ssl.py
> externals\openssl-1.0.1j
> 
> Victor
>

From zachary.ware+pydev at gmail.com  Tue Jan 13 23:19:46 2015
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Tue, 13 Jan 2015 16:19:46 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAKJDb-Pw55wCQQt6Zi5xdAQi0pR6szRZ0PWs0hzDF=aMSVuqDg@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <CAKJDb-Pw55wCQQt6Zi5xdAQi0pR6szRZ0PWs0hzDF=aMSVuqDg@mail.gmail.com>
Message-ID: <CAKJDb-NP0KGKi8EmzLjCyNdP5LbsSn-RzC7Mp1THNVXWzK8HUg@mail.gmail.com>

On Tue, Jan 13, 2015 at 4:15 PM, Zachary Ware
<zachary.ware+pydev at gmail.com> wrote:
> """
> Quick Start Guide
> -----------------
>
> 1.  Install Microsoft Visual Studio 2015, any edition.

Note that this isn't precisely true; any VS 2010 SP1 or newer *should*
work, as you already know :).  This just says 2015 because that's the
'official' version to use.

-- 
Zach

From breamoreboy at yahoo.co.uk  Tue Jan 13 23:27:30 2015
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 13 Jan 2015 22:27:30 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
Message-ID: <m9464k$b6p$1@ger.gmane.org>

On 13/01/2015 22:04, Victor Stinner wrote:

> +* Type: PCbuild\win32\python_d.exe PCbuild\prepare_ssl.py
> externals\openssl-1.0.1j
>

See also http://bugs.python.org/issue23212 "Update Windows and OS X 
installer copies of OpenSSL to 1.0.1k"

-- 
My fellow Pythonistas, ask not what our language can do for you, ask
what you can do for our language.

Mark Lawrence


From victor.stinner at gmail.com  Tue Jan 13 23:36:47 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 13 Jan 2015 23:36:47 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>

2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
> Technically, Python 3.5 requires Visual Studio 2015

For me, it's *very* difficult to find how to install Visual Studio.
There are many different websites and web pages which mention Visual
Studio with a lot of versions and "flavors" (Express, Community,
Ultimate, etc.).

Visual Studio 2015 was not released yet :-/

My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
2008 Express and VS 2010 Express installed. I understood that
"Ultimate" includes a *lot* of things, not only a C compiler.

I found a "free" Visual Studio which is in fact Visual Studio 2013
Community and I read that it's not free.

I sent an email to Brian Curtin to ask to renew my MSDN account. He
didn't reply yet.

Victor

From brian at python.org  Tue Jan 13 23:42:41 2015
From: brian at python.org (Brian Curtin)
Date: Tue, 13 Jan 2015 16:42:41 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
Message-ID: <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>

On Tue, Jan 13, 2015 at 4:36 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> 2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
>> Technically, Python 3.5 requires Visual Studio 2015
>
> For me, it's *very* difficult to find how to install Visual Studio.
> There are many different websites and web pages which mention Visual
> Studio with a lot of versions and "flavors" (Express, Community,
> Ultimate, etc.).
>
> Visual Studio 2015 was not released yet :-/
>
> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
> 2008 Express and VS 2010 Express installed. I understood that
> "Ultimate" includes a *lot* of things, not only a C compiler.
>
> I found a "free" Visual Studio which is in fact Visual Studio 2013
> Community and I read that it's not free.
>
> I sent an email to Brian Curtin to ask to renew my MSDN account. He
> didn't reply yet.

I saw that and will send it on, but it's still going to take some time
to process - usually a week or so.

In the meantime, the first result searching for Visual Studio 2015
came up with http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx,
which seems to give you VS2015. I haven't tried to run it since I'm
not on Windows at the moment, but it looks correct.

From victor.stinner at gmail.com  Tue Jan 13 23:45:28 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 13 Jan 2015 23:45:28 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
Message-ID: <CAMpsgwZqip609LSv4C7d+XBjfidFBbmuPa0OYtn1-25gyzHhEQ@mail.gmail.com>

2015-01-13 23:42 GMT+01:00 Brian Curtin <brian at python.org>:
> In the meantime, the first result searching for Visual Studio 2015
> came up with http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx,
> which seems to give you VS2015. I haven't tried to run it since I'm
> not on Windows at the moment, but it looks correct.

I only see the "Ultime" flavor which contains a lot of things that I
don't need. It says that it requires 20 GB of disk, I don't have
enough free disk space (12 GB or something like that).

Is there a lighter flavor available?

If VS 2010 still works, I prefer to keep it right now.

Victor

From mal at egenix.com  Tue Jan 13 23:46:55 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 13 Jan 2015 23:46:55 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>	<DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>	<CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
Message-ID: <54B5A05F.1040208@egenix.com>

On 13.01.2015 23:42, Brian Curtin wrote:
> On Tue, Jan 13, 2015 at 4:36 PM, Victor Stinner
> <victor.stinner at gmail.com> wrote:
>> 2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
>>> Technically, Python 3.5 requires Visual Studio 2015
>>
>> For me, it's *very* difficult to find how to install Visual Studio.
>> There are many different websites and web pages which mention Visual
>> Studio with a lot of versions and "flavors" (Express, Community,
>> Ultimate, etc.).
>>
>> Visual Studio 2015 was not released yet :-/
>>
>> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
>> 2008 Express and VS 2010 Express installed. I understood that
>> "Ultimate" includes a *lot* of things, not only a C compiler.
>>
>> I found a "free" Visual Studio which is in fact Visual Studio 2013
>> Community and I read that it's not free.
>>
>> I sent an email to Brian Curtin to ask to renew my MSDN account. He
>> didn't reply yet.
> 
> I saw that and will send it on, but it's still going to take some time
> to process - usually a week or so.
> 
> In the meantime, the first result searching for Visual Studio 2015
> came up with http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx,
> which seems to give you VS2015. I haven't tried to run it since I'm
> not on Windows at the moment, but it looks correct.

Just a note of caution: for older preview releases of VS the
only way to get back to a clean system was to reinstall
Windows.

I don't know whether this will be different with VS 2015,
but if you care for your VM, you should probably create
a snapshot before installing VS 2015 preview to make it
easy to revert back, e.g. to install the final VS 2015
version.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...        http://egenix.com/go69

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From victor.stinner at gmail.com  Tue Jan 13 23:50:28 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 13 Jan 2015 23:50:28 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <54B5A05F.1040208@egenix.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
Message-ID: <CAMpsgwayO5w1=aoCmfO4DnCF2SszdHy0JxaoOHCNH8emVfrBig@mail.gmail.com>

2015-01-13 23:46 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
> Just a note of caution: for older preview releases of VS the
> only way to get back to a clean system was to reinstall
> Windows.

Does it mean that it's not possible to have VS 2008 and VS 2015
installed at the same time?

VS 2008 is required to build Python 2.7.

Victor

From Steve.Dower at microsoft.com  Tue Jan 13 23:44:05 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 13 Jan 2015 22:44:05 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
Message-ID: <DM2PR0301MB073475DD196208599368038CF5400@DM2PR0301MB0734.namprd03.prod.outlook.com>

Victor Stinner wrote:
> 2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
>> Technically, Python 3.5 requires Visual Studio 2015
> 
> For me, it's *very* difficult to find how to install Visual Studio.
> There are many different websites and web pages which mention Visual Studio with
> a lot of versions and "flavors" (Express, Community, Ultimate, etc.).

http://www.visualstudio.com/ is the only website to look at, and it may redirect you to somewhere on microsoft.com. Anywhere else is sketchy.

> Visual Studio 2015 was not released yet :-/

Yes, the timing is a little bit awkward, which is why I'm keen to make sure VS 2010 keeps working for now.

> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
> 2008 Express and VS 2010 Express installed. I understood that "Ultimate"
> includes a *lot* of things, not only a C compiler.

I appreciate this, and I'm constantly trying to get a version released that is just the compiler and nothing else. Unfortunately, I'm one lone low-level engineer, and releasing a different package like this is a high-level decision that I have no control over. Because VS 2015 is in preview, the only available version is Ultimate, but when the RC is released I'd expect to be able to choose the more specific versions.

> I found a "free" Visual Studio which is in fact Visual Studio 2013 Community and
> I read that it's not free.

Community Edition is certainly free (as in $0 - if you want to get ideological about "free" then you'll make it very hard for yourself to use any software produced by a business). The only requirement is that you log in with a Microsoft (a.k.a Live.com/Hotmail) account. It's a dumb requirement, but again, this is a high-level decision that I get no say in.

> I sent an email to Brian Curtin to ask to renew my MSDN account. He didn't reply
> yet.

Give him a chance, I'm sure he's just busy :)

> Victor

Cheers,
Steve

From victor.stinner at gmail.com  Wed Jan 14 00:02:20 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 14 Jan 2015 00:02:20 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAKJDb-Pw55wCQQt6Zi5xdAQi0pR6szRZ0PWs0hzDF=aMSVuqDg@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <CAKJDb-Pw55wCQQt6Zi5xdAQi0pR6szRZ0PWs0hzDF=aMSVuqDg@mail.gmail.com>
Message-ID: <CAMpsgwYGxZcccbpAuYsC9Bi992G99rZv6HXf+oDv5wiBTehgcw@mail.gmail.com>

2015-01-13 23:15 GMT+01:00 Zachary Ware <zachary.ware+pydev at gmail.com>:
> The first line of the section you linked to is "The readme included in
> the solution has more details, especially on what additional software
> is required to build which parts of Python.", and 'readme' is a link
> to the readme on h.p.o. :)

Ok, I didn't read the full section. But we should show the readme link
in bold or maybe in a "seealso" section. Right now, it's easy to miss
this important link.

> Quick Start Guide
> -----------------
>
> 1.  Install Microsoft Visual Studio 2015, any edition.
> 2.  Install Subversion, and make sure 'svn.exe' is on your PATH.
> 3.  Run "build.bat -e" to build Python in 32-bit Release configuration.

Oh wow, it's much simpler that my procedure :-) It worked for me, in a
few minutes I got a working "import ssl". Thanks. I'm now able to test
my asyncio patch on Windows ;-)
http://bugs.python.org/issue22560

(With this patch, it will be possible to use SSL with the proactor
(IOCP) event loop which is more efficient and supports more defaults
than the default selector event loop.)

Note: build.bat is in the PCbuild directory, and the command doesn't
need to be executed in the Visual Studio/Windows SDK shell. It works
in the classical cmd.exe shell.

Victor

From Steve.Dower at microsoft.com  Tue Jan 13 23:55:42 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Tue, 13 Jan 2015 22:55:42 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwayO5w1=aoCmfO4DnCF2SszdHy0JxaoOHCNH8emVfrBig@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwayO5w1=aoCmfO4DnCF2SszdHy0JxaoOHCNH8emVfrBig@mail.gmail.com>
Message-ID: <DM2PR0301MB073482E95F0371C2DCAADD1EF5400@DM2PR0301MB0734.namprd03.prod.outlook.com>

Victor Stinner wrote:
> 2015-01-13 23:46 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
>> Just a note of caution: for older preview releases of VS the only way 
>> to get back to a clean system was to reinstall Windows.
>
> Does it mean that it's not possible to have VS 2008 and VS 2015 installed at the same time?

It's possible, and that configuration will be fine.

Until VS 2015 RC is released, there's no guarantee that it will uninstall or upgrade cleanly. The testing so far looks like it'll be fine, but there's no guarantee.

There's also a chance that it will interfere with VS 2013, but any earlier versions will be fine.

Cheers,
Steve

> VS 2008 is required to build Python 2.7.
>
> Victor

From ethan at stoneleaf.us  Wed Jan 14 08:32:19 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 13 Jan 2015 23:32:19 -0800
Subject: [Python-Dev] C ABI and #ifndef Py_LIMITED_API
Message-ID: <54B61B83.4020105@stoneleaf.us>

In the CPython source code I see

#ifndef Py_LIMITED_API

Is there a section in the docs that explains the purpose?  If not, can someone give me the cliff notes version?

Many thanks.

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150113/89fca9b0/attachment.sig>

From g.brandl at gmx.net  Wed Jan 14 09:13:30 2015
From: g.brandl at gmx.net (Georg Brandl)
Date: Wed, 14 Jan 2015 09:13:30 +0100
Subject: [Python-Dev] C ABI and #ifndef Py_LIMITED_API
In-Reply-To: <54B61B83.4020105@stoneleaf.us>
References: <54B61B83.4020105@stoneleaf.us>
Message-ID: <m958fa$ngc$1@ger.gmane.org>

On 01/14/2015 08:32 AM, Ethan Furman wrote:
> In the CPython source code I see
> 
> #ifndef Py_LIMITED_API
> 
> Is there a section in the docs that explains the purpose?  If not, can
> someone give me the cliff notes version?

PEP 384, and in particular [1] should get you started.

cheers,
Georg

[1]
https://www.python.org/dev/peps/pep-0384/#header-files-and-preprocessor-definitions


From mal at egenix.com  Wed Jan 14 09:29:44 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 14 Jan 2015 09:29:44 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwayO5w1=aoCmfO4DnCF2SszdHy0JxaoOHCNH8emVfrBig@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>	<DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>	<CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>	<CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>	<54B5A05F.1040208@egenix.com>
 <CAMpsgwayO5w1=aoCmfO4DnCF2SszdHy0JxaoOHCNH8emVfrBig@mail.gmail.com>
Message-ID: <54B628F8.2000000@egenix.com>

On 13.01.2015 23:50, Victor Stinner wrote:
> 2015-01-13 23:46 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
>> Just a note of caution: for older preview releases of VS the
>> only way to get back to a clean system was to reinstall
>> Windows.
> 
> Does it mean that it's not possible to have VS 2008 and VS 2015
> installed at the same time?
> 
> VS 2008 is required to build Python 2.7.

Steve already answered this basically. It is well possible to
have multiple VS versions installed at the same time. Before
compiling you just need to run the right environment setup batch file
to prepare everything.

I was just referring to *preview* versions of VS. If you use
one of those, chances are that you'll have a hard time upgrading
to the final version of VS. My recommendation is not to use preview
versions on systems that you cannot easily revert to a state
before the preview install.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 14 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________
2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
2015-01-20: Python Meeting Duesseldorf ...        http://egenix.com/go69

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From msabramo at gmail.com  Wed Jan 14 17:10:53 2015
From: msabramo at gmail.com (Marc Abramowitz)
Date: Wed, 14 Jan 2015 08:10:53 -0800
Subject: [Python-Dev] Fix for `python setup.py check --restructuredtext`
Message-ID: <CAAgNOZ0F7D7CKSBtgDAtcCbg_cCrqfJjX4Z+ktcB+uvEAHzOrA@mail.gmail.com>

Looking for a core developer who can review a distutils change -- it's a
very small change:

http://bugs.python.org/issue23063

Let me know how I can help (e.g.: if I can review or test something to
offset time spent).

Cheers,
Marc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/60cca3f9/attachment.html>

From demianbrecht at gmail.com  Wed Jan 14 17:32:23 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 08:32:23 -0800
Subject: [Python-Dev] Overriding stdlib http package
Message-ID: <54B69A17.9060209@gmail.com>

Hi all,

As part of the work I'm doing on httplib3 (now that I've actually gotten
a bit of time), one of the things I'm trying to get done is injection of
httplib3 over http in order to not have to modify all import paths in
modules and such. Here's the gist of what I have so far:
https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.

It's greatly simplified over importlib2's inject mechanism, but I'm
assuming that's largely due to requirements of that package (i.e. Python
2) in contrast to this one.

My questions are: Does this look sane? Is there anything that I might be
not accounting for? It /does/ seem to work as expected when running
tests, but I'm curious if there's anything that I might be missing that
might jump out at someone more intimately familiar with the mechanics of
importlib.

Thanks,
Demian

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/63497b03/attachment.sig>

From graffatcolmingov at gmail.com  Wed Jan 14 17:37:12 2015
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Wed, 14 Jan 2015 10:37:12 -0600
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <54B69A17.9060209@gmail.com>
References: <54B69A17.9060209@gmail.com>
Message-ID: <CAN-Kwu2y91ut5Agssxi-+4LNZh_f6yg83+vzTy=w+OkMCJNrcg@mail.gmail.com>

I think this belongs on python-list, not python-dev.

On Wed, Jan 14, 2015 at 10:32 AM, Demian Brecht <demianbrecht at gmail.com> wrote:
> Hi all,
>
> As part of the work I'm doing on httplib3 (now that I've actually gotten
> a bit of time), one of the things I'm trying to get done is injection of
> httplib3 over http in order to not have to modify all import paths in
> modules and such. Here's the gist of what I have so far:
> https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.
>
> It's greatly simplified over importlib2's inject mechanism, but I'm
> assuming that's largely due to requirements of that package (i.e. Python
> 2) in contrast to this one.
>
> My questions are: Does this look sane? Is there anything that I might be
> not accounting for? It /does/ seem to work as expected when running
> tests, but I'm curious if there's anything that I might be missing that
> might jump out at someone more intimately familiar with the mechanics of
> importlib.
>
> Thanks,
> Demian
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com
>

From demianbrecht at gmail.com  Wed Jan 14 17:32:58 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 08:32:58 -0800
Subject: [Python-Dev] Overriding stdlib http package
Message-ID: <54B69A3A.6010901@gmail.com>

Hi all,

As part of the work I'm doing on httplib3 (now that I've actually gotten
a bit of time), one of the things I'm trying to get done is injection of
httplib3 over http in order to not have to modify all import paths in
modules and such. Here's the gist of what I have so far:
https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.

It's greatly simplified over importlib2's inject mechanism, but I'm
assuming that's largely due to requirements of that package (i.e. Python
2) in contrast to this one.

My question is: Does this look sane? Is there anything that I might be
not accounting for? It /does/ seem to work as expected when running
tests, but I'm curious if there's anything that I might be missing that
might jump out at someone more intimately familiar with the mechanics of
importlib.

Thanks,
Demian


-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/3e719476/attachment.sig>

From demianbrecht at gmail.com  Wed Jan 14 17:39:45 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 08:39:45 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <54B69A3A.6010901@gmail.com>
References: <54B69A3A.6010901@gmail.com>
Message-ID: <54B69BD1.8080607@gmail.com>

Apologies for the double send, apparently Thunderbird got confused when
going through a 4G dead zone.

I should mention that I'm aware that any module that previously imported
the stdlib version would retain that version rather than a reference to
the new one, but I'm okay with that as this is a very specific use case:
Having imports across tests and package modules use httplib3 to
facilitate merging changes back upstream.


On 2015-01-14 8:32 AM, Demian Brecht wrote:
> Hi all,
> 
> As part of the work I'm doing on httplib3 (now that I've actually gotten
> a bit of time), one of the things I'm trying to get done is injection of
> httplib3 over http in order to not have to modify all import paths in
> modules and such. Here's the gist of what I have so far:
> https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.
> 
> It's greatly simplified over importlib2's inject mechanism, but I'm
> assuming that's largely due to requirements of that package (i.e. Python
> 2) in contrast to this one.
> 
> My question is: Does this look sane? Is there anything that I might be
> not accounting for? It /does/ seem to work as expected when running
> tests, but I'm curious if there's anything that I might be missing that
> might jump out at someone more intimately familiar with the mechanics of
> importlib.
> 
> Thanks,
> Demian
> 
> 

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/3ac7a011/attachment.sig>

From ethan at stoneleaf.us  Wed Jan 14 17:24:15 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Wed, 14 Jan 2015 08:24:15 -0800
Subject: [Python-Dev] C ABI and #ifndef Py_LIMITED_API
In-Reply-To: <m958fa$ngc$1@ger.gmane.org>
References: <54B61B83.4020105@stoneleaf.us> <m958fa$ngc$1@ger.gmane.org>
Message-ID: <54B6982F.5080403@stoneleaf.us>

On 01/14/2015 12:13 AM, Georg Brandl wrote:
> On 01/14/2015 08:32 AM, Ethan Furman wrote:
>>
>> In the CPython source code I see
>>
>> #ifndef Py_LIMITED_API
>>
>> Is there a section in the docs that explains the purpose?  If not, can
>> someone give me the cliff notes version?
> 
> PEP 384, and in particular [1] should get you started.
> 
> [1]
> https://www.python.org/dev/peps/pep-0384/#header-files-and-preprocessor-definitions

Nice, that answered other questions as well!  :)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/e495fec8/attachment.sig>

From demianbrecht at gmail.com  Wed Jan 14 17:49:56 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 16:49:56 +0000
Subject: [Python-Dev] Overriding stdlib http package
References: <54B69A17.9060209@gmail.com>
 <CAN-Kwu2y91ut5Agssxi-+4LNZh_f6yg83+vzTy=w+OkMCJNrcg@mail.gmail.com>
 <CAE+T62aZ_HNS52izqHfWkFEACgdiNjZ6pawwAmjqSVy=KMaBew@mail.gmail.com>
Message-ID: <CAE+T62aV+tk4c7PK4h7AiajnFNL2W6niGfEvLzd4DUCg7J-BjA@mail.gmail.com>

I had considered that but thought that dev might be more appropriate as
it's related to overriding a stdlib module in order to work on that module
out of band with cpython (with the intention of merging back upstream). I
would imagine those on the dev list may be better suited to answer.

On Wed, Jan 14, 2015, 08:37 Ian Cordasco <graffatcolmingov at gmail.com> wrote:

I think this belongs on python-list, not python-dev.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/dbcfbf0d/attachment.html>

From solipsis at pitrou.net  Wed Jan 14 17:54:47 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 14 Jan 2015 17:54:47 +0100
Subject: [Python-Dev] Overriding stdlib http package
References: <54B69A17.9060209@gmail.com>
Message-ID: <20150114175447.5d736cb4@fsol>

On Wed, 14 Jan 2015 08:32:23 -0800
Demian Brecht <demianbrecht at gmail.com> wrote:
> Hi all,
> 
> As part of the work I'm doing on httplib3 (now that I've actually gotten
> a bit of time), one of the things I'm trying to get done is injection of
> httplib3 over http in order to not have to modify all import paths in
> modules and such. Here's the gist of what I have so far:
> https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.

What don't you simply monkeypatch sys.modules, e.g.:

  import myhttplib

  sys.modules['http'] = myhttplib

or doesn't it work as desired?

Regards

Antoine.



From tseaver at palladion.com  Wed Jan 14 18:04:22 2015
From: tseaver at palladion.com (Tres Seaver)
Date: Wed, 14 Jan 2015 12:04:22 -0500
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <20150114175447.5d736cb4@fsol>
References: <54B69A17.9060209@gmail.com> <20150114175447.5d736cb4@fsol>
Message-ID: <m967im$r4n$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/14/2015 11:54 AM, Antoine Pitrou wrote:
> On Wed, 14 Jan 2015 08:32:23 -0800 Demian Brecht
> <demianbrecht at gmail.com> wrote:
>> Hi all,
>> 
>> As part of the work I'm doing on httplib3 (now that I've actually
>> gotten a bit of time), one of the things I'm trying to get done is
>> injection of httplib3 over http in order to not have to modify all
>> import paths in modules and such. Here's the gist of what I have so
>> far: https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.
> 
> What don't you simply monkeypatch sys.modules, e.g.:
> 
> import myhttplib
> 
> sys.modules['http'] = myhttplib
> 
> or doesn't it work as desired?

Doesn't that leave any prior imports broken (using the original module)?


Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlS2oZYACgkQ+gerLs4ltQ457gCfTSuwfOUHOivoQAUncq6VbxdQ
YOkAoLec1hghar8IULuaz5W0MTXOtQm/
=tvv7
-----END PGP SIGNATURE-----


From solipsis at pitrou.net  Wed Jan 14 18:20:41 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 14 Jan 2015 18:20:41 +0100
Subject: [Python-Dev] Overriding stdlib http package
References: <54B69A17.9060209@gmail.com> <20150114175447.5d736cb4@fsol>
 <m967im$r4n$1@ger.gmane.org>
Message-ID: <20150114182041.1f59896e@fsol>

On Wed, 14 Jan 2015 12:04:22 -0500
Tres Seaver <tseaver at palladion.com> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
> 
> On 01/14/2015 11:54 AM, Antoine Pitrou wrote:
> > On Wed, 14 Jan 2015 08:32:23 -0800 Demian Brecht
> > <demianbrecht at gmail.com> wrote:
> >> Hi all,
> >> 
> >> As part of the work I'm doing on httplib3 (now that I've actually
> >> gotten a bit of time), one of the things I'm trying to get done is
> >> injection of httplib3 over http in order to not have to modify all
> >> import paths in modules and such. Here's the gist of what I have so
> >> far: https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.
> > 
> > What don't you simply monkeypatch sys.modules, e.g.:
> > 
> > import myhttplib
> > 
> > sys.modules['http'] = myhttplib
> > 
> > or doesn't it work as desired?
> 
> Doesn't that leave any prior imports broken (using the original module)?

Not sure. Any fiddling with the import system is better done at
startup, anyway.

Regards

Antoine.



From demianbrecht at gmail.com  Wed Jan 14 18:37:21 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 09:37:21 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <20150114175447.5d736cb4@fsol>
References: <54B69A17.9060209@gmail.com> <20150114175447.5d736cb4@fsol>
Message-ID: <54B6A951.70002@gmail.com>

Hm, I /did/ try that but ran into issues. Swapping the custom finder for
the monkey patch now seems to work as expected though. Could be that I
was doing something else at the time that caused it not to work.

I'll keep running with that and will ping the thread if the issues
surface again.

Thanks!

On 2015-01-14 8:54 AM, Antoine Pitrou wrote:
> What don't you simply monkeypatch sys.modules, e.g.:
> 
>   import myhttplib
> 
>   sys.modules['http'] = myhttplib
> 
> or doesn't it work as desired?

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/e5656ca4/attachment.sig>

From contact at ionelmc.ro  Wed Jan 14 19:38:26 2015
From: contact at ionelmc.ro (=?UTF-8?Q?Ionel_Cristian_M=C4=83rie=C8=99?=)
Date: Wed, 14 Jan 2015 20:38:26 +0200
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <54B69A17.9060209@gmail.com>
References: <54B69A17.9060209@gmail.com>
Message-ID: <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>

You could do the sys.modules patch as Antoine suggested in a .pth file, so
that it's triggered at startup.
Eg, very similar:
https://github.com/xando/subprocess.run/blob/ab02d165802b2ad57dd0d16c1169ab05ed312ef1/subprocess.run.pth


Thanks,
-- Ionel Cristian M?rie?, blog.ionelmc.ro

On Wed, Jan 14, 2015 at 6:32 PM, Demian Brecht <demianbrecht at gmail.com>
wrote:

> Hi all,
>
> As part of the work I'm doing on httplib3 (now that I've actually gotten
> a bit of time), one of the things I'm trying to get done is injection of
> httplib3 over http in order to not have to modify all import paths in
> modules and such. Here's the gist of what I have so far:
> https://gist.github.com/demianbrecht/bc6530a40718e4fcbf90.
>
> It's greatly simplified over importlib2's inject mechanism, but I'm
> assuming that's largely due to requirements of that package (i.e. Python
> 2) in contrast to this one.
>
> My questions are: Does this look sane? Is there anything that I might be
> not accounting for? It /does/ seem to work as expected when running
> tests, but I'm curious if there's anything that I might be missing that
> might jump out at someone more intimately familiar with the mechanics of
> importlib.
>
> Thanks,
> Demian
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/contact%40ionelmc.ro
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/0258627b/attachment-0001.html>

From guido at python.org  Wed Jan 14 20:35:56 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Jan 2015 11:35:56 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
Message-ID: <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>

Why do you want to hack the existing http modules?

This is not a rhetorical question. The answer may lead us to redesign the
existing http modules to be more flexible so that the higher-level problem
you are trying to solve by hacking http import can be solved instead by
using an interface provided by the stdlib http module.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/3e369496/attachment.html>

From demianbrecht at gmail.com  Wed Jan 14 21:07:53 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 12:07:53 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
Message-ID: <54B6CC99.2040501@gmail.com>

On 2015-01-14 11:35 AM, Guido van Rossum wrote:
> Why do you want to hack the existing http modules?
> 
> This is not a rhetorical question. The answer may lead us to redesign the
> existing http modules to be more flexible so that the higher-level problem
> you are trying to solve by hacking http import can be solved instead by
> using an interface provided by the stdlib http module.

Sorry, this venture began in core-mentorship, so a little context may be
of use: My end goal is to become a maintainer of the http package.
As I'm not a core dev, Nick had suggested making a friendly fork of the
package in order to facilitate progress without being bound to the
non-core dev contributor workflow (it can, at times, be a little painful
getting reviews and such completed on orphaned packages).

So, the question that I was trying to answer isn't directly related to
the http package in particular, but how to override stdlib modules in
general with a third party package in order to facilitate out of band
development while making minimal changes to package code (i.e. changing
all absolute import package names in test and module code) to ease
upstream merging.

That all said, this would likely be a moot issue if I had commit
privileges ;) But it might be nice to figure out a good workflow should
this come up again with any other new contributors looking to take
ownership of an orphaned module.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/83ce11ae/attachment.sig>

From guido at python.org  Wed Jan 14 21:25:15 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 14 Jan 2015 12:25:15 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <54B6CC99.2040501@gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
Message-ID: <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>

Aha. Glad I asked. You would arguably get a more useful response if you
asked on core-mentorship and explained some of that background (for those
of us who rely on external memory :-).

The stdlib intentionally makes what you are trying to do hard (so library
writers don't have to worry about stdlib modules being overridden with
hacks at the whim of other library writers or app writers).

I'm not sure how commit privileges would help you -- can't you just fork
the CPython (I'm sure there's already a Bitbucket mirror that you can fork
easily) and do your work there? Even with commit privileges you wouldn't be
committing partial work unreviewed.

On Wed, Jan 14, 2015 at 12:07 PM, Demian Brecht <demianbrecht at gmail.com>
wrote:

> On 2015-01-14 11:35 AM, Guido van Rossum wrote:
> > Why do you want to hack the existing http modules?
> >
> > This is not a rhetorical question. The answer may lead us to redesign the
> > existing http modules to be more flexible so that the higher-level
> problem
> > you are trying to solve by hacking http import can be solved instead by
> > using an interface provided by the stdlib http module.
>
> Sorry, this venture began in core-mentorship, so a little context may be
> of use: My end goal is to become a maintainer of the http package.
> As I'm not a core dev, Nick had suggested making a friendly fork of the
> package in order to facilitate progress without being bound to the
> non-core dev contributor workflow (it can, at times, be a little painful
> getting reviews and such completed on orphaned packages).
>
> So, the question that I was trying to answer isn't directly related to
> the http package in particular, but how to override stdlib modules in
> general with a third party package in order to facilitate out of band
> development while making minimal changes to package code (i.e. changing
> all absolute import package names in test and module code) to ease
> upstream merging.
>
> That all said, this would likely be a moot issue if I had commit
> privileges ;) But it might be nice to figure out a good workflow should
> this come up again with any other new contributors looking to take
> ownership of an orphaned module.
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/24b878a9/attachment.html>

From demianbrecht at gmail.com  Wed Jan 14 22:00:03 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 13:00:03 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
Message-ID: <54B6D8D3.4090706@gmail.com>

On 2015-01-14 12:25 PM, Guido van Rossum wrote:
> I'm not sure how commit privileges would help you -- can't you just fork
> the CPython (I'm sure there's already a Bitbucket mirror that you can fork
> easily) and do your work there? Even with commit privileges you wouldn't be
> committing partial work unreviewed.

The friendly module fork allows for others to easily (or at least the
intention is to do it easily) use the module with the new, backwards
compatible features as a drop in replacement for the stdlib module.
Giving others the ability to do this would lend itself to the adoption
of the module and bug reports and such before upstream patches are produced.

That said, the main downside to the friendly fork is the patch
submission process: After changes have been merged to the fork, there's
bound to be churn during the upstream patch submission, which would
likely lead to something that looks like:

> Implement feature/bug fix [1]
> Commit changes to httlib3
> Generate patch for CPython
> Import patch to local CPython
> Run unit tests [1]
> Generate hg patch (patchA) for submission to bug tracker
> Upload patchA
> patchA is reviewed
> Implement review changes and generate patchB [1]
> Upload patchB
> [...wait for merge...]
> Merge delta of patchB and patchA to httplib3
> Test/upload new PyPI package

I see commit privileges helping in two ways:

1. I've experienced lag on a few occasions between review and merge. I'm
assuming that this is largely due to a lack of dotted line maintainer of
the http package (although I believe that the general consensus is that
Senthil is the de facto maintainer of the package). Commit privileges
would help in getting the patches merged once reviews are complete.

2. It would help my own workflow. While feature development can be done
in httplib3, I do also tend to swap between issues in the bug tracker
and large feature work. Because I have two lines of work (CPython/bug
tracker and Github), I run into issues around where these changes should
be made: Should the bug fixes live in CPython/bug tracker or should I
fix the issue in httplib3 and go through the submission workflow above?
Either way, I'm signing myself up for a good deal of headache managing
the httplib3 work, especially when development work across feature
branches is dependent on patches submitted to CPython.


I definitely don't mind the extra work if there are no other options,
but my end goal is to be a maintainer of the http package and core
developer, not to maintain a third party fork.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/1df18b36/attachment.sig>

From brett at python.org  Wed Jan 14 22:19:07 2015
From: brett at python.org (Brett Cannon)
Date: Wed, 14 Jan 2015 21:19:07 +0000
Subject: [Python-Dev] Overriding stdlib http package
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
Message-ID: <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>

On Wed Jan 14 2015 at 4:08:52 PM Demian Brecht <demianbrecht at gmail.com>
wrote:

> On 2015-01-14 12:25 PM, Guido van Rossum wrote:
> > I'm not sure how commit privileges would help you -- can't you just fork
> > the CPython (I'm sure there's already a Bitbucket mirror that you can
> fork
> > easily) and do your work there? Even with commit privileges you wouldn't
> be
> > committing partial work unreviewed.
>
> The friendly module fork allows for others to easily (or at least the
> intention is to do it easily) use the module with the new, backwards
> compatible features as a drop in replacement for the stdlib module.
>

But as Guido pointed out, we _like_ it being difficult to do because we
don't want this kind of substitution happening as code ends up depending on
bugs and quirks that you may fix.


> Giving others the ability to do this would lend itself to the adoption
> of the module and bug reports and such before upstream patches are
> produced.
>
> That said, the main downside to the friendly fork is the patch
> submission process: After changes have been merged to the fork, there's
> bound to be churn during the upstream patch submission, which would
> likely lead to something that looks like:
>
> > Implement feature/bug fix [1]
> > Commit changes to httlib3
> > Generate patch for CPython
> > Import patch to local CPython
> > Run unit tests [1]
> > Generate hg patch (patchA) for submission to bug tracker
> > Upload patchA
> > patchA is reviewed
> > Implement review changes and generate patchB [1]
> > Upload patchB
> > [...wait for merge...]
> > Merge delta of patchB and patchA to httplib3
> > Test/upload new PyPI package
>
> I see commit privileges helping in two ways:
>
> 1. I've experienced lag on a few occasions between review and merge. I'm
> assuming that this is largely due to a lack of dotted line maintainer of
> the http package (although I believe that the general consensus is that
> Senthil is the de facto maintainer of the package). Commit privileges
> would help in getting the patches merged once reviews are complete.
>
> 2. It would help my own workflow. While feature development can be done
> in httplib3, I do also tend to swap between issues in the bug tracker
> and large feature work. Because I have two lines of work (CPython/bug
> tracker and Github), I run into issues around where these changes should
> be made: Should the bug fixes live in CPython/bug tracker or should I
> fix the issue in httplib3 and go through the submission workflow above?
> Either way, I'm signing myself up for a good deal of headache managing
> the httplib3 work, especially when development work across feature
> branches is dependent on patches submitted to CPython.
>
>
> I definitely don't mind the extra work if there are no other options,
> but my end goal is to be a maintainer of the http package and core
> developer, not to maintain a third party fork.
>

How many other modules are dependent on the http module in the stdlib that
are going to be affected by your changes? One option is you fork http
**and** and modules in the stdlib that are dependent on it. You don't
really have to change the other modules beyond their import statement of
using http -- you can even do `import http3 as http` or something to
minimize the changes -- but you at least don't have to monkeypatch
sys.modules for others to gain from your http changes. Plus as you patch
stuff in http you may find you have/want to patch other dependent modules
as well and so you will have already done that.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/9947611b/attachment.html>

From donald at stufft.io  Wed Jan 14 22:35:30 2015
From: donald at stufft.io (Donald Stufft)
Date: Wed, 14 Jan 2015 16:35:30 -0500
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
Message-ID: <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>


> On Jan 14, 2015, at 4:19 PM, Brett Cannon <brett at python.org> wrote:
> 
> But as Guido pointed out, we _like_ it being difficult to do because we don't want this kind of substitution happening as code ends up depending on bugs and quirks that you may fix.


Not all of us, I hate the default order of sys.path.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/c6e81fab/attachment.html>

From demianbrecht at gmail.com  Wed Jan 14 22:58:13 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Wed, 14 Jan 2015 13:58:13 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
Message-ID: <54B6E675.8020209@gmail.com>

On 2015-01-14 1:19 PM, Brett Cannon wrote:
> But as Guido pointed out, we _like_ it being difficult to do because we
> don't want this kind of substitution happening as code ends up depending on
> bugs and quirks that you may fix.

I can understand the reasoning.

> How many other modules are dependent on the http module in the stdlib that
> are going to be affected by your changes? One option is you fork http
> **and** and modules in the stdlib that are dependent on it. You don't
> really have to change the other modules beyond their import statement of
> using http -- you can even do `import http3 as http` or something to
> minimize the changes -- but you at least don't have to monkeypatch
> sys.modules for others to gain from your http changes. Plus as you patch
> stuff in http you may find you have/want to patch other dependent modules
> as well and so you will have already done that.

It looks like there are 5 other modules dependent on the http package.
If I understand what you're proposing, it pretty much defeats the
purpose of what I'm trying to accomplish with a standalone httplib3 package.

That said, considering the points that you and Guido have both made, I
think that the best course of action is to either just fork CPython as a
whole or to continue with httplib3 but abandon overriding sys.modules,
develop features detached from the stdlib and worry about fixing
dependencies when integrating changes upstream.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/c4eda66d/attachment.sig>

From larry at hastings.org  Thu Jan 15 05:02:24 2015
From: larry at hastings.org (Larry Hastings)
Date: Wed, 14 Jan 2015 23:02:24 -0500
Subject: [Python-Dev] Schedule for 3.4.3, revised schedule for 3.5.0a1
Message-ID: <54B73BD0.8000207@hastings.org>



Python 3.5.0a1 is currently scheduled to be released February 1. Since 
I'll be on the road that day, the 3.5 team has agreed to push the 
release back a week.  3.5.0a1 will be tagged Saturday February 7 and 
released Sunday February 8.  This doesn't change any of the other 
release dates for 3.5..

Since it's about time for a 3.4.3 anyway, we're going to push that out 
at the same time.  3.4.3rc1 will be tagged Saturday February 7 and 
released Sunday February 8.  3.4.3 final will follow two weeks later, 
tagged Saturday February 21 and released Sunday February 22.

Get your bug fixes (3.4) and crazy new functionality (3.5) in now!


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150114/a7933056/attachment-0001.html>

From brett at python.org  Thu Jan 15 17:30:52 2015
From: brett at python.org (Brett Cannon)
Date: Thu, 15 Jan 2015 16:30:52 +0000
Subject: [Python-Dev] Overriding stdlib http package
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
 <54B6E675.8020209@gmail.com>
Message-ID: <CAP1=2W6Ck94JgNXM62iOnZDqdJBP2-G6JPMtsPa-ZK9PLHHRgQ@mail.gmail.com>

On Wed Jan 14 2015 at 4:58:20 PM Demian Brecht <demianbrecht at gmail.com>
wrote:

> On 2015-01-14 1:19 PM, Brett Cannon wrote:
> > But as Guido pointed out, we _like_ it being difficult to do because we
> > don't want this kind of substitution happening as code ends up depending
> on
> > bugs and quirks that you may fix.
>
> I can understand the reasoning.
>
> > How many other modules are dependent on the http module in the stdlib
> that
> > are going to be affected by your changes? One option is you fork http
> > **and** and modules in the stdlib that are dependent on it. You don't
> > really have to change the other modules beyond their import statement of
> > using http -- you can even do `import http3 as http` or something to
> > minimize the changes -- but you at least don't have to monkeypatch
> > sys.modules for others to gain from your http changes. Plus as you patch
> > stuff in http you may find you have/want to patch other dependent modules
> > as well and so you will have already done that.
>
> It looks like there are 5 other modules dependent on the http package.
> If I understand what you're proposing, it pretty much defeats the
> purpose of what I'm trying to accomplish with a standalone httplib3
> package.
>
> That said, considering the points that you and Guido have both made, I
> think that the best course of action is to either just fork CPython as a
> whole or to continue with httplib3 but abandon overriding sys.modules,
> develop features detached from the stdlib and worry about fixing
> dependencies when integrating changes upstream.
>

If I were you I would fork and then for bugfixes send them upstream to us
while you develop API additions independently. That way if your fork gains
traction you can come to us and say "my fork has a stable API, has existed
for (at least) a year, and the community seems to have rallied behind it",
at which point we can look at drawing it in. And if you fix enough bugs we
might make you maintainer anyway while you work out API design with the
community outside of the stdlib.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150115/953169c4/attachment.html>

From victor.stinner at gmail.com  Thu Jan 15 22:30:31 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 15 Jan 2015 22:30:31 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <54B5A05F.1040208@egenix.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
Message-ID: <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>

Hi,

I installed the SP1 for Visual Studio 2010, and it looks like that it
broke my Windows SDK 7.1 (setenv was missing, cl.exe was also
missing). I uninstalled the SDK 7.1, and then I saw that a patch is
required to use Windows SDK 7.1 with Visual Studio 2010 SP1. Ah. Too
late.

I don't understand the link between the SDK and Visual Studio. There
are not separated directories?

And now I cannot find Windows SDK 7.1 anymore. It looks like it
disappeared from microsoft.com. The SDK 7.1 was released in 2010, so
it's now quite old, but it worked well!

Can I use the SDK 8.0 or 8.1 to build Python extensions for Python 3.3 and 3.4?

It took me several hours to have a working platform to build my Python
extensions for Python 2.7, 3.3 and 3.4, in 32 and 64 bits with
automated scripts to run all commands. And now it doesn't work anymore
:-(

Victor

2015-01-13 23:46 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
> On 13.01.2015 23:42, Brian Curtin wrote:
>> On Tue, Jan 13, 2015 at 4:36 PM, Victor Stinner
>> <victor.stinner at gmail.com> wrote:
>>> 2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
>>>> Technically, Python 3.5 requires Visual Studio 2015
>>>
>>> For me, it's *very* difficult to find how to install Visual Studio.
>>> There are many different websites and web pages which mention Visual
>>> Studio with a lot of versions and "flavors" (Express, Community,
>>> Ultimate, etc.).
>>>
>>> Visual Studio 2015 was not released yet :-/
>>>
>>> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
>>> 2008 Express and VS 2010 Express installed. I understood that
>>> "Ultimate" includes a *lot* of things, not only a C compiler.
>>>
>>> I found a "free" Visual Studio which is in fact Visual Studio 2013
>>> Community and I read that it's not free.
>>>
>>> I sent an email to Brian Curtin to ask to renew my MSDN account. He
>>> didn't reply yet.
>>
>> I saw that and will send it on, but it's still going to take some time
>> to process - usually a week or so.
>>
>> In the meantime, the first result searching for Visual Studio 2015
>> came up with http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx,
>> which seems to give you VS2015. I haven't tried to run it since I'm
>> not on Windows at the moment, but it looks correct.
>
> Just a note of caution: for older preview releases of VS the
> only way to get back to a clean system was to reinstall
> Windows.
>
> I don't know whether this will be different with VS 2015,
> but if you care for your VM, you should probably create
> a snapshot before installing VS 2015 preview to make it
> easy to revert back, e.g. to install the final VS 2015
> version.
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 13 2015)
>>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
> 2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
> 2015-01-20: Python Meeting Duesseldorf ...        http://egenix.com/go69
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/

From rymg19 at gmail.com  Thu Jan 15 22:39:28 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 15 Jan 2015 15:39:28 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
Message-ID: <CAO41-mPDTAwcBUzn4-9s9Udkt0jVkNfCnGCj8+QKmcwuCL=2bA@mail.gmail.com>

On Thu, Jan 15, 2015 at 3:30 PM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> Hi,
>
> I installed the SP1 for Visual Studio 2010, and it looks like that it
> broke my Windows SDK 7.1 (setenv was missing, cl.exe was also
> missing). I uninstalled the SDK 7.1, and then I saw that a patch is
> required to use Windows SDK 7.1 with Visual Studio 2010 SP1. Ah. Too
> late.
>
> I don't understand the link between the SDK and Visual Studio. There
> are not separated directories?
>
> And now I cannot find Windows SDK 7.1 anymore. It looks like it
> disappeared from microsoft.com. The SDK 7.1 was released in 2010, so
> it's now quite old, but it worked well!
>

http://www.mathworks.com/matlabcentral/answers/101105-how-do-i-install-microsoft-windows-sdk-7-1
http://www.microsoft.com/en-us/download/details.aspx?id=8279

Both are about Windows SDK 7.1. The latter is a download link; the former
says what to do if you have VS 2010 installed.


>
> Can I use the SDK 8.0 or 8.1 to build Python extensions for Python 3.3 and
> 3.4?
>
> It took me several hours to have a working platform to build my Python
> extensions for Python 2.7, 3.3 and 3.4, in 32 and 64 bits with
> automated scripts to run all commands. And now it doesn't work anymore
> :-(
>
> Victor
>
> 2015-01-13 23:46 GMT+01:00 M.-A. Lemburg <mal at egenix.com>:
> > On 13.01.2015 23:42, Brian Curtin wrote:
> >> On Tue, Jan 13, 2015 at 4:36 PM, Victor Stinner
> >> <victor.stinner at gmail.com> wrote:
> >>> 2015-01-13 23:18 GMT+01:00 Steve Dower <Steve.Dower at microsoft.com>:
> >>>> Technically, Python 3.5 requires Visual Studio 2015
> >>>
> >>> For me, it's *very* difficult to find how to install Visual Studio.
> >>> There are many different websites and web pages which mention Visual
> >>> Studio with a lot of versions and "flavors" (Express, Community,
> >>> Ultimate, etc.).
> >>>
> >>> Visual Studio 2015 was not released yet :-/
> >>>
> >>> My VM has only a disk of 40 GB. Only 12 GB are free. I already have VS
> >>> 2008 Express and VS 2010 Express installed. I understood that
> >>> "Ultimate" includes a *lot* of things, not only a C compiler.
> >>>
> >>> I found a "free" Visual Studio which is in fact Visual Studio 2013
> >>> Community and I read that it's not free.
> >>>
> >>> I sent an email to Brian Curtin to ask to renew my MSDN account. He
> >>> didn't reply yet.
> >>
> >> I saw that and will send it on, but it's still going to take some time
> >> to process - usually a week or so.
> >>
> >> In the meantime, the first result searching for Visual Studio 2015
> >> came up with
> http://www.visualstudio.com/en-us/downloads/visual-studio-2015-downloads-vs.aspx
> ,
> >> which seems to give you VS2015. I haven't tried to run it since I'm
> >> not on Windows at the moment, but it looks correct.
> >
> > Just a note of caution: for older preview releases of VS the
> > only way to get back to a clean system was to reinstall
> > Windows.
> >
> > I don't know whether this will be different with VS 2015,
> > but if you care for your VM, you should probably create
> > a snapshot before installing VS 2015 preview to make it
> > easy to revert back, e.g. to install the final VS 2015
> > version.
> >
> > --
> > Marc-Andre Lemburg
> > eGenix.com
> >
> > Professional Python Services directly from the Source  (#1, Jan 13 2015)
> >>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> > ________________________________________________________________________
> > 2015-01-09: Released eGenix pyOpenSSL 0.13.7 ...  http://egenix.com/go68
> > 2015-01-20: Python Meeting Duesseldorf ...        http://egenix.com/go69
> >
> > ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
> >
> >    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
> >     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
> >            Registered at Amtsgericht Duesseldorf: HRB 46611
> >                http://www.egenix.com/company/contact/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150115/52f07e60/attachment.html>

From victor.stinner at gmail.com  Thu Jan 15 22:58:29 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 15 Jan 2015 22:58:29 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAO41-mPDTAwcBUzn4-9s9Udkt0jVkNfCnGCj8+QKmcwuCL=2bA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
 <CAO41-mPDTAwcBUzn4-9s9Udkt0jVkNfCnGCj8+QKmcwuCL=2bA@mail.gmail.com>
Message-ID: <CAMpsgwZs0vhH3HeEwYkoZuddD3rT-PPTFLw+WzAQyY8MzAuxHg@mail.gmail.com>

2015-01-15 22:39 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
> http://www.microsoft.com/en-us/download/details.aspx?id=8279

"Microsoft Windows SDK for Windows 7 and .NET Framework 4"

Are you sure that it is SDK 7.1, and not 7.0?

--

The SDK 7.0 works for Python 2.7 which is compiled with Visual Studio 2008.

I used the SDK 7.1 for Python 3.3 and 3.4 which are compiled with
Visual Studio 2010.

It looks likt SDK 8 is more for Visual Studio 2012.

If you use the wrong SDK, you will depend on a "MSVCRxxx.dll" which is
not provided by Python x.x (ex: MSVCR100.dll for SDK 7.1/Python 3.3 &
3.4).

Victor

From zachary.ware+pydev at gmail.com  Thu Jan 15 23:26:09 2015
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Thu, 15 Jan 2015 16:26:09 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
Message-ID: <CAKJDb-M4WNmt9CVbTJFygQpa6qO0o2tCog1znaRxLGj7gLLZoA@mail.gmail.com>

On Thu, Jan 15, 2015 at 3:30 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Hi,
>
> I installed the SP1 for Visual Studio 2010, and it looks like that it
> broke my Windows SDK 7.1 (setenv was missing, cl.exe was also
> missing). I uninstalled the SDK 7.1, and then I saw that a patch is
> required to use Windows SDK 7.1 with Visual Studio 2010 SP1. Ah. Too
> late.

Doing a 'repair' on VS2010 might get the SDK back for you; I'm not
sure.  I believe this link should be what's needed to fix the SDK
after installing VS2010SP1:
http://www.microsoft.com/en-us/download/details.aspx?id=4422

> I don't understand the link between the SDK and Visual Studio. There
> are not separated directories?

If I'm not mistaken, Visual Studio uses the SDK for all of its
building and will install its preferred SDK as a dependency.  It can
use other SDKs as well, though.

> And now I cannot find Windows SDK 7.1 anymore. It looks like it
> disappeared from microsoft.com. The SDK 7.1 was released in 2010, so
> it's now quite old, but it worked well!

Web installer:
http://www.microsoft.com/en-us/download/details.aspx?id=8279
ISOs:
http://www.microsoft.com/en-us/download/details.aspx?id=8442

Extension building in general is still a mess on Windows, I hope the
links above are enough to get you going again!

-- 
Zach

From p.f.moore at gmail.com  Thu Jan 15 23:34:53 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 15 Jan 2015 22:34:53 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAKJDb-M4WNmt9CVbTJFygQpa6qO0o2tCog1znaRxLGj7gLLZoA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
 <CAKJDb-M4WNmt9CVbTJFygQpa6qO0o2tCog1znaRxLGj7gLLZoA@mail.gmail.com>
Message-ID: <CACac1F_3vd4Ter0VDcjXKA-iKAtZX8OdiBrQdBHLwL-hmTC8LA@mail.gmail.com>

On 15 January 2015 at 22:26, Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
> Extension building in general is still a mess on Windows, I hope the
> links above are enough to get you going again!

For building extensions, I have a powershell script that, starting
with a clean machine, downloads and installs everything needed to
build extensions for Python 2.7-3.4 (Python, 32 and 64-bit, SDK
compilers and Visual C for Python 2.7, and some support packages).
It's available at https://github.com/pfmoore/pybuild It's pretty
fragile (largely because the SDK installs are pretty fragile, but also
because it doesn't check if things it wants to install are already
there), but it's good for setting up a new VM from scratch.

It isn't designed for building Python, and I've no idea how well it
would work for that. But you might be able to pick out some parts of
it that would be useful (if nothing else, it includes direct download
URLs for the various components needed).

Paul

From Steve.Dower at microsoft.com  Thu Jan 15 23:24:45 2015
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Thu, 15 Jan 2015 22:24:45 +0000
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwZs0vhH3HeEwYkoZuddD3rT-PPTFLw+WzAQyY8MzAuxHg@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
 <CAO41-mPDTAwcBUzn4-9s9Udkt0jVkNfCnGCj8+QKmcwuCL=2bA@mail.gmail.com>
 <CAMpsgwZs0vhH3HeEwYkoZuddD3rT-PPTFLw+WzAQyY8MzAuxHg@mail.gmail.com>
Message-ID: <DM2PR0301MB07342872429EFDB10CBAC1ACF54E0@DM2PR0301MB0734.namprd03.prod.outlook.com>

Victor Stinner wrote:
> 2015-01-15 22:39 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
> > http://www.microsoft.com/en-us/download/details.aspx?id=8279
> 
> "Microsoft Windows SDK for Windows 7 and .NET Framework 4"
> 
> Are you sure that it is SDK 7.1, and not 7.0?

The naming is horrible, and the bugs between VS and the SDK are also horrible. I'm sorry.

SDK 7.0 == "Windows SDK for Windows 7 and .NET Framework 3.5"
SDK 7.1 == "Windows SDK for Windows 7 and .NET Framework 4"

So this is the correct link for building Python 3.3 and 3.4. It's also the one you want to keep building the latest code with the old compiler.

> --
> 
> The SDK 7.0 works for Python 2.7 which is compiled with Visual Studio 2008.
> 
> I used the SDK 7.1 for Python 3.3 and 3.4 which are compiled with Visual Studio
> 2010.
> 
> It looks likt SDK 8 is more for Visual Studio 2012.
> 
> If you use the wrong SDK, you will depend on a "MSVCRxxx.dll" which is not
> provided by Python x.x (ex: MSVCR100.dll for SDK 7.1/Python 3.3 & 3.4).

This information is all entirely correct. Worse, if you depend on the wrong runtime version, you'll get unexpected crashes and other errors.

Cheers,
Steve

> Victor

From victor.stinner at gmail.com  Thu Jan 15 23:43:37 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 15 Jan 2015 23:43:37 +0100
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CACac1F_3vd4Ter0VDcjXKA-iKAtZX8OdiBrQdBHLwL-hmTC8LA@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
 <CAKJDb-M4WNmt9CVbTJFygQpa6qO0o2tCog1znaRxLGj7gLLZoA@mail.gmail.com>
 <CACac1F_3vd4Ter0VDcjXKA-iKAtZX8OdiBrQdBHLwL-hmTC8LA@mail.gmail.com>
Message-ID: <CAMpsgwZFfGFMtwCJSFGta9pcdsAO1OsK2-aRqqehXsuRy_qh9g@mail.gmail.com>

Oh by the way, the tool that I wrote to build wheel packages on Windows is here:
https://code.google.com/p/tulip/source/browse/release.py

It was too annoying to have to open 6 times the Windows SDK shell, and
type each time between 2 and 4 commands.

release.py help:
------
Usage: release.py [options] command

Options:
  -h, --help            show this help message and exit
  -v, --verbose         verbose
  -t TAG, --tag=TAG     Mercurial tag or revision, required to release
  -p PYTHON, --python=PYTHON
                        Only build/test one specific Python version, ex:
                        "2.7:32"
  -C, --no-compile      Don't compile the module, this options implies
                        --running
  -r, --running         Only use the running Python version
  --ignore              Ignore local changes

Commands:

- build: build asyncio in place, imply --running
- test: run tests
- test_wheel: test building wheel packages
- release: run tests and publish wheel packages,
  require the --tag option
- clean: cleanup the project
------

I wrote the tool for Tulip but it should be easy to make it more generic.

Victor

2015-01-15 23:34 GMT+01:00 Paul Moore <p.f.moore at gmail.com>:
> On 15 January 2015 at 22:26, Zachary Ware <zachary.ware+pydev at gmail.com> wrote:
>> Extension building in general is still a mess on Windows, I hope the
>> links above are enough to get you going again!
>
> For building extensions, I have a powershell script that, starting
> with a clean machine, downloads and installs everything needed to
> build extensions for Python 2.7-3.4 (Python, 32 and 64-bit, SDK
> compilers and Visual C for Python 2.7, and some support packages).
> It's available at https://github.com/pfmoore/pybuild It's pretty
> fragile (largely because the SDK installs are pretty fragile, but also
> because it doesn't check if things it wants to install are already
> there), but it's good for setting up a new VM from scratch.
>
> It isn't designed for building Python, and I've no idea how well it
> would work for that. But you might be able to pick out some parts of
> it that would be useful (if nothing else, it includes direct download
> URLs for the various components needed).
>
> Paul
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com

From rymg19 at gmail.com  Thu Jan 15 23:11:59 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Thu, 15 Jan 2015 16:11:59 -0600
Subject: [Python-Dev] Compile Python on Windows (OpenSSL)
In-Reply-To: <CAMpsgwZs0vhH3HeEwYkoZuddD3rT-PPTFLw+WzAQyY8MzAuxHg@mail.gmail.com>
References: <CAMpsgwYwXT8z3bdF6U-pd_qbW7wXrcVanEv0AdHSBhCqo51oUQ@mail.gmail.com>
 <DM2PR0301MB073410975D9093596209DCC3F5400@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <CAMpsgwZJLWY6r0tx7PmOwUizSf4hC_6KskPn=7wpG3RXNF79kA@mail.gmail.com>
 <CAD+XWwp_mAXkUKVqMzbZkpyisTXp3hOnjw-3c4fBbXMgKO8oWA@mail.gmail.com>
 <54B5A05F.1040208@egenix.com>
 <CAMpsgwae0_jsKCtYvqxSr+s+M1nLcZ8=q-A7fxA0sLyogBZocA@mail.gmail.com>
 <CAO41-mPDTAwcBUzn4-9s9Udkt0jVkNfCnGCj8+QKmcwuCL=2bA@mail.gmail.com>
 <CAMpsgwZs0vhH3HeEwYkoZuddD3rT-PPTFLw+WzAQyY8MzAuxHg@mail.gmail.com>
Message-ID: <CAO41-mMO_esJ5bMuZ12rakq92=ab0VPM3DVpZa3LGxgYPtNi_w@mail.gmail.com>

If you expand the Details section, it says the version is 7.1.

On Thu, Jan 15, 2015 at 3:58 PM, Victor Stinner <victor.stinner at gmail.com>
wrote:

> 2015-01-15 22:39 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
> > http://www.microsoft.com/en-us/download/details.aspx?id=8279
>
> "Microsoft Windows SDK for Windows 7 and .NET Framework 4"
>
> Are you sure that it is SDK 7.1, and not 7.0?
>
> --
>
> The SDK 7.0 works for Python 2.7 which is compiled with Visual Studio 2008.
>
> I used the SDK 7.1 for Python 3.3 and 3.4 which are compiled with
> Visual Studio 2010.
>
> It looks likt SDK 8 is more for Visual Studio 2012.
>
> If you use the wrong SDK, you will depend on a "MSVCRxxx.dll" which is
> not provided by Python x.x (ex: MSVCR100.dll for SDK 7.1/Python 3.3 &
> 3.4).
>
> Victor
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150115/16a1b1a3/attachment.html>

From status at bugs.python.org  Fri Jan 16 18:07:02 2015
From: status at bugs.python.org (Python tracker)
Date: Fri, 16 Jan 2015 18:07:02 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20150116170702.62102560CF@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2015-01-09 - 2015-01-16)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4736 (+10)
  closed 30277 (+33)
  total  35013 (+43)

Open issues with patches: 2220 


Issues opened (32)
==================

#22038: Implement atomic operations on non-x86 platforms
http://bugs.python.org/issue22038  reopened by haypo

#23211: test.test_logging.SMTPHandlerTest failing on Snow Leopard
http://bugs.python.org/issue23211  opened by geoffreyspear

#23212: Update Windows and OS X installer copies of OpenSSL to 1.0.1k
http://bugs.python.org/issue23212  opened by ned.deily

#23213: subprocess communicate() hangs when stderr isn't closed
http://bugs.python.org/issue23213  opened by whissi

#23214: BufferedReader.read1(size) signature incompatible with Buffere
http://bugs.python.org/issue23214  opened by vadmium

#23215: MemoryError with custom error handlers and multibyte codecs
http://bugs.python.org/issue23215  opened by alexer

#23216: IDLE grep/find/replace source code needs docstrings
http://bugs.python.org/issue23216  opened by Al.Sweigart

#23217: help() function incorrectly captures comment preceding a neste
http://bugs.python.org/issue23217  opened by rhettinger

#23218: Modernize the IDLE Find/Replace/Find in Files dialogs
http://bugs.python.org/issue23218  opened by Al.Sweigart

#23220: IDLE does not display \b backspace correctly.
http://bugs.python.org/issue23220  opened by Al.Sweigart

#23224: LZMADecompressor object is only initialized in __init__
http://bugs.python.org/issue23224  opened by vadmium

#23226: Add float linspace recipe to docs
http://bugs.python.org/issue23226  opened by abarnert

#23227: Generator's finally block not run if close() called before fir
http://bugs.python.org/issue23227  opened by sjdrake

#23228: Crashes when tarfile contains a symlink and unpack directory c
http://bugs.python.org/issue23228  opened by mvo

#23229: add inf, nan, infj, nanj to cmath module
http://bugs.python.org/issue23229  opened by mark.dickinson

#23231: Fix codecs.iterencode/decode() by allowing data parameter to b
http://bugs.python.org/issue23231  opened by vadmium

#23232: 'codecs' module functionality + its docs -- concerning custom 
http://bugs.python.org/issue23232  opened by zuo

#23233: TypeError in ./setup.py
http://bugs.python.org/issue23233  opened by serhiy.storchaka

#23235: run_tests.py doesn't set test.support.verbose correctly
http://bugs.python.org/issue23235  opened by berker.peksag

#23236: asyncio: add timeout to StreamReader read methods
http://bugs.python.org/issue23236  opened by haypo

#23237: Interrupts are lost during readline PyOS_InputHook processing 
http://bugs.python.org/issue23237  opened by Michiel.de.Hoon

#23238: http.server failed to decode '+' to ' ' when calling cgi scrip
http://bugs.python.org/issue23238  opened by memkmemk

#23239: SSL match_hostname does not accept IP Address
http://bugs.python.org/issue23239  opened by ??d??m.Zsigmond

#23241: shutil should accept pathlib types
http://bugs.python.org/issue23241  opened by mkesper

#23243: asyncio: emit ResourceWarning warnings if transports/event loo
http://bugs.python.org/issue23243  opened by haypo

#23245: urllib2: urlopen() gets exception(kwargs bug?)
http://bugs.python.org/issue23245  opened by Douman

#23246: distutils fails to locate vcvarsall with Visual C++ Compiler f
http://bugs.python.org/issue23246  opened by Gregory.Szorc

#23247: Multibyte codec StreamWriter.reset() crashes
http://bugs.python.org/issue23247  opened by vadmium

#23248: Update ssl data
http://bugs.python.org/issue23248  opened by pitrou

#23249: test_win32 fails on aarch64
http://bugs.python.org/issue23249  opened by rkuska

#23250: http.cookies HttpOnly attribute does not use suggested case-st
http://bugs.python.org/issue23250  opened by jdufresne

#23251: mention in time.sleep() docs that it does not block other Pyth
http://bugs.python.org/issue23251  opened by akira



Most recent 15 issues with no replies (15)
==========================================

#23251: mention in time.sleep() docs that it does not block other Pyth
http://bugs.python.org/issue23251

#23250: http.cookies HttpOnly attribute does not use suggested case-st
http://bugs.python.org/issue23250

#23249: test_win32 fails on aarch64
http://bugs.python.org/issue23249

#23248: Update ssl data
http://bugs.python.org/issue23248

#23247: Multibyte codec StreamWriter.reset() crashes
http://bugs.python.org/issue23247

#23227: Generator's finally block not run if close() called before fir
http://bugs.python.org/issue23227

#23218: Modernize the IDLE Find/Replace/Find in Files dialogs
http://bugs.python.org/issue23218

#23217: help() function incorrectly captures comment preceding a neste
http://bugs.python.org/issue23217

#23216: IDLE grep/find/replace source code needs docstrings
http://bugs.python.org/issue23216

#23215: MemoryError with custom error handlers and multibyte codecs
http://bugs.python.org/issue23215

#23207: logging.basicConfig does not validate keyword arguments
http://bugs.python.org/issue23207

#23202: pyvenv does not fail like documented when a venv already exist
http://bugs.python.org/issue23202

#23183: timeit CLI best of 3: undocumented output format
http://bugs.python.org/issue23183

#23182: Update grammar tests to use new style for annotated function d
http://bugs.python.org/issue23182

#23169: Reflect that PreReq and BuildPreReq are deprecated in the late
http://bugs.python.org/issue23169



Most recent 15 issues waiting for review (15)
=============================================

#23251: mention in time.sleep() docs that it does not block other Pyth
http://bugs.python.org/issue23251

#23250: http.cookies HttpOnly attribute does not use suggested case-st
http://bugs.python.org/issue23250

#23249: test_win32 fails on aarch64
http://bugs.python.org/issue23249

#23248: Update ssl data
http://bugs.python.org/issue23248

#23243: asyncio: emit ResourceWarning warnings if transports/event loo
http://bugs.python.org/issue23243

#23236: asyncio: add timeout to StreamReader read methods
http://bugs.python.org/issue23236

#23235: run_tests.py doesn't set test.support.verbose correctly
http://bugs.python.org/issue23235

#23233: TypeError in ./setup.py
http://bugs.python.org/issue23233

#23231: Fix codecs.iterencode/decode() by allowing data parameter to b
http://bugs.python.org/issue23231

#23228: Crashes when tarfile contains a symlink and unpack directory c
http://bugs.python.org/issue23228

#23226: Add float linspace recipe to docs
http://bugs.python.org/issue23226

#23218: Modernize the IDLE Find/Replace/Find in Files dialogs
http://bugs.python.org/issue23218

#23216: IDLE grep/find/replace source code needs docstrings
http://bugs.python.org/issue23216

#23215: MemoryError with custom error handlers and multibyte codecs
http://bugs.python.org/issue23215

#23211: test.test_logging.SMTPHandlerTest failing on Snow Leopard
http://bugs.python.org/issue23211



Top 10 most discussed issues (10)
=================================

#23229: add inf, nan, infj, nanj to cmath module
http://bugs.python.org/issue23229  20 msgs

#22038: Implement atomic operations on non-x86 platforms
http://bugs.python.org/issue22038  13 msgs

#22932: email.utils.formatdate uses unreliable time.timezone constant
http://bugs.python.org/issue22932   8 msgs

#23193: Please support "numeric_owner" in tarfile
http://bugs.python.org/issue23193   6 msgs

#23245: urllib2: urlopen() gets exception(kwargs bug?)
http://bugs.python.org/issue23245   6 msgs

#1602: windows console doesn't print or input Unicode
http://bugs.python.org/issue1602   5 msgs

#13881: Stream encoder for zlib_codec doesn't use the incremental enco
http://bugs.python.org/issue13881   5 msgs

#14014: codecs.StreamWriter.reset contract not fulfilled
http://bugs.python.org/issue14014   5 msgs

#20132: Many incremental codecs don???t handle fragmented data
http://bugs.python.org/issue20132   5 msgs

#23201: Decimal(0)**0 is an error, 0**0 is 1, but Decimal(0) == 0
http://bugs.python.org/issue23201   5 msgs



Issues closed (33)
==================

#11835: python (x64) ctypes incorrectly pass structures parameter
http://bugs.python.org/issue11835  closed by berker.peksag

#19777: Provide a home() classmethod on Path objects
http://bugs.python.org/issue19777  closed by pitrou

#20467: Confusing wording about __init__
http://bugs.python.org/issue20467  closed by ethan.furman

#21581: Consider dropping importlib.abc.Loader.create_module()
http://bugs.python.org/issue21581  closed by brett.cannon

#21896: Unexpected ConnectionResetError in urllib.request against a va
http://bugs.python.org/issue21896  closed by ned.deily

#21902: Docstring of math.acosh, asinh, and atanh
http://bugs.python.org/issue21902  closed by mark.dickinson

#22560: New SSL implementation based on ssl.MemoryBIO
http://bugs.python.org/issue22560  closed by haypo

#22952: multiprocessing doc introduction not in affirmative tone
http://bugs.python.org/issue22952  closed by pitrou

#22988: No error when yielding from `finally`
http://bugs.python.org/issue22988  closed by ethan.furman

#22997: Minor improvements to "Functional API" section of Enum documen
http://bugs.python.org/issue22997  closed by ethan.furman

#23013: Tweak wording for importlib.util.LazyLoader in regards to Load
http://bugs.python.org/issue23013  closed by brett.cannon

#23018: Add version info to python
http://bugs.python.org/issue23018  closed by steve.dower

#23063: `python setup.py check --restructuredtext --strict --metadata`
http://bugs.python.org/issue23063  closed by python-dev

#23160: Respect the environment variable SVNROOT in external-common.ba
http://bugs.python.org/issue23160  closed by python-dev

#23173: asyncio: kill the subprocess if the creation failed
http://bugs.python.org/issue23173  closed by haypo

#23181: Unicode "code point" should be two words in documentation
http://bugs.python.org/issue23181  closed by python-dev

#23185: add inf and nan to math module
http://bugs.python.org/issue23185  closed by mark.dickinson

#23189: Set docstrings to empty string when optimizing with -OO.
http://bugs.python.org/issue23189  closed by rhettinger

#23197: asyncio: check if a future is cancelled before calling set_res
http://bugs.python.org/issue23197  closed by haypo

#23198: asyncio: refactor StreamReader
http://bugs.python.org/issue23198  closed by haypo

#23206: json.dumps(ensure_ascii=False) is ~10x slower than json.dumps(
http://bugs.python.org/issue23206  closed by pitrou

#23209: asyncio: break some cycles
http://bugs.python.org/issue23209  closed by haypo

#23210: remove the word "virtual" from help(range)
http://bugs.python.org/issue23210  closed by python-dev

#23219: asyncio: cancelling wait_for(task, timeout) must also cancel t
http://bugs.python.org/issue23219  closed by python-dev

#23221: "a(n) the", "the a(n)" typos
http://bugs.python.org/issue23221  closed by python-dev

#23222: open doesn't provide traceback for int argument
http://bugs.python.org/issue23222  closed by SilentGhost

#23223: subprocess32 unable to be installed via pip
http://bugs.python.org/issue23223  closed by haypo

#23225: selectors: raise an exception if the selector is closed
http://bugs.python.org/issue23225  closed by haypo

#23230: Bug parsing integers with zero padding
http://bugs.python.org/issue23230  closed by r.david.murray

#23234: refactor subprocess: use new OSError exceptions, factorize std
http://bugs.python.org/issue23234  closed by haypo

#23240: pip 6.0.6- pip install command is broken
http://bugs.python.org/issue23240  closed by brett.cannon

#23242: asyncio: Process must close the transport when the process exi
http://bugs.python.org/issue23242  closed by haypo

#23244: doc typo in 'Glossary': 'generator'
http://bugs.python.org/issue23244  closed by python-dev

From techtonik at gmail.com  Sat Jan 17 11:01:30 2015
From: techtonik at gmail.com (anatoly techtonik)
Date: Sat, 17 Jan 2015 13:01:30 +0300
Subject: [Python-Dev] hg vs Github [was: PEP 481 - Migrate Some
 Supporting Repositories to Git and Github]
In-Reply-To: <547ca751.8524e00a.5a37.ffff8c58@mx.google.com>
References: <slrnm7o6rb.mik.mcepl@mitmanek.ceplovi.cz>
 <547ca751.8524e00a.5a37.ffff8c58@mx.google.com>
Message-ID: <CAPkN8xK93gm_eWJQzeANFkhug4=GNhLg1VhbuwTVCYzmpne1PQ@mail.gmail.com>

On Mon, Dec 1, 2014 at 8:37 PM, Jim J. Jewett <jimjjewett at gmail.com> wrote:
>
>> What I really don't understand is why this discussion is hg v.
>> GitHub, when it should be hg v. git. Particular hosting is
>> a secondary issue
>
> I think even the proponents concede that git isn't better enough
> to justify a switch in repositories.
>
> They do claim that GitHub (the whole environment; not just the
> hosting) is so much better that a switch to GitHub is justified.
>
> Github + hg offers far fewer benefits than Github + git, so also
> switching to git is part of the price.  Whether that is an
> intolerable markup or a discount is disputed, as are the value
> of several other costs and benefits.

git - GitHub < hg + hgweb
GitHub + hg < hg + hgweb
GitHub + git > hg + hgweb
GitHub + git <= hg + hgweb + Roundup
GitHub + git < hg + hgweb + Roundup + Rietveld

(GitHub + git) * Python = 0
(hg + hgweb) * Python = 1


There is also a question of support. Obviously Roundup will degrade
over time if there is economic support for coordination action for
people working on it, no matter how strong is their open source and
community spirits. It was fun back in the days, but now time control
and "human resource mining" techniques are so perfect that I hardly
imagine someone to have free time to work on community open
source projects.

With no time to support our own dogfood, the community will have
to switch to GitHub sooner or later, because that is the thing that
saves time so valuable. GitLab could be a open source alternative
if Python community chooses to be more tolerant to use Ruby tools
dogfood in its ecosystem.

From wes.turner at gmail.com  Sat Jan 17 22:05:27 2015
From: wes.turner at gmail.com (Wes Turner)
Date: Sat, 17 Jan 2015 15:05:27 -0600
Subject: [Python-Dev] hg vs Github [was: PEP 481 - Migrate Some
 Supporting Repositories to Git and Github]
In-Reply-To: <547ca751.8524e00a.5a37.ffff8c58@mx.google.com>
References: <slrnm7o6rb.mik.mcepl@mitmanek.ceplovi.cz>
 <547ca751.8524e00a.5a37.ffff8c58@mx.google.com>
Message-ID: <CACfEFw-v2+m7jW=_a7tDWqcZd4HVYsKj51qNm5BB+EqW5U=sSQ@mail.gmail.com>

Links to OT (pending a mailman upgrade with auto-appended links):

https://mail.python.org/pipermail/python-dev/2014-November/137283.html

https://mail.python.org/pipermail/python-dev/2014-December/thread.html#137381

http://westurner.github.io/wiki/ideas#open-source-mailing-list-extractor
(sphinx + github wiki repo)
On Dec 1, 2014 11:37 AM, "Jim J. Jewett" <jimjjewett at gmail.com> wrote:

>
>
>
> M. Cepl asked:
>
> > What I really don't understand is why this discussion is hg v.
> > GitHub, when it should be hg v. git. Particular hosting is
> > a secondary issue
>
> I think even the proponents concede that git isn't better enough
> to justify a switch in repositories.
>
> They do claim that GitHub (the whole environment; not just the
> hosting) is so much better that a switch to GitHub is justified.
>
> Github + hg offers far fewer benefits than Github + git, so also
> switching to git is part of the price.  Whether that is an
> intolerable markup or a discount is disputed, as are the value
> of several other costs and benefits.
>
> -jJ
>
> --
>
> If there are still threading problems with my replies, please
> email me with details, so that I can try to resolve them.  -jJ
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/wes.turner%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150117/7a167fea/attachment.html>

From ethan at stoneleaf.us  Sun Jan 18 04:12:38 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 17 Jan 2015 19:12:38 -0800
Subject: [Python-Dev] PEP 461 and __imod__
Message-ID: <54BB24A6.7050509@stoneleaf.us>

I have % (__mod__) working for both bytes and bytearray (yay!), but I cannot find nor figure out how to tell the
bytearray object that 'xyz' is the __imod__ function.

Any pointers?

--
~Ethan~



-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150117/16fe3973/attachment.sig>

From ethan at stoneleaf.us  Sun Jan 18 04:53:25 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Sat, 17 Jan 2015 19:53:25 -0800
Subject: [Python-Dev] PEP 461 and __imod__
In-Reply-To: <54BB24A6.7050509@stoneleaf.us>
References: <54BB24A6.7050509@stoneleaf.us>
Message-ID: <54BB2E35.2060300@stoneleaf.us>

On 01/17/2015 07:12 PM, Ethan Furman wrote:
>
> I have % (__mod__) working for both bytes and bytearray (yay!), but I cannot find
> nor figure out how to tell the bytearray object that 'xyz' is the __imod__ function.
> 
> Any pointers?

I think I found a clue in Objects/floatobject.c:

static PyNumberMethods float_as_number = {
    float_add,          /*nb_add*/
    float_sub,          /*nb_subtract*/
    float_mul,          /*nb_multiply*/
    float_rem,          /*nb_remainder*/
    float_divmod,       /*nb_divmod*/
    float_pow,          /*nb_power*/
    (unaryfunc)float_neg, /*nb_negative*/
    (unaryfunc)float_float, /*nb_positive*/
    (unaryfunc)float_abs, /*nb_absolute*/
    (inquiry)float_bool, /*nb_bool*/
    0,                  /*nb_invert*/
    0,                  /*nb_lshift*/
    0,                  /*nb_rshift*/
    0,                  /*nb_and*/
    0,                  /*nb_xor*/
    0,                  /*nb_or*/
    float_trunc,        /*nb_int*/
    0,                  /*nb_reserved*/
    float_float,        /*nb_float*/
    0,                  /* nb_inplace_add */
    0,                  /* nb_inplace_subtract */
    0,                  /* nb_inplace_multiply */
    0,                  /* nb_inplace_remainder */
    0,                  /* nb_inplace_power */
    0,                  /* nb_inplace_lshift */
    0,                  /* nb_inplace_rshift */
    0,                  /* nb_inplace_and */
    0,                  /* nb_inplace_xor */
    0,                  /* nb_inplace_or */
    float_floor_div, /* nb_floor_divide */
    float_div,          /* nb_true_divide */
    0,                  /* nb_inplace_floor_divide */
    0,                  /* nb_inplace_true_divide */
};

Guess I'll see if it seg faults.  ;)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150117/7fbbb087/attachment.sig>

From info at martin-thoma.de  Sun Jan 18 16:08:04 2015
From: info at martin-thoma.de (Martin Thoma)
Date: Sun, 18 Jan 2015 16:08:04 +0100
Subject: [Python-Dev] [PyPI] Why are some packages on pypi.python.org/simple,
	but have no page?
Message-ID: <CALZHTuASfNZO_oe2KMQg+_uigmbFjvHyTU0NTmqWqnfOUAC=rw@mail.gmail.com>

Hello Python developers,

Could anybody please answer the following question?
(I have asked it on http://stackoverflow.com/q/28010799/562769, but Steve
Barnes thinks I should ask it here)

I am currently analyzing packages on PyPI. I use
https://pypi.python.org/simple/ to get all package names and
https://pypi.python.org/pypi/numpy/json and similar to get the metadata.

However, there are 514 packages (e.g. abu.rpc, acid, about-pandoc,
about-numtest, ...) which do not have the https://pypi.python.org/pypi/
site, but are on https://pypi.python.org/simple/.

Why is that the case?

Best regards,
Martin
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150118/13a23bb9/attachment.html>

From graffatcolmingov at gmail.com  Sun Jan 18 17:14:11 2015
From: graffatcolmingov at gmail.com (Ian Cordasco)
Date: Sun, 18 Jan 2015 10:14:11 -0600
Subject: [Python-Dev] [PyPI] Why are some packages on
 pypi.python.org/simple, but have no page?
In-Reply-To: <CALZHTuASfNZO_oe2KMQg+_uigmbFjvHyTU0NTmqWqnfOUAC=rw@mail.gmail.com>
References: <CALZHTuASfNZO_oe2KMQg+_uigmbFjvHyTU0NTmqWqnfOUAC=rw@mail.gmail.com>
Message-ID: <CAN-Kwu0SQVOA=+eXGCY3Nd7Q2C-ABdA1fNZHJnnx57vhSRvyfA@mail.gmail.com>

Taking one of your examples: https://pypi.python.org/simple/acid/ 404s
(I didn't bother checkin the other three). So there are links on
/simple but no content for them. So I think your question is better
asked, why are there links on /simple that lead to 404s.

On Sun, Jan 18, 2015 at 9:08 AM, Martin Thoma <info at martin-thoma.de> wrote:
> Hello Python developers,
>
> Could anybody please answer the following question?
> (I have asked it on http://stackoverflow.com/q/28010799/562769, but Steve
> Barnes thinks I should ask it here)
>
> I am currently analyzing packages on PyPI. I use
> https://pypi.python.org/simple/ to get all package names and
> https://pypi.python.org/pypi/numpy/json and similar to get the metadata.
>
> However, there are 514 packages (e.g. abu.rpc, acid, about-pandoc,
> about-numtest, ...) which do not have the https://pypi.python.org/pypi/
> site, but are on https://pypi.python.org/simple/.
>
> Why is that the case?
>
> Best regards,
> Martin
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/graffatcolmingov%40gmail.com
>

From chaselton at gmail.com  Mon Jan 19 15:51:58 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Mon, 19 Jan 2015 08:51:58 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
Message-ID: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>

Hello,
I'm struggling with a build issue on Android; I've posted to the
general python list with no result, so I'm re-posting here in hopes
that someone can help.  If this is the wrong place feel free to let me
know.

I'm attempting to build Python 2.7.8 on my Android device; I'm using
an environment that simulates a Linux filesystem within the Android
terminal using a port of fakechroot.  Within that environment I've
ported and/or bootstrapped a number of Linux utilities (curl, git,
openssl, gcc)

I run ./configure, then make, and the executable and library are
built.  The problem occurs when build_ext is run; the newly built
python executable builds, then links _struct, and immediately
afterwards I get an 'undefined reference to dlopen' error.

If I run ./python setup.py --verbose -library-dirs /path/to/lib
--libraries='c dl m' -f, the 'undefined reference to dlopen' error is
thrown again.

If I run ./python setup.py --verbose -library-dirs /path/to/lib
--libraries='-lc -ldl -lm' -f the build continues past _struct...even
though ld throws the expected 'unable to find -l-lc' and other errors.

Let me know if you need me to provide additional information.  Any
help would be greatly appreciated.

Cyd

From ethan at stoneleaf.us  Mon Jan 19 17:36:55 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 19 Jan 2015 08:36:55 -0800
Subject: [Python-Dev] bytes & bytearray
Message-ID: <54BD32A7.5070206@stoneleaf.us>

I was checking the documentation [1] to see where to put the new information about bytes and bytearray %-formatting, and
noticed that /every/ operation that could modify a bytearray object in place (e.g. center, capitalize, strip, etc.)
instead returns a new copy.

The section just prior to that [2] does say, "As bytearray objects are mutable, they support the mutable sequence
operations ...".

So it seems that when bytearray is treated as ascii data a new bytearry is returned, and when bytearray is treated as a
container it is modified in place:

New:

  bytearray(b'Chapter Title').center()
  bytearray(b' Chapter Title ').replace(b' ', b'- * - ')

In-place:

  bytearray(b'abc'][1] = ord(b'z')
  bytearray(b'def'] += b'ghi'
  bytearray(b'123'] *= 3

I now have a minor dilemma:  %-formatting is an ascii operation, but it also has an in-place operator (%=) . . . so does
%= modify the bytearray in place just like += and *= do, or does it return a new bytearray just like all the named ascii
operations do?  I do not know which is more surprising: having one of the in-place operators not work in place, or
having an unnamed ascii-operation not return a new copy.

Thoughts?

--
~Ethan~


[1] https://docs.python.org/3/library/stdtypes.html#bytes-and-bytearray-operations
[2] https://docs.python.org/3/library/stdtypes.html#bytearray-objects

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150119/9a7be275/attachment.sig>

From eric at trueblade.com  Mon Jan 19 18:01:36 2015
From: eric at trueblade.com (Eric V. Smith)
Date: Mon, 19 Jan 2015 12:01:36 -0500
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <54BD32A7.5070206@stoneleaf.us>
References: <54BD32A7.5070206@stoneleaf.us>
Message-ID: <54BD3870.5020808@trueblade.com>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 1/19/2015 11:36 AM, Ethan Furman wrote:
> I was checking the documentation [1] to see where to put the new
> information about bytes and bytearray %-formatting, and noticed
> that /every/ operation that could modify a bytearray object in
> place (e.g. center, capitalize, strip, etc.) instead returns a new
> copy.
> 
> The section just prior to that [2] does say, "As bytearray objects
> are mutable, they support the mutable sequence operations ...".
> 
> So it seems that when bytearray is treated as ascii data a new
> bytearry is returned, and when bytearray is treated as a container
> it is modified in place:
> 
> New:
> 
> bytearray(b'Chapter Title').center() bytearray(b' Chapter Title
> ').replace(b' ', b'- * - ')
> 
> In-place:
> 
> bytearray(b'abc'][1] = ord(b'z') bytearray(b'def'] += b'ghi' 
> bytearray(b'123'] *= 3
> 
> I now have a minor dilemma:  %-formatting is an ascii operation,
> but it also has an in-place operator (%=) . . . so does %= modify
> the bytearray in place just like += and *= do, or does it return a
> new bytearray just like all the named ascii operations do?  I do
> not know which is more surprising: having one of the in-place
> operators not work in place, or having an unnamed ascii-operation
> not return a new copy.
> 
> Thoughts?

I'd return a new copy (or, not implement it at all and let the default
machinery call __mod__, and do the assignment). That's the least
surprising to me, since the goal is to be compatible with 2.7's str
%-formatting.

- -- 
Eric.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1

iQEcBAEBAgAGBQJUvThwAAoJENxauZFcKtNx/noIAIfIW2Ir4P9bQ9s7bdyJCvMj
RlqfBhWVOjWZ52gK/CdTvpZZDlDUf+gh1JkbvggNvIQHUTy0IY9IxryWvNemfSce
nzZxUhNKzx77f/oGTeGgqBBMZGvCsRCqYFbLBME5LDweiHAIL2MVGqF/SwkobrGq
ruJjSBtNAl28IgpDj7kM7tT7/iXINk6XkVatNa2OxV2FOyYiIz+7Vs2HGpkltzhW
g4qqGEvEpLOl1oRtmI/A3TDFjQgyHc1MKVax6PH/Nq2OMfnoS4hg+jTAzE6Eairh
SmWyZUMjpTeHpCmUgx03WLh8iRTokfE2LN2KJuBN18iAT5EqC6sNTKZm9HX5odw=
=D9zy
-----END PGP SIGNATURE-----

From ethan at stoneleaf.us  Mon Jan 19 18:19:32 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 19 Jan 2015 09:19:32 -0800
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <54BD3870.5020808@trueblade.com>
References: <54BD32A7.5070206@stoneleaf.us> <54BD3870.5020808@trueblade.com>
Message-ID: <54BD3CA4.9050007@stoneleaf.us>

On 01/19/2015 09:01 AM, Eric V. Smith wrote:
> 
> I'd return a new copy (or, not implement it at all and let the default
> machinery call __mod__, and do the assignment). That's the least
> surprising to me, since the goal is to be compatible with 2.7's str
> %-formatting.

That certainly feels like the winning argument.  :)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150119/031d6b55/attachment.sig>

From solipsis at pitrou.net  Mon Jan 19 18:54:54 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 19 Jan 2015 18:54:54 +0100
Subject: [Python-Dev] bytes & bytearray
References: <54BD32A7.5070206@stoneleaf.us>
	<54BD3870.5020808@trueblade.com>
Message-ID: <20150119185454.02e0d419@fsol>

On Mon, 19 Jan 2015 12:01:36 -0500
"Eric V. Smith" <eric at trueblade.com> wrote:
> 
> I'd return a new copy (or, not implement it at all and let the default
> machinery call __mod__, and do the assignment). That's the least
> surprising to me, since the goal is to be compatible with 2.7's str
> %-formatting.

+1 for a copy. There's no reasonable use case to make the formatting
in-place, and it wouldn't buy any significant performance win anyway.

Regards

Antoine.



From pmiscml at gmail.com  Mon Jan 19 20:43:40 2015
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Mon, 19 Jan 2015 21:43:40 +0200
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <54BD32A7.5070206@stoneleaf.us>
References: <54BD32A7.5070206@stoneleaf.us>
Message-ID: <20150119214340.44fe0e7f@x230>

Hello,

On Mon, 19 Jan 2015 08:36:55 -0800
Ethan Furman <ethan at stoneleaf.us> wrote:

> I was checking the documentation [1] to see where to put the new
> information about bytes and bytearray %-formatting, and noticed
> that /every/ operation that could modify a bytearray object in place
> (e.g. center, capitalize, strip, etc.) instead returns a new copy.

Well, those "operations" come from string methods. String methods
always return a copy, so I'm not sure the same methods, applied to
bytearray *could* reasonably be inplace operations, without surprising
user a lot.

But at the same, a usecase of inplace operations on bytearrays is
valid, and I'd like to take a chance to branch the topic to discuss
how they possibly could be implemented.  

> 
> The section just prior to that [2] does say, "As bytearray objects
> are mutable, they support the mutable sequence operations ...".
> 
> So it seems that when bytearray is treated as ascii data a new
> bytearry is returned, and when bytearray is treated as a container it
> is modified in place:

Per above, I'd formulate it differently: methods inherited from string
always return a new instance, while some *operators* modify it inplace
(and yes, those are usually the same operators that modify other
containers inplace).

So, suppose there's a requirement to support inplace operations
(methods) on bytearray, what would be Pythonic way to implement it?

Something like:

b.lower_inplace()
b.lower_i()

, or maybe

import bytearray_ops
bytearray_ops.lower(b)

?

> New:
> 
>   bytearray(b'Chapter Title').center()
>   bytearray(b' Chapter Title ').replace(b' ', b'- * - ')
> 
> In-place:
> 
>   bytearray(b'abc'][1] = ord(b'z')
>   bytearray(b'def'] += b'ghi'
>   bytearray(b'123'] *= 3


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From guido at python.org  Mon Jan 19 23:03:20 2015
From: guido at python.org (Guido van Rossum)
Date: Mon, 19 Jan 2015 14:03:20 -0800
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <20150119214340.44fe0e7f@x230>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
Message-ID: <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>

On Mon, Jan 19, 2015 at 11:43 AM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

> [...]
> So, suppose there's a requirement to support inplace operations
> (methods) on bytearray, what would be Pythonic way to implement it?
>
> Something like:
>
> b.lower_inplace()
> b.lower_i()
>
> , or maybe
>
> import bytearray_ops
> bytearray_ops.lower(b)
>
> ?


Please don't go there. The use cases are too rare.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150119/a31adadf/attachment.html>

From pmiscml at gmail.com  Mon Jan 19 23:22:02 2015
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Tue, 20 Jan 2015 00:22:02 +0200
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
 <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
Message-ID: <20150120002202.07c99774@x230>

Hello,

On Mon, 19 Jan 2015 14:03:20 -0800
Guido van Rossum <guido at python.org> wrote:

> On Mon, Jan 19, 2015 at 11:43 AM, Paul Sokolovsky <pmiscml at gmail.com>
> wrote:
> 
> > [...]
> > So, suppose there's a requirement to support inplace operations
> > (methods) on bytearray, what would be Pythonic way to implement it?
> >
> > Something like:
> >
> > b.lower_inplace()
> > b.lower_i()
> >
> > , or maybe
> >
> > import bytearray_ops
> > bytearray_ops.lower(b)
> >
> > ?
> 
> 
> Please don't go there. The use cases are too rare.

recv_into() an HTTP request and .lower() it inplace to ease parsing?
But I mostly raise that with MicroPython hat on, there it may be not
necessarily superfluous. But then, having a separate module for such
operations doesn't seem too bad, except that it would be retrograde
action after "string" module deprecation in favor of methods.

Anyway, I targetted that question for python-ideas on some easy day, and
just took a quick chance here seeing someone else talking about inplace
bytearray operations ;-).

> 
> -- 
> --Guido van Rossum (python.org/~guido)


-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From chaselton at gmail.com  Tue Jan 20 00:23:38 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Mon, 19 Jan 2015 17:23:38 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
Message-ID: <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>

On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> Hello,
> I'm struggling with a build issue on Android; I've posted to the
> general python list with no result, so I'm re-posting here in hopes
> that someone can help.  If this is the wrong place feel free to let me
> know.
>
> I'm attempting to build Python 2.7.8 on my Android device; I'm using
> an environment that simulates a Linux filesystem within the Android
> terminal using a port of fakechroot.  Within that environment I've
> ported and/or bootstrapped a number of Linux utilities (curl, git,
> openssl, gcc)
>
> I run ./configure, then make, and the executable and library are
> built.  The problem occurs when build_ext is run; the newly built
> python executable builds, then links _struct, and immediately
> afterwards I get an 'undefined reference to dlopen' error.
>
> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> --libraries='c dl m' -f, the 'undefined reference to dlopen' error is
> thrown again.
>
> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> --libraries='-lc -ldl -lm' -f the build continues past _struct...even
> though ld throws the expected 'unable to find -l-lc' and other errors.
>
> Let me know if you need me to provide additional information.  Any
> help would be greatly appreciated.
>
> Cyd


Additionally I took a strace of the error producing command. The
following is (hopefully) a relevant portion minus the various 'no such
file' lines before the correct lib is found (which it always is)

16513 open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.py",
O_RDONLY|O_LARGEFILE) = 3           16513
open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.pyc",
O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
 = 16525
16513 wait4(16525,  <unfinished ...>
16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
O_RDONLY|O_LARGEFILE) = 3
16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
"-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars */])
= 0

*snip call to libc intercepted by libfakechroot*

16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
16525 open("/dev/__properties__",
O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
16525 open("build/temp.linux-armv7l-2.7/multiarch",
O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
16525 fork()                            = 16526
16525 wait4(-1,  <unfinished ...>
16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE, 0666)
= -1 EACCES (Permission denied)

See attached for remainder
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sout
Type: application/octet-stream
Size: 217490 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150119/e0016c73/attachment-0001.obj>

From mistersheik at gmail.com  Tue Jan 20 01:40:13 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Mon, 19 Jan 2015 19:40:13 -0500
Subject: [Python-Dev] Why are generated files in the repository?
Message-ID: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>

Hi everyone,

I tried to work on PEP 448 and updated the latest patch to Python 3.5.  I
uploaded the new diff here: http://bugs.python.org/issue2292.  I don't know
how to debug further.  Is there a way to view the compiled output despite
Python not starting up?

I was also wondering why files like Python/graminit.c are in the
respository?  They generate spurious merge conflicts.

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150119/78264b69/attachment.html>

From benjamin at python.org  Tue Jan 20 01:53:25 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 19 Jan 2015 19:53:25 -0500
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
Message-ID: <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>



On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
> I was also wondering why files like Python/graminit.c are in the
> respository?  They generate spurious merge conflicts.

Convenience mostly.

From greg.ewing at canterbury.ac.nz  Tue Jan 20 06:15:02 2015
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 20 Jan 2015 18:15:02 +1300
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
 <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
Message-ID: <54BDE456.70506@canterbury.ac.nz>

Guido van Rossum wrote:
> On Mon, Jan 19, 2015 at 11:43 AM, Paul Sokolovsky <pmiscml at gmail.com 
> <mailto:pmiscml at gmail.com>> wrote:
> 
>     b.lower_inplace()
>     b.lower_i()
> 
> Please don't go there. The use cases are too rare.

And if you have such a use case, it's not too
hard to do

   b[:] = b.lower()

-- 
Greg

From pmiscml at gmail.com  Tue Jan 20 10:48:10 2015
From: pmiscml at gmail.com (Paul Sokolovsky)
Date: Tue, 20 Jan 2015 11:48:10 +0200
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <54BDE456.70506@canterbury.ac.nz>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
 <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
 <54BDE456.70506@canterbury.ac.nz>
Message-ID: <20150120114810.4ea87715@x230>

Hello,

On Tue, 20 Jan 2015 18:15:02 +1300
Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:

> Guido van Rossum wrote:
> > On Mon, Jan 19, 2015 at 11:43 AM, Paul Sokolovsky
> > <pmiscml at gmail.com <mailto:pmiscml at gmail.com>> wrote:
> > 
> >     b.lower_inplace()
> >     b.lower_i()
> > 
> > Please don't go there. The use cases are too rare.
> 
> And if you have such a use case, it's not too
> hard to do
> 
>    b[:] = b.lower()

The point of inplace operations (memoryview's, other stuff already in
Python) is to avoid unneeded memory allocation and copying. For 1Tb
bytearray with 1Tb of RAM, it will be very hard to do. (Ditto for 100K
bytearray with 150K RAM.)



-- 
Best regards,
 Paul                          mailto:pmiscml at gmail.com

From mistersheik at gmail.com  Tue Jan 20 15:35:27 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 09:35:27 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
Message-ID: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>

I get error:

TypeError: init_builtin() takes exactly 1 argument (0 given)

The only source file that can generate that error
is Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:

        PyErr_Format(PyExc_TypeError,
                     "call takes exactly %d arguments XYZABC (%zd given)",
                     inargs_index, actual_args);

I do not see any difference after make clean and a full rebuild.  How is
this possible?  I need to debug the arguments passed.

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/2c1f0773/attachment.html>

From brett at python.org  Tue Jan 20 15:51:44 2015
From: brett at python.org (Brett Cannon)
Date: Tue, 20 Jan 2015 14:51:44 +0000
Subject: [Python-Dev] How do I ensure that my code is being executed?
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
Message-ID: <CAP1=2W4=ckmP_EXpVL2oGY0xYsD0_rUCykad-kX+CmdxVym8Ug@mail.gmail.com>

This is a mailing to discuss the development *of* Python, not its *use*.
You should be able to get help from python-list or #python on IRC.

On Tue Jan 20 2015 at 9:44:48 AM Neil Girdhar <mistersheik at gmail.com> wrote:

> I get error:
>
> TypeError: init_builtin() takes exactly 1 argument (0 given)
>
> The only source file that can generate that error
> is Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
>
>         PyErr_Format(PyExc_TypeError,
>                      "call takes exactly %d arguments XYZABC (%zd given)",
>                      inargs_index, actual_args);
>
> I do not see any difference after make clean and a full rebuild.  How is
> this possible?  I need to debug the arguments passed.
>
> Best,
>
> Neil
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/1d7fdb5e/attachment.html>

From stefanrin at gmail.com  Tue Jan 20 15:52:18 2015
From: stefanrin at gmail.com (Stefan Ring)
Date: Tue, 20 Jan 2015 15:52:18 +0100
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
Message-ID: <CAAxjCEzP-VcN2OqEqoc2VUYk0B=C3ZK0YWucVGJcJ_-=8mk2pw@mail.gmail.com>

On Tue, Jan 20, 2015 at 3:35 PM, Neil Girdhar <mistersheik at gmail.com> wrote:
> I get error:
>
> TypeError: init_builtin() takes exactly 1 argument (0 given)
>
> The only source file that can generate that error is
> Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
>
>         PyErr_Format(PyExc_TypeError,
>                      "call takes exactly %d arguments XYZABC (%zd given)",
>                      inargs_index, actual_args);
>
> I do not see any difference after make clean and a full rebuild.  How is
> this possible?  I need to debug the arguments passed.

The message says "argument", the source code says "arguments" (I
suppose that you only added the XYZABC), so this cannot be source of
this exception.

grep for "given" in ceval.c

From skip.montanaro at gmail.com  Tue Jan 20 15:53:04 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 20 Jan 2015 08:53:04 -0600
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
Message-ID: <CANc-5UyxgmySv1F2pz87E+cWi2sh9wEA3R1f+AcidoWLD-73-g@mail.gmail.com>

On Tue, Jan 20, 2015 at 8:35 AM, Neil Girdhar <mistersheik at gmail.com> wrote:
>
> I get error:
>
> TypeError: init_builtin() takes exactly 1 argument (0 given)
>
> The only source file that can generate that error is Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
>
>         PyErr_Format(PyExc_TypeError,
>                      "call takes exactly %d arguments XYZABC (%zd given)",
>                      inargs_index, actual_args);
>
> I do not see any difference after make clean and a full rebuild.  How is this possible?  I need to debug the arguments passed.

Neil,

I'm a little bit confused. Why are you modifying the Python
interpreter to see if your code (presumably not part of the Python
interpreter) is being executed? I will take a stab at your question
though, and suggest you aren't actually running the interpreter you
just built.

Can you provide some more context for your question?

One last thing. Are you working on Python itself
(python-dev at python.org is the right place to ask questions) or using
Python to develop an application (python-dev is not the right place,
try python-list at python.org)?

Skip

From benjamin at python.org  Tue Jan 20 15:53:49 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 20 Jan 2015 09:53:49 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAP1=2W4=ckmP_EXpVL2oGY0xYsD0_rUCykad-kX+CmdxVym8Ug@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAP1=2W4=ckmP_EXpVL2oGY0xYsD0_rUCykad-kX+CmdxVym8Ug@mail.gmail.com>
Message-ID: <1421765629.3455027.216264349.5F0E8205@webmail.messagingengine.com>



On Tue, Jan 20, 2015, at 09:51, Brett Cannon wrote:
> This is a mailing to discuss the development *of* Python, not its *use*.
> You should be able to get help from python-list or #python on IRC.

To be fair, he's asking to debug his patch in
https://bugs.python.org/issue2292

From brett at python.org  Tue Jan 20 15:55:32 2015
From: brett at python.org (Brett Cannon)
Date: Tue, 20 Jan 2015 14:55:32 +0000
Subject: [Python-Dev] How do I ensure that my code is being executed?
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAP1=2W4=ckmP_EXpVL2oGY0xYsD0_rUCykad-kX+CmdxVym8Ug@mail.gmail.com>
 <1421765629.3455027.216264349.5F0E8205@webmail.messagingengine.com>
Message-ID: <CAP1=2W45TY6hEbFo84fDQXSp_QWJtnG=tf8RKmcZ6FbRbpm+vA@mail.gmail.com>

On Tue Jan 20 2015 at 9:53:52 AM Benjamin Peterson <benjamin at python.org>
wrote:

>
>
> On Tue, Jan 20, 2015, at 09:51, Brett Cannon wrote:
> > This is a mailing to discuss the development *of* Python, not its *use*.
> > You should be able to get help from python-list or #python on IRC.
>
> To be fair, he's asking to debug his patch in
> https://bugs.python.org/issue2292


Ah, sorry about that. The issue wasn't referenced in the email.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/2402859b/attachment.html>

From mistersheik at gmail.com  Tue Jan 20 15:54:43 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 09:54:43 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CANc-5UyxgmySv1F2pz87E+cWi2sh9wEA3R1f+AcidoWLD-73-g@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CANc-5UyxgmySv1F2pz87E+cWi2sh9wEA3R1f+AcidoWLD-73-g@mail.gmail.com>
Message-ID: <CAA68w_==04GL5oEMH9BeH0e4Be6RznPkq9Ks4JQ4uqYEphuNMw@mail.gmail.com>

Hi Skip,

I'm trying to finish the implementation of PEP 448.  I have updated the
patch to 3.5, fixed the grammar, and the ast.  There is a bug with the
argument counting or unpacking, which I can't seem to locate.

Best,

Neil

On Tue, Jan 20, 2015 at 9:53 AM, Skip Montanaro <skip.montanaro at gmail.com>
wrote:

> On Tue, Jan 20, 2015 at 8:35 AM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
> >
> > I get error:
> >
> > TypeError: init_builtin() takes exactly 1 argument (0 given)
> >
> > The only source file that can generate that error is
> Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
> >
> >         PyErr_Format(PyExc_TypeError,
> >                      "call takes exactly %d arguments XYZABC (%zd
> given)",
> >                      inargs_index, actual_args);
> >
> > I do not see any difference after make clean and a full rebuild.  How is
> this possible?  I need to debug the arguments passed.
>
> Neil,
>
> I'm a little bit confused. Why are you modifying the Python
> interpreter to see if your code (presumably not part of the Python
> interpreter) is being executed? I will take a stab at your question
> though, and suggest you aren't actually running the interpreter you
> just built.
>
> Can you provide some more context for your question?
>
> One last thing. Are you working on Python itself
> (python-dev at python.org is the right place to ask questions) or using
> Python to develop an application (python-dev is not the right place,
> try python-list at python.org)?
>
> Skip
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/bef77925/attachment-0001.html>

From mistersheik at gmail.com  Tue Jan 20 15:57:05 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 09:57:05 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAP1=2W45TY6hEbFo84fDQXSp_QWJtnG=tf8RKmcZ6FbRbpm+vA@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAP1=2W4=ckmP_EXpVL2oGY0xYsD0_rUCykad-kX+CmdxVym8Ug@mail.gmail.com>
 <1421765629.3455027.216264349.5F0E8205@webmail.messagingengine.com>
 <CAP1=2W45TY6hEbFo84fDQXSp_QWJtnG=tf8RKmcZ6FbRbpm+vA@mail.gmail.com>
Message-ID: <CAA68w_kNkS3Q-+_enby5XmvhR_t8aWVVXOJ74Rq1e58eMwOYNw@mail.gmail.com>

Sorry, I should have provided more context.

Best,

Neil

On Tue, Jan 20, 2015 at 9:55 AM, Brett Cannon <brett at python.org> wrote:

>
>
> On Tue Jan 20 2015 at 9:53:52 AM Benjamin Peterson <benjamin at python.org>
> wrote:
>
>>
>>
>> On Tue, Jan 20, 2015, at 09:51, Brett Cannon wrote:
>> > This is a mailing to discuss the development *of* Python, not its *use*.
>> > You should be able to get help from python-list or #python on IRC.
>>
>> To be fair, he's asking to debug his patch in
>> https://bugs.python.org/issue2292
>
>
> Ah, sorry about that. The issue wasn't referenced in the email.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/ff5f3d2e/attachment-0001.html>

From mistersheik at gmail.com  Tue Jan 20 15:59:27 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 09:59:27 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAAxjCEzP-VcN2OqEqoc2VUYk0B=C3ZK0YWucVGJcJ_-=8mk2pw@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAAxjCEzP-VcN2OqEqoc2VUYk0B=C3ZK0YWucVGJcJ_-=8mk2pw@mail.gmail.com>
Message-ID: <CAA68w_mw5pqGp2FuLazVHiL7eaGC-j4VHAsmDTkVWQV5djZ2NA@mail.gmail.com>

Good eye!  I did the following grep:

~/cpython: grep -R takes.exac *
Doc/c-api/bytes.rst:   Identical to :c:func:`PyBytes_FromFormat` except
that it takes exactly two
Doc/c-api/unicode.rst:   Identical to :c:func:`PyUnicode_FromFormat` except
that it takes exactly two
Doc/library/unittest.mock.rst:   TypeError: <lambda>() takes exactly 3
arguments (1 given)
Doc/whatsnew/2.0.rst:The ``\x`` escape in string literals now takes exactly
2 hex digits.  Previously
Lib/test/test_compileall.py:    def test_d_takes_exactly_one_dir(self):
Lib/test/test_inspect.py:        # f1 takes exactly 2 arguments
Lib/test/test_inspect.py:            # f1/f2 takes exactly/at most 2
arguments
Lib/tkinter/__init__.py:        # TypeError: setvar() takes exactly 3
arguments (2 given)
Modules/_ctypes/_ctypes.c:                     "call takes exactly %d
arguments xxx (%zd given)",
Objects/methodobject.c:                "%.200s() takes exactly one argument
(%zd given)",
Binary file Objects/methodobject.o matches
Binary file Programs/_freeze_importlib matches
Binary file Programs/_testembed matches
Python/ceval.c:                     "%.200s() takes exactly one argument
(%d given)",
Python/ceval.c.orig:                     "%.200s() takes exactly one
argument (%d given)",
Binary file Python/ceval.o matches
Binary file libpython3.5dm.a matches
Binary file python.exe matches

I'll keep searching?

On Tue, Jan 20, 2015 at 9:52 AM, Stefan Ring <stefanrin at gmail.com> wrote:

> On Tue, Jan 20, 2015 at 3:35 PM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
> > I get error:
> >
> > TypeError: init_builtin() takes exactly 1 argument (0 given)
> >
> > The only source file that can generate that error is
> > Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
> >
> >         PyErr_Format(PyExc_TypeError,
> >                      "call takes exactly %d arguments XYZABC (%zd
> given)",
> >                      inargs_index, actual_args);
> >
> > I do not see any difference after make clean and a full rebuild.  How is
> > this possible?  I need to debug the arguments passed.
>
> The message says "argument", the source code says "arguments" (I
> suppose that you only added the XYZABC), so this cannot be source of
> this exception.
>
> grep for "given" in ceval.c
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/aee27ce8/attachment-0001.html>

From mistersheik at gmail.com  Tue Jan 20 16:01:51 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 10:01:51 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAA68w_mw5pqGp2FuLazVHiL7eaGC-j4VHAsmDTkVWQV5djZ2NA@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAAxjCEzP-VcN2OqEqoc2VUYk0B=C3ZK0YWucVGJcJ_-=8mk2pw@mail.gmail.com>
 <CAA68w_mw5pqGp2FuLazVHiL7eaGC-j4VHAsmDTkVWQV5djZ2NA@mail.gmail.com>
Message-ID: <CAA68w_kKiyqfRwD0YgZHsY3-rf9HxKOwNqA_SHjzMJQhq_UsEw@mail.gmail.com>

Okay, found it thanks.

On Tue, Jan 20, 2015 at 9:59 AM, Neil Girdhar <mistersheik at gmail.com> wrote:

> Good eye!  I did the following grep:
>
> ~/cpython: grep -R takes.exac *
> Doc/c-api/bytes.rst:   Identical to :c:func:`PyBytes_FromFormat` except
> that it takes exactly two
> Doc/c-api/unicode.rst:   Identical to :c:func:`PyUnicode_FromFormat`
> except that it takes exactly two
> Doc/library/unittest.mock.rst:   TypeError: <lambda>() takes exactly 3
> arguments (1 given)
> Doc/whatsnew/2.0.rst:The ``\x`` escape in string literals now takes
> exactly 2 hex digits.  Previously
> Lib/test/test_compileall.py:    def test_d_takes_exactly_one_dir(self):
> Lib/test/test_inspect.py:        # f1 takes exactly 2 arguments
> Lib/test/test_inspect.py:            # f1/f2 takes exactly/at most 2
> arguments
> Lib/tkinter/__init__.py:        # TypeError: setvar() takes exactly 3
> arguments (2 given)
> Modules/_ctypes/_ctypes.c:                     "call takes exactly %d
> arguments xxx (%zd given)",
> Objects/methodobject.c:                "%.200s() takes exactly one
> argument (%zd given)",
> Binary file Objects/methodobject.o matches
> Binary file Programs/_freeze_importlib matches
> Binary file Programs/_testembed matches
> Python/ceval.c:                     "%.200s() takes exactly one argument
> (%d given)",
> Python/ceval.c.orig:                     "%.200s() takes exactly one
> argument (%d given)",
> Binary file Python/ceval.o matches
> Binary file libpython3.5dm.a matches
> Binary file python.exe matches
>
> I'll keep searching?
>
> On Tue, Jan 20, 2015 at 9:52 AM, Stefan Ring <stefanrin at gmail.com> wrote:
>
>> On Tue, Jan 20, 2015 at 3:35 PM, Neil Girdhar <mistersheik at gmail.com>
>> wrote:
>> > I get error:
>> >
>> > TypeError: init_builtin() takes exactly 1 argument (0 given)
>> >
>> > The only source file that can generate that error is
>> > Modules/_ctypes/_ctypes.c, but when I make changes to that file such as:
>> >
>> >         PyErr_Format(PyExc_TypeError,
>> >                      "call takes exactly %d arguments XYZABC (%zd
>> given)",
>> >                      inargs_index, actual_args);
>> >
>> > I do not see any difference after make clean and a full rebuild.  How is
>> > this possible?  I need to debug the arguments passed.
>>
>> The message says "argument", the source code says "arguments" (I
>> suppose that you only added the XYZABC), so this cannot be source of
>> this exception.
>>
>> grep for "given" in ceval.c
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/e84ce72b/attachment-0001.html>

From mistersheik at gmail.com  Tue Jan 20 16:09:27 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 10:09:27 -0500
Subject: [Python-Dev] How do I ensure that my code is being executed?
In-Reply-To: <CAA68w_kKiyqfRwD0YgZHsY3-rf9HxKOwNqA_SHjzMJQhq_UsEw@mail.gmail.com>
References: <CAA68w_=hodtbTwQHuoxT3T3VtnzbvGZMnv663gRkrd420vhXxQ@mail.gmail.com>
 <CAAxjCEzP-VcN2OqEqoc2VUYk0B=C3ZK0YWucVGJcJ_-=8mk2pw@mail.gmail.com>
 <CAA68w_mw5pqGp2FuLazVHiL7eaGC-j4VHAsmDTkVWQV5djZ2NA@mail.gmail.com>
 <CAA68w_kKiyqfRwD0YgZHsY3-rf9HxKOwNqA_SHjzMJQhq_UsEw@mail.gmail.com>
Message-ID: <CAA68w_kv8br==1Jx+gDYU559Fy-rusar0zk_jtVoz1ZeUgo3jg@mail.gmail.com>

(in fact, it was Python/getargs.c)

On Tue, Jan 20, 2015 at 10:01 AM, Neil Girdhar <mistersheik at gmail.com>
wrote:

> Okay, found it thanks.
>
> On Tue, Jan 20, 2015 at 9:59 AM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
>
>> Good eye!  I did the following grep:
>>
>> ~/cpython: grep -R takes.exac *
>> Doc/c-api/bytes.rst:   Identical to :c:func:`PyBytes_FromFormat` except
>> that it takes exactly two
>> Doc/c-api/unicode.rst:   Identical to :c:func:`PyUnicode_FromFormat`
>> except that it takes exactly two
>> Doc/library/unittest.mock.rst:   TypeError: <lambda>() takes exactly 3
>> arguments (1 given)
>> Doc/whatsnew/2.0.rst:The ``\x`` escape in string literals now takes
>> exactly 2 hex digits.  Previously
>> Lib/test/test_compileall.py:    def test_d_takes_exactly_one_dir(self):
>> Lib/test/test_inspect.py:        # f1 takes exactly 2 arguments
>> Lib/test/test_inspect.py:            # f1/f2 takes exactly/at most 2
>> arguments
>> Lib/tkinter/__init__.py:        # TypeError: setvar() takes exactly 3
>> arguments (2 given)
>> Modules/_ctypes/_ctypes.c:                     "call takes exactly %d
>> arguments xxx (%zd given)",
>> Objects/methodobject.c:                "%.200s() takes exactly one
>> argument (%zd given)",
>> Binary file Objects/methodobject.o matches
>> Binary file Programs/_freeze_importlib matches
>> Binary file Programs/_testembed matches
>> Python/ceval.c:                     "%.200s() takes exactly one argument
>> (%d given)",
>> Python/ceval.c.orig:                     "%.200s() takes exactly one
>> argument (%d given)",
>> Binary file Python/ceval.o matches
>> Binary file libpython3.5dm.a matches
>> Binary file python.exe matches
>>
>> I'll keep searching?
>>
>> On Tue, Jan 20, 2015 at 9:52 AM, Stefan Ring <stefanrin at gmail.com> wrote:
>>
>>> On Tue, Jan 20, 2015 at 3:35 PM, Neil Girdhar <mistersheik at gmail.com>
>>> wrote:
>>> > I get error:
>>> >
>>> > TypeError: init_builtin() takes exactly 1 argument (0 given)
>>> >
>>> > The only source file that can generate that error is
>>> > Modules/_ctypes/_ctypes.c, but when I make changes to that file such
>>> as:
>>> >
>>> >         PyErr_Format(PyExc_TypeError,
>>> >                      "call takes exactly %d arguments XYZABC (%zd
>>> given)",
>>> >                      inargs_index, actual_args);
>>> >
>>> > I do not see any difference after make clean and a full rebuild.  How
>>> is
>>> > this possible?  I need to debug the arguments passed.
>>>
>>> The message says "argument", the source code says "arguments" (I
>>> suppose that you only added the XYZABC), so this cannot be source of
>>> this exception.
>>>
>>> grep for "given" in ceval.c
>>>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/7a336485/attachment.html>

From mistersheik at gmail.com  Tue Jan 20 17:34:33 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 11:34:33 -0500
Subject: [Python-Dev] =?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
	=?utf-8?q?uestion_regarding_test=5Fast?=
Message-ID: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>

My question first:
test_ast is mostly generated code, but I can't find where it is being
generated.  I am pretty sure I know how to fix most of the introduced
problems.  Who is generating test_ast??

Update:

So far, I've done the following:

Updated the patch to 3.5
Fixed the grammar to accept final commas in argument lists always, and to
work with the already implemented code.
Fixed the ast to accept what it needs to accept and reject according to the
limitation laid down by Guido.
Fixed the parsing library.

Fixed these tests:
test_ast.py
test_extcall.py
test_grammar.py
test_syntax.py
test_unpack_ex.py

As far as I can tell, all I have left is to fix test_ast and possibly write
some more tests (there are already some new tests and some of the old
negative tests expecting SyntaxError are now positive tests).

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/74031990/attachment.html>

From benjamin at python.org  Tue Jan 20 17:36:05 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 20 Jan 2015 11:36:05 -0500
Subject: [Python-Dev]
 =?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
 =?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
Message-ID: <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>



On Tue, Jan 20, 2015, at 11:34, Neil Girdhar wrote:
> My question first:
> test_ast is mostly generated code, but I can't find where it is being
> generated.  I am pretty sure I know how to fix most of the introduced
> problems.  Who is generating test_ast??

It generates itself.

From guido at python.org  Tue Jan 20 17:40:10 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Jan 2015 08:40:10 -0800
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <20150120114810.4ea87715@x230>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
 <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
 <54BDE456.70506@canterbury.ac.nz> <20150120114810.4ea87715@x230>
Message-ID: <CAP7+vJL8ED6qBwa69ExXGkuZA_08QykTeYV91aFPZCr5hFuTNw@mail.gmail.com>

On Tue, Jan 20, 2015 at 1:48 AM, Paul Sokolovsky <pmiscml at gmail.com> wrote:

> The point of inplace operations (memoryview's, other stuff already in
> Python) is to avoid unneeded memory allocation and copying. For 1Tb
> bytearray with 1Tb of RAM, it will be very hard to do. (Ditto for 100K
> bytearray with 150K RAM.)
>

So you'll have to figure a better way to do it. We're not going to add
inplace_lower(), and that's the final word. (Of course you can add it to
microPython as an extension of the language.)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/8c914058/attachment.html>

From donald at stufft.io  Tue Jan 20 17:42:55 2015
From: donald at stufft.io (Donald Stufft)
Date: Tue, 20 Jan 2015 11:42:55 -0500
Subject: [Python-Dev] Possible "REMOTE HOST IDENTIFICATION HAS CHANGED!"
	Error.
Message-ID: <2584297A-9F79-4F6F-85D3-843A132226D9@stufft.io>

Just a heads up that people might see a "REMOTE HOST IDENTIFICATION HAS
CHANGED!" error when connecting to hg.python.org's SSH (or any other PSF
machine). The reason for this is that previously we allowed RSA, ECDSA, and 
ED25519 host keys. However ECDSA relies on having an unbiased random number
generator on every connection and any bias in the random numbers can leak the
private key. Since these are running on VMs where we don't know for sure what
the quality is of the random numbers I've disabled the ECDSA host key.

The impact of this is if you had previously connected to a PSF machine, and
your client had the ECDSA key in your ~/.ssh/known_hosts file, that you'll
see an error like:

    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
    Someone could be eavesdropping on you right now (man-in-the-middle attack)!
    It is also possible that a host key has just been changed.

The remediation is to remove the ECDSA for the PSF servers from your known
hosts and connect again and accept either the RSA or the ED25519 key when it
presents it.

The fingerprints for hg.python.org for both of those keys are:

$ ssh-keygen -lf /etc/ssh/ssh_host_rsa_key.pub
2048 a0:12:52:50:4a:4b:db:43:ac:65:26:b6:6f:0a:f7:b8 /etc/ssh/ssh_host_rsa_key.pub (RSA)
$ ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub
256 1d:02:d1:d2:7b:a1:cb:e0:51:65:25:d7:19:dd:4e:74 /etc/ssh/ssh_host_ed25519_key.pub (ED25519)

Sorry for any inconvience this causes!

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From trent at snakebite.org  Tue Jan 20 17:51:31 2015
From: trent at snakebite.org (Trent Nelson)
Date: Tue, 20 Jan 2015 11:51:31 -0500
Subject: [Python-Dev] bytes & bytearray
In-Reply-To: <20150120114810.4ea87715@x230>
References: <54BD32A7.5070206@stoneleaf.us> <20150119214340.44fe0e7f@x230>
 <CAP7+vJK+dZfz7U8B7HkL6SGJphAAQQ2sqS9sqG_BsuWC87KAwQ@mail.gmail.com>
 <54BDE456.70506@canterbury.ac.nz> <20150120114810.4ea87715@x230>
Message-ID: <20150120165130.GA10241@snakebite.org>

On Tue, Jan 20, 2015 at 11:48:10AM +0200, Paul Sokolovsky wrote:
> Hello,
> 
> On Tue, 20 Jan 2015 18:15:02 +1300
> Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> 
> > Guido van Rossum wrote:
> > > On Mon, Jan 19, 2015 at 11:43 AM, Paul Sokolovsky
> > > <pmiscml at gmail.com <mailto:pmiscml at gmail.com>> wrote:
> > > 
> > >     b.lower_inplace()
> > >     b.lower_i()
> > > 
> > > Please don't go there. The use cases are too rare.
> > 
> > And if you have such a use case, it's not too
> > hard to do
> > 
> >    b[:] = b.lower()
> 
> The point of inplace operations (memoryview's, other stuff already in
> Python) is to avoid unneeded memory allocation and copying. For 1Tb
> bytearray with 1Tb of RAM, it will be very hard to do. (Ditto for 100K
> bytearray with 150K RAM.)

You can just loop through the bytearray and assign elements.  I use
something along the lines of this for PyParallel where I'm operating on
bytearrays that are backed by underlying socket buffers, where I don't
want to do any memory allocations/reallocations:

    def toupper_bytes(data):
        assert isinstance(data, bytearray)
        a = ord('a')
        z = ord('z')
        for i in range(0, len(data)):
            c = data[i]
            if c >= a and c <= z:
                data[i] = c - 32

Low overhead, mostly stays within the same ceval frame.  Should be a
walk in the park for PyPy, Cython or Numba to optimize, too.

    Trent.

From mistersheik at gmail.com  Tue Jan 20 18:06:29 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 12:06:29 -0500
Subject: [Python-Dev]
	=?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
	=?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>
Message-ID: <CAA68w_nfyVWSJqq77s5OOrQ8azjuzVBBMj2St=4LJRfS-cVyRg@mail.gmail.com>

Hi Benjamin,

I'm having trouble finding where it is generating the lines below

#### EVERYTHING BELOW IS GENERATED #####

Neither a call to test_ast nor a make (in case it's generated somewhere
else) regenerate those lines if they have been removed.

How were those lines generated?

Best,
Neil


On Tue, Jan 20, 2015 at 11:36 AM, Benjamin Peterson <benjamin at python.org>
wrote:

>
>
> On Tue, Jan 20, 2015, at 11:34, Neil Girdhar wrote:
> > My question first:
> > test_ast is mostly generated code, but I can't find where it is being
> > generated.  I am pretty sure I know how to fix most of the introduced
> > problems.  Who is generating test_ast??
>
> It generates itself.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/77b16fc8/attachment.html>

From benjamin at python.org  Tue Jan 20 18:09:39 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 20 Jan 2015 12:09:39 -0500
Subject: [Python-Dev]
 =?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
 =?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <CAA68w_nfyVWSJqq77s5OOrQ8azjuzVBBMj2St=4LJRfS-cVyRg@mail.gmail.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>
 <CAA68w_nfyVWSJqq77s5OOrQ8azjuzVBBMj2St=4LJRfS-cVyRg@mail.gmail.com>
Message-ID: <1421773779.3497784.216329401.3D487CE9@webmail.messagingengine.com>

$ ./python Lib/test/test_ast.py -g
exec_results = [
('Module', [('Expr', (1, 0), ('NameConstant', (1, 0), None))]),
('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
None, []), [('Pass', (1, 9))], [], None)]),
('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
'a', None)], None, [], [], None, []), [('Pass', (1, 10))], [], None)]),
('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
'a', None)], None, [], [], None, [('Num', (1, 8), 0)]), [('Pass', (1,
12))], [], None)]),
('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], ('arg', (1,
7), 'args', None), [], [], None, []), [('Pass', (1, 14))], [], None)]),
('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
('arg', (1, 8), 'kwargs', None), []), [('Pass', (1, 17))], [], None)]),
....

On Tue, Jan 20, 2015, at 12:06, Neil Girdhar wrote:
> Hi Benjamin,
> 
> I'm having trouble finding where it is generating the lines below
> 
> #### EVERYTHING BELOW IS GENERATED #####
> 
> Neither a call to test_ast nor a make (in case it's generated somewhere
> else) regenerate those lines if they have been removed.
> 
> How were those lines generated?
> 
> Best,
> Neil
> 
> 
> On Tue, Jan 20, 2015 at 11:36 AM, Benjamin Peterson <benjamin at python.org>
> wrote:
> 
> >
> >
> > On Tue, Jan 20, 2015, at 11:34, Neil Girdhar wrote:
> > > My question first:
> > > test_ast is mostly generated code, but I can't find where it is being
> > > generated.  I am pretty sure I know how to fix most of the introduced
> > > problems.  Who is generating test_ast??
> >
> > It generates itself.
> >

From mistersheik at gmail.com  Tue Jan 20 18:10:07 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 12:10:07 -0500
Subject: [Python-Dev]
	=?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
	=?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <1421773779.3497784.216329401.3D487CE9@webmail.messagingengine.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>
 <CAA68w_nfyVWSJqq77s5OOrQ8azjuzVBBMj2St=4LJRfS-cVyRg@mail.gmail.com>
 <1421773779.3497784.216329401.3D487CE9@webmail.messagingengine.com>
Message-ID: <CAA68w_=X3-Zx0Md9_HcOXe9s7a6YoOz2GQc3Ew7Eve9NjXfZgQ@mail.gmail.com>

Thanks!

On Tue, Jan 20, 2015 at 12:09 PM, Benjamin Peterson <benjamin at python.org>
wrote:

> $ ./python Lib/test/test_ast.py -g
> exec_results = [
> ('Module', [('Expr', (1, 0), ('NameConstant', (1, 0), None))]),
> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
> None, []), [('Pass', (1, 9))], [], None)]),
> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
> 'a', None)], None, [], [], None, []), [('Pass', (1, 10))], [], None)]),
> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
> 'a', None)], None, [], [], None, [('Num', (1, 8), 0)]), [('Pass', (1,
> 12))], [], None)]),
> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], ('arg', (1,
> 7), 'args', None), [], [], None, []), [('Pass', (1, 14))], [], None)]),
> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
> ('arg', (1, 8), 'kwargs', None), []), [('Pass', (1, 17))], [], None)]),
> ....
>
> On Tue, Jan 20, 2015, at 12:06, Neil Girdhar wrote:
> > Hi Benjamin,
> >
> > I'm having trouble finding where it is generating the lines below
> >
> > #### EVERYTHING BELOW IS GENERATED #####
> >
> > Neither a call to test_ast nor a make (in case it's generated somewhere
> > else) regenerate those lines if they have been removed.
> >
> > How were those lines generated?
> >
> > Best,
> > Neil
> >
> >
> > On Tue, Jan 20, 2015 at 11:36 AM, Benjamin Peterson <benjamin at python.org
> >
> > wrote:
> >
> > >
> > >
> > > On Tue, Jan 20, 2015, at 11:34, Neil Girdhar wrote:
> > > > My question first:
> > > > test_ast is mostly generated code, but I can't find where it is being
> > > > generated.  I am pretty sure I know how to fix most of the introduced
> > > > problems.  Who is generating test_ast??
> > >
> > > It generates itself.
> > >
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/ed42f557/attachment.html>

From mistersheik at gmail.com  Tue Jan 20 18:41:05 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Tue, 20 Jan 2015 12:41:05 -0500
Subject: [Python-Dev]
	=?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
	=?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <CAA68w_=X3-Zx0Md9_HcOXe9s7a6YoOz2GQc3Ew7Eve9NjXfZgQ@mail.gmail.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <1421771765.3486982.216313333.7ED0D828@webmail.messagingengine.com>
 <CAA68w_nfyVWSJqq77s5OOrQ8azjuzVBBMj2St=4LJRfS-cVyRg@mail.gmail.com>
 <1421773779.3497784.216329401.3D487CE9@webmail.messagingengine.com>
 <CAA68w_=X3-Zx0Md9_HcOXe9s7a6YoOz2GQc3Ew7Eve9NjXfZgQ@mail.gmail.com>
Message-ID: <CAA68w_=1ajcXFjTurY2t6WA7qXXX9qz5xQRdkCxoyMz+VhjbXQ@mail.gmail.com>

Okay, I think it's ready for a code review.  Would anyone be kind enough to
offer comments?

On Tue, Jan 20, 2015 at 12:10 PM, Neil Girdhar <mistersheik at gmail.com>
wrote:

> Thanks!
>
> On Tue, Jan 20, 2015 at 12:09 PM, Benjamin Peterson <benjamin at python.org>
> wrote:
>
>> $ ./python Lib/test/test_ast.py -g
>> exec_results = [
>> ('Module', [('Expr', (1, 0), ('NameConstant', (1, 0), None))]),
>> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
>> None, []), [('Pass', (1, 9))], [], None)]),
>> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
>> 'a', None)], None, [], [], None, []), [('Pass', (1, 10))], [], None)]),
>> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [('arg', (1, 6),
>> 'a', None)], None, [], [], None, [('Num', (1, 8), 0)]), [('Pass', (1,
>> 12))], [], None)]),
>> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], ('arg', (1,
>> 7), 'args', None), [], [], None, []), [('Pass', (1, 14))], [], None)]),
>> ('Module', [('FunctionDef', (1, 0), 'f', ('arguments', [], None, [], [],
>> ('arg', (1, 8), 'kwargs', None), []), [('Pass', (1, 17))], [], None)]),
>> ....
>>
>> On Tue, Jan 20, 2015, at 12:06, Neil Girdhar wrote:
>> > Hi Benjamin,
>> >
>> > I'm having trouble finding where it is generating the lines below
>> >
>> > #### EVERYTHING BELOW IS GENERATED #####
>> >
>> > Neither a call to test_ast nor a make (in case it's generated somewhere
>> > else) regenerate those lines if they have been removed.
>> >
>> > How were those lines generated?
>> >
>> > Best,
>> > Neil
>> >
>> >
>> > On Tue, Jan 20, 2015 at 11:36 AM, Benjamin Peterson <
>> benjamin at python.org>
>> > wrote:
>> >
>> > >
>> > >
>> > > On Tue, Jan 20, 2015, at 11:34, Neil Girdhar wrote:
>> > > > My question first:
>> > > > test_ast is mostly generated code, but I can't find where it is
>> being
>> > > > generated.  I am pretty sure I know how to fix most of the
>> introduced
>> > > > problems.  Who is generating test_ast??
>> > >
>> > > It generates itself.
>> > >
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/dd9b413b/attachment-0001.html>

From guido at python.org  Tue Jan 20 18:48:40 2015
From: guido at python.org (Guido van Rossum)
Date: Tue, 20 Jan 2015 09:48:40 -0800
Subject: [Python-Dev] [Python-ideas] What's going on with PEP 448 -
 Additional Unpacking Generalizations ?
In-Reply-To: <CAA68w_=MY+CmqpZ5Ni=FwLFpjAGhnJ_yrZu+wmAW+fS74dD1Gg@mail.gmail.com>
References: <f97552ab-bfb8-434b-bc68-5120d394682e@googlegroups.com>
 <CAPTjJmpGkgh0onpJVBYB4KXOi-n8HRDx2R55XJFteO=ty3FRVQ@mail.gmail.com>
 <CAP7+vJKKmwuWq+4Ps2drFuQA5ParxdPKe_XDuAFATiQrZ6TZMQ@mail.gmail.com>
 <43272cca-20ea-4337-bd2a-39bdf981048f@googlegroups.com>
 <CAPTjJmrjEm2GuRC=JNeEb7oRZ6xJAYPVHw9Fp0oAOWePPrRRbA@mail.gmail.com>
 <CAA68w_=_Gt1CsT_S4SHjK5fKKxSUHCt0J=4zSnjU5jMqQ6L=FA@mail.gmail.com>
 <CAPTjJmrUu72Z=UBa6ej7+zV34O-hco-XovYrZ77pG8O29nBnwQ@mail.gmail.com>
 <CAPTjJmrOCo5pzXCUe_nQ44X80FQMfb15QqiTx6AU14Wx7RDSwA@mail.gmail.com>
 <CAA68w_n6PB_rMVdKxG5v2CcoDSJsSfC3+goCJ6mNFanhAUO7tA@mail.gmail.com>
 <26294fbf-d883-410c-b7ec-cd82697ddd71@googlegroups.com>
 <CAN1F8qUzFUSdaKg_WP6d_2BqHGKY1iW_18M+9zPGEM=7kkF5jg@mail.gmail.com>
 <CAN1F8qVLEsm6PxwfHAho7v0ktMmK+Ym3V78e+nFhar4j67LuJQ@mail.gmail.com>
 <CAA68w_kTY5r6UCHZnYiBnA3ZbncJYA0eytysfo=d+RzgmTht+g@mail.gmail.com>
 <CAP7+vJJ-D-xG9NPDo1TH33YFE8O_aB5hwf5FCdgQnSYzo6OTUw@mail.gmail.com>
 <CAN1F8qUXeCPeUknp863XEk=JNAtc+75N5B_r87CA6vy2JtjEzw@mail.gmail.com>
 <CAA68w_m1Qo0dfQkqpoP=egHC8kpF18a8iT+5mJGm08gabqsXvg@mail.gmail.com>
 <CAP7+vJKBLN0jzQ-yr=qRL97B+1N+y819bW_nXbn+CLdJv4+2Dw@mail.gmail.com>
 <CAA68w_=MY+CmqpZ5Ni=FwLFpjAGhnJ_yrZu+wmAW+fS74dD1Gg@mail.gmail.com>
Message-ID: <CAP7+vJJszvJJ-zNQ5T7rk9+yJ2J4G_2j7TPE-3Re0ABXYe9Xmw@mail.gmail.com>

Thanks very much for your work! I am CC'ing python-dev to see if there are
any last calls for PEP 448. Assuming no material objection appear to the
new syntax and semantics, I can approve the PEP later this week. To get it
committed, you need one of the active committers to give you a code review
(waiting for me would mean waiting forever). Maybe Antoine, Benjamin or
Victor?

On Tue, Jan 20, 2015 at 9:39 AM, Neil Girdhar <mistersheik at gmail.com> wrote:

> I believe I have finally finished the work on the patch for PEP 448 (
> http://bugs.python.org/issue2292).  How do we get the PEP approved?  What
> else would we need to check it into Python?
>
> Best,
>
> Neil
>
> On Tue, Jan 20, 2015 at 12:03 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>> OK. I don't like arg unpackings after keyword args, for the same reason
>> plain positional args aren't allowed after keyword args, but I guess I
>> didn't pay attention when it was introduced, so we're stuck with it now,
>> it's not the end of the world, and at least the definition is clear
>> (collect all positional args first, then handle keyword args).
>>
>> On Tue, Jan 20, 2015 at 8:51 AM, Neil Girdhar <mistersheik at gmail.com>
>> wrote:
>>
>>>
>>>
>>> Okay, so: positional arguments neither follow keyword arguments nor
>>> keyword argument unpackings; iterable argument unpackings never follow
>>> keyword argument unpackings.
>>>
>>>
>>>
>>> On Tue, Jan 20, 2015 at 11:47 AM, Joshua Landau <joshua at landau.ws>
>>> wrote:
>>>
>>>> On 20 January 2015 at 16:38, Guido van Rossum <guido at python.org> wrote:
>>>> > The PEP hasn't been accepted yet AFAIK...  I'm generally okay with
>>>> allowing
>>>> > multiple *x things (except in an *unpack* position of course) but I
>>>> still
>>>> > don't think we should be mixing positional and keyword args. So, no
>>>> f(a,
>>>> > b=2, c), nor f(a, b=2, *c).
>>>> >
>>>>
>>>> f(a, b=2, *c) is currently legal as both a call and as a definition:
>>>>
>>>>     a, *c = 1, 2, 3
>>>>
>>>>     def f(*args, **kwargs):
>>>>         print(args, kwargs)
>>>>
>>>>     f(a, b=2, *c)
>>>>     #>>> (1, 2, 3) {'b': 2}
>>>>
>>>>     def f(a, b=2, *c):
>>>>         print(a, b, c)
>>>>
>>>>     f(1, 2, 3)
>>>>     #>>> 1 2 (3,)
>>>>
>>>> So I imagine that's staying (or, at least, this PEP isn't removing
>>>> it). I don't think anyone is (yet) arguing for f(a, b=2, c).
>>>> _______________________________________________
>>>> Python-ideas mailing list
>>>> Python-ideas at python.org
>>>> https://mail.python.org/mailman/listinfo/python-ideas
>>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>>
>>>> --
>>>>
>>>> ---
>>>> You received this message because you are subscribed to a topic in the
>>>> Google Groups "python-ideas" group.
>>>> To unsubscribe from this topic, visit
>>>> https://groups.google.com/d/topic/python-ideas/J99EFY1D1nI/unsubscribe.
>>>> To unsubscribe from this group and all its topics, send an email to
>>>> python-ideas+unsubscribe at googlegroups.com.
>>>> For more options, visit https://groups.google.com/d/optout.
>>>>
>>>
>>>
>>> _______________________________________________
>>> Python-ideas mailing list
>>> Python-ideas at python.org
>>> https://mail.python.org/mailman/listinfo/python-ideas
>>> Code of Conduct: http://python.org/psf/codeofconduct/
>>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150120/7efd296e/attachment.html>

From mistersheik at gmail.com  Wed Jan 21 11:16:26 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Wed, 21 Jan 2015 05:16:26 -0500
Subject: [Python-Dev] Why does STORE_MAP not take a parameter?
Message-ID: <CAA68w_manHmXq--6-5H1q6JoiOi3xf2njJ=6ZwCr6GiaCJPfFg@mail.gmail.com>

Building argument lists and dicts in python entails the following opcode
pattern:

  1           0 BUILD_MAP                3
              3 LOAD_CONST               0 (2)
              6 LOAD_CONST               1 (1)
              9 STORE_MAP
             10 LOAD_CONST               2 (4)
             13 LOAD_CONST               3 (3)
             16 STORE_MAP
             17 LOAD_CONST               4 (6)
             20 LOAD_CONST               5 (5)
             23 STORE_MAP

Building lists on the other hand works like this:

  1           0 LOAD_CONST               0 (1)
              3 LOAD_CONST               1 (2)
              6 LOAD_CONST               2 (3)
              9 BUILD_LIST               3


Why not have BUILD_MAP work like BUILD_LIST?   I.e., STORE_MAP takes a
parameter n and adds the last n pairs of stack elements into the n-1 stack
element (the dictionary).

This means that the data would live on the stack until it is all suddenly
transferred to the dict, but I'm not sure about the effect on performance
(whether positive or negative).

It does save on instruction cache misses.

It should be an easy change to implement since all calls to BUILD_MAP could
be replaced with "BUILD_MAP 1" and then optimized later.

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150121/3130a8fe/attachment.html>

From PHIL at mail.etsu.edu  Wed Jan 21 12:07:46 2015
From: PHIL at mail.etsu.edu (Pfeiffer, Phillip E., IV)
Date: Wed, 21 Jan 2015 11:07:46 +0000
Subject: [Python-Dev] incorrect docstring for sys.int_info.sizeof_digit?
Message-ID: <CFCA2EF1B5196F488C2E59334AFDB90E051F12@ETSUMB2.etsu.edu>

Apologies if this has already been reported; I couldn't find a readily searchable archive for this mailing list (and apologies if I've just missed it).

Issue: the docstring for sys.int_info_.sizeof_digit appears to be amiss; see below.

------
Microsoft Windows [Version 6.1.7601]
Copyright (c) 2009 Microsoft Corporation.  All rights reserved.

C:\Windows\system32>python
Python 3.4.2 (v3.4.2:ab2c023a9432, Oct  6 2014, 22:16:31) [MSC v.1600 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> from sys import int_info
>>> print(sys.int_info.sizeof_digit.__doc__)
int(x=0) -> integer
int(x, base=10) -> integer

Convert a number or string to an integer, or return 0 if no arguments
are given.  If x is a number, return x.__int__().  For floating point
numbers, this truncates towards zero.

If x is not a number or if base is given, then x must be a string,
bytes, or bytearray instance representing an integer literal in the
given base.  The literal can be preceded by '+' or '-' and be surrounded
by whitespace.  The base defaults to 10.  Valid bases are 0 and 2-36.
Base 0 means to interpret the base from the string as an integer literal.
>>> int('0b100', base=0)
4
>>>


-- Phil

===================
 Phil Pfeiffer/ East TN State U.                |  Kindness in thought leads to wisdom.
 Dept. of Computing / P.O. Box 70711   |  Kindness in speech leads to eloquence.
 Johnson City, TN.  37614-1266              |  Kindness in action leads to love.
phil (at) etsu.edu  (423) 439-5355           |                            -- Lao-Tsu

From PHIL at mail.etsu.edu  Wed Jan 21 12:16:26 2015
From: PHIL at mail.etsu.edu (Pfeiffer, Phillip E., IV)
Date: Wed, 21 Jan 2015 11:16:26 +0000
Subject: [Python-Dev] sorry for the last e-mail
Message-ID: <CFCA2EF1B5196F488C2E59334AFDB90E051F3E@ETSUMB2.etsu.edu>

Apologies for being stupid.  I've been away from Python for awhile, and just realized that I'm simply seeing the docstring for int when I check out sys.int_info.whatever, where .whatever is an int-valued attribute.

While it would be nice to have a clearer description of the sys.int_info's attributes in the sys.int_info docstring, that's a different kettle of fish.

Again, apologies.

-- Phil

===================
 Phil Pfeiffer/ East TN State U.                |  Kindness in thought leads to wisdom.
 Dept. of Computing / P.O. Box 70711   |  Kindness in speech leads to eloquence.
 Johnson City, TN.  37614-1266              |  Kindness in action leads to love.
phil (at) etsu.edu  (423) 439-5355           |                            -- Lao-Tsu

From mail at timgolden.me.uk  Wed Jan 21 15:53:19 2015
From: mail at timgolden.me.uk (Tim Golden)
Date: Wed, 21 Jan 2015 14:53:19 +0000
Subject: [Python-Dev] incorrect docstring for sys.int_info.sizeof_digit?
In-Reply-To: <CFCA2EF1B5196F488C2E59334AFDB90E051F12@ETSUMB2.etsu.edu>
References: <CFCA2EF1B5196F488C2E59334AFDB90E051F12@ETSUMB2.etsu.edu>
Message-ID: <54BFBD5F.1000903@timgolden.me.uk>

On 21/01/2015 11:07, Pfeiffer, Phillip E., IV wrote:
> Apologies if this has already been reported; I couldn't find a
> readily searchable archive for this mailing list (and apologies if
> I've just missed it).

Depending on "readily searchable", this isn't too bad:

http://markmail.org/search/?q=list%3Aorg.python.python-dev+integer+docstring

TJG

From rdmurray at bitdance.com  Wed Jan 21 16:00:50 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 21 Jan 2015 10:00:50 -0500
Subject: [Python-Dev] incorrect docstring for sys.int_info.sizeof_digit?
In-Reply-To: <54BFBD5F.1000903@timgolden.me.uk>
References: <CFCA2EF1B5196F488C2E59334AFDB90E051F12@ETSUMB2.etsu.edu>
 <54BFBD5F.1000903@timgolden.me.uk>
Message-ID: <20150121150050.D6482250EF1@webabinitio.net>

On Wed, 21 Jan 2015 14:53:19 +0000, Tim Golden <mail at timgolden.me.uk> wrote:
> On 21/01/2015 11:07, Pfeiffer, Phillip E., IV wrote:
> > Apologies if this has already been reported; I couldn't find a
> > readily searchable archive for this mailing list (and apologies if
> > I've just missed it).
> 
> Depending on "readily searchable", this isn't too bad:
> 
> http://markmail.org/search/?q=list%3Aorg.python.python-dev+integer+docstring

But, if you are searching to see if a bug has been reported, you want to
search the tracker at bugs.python.org, not the python-dev mailing list.

--David

From chaselton at gmail.com  Wed Jan 21 23:43:19 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Wed, 21 Jan 2015 16:43:19 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
Message-ID: <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>

On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com> wrote:
> On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>> Hello,
>> I'm struggling with a build issue on Android; I've posted to the
>> general python list with no result, so I'm re-posting here in hopes
>> that someone can help.  If this is the wrong place feel free to let me
>> know.
>>
>> I'm attempting to build Python 2.7.8 on my Android device; I'm using
>> an environment that simulates a Linux filesystem within the Android
>> terminal using a port of fakechroot.  Within that environment I've
>> ported and/or bootstrapped a number of Linux utilities (curl, git,
>> openssl, gcc)
>>
>> I run ./configure, then make, and the executable and library are
>> built.  The problem occurs when build_ext is run; the newly built
>> python executable builds, then links _struct, and immediately
>> afterwards I get an 'undefined reference to dlopen' error.
>>
>> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> --libraries='c dl m' -f, the 'undefined reference to dlopen' error is
>> thrown again.
>>
>> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> --libraries='-lc -ldl -lm' -f the build continues past _struct...even
>> though ld throws the expected 'unable to find -l-lc' and other errors.
>>
>> Let me know if you need me to provide additional information.  Any
>> help would be greatly appreciated.
>>
>> Cyd
>
>
> Additionally I took a strace of the error producing command. The
> following is (hopefully) a relevant portion minus the various 'no such
> file' lines before the correct lib is found (which it always is)
>
> 16513 open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.py",
> O_RDONLY|O_LARGEFILE) = 3           16513
> open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.pyc",
> O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
>  = 16525
> 16513 wait4(16525,  <unfinished ...>
> 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
> O_RDONLY|O_LARGEFILE) = 3
> 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
> "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
> build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars */])
> = 0
>
> *snip call to libc intercepted by libfakechroot*
>
> 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
> 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
> 16525 open("/dev/__properties__",
> O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
> 16525 open("build/temp.linux-armv7l-2.7/multiarch",
> O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
> O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> 16525 fork()                            = 16526
> 16525 wait4(-1,  <unfinished ...>
> 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE, 0666)
> = -1 EACCES (Permission denied)
>
> See attached for remainder

Should I be posting this issue elsewhere?
Is more information needed?

From guido at python.org  Wed Jan 21 23:53:16 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 21 Jan 2015 14:53:16 -0800
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
Message-ID: <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>

Maybe try a list focused on Android development? Few people in the Python
core development community have any Android experience. But the issues and
context you offer seem to be related to the Android world, not to Python.
(dlopen is used by a lot of systems, not just Python.)

On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com> wrote:
> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >> Hello,
> >> I'm struggling with a build issue on Android; I've posted to the
> >> general python list with no result, so I'm re-posting here in hopes
> >> that someone can help.  If this is the wrong place feel free to let me
> >> know.
> >>
> >> I'm attempting to build Python 2.7.8 on my Android device; I'm using
> >> an environment that simulates a Linux filesystem within the Android
> >> terminal using a port of fakechroot.  Within that environment I've
> >> ported and/or bootstrapped a number of Linux utilities (curl, git,
> >> openssl, gcc)
> >>
> >> I run ./configure, then make, and the executable and library are
> >> built.  The problem occurs when build_ext is run; the newly built
> >> python executable builds, then links _struct, and immediately
> >> afterwards I get an 'undefined reference to dlopen' error.
> >>
> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >> --libraries='c dl m' -f, the 'undefined reference to dlopen' error is
> >> thrown again.
> >>
> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >> --libraries='-lc -ldl -lm' -f the build continues past _struct...even
> >> though ld throws the expected 'unable to find -l-lc' and other errors.
> >>
> >> Let me know if you need me to provide additional information.  Any
> >> help would be greatly appreciated.
> >>
> >> Cyd
> >
> >
> > Additionally I took a strace of the error producing command. The
> > following is (hopefully) a relevant portion minus the various 'no such
> > file' lines before the correct lib is found (which it always is)
> >
> > 16513
> open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.py",
> > O_RDONLY|O_LARGEFILE) = 3           16513
> >
> open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.pyc",
> > O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
> >  = 16525
> > 16513 wait4(16525,  <unfinished ...>
> > 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
> > O_RDONLY|O_LARGEFILE) = 3
> > 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
> > "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
> > build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars */])
> > = 0
> >
> > *snip call to libc intercepted by libfakechroot*
> >
> > 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
> > 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
> > 16525 open("/dev/__properties__",
> > O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
> > 16525 open("build/temp.linux-armv7l-2.7/multiarch",
> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> > 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> > 16525 fork()                            = 16526
> > 16525 wait4(-1,  <unfinished ...>
> > 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE, 0666)
> > = -1 EACCES (Permission denied)
> >
> > See attached for remainder
>
> Should I be posting this issue elsewhere?
> Is more information needed?
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150121/cc34d472/attachment.html>

From greg.ewing at canterbury.ac.nz  Thu Jan 22 00:57:30 2015
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 22 Jan 2015 12:57:30 +1300
Subject: [Python-Dev] Why does STORE_MAP not take a parameter?
In-Reply-To: <CAA68w_manHmXq--6-5H1q6JoiOi3xf2njJ=6ZwCr6GiaCJPfFg@mail.gmail.com>
References: <CAA68w_manHmXq--6-5H1q6JoiOi3xf2njJ=6ZwCr6GiaCJPfFg@mail.gmail.com>
Message-ID: <54C03CEA.2080305@canterbury.ac.nz>

On 01/21/2015 11:16 PM, Neil Girdhar wrote:
> Why not have BUILD_MAP work like BUILD_LIST?   I.e., STORE_MAP takes a
> parameter n and adds the last n pairs of stack elements into the n-1 stack
> element (the dictionary).

It probably wouldn't make much difference. Building a list is
substantially cheaper if you have all the items on hand and can
copy them in en masse. But adding an item to a dict entails
quite a lot of overhead, since you need to hash the key, look
for a free slot, etc. and this would likely swamp any gain
from executing less bytecodes. And on the other side of the
equation, evaluating the items one at a time requires less
stack space, so the stack frame can be smaller.

But as always, you can't be sure without measuring it, and
this would be a good thing for someone interested to try out.

-- 
Greg


From agriff at tin.it  Thu Jan 22 09:11:52 2015
From: agriff at tin.it (Andrea Griffini)
Date: Thu, 22 Jan 2015 09:11:52 +0100
Subject: [Python-Dev] Why does STORE_MAP not take a parameter?
In-Reply-To: <54C03CEA.2080305@canterbury.ac.nz>
References: <CAA68w_manHmXq--6-5H1q6JoiOi3xf2njJ=6ZwCr6GiaCJPfFg@mail.gmail.com>
 <54C03CEA.2080305@canterbury.ac.nz>
Message-ID: <CALy7ODt2iniPCfA=khC=vMM96B2tDpuyNY9Dahok4Gz6yj_52g@mail.gmail.com>

In a function call with named arguments the code generated doesn't follow
that pattern:

dis.dis(lambda : f(a=1,b=2,c=3))

displays:

  1           0 LOAD_GLOBAL              0 (foo)
              3 LOAD_CONST               1 ('a')
              6 LOAD_CONST               2 (1)
              9 LOAD_CONST               3 ('b')
             12 LOAD_CONST               4 (2)
             15 LOAD_CONST               5 ('c')
             18 LOAD_CONST               6 (3)
             21 CALL_FUNCTION          768

             24 RETURN_VALUE


so the problem is only about inlined dicts (somewhat common but not
ubiquitous as they are in Javascript).

Andrea

On Thu, Jan 22, 2015 at 12:57 AM, Greg Ewing <greg.ewing at canterbury.ac.nz>
wrote:

> On 01/21/2015 11:16 PM, Neil Girdhar wrote:
>
>> Why not have BUILD_MAP work like BUILD_LIST?   I.e., STORE_MAP takes a
>> parameter n and adds the last n pairs of stack elements into the n-1 stack
>> element (the dictionary).
>>
>
> It probably wouldn't make much difference. Building a list is
> substantially cheaper if you have all the items on hand and can
> copy them in en masse. But adding an item to a dict entails
> quite a lot of overhead, since you need to hash the key, look
> for a free slot, etc. and this would likely swamp any gain
> from executing less bytecodes. And on the other side of the
> equation, evaluating the items one at a time requires less
> stack space, so the stack frame can be smaller.
>
> But as always, you can't be sure without measuring it, and
> this would be a good thing for someone interested to try out.
>
> --
> Greg
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> agriff%40tin.it
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150122/397699db/attachment.html>

From brett at python.org  Thu Jan 22 15:33:53 2015
From: brett at python.org (Brett Cannon)
Date: Thu, 22 Jan 2015 14:33:53 +0000
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
 <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>
Message-ID: <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>

A mobile SIG is being formed, but it doesn't have a mailing list yet, else
that would be a good place to ask this question.

On Wed Jan 21 2015 at 5:54:39 PM Guido van Rossum <guido at python.org> wrote:

> Maybe try a list focused on Android development? Few people in the Python
> core development community have any Android experience. But the issues and
> context you offer seem to be related to the Android world, not to Python.
> (dlopen is used by a lot of systems, not just Python.)
>
> On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>
>> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com>
>> wrote:
>> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton <chaselton at gmail.com>
>> wrote:
>> >> Hello,
>> >> I'm struggling with a build issue on Android; I've posted to the
>> >> general python list with no result, so I'm re-posting here in hopes
>> >> that someone can help.  If this is the wrong place feel free to let me
>> >> know.
>> >>
>> >> I'm attempting to build Python 2.7.8 on my Android device; I'm using
>> >> an environment that simulates a Linux filesystem within the Android
>> >> terminal using a port of fakechroot.  Within that environment I've
>> >> ported and/or bootstrapped a number of Linux utilities (curl, git,
>> >> openssl, gcc)
>> >>
>> >> I run ./configure, then make, and the executable and library are
>> >> built.  The problem occurs when build_ext is run; the newly built
>> >> python executable builds, then links _struct, and immediately
>> >> afterwards I get an 'undefined reference to dlopen' error.
>> >>
>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> >> --libraries='c dl m' -f, the 'undefined reference to dlopen' error is
>> >> thrown again.
>> >>
>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> >> --libraries='-lc -ldl -lm' -f the build continues past _struct...even
>> >> though ld throws the expected 'unable to find -l-lc' and other errors.
>> >>
>> >> Let me know if you need me to provide additional information.  Any
>> >> help would be greatly appreciated.
>> >>
>> >> Cyd
>> >
>> >
>> > Additionally I took a strace of the error producing command. The
>> > following is (hopefully) a relevant portion minus the various 'no such
>> > file' lines before the correct lib is found (which it always is)
>> >
>> > 16513
>> open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.py",
>> > O_RDONLY|O_LARGEFILE) = 3           16513
>> >
>> open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.pyc",
>> > O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
>> >  = 16525
>> > 16513 wait4(16525,  <unfinished ...>
>> > 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
>> > O_RDONLY|O_LARGEFILE) = 3
>> > 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
>> > "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
>> > build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars */])
>> > = 0
>> >
>> > *snip call to libc intercepted by libfakechroot*
>> >
>> > 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
>> > 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
>> > 16525 open("/dev/__properties__",
>> > O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
>> > 16525 open("build/temp.linux-armv7l-2.7/multiarch",
>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>> > 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>> > 16525 fork()                            = 16526
>> > 16525 wait4(-1,  <unfinished ...>
>> > 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE, 0666)
>> > = -1 EACCES (Permission denied)
>> >
>> > See attached for remainder
>>
>> Should I be posting this issue elsewhere?
>> Is more information needed?
>>
> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>>
> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>  _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150122/cf5c0467/attachment.html>

From walter at livinglogic.de  Thu Jan 22 18:23:19 2015
From: walter at livinglogic.de (Walter =?utf-8?q?D=C3=B6rwald?=)
Date: Thu, 22 Jan 2015 18:23:19 +0100
Subject: [Python-Dev]
 =?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94__?=
 =?utf-8?q?Question_regarding_test=5Fast?=
In-Reply-To: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
Message-ID: <C75ED11D-D1D9-4A91-ACA3-D3D6CF121D33@livinglogic.de>

On 20 Jan 2015, at 17:34, Neil Girdhar wrote:

> My question first:
> test_ast is mostly generated code, but I can't find where it is being
> generated.  I am pretty sure I know how to fix most of the introduced
> problems.  Who is generating test_ast??
>
> Update:
>
> So far, I've done the following:
>
> Updated the patch to 3.5
> Fixed the grammar to accept final commas in argument lists always, and to
> work with the already implemented code.
> Fixed the ast to accept what it needs to accept and reject according to the
> limitation laid down by Guido.
> Fixed the parsing library.
>
> Fixed these tests:
> test_ast.py
> test_extcall.py
> test_grammar.py
> test_syntax.py
> test_unpack_ex.py
>
> As far as I can tell, all I have left is to fix test_ast and possibly write
> some more tests (there are already some new tests and some of the old
> negative tests expecting SyntaxError are now positive tests).

inspect.signature might need an update.

Servus,
   Walter

From mistersheik at gmail.com  Thu Jan 22 23:03:10 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Thu, 22 Jan 2015 17:03:10 -0500
Subject: [Python-Dev]
	=?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94_Q?=
	=?utf-8?q?uestion_regarding_test=5Fast?=
In-Reply-To: <C75ED11D-D1D9-4A91-ACA3-D3D6CF121D33@livinglogic.de>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <C75ED11D-D1D9-4A91-ACA3-D3D6CF121D33@livinglogic.de>
Message-ID: <CAA68w_mgThXs=6cieTPAiTnsPRN5eertbEmboopguNmONrVXzg@mail.gmail.com>

Thanks for taking a look.  I looked at inspect and I can't see anything
that needs to change since it's the caller rather than the receiver who has
more options after this PEP.  Did you see anything in particular?

Best,

Neil

On Thu, Jan 22, 2015 at 12:23 PM, Walter D?rwald <walter at livinglogic.de>
wrote:

> On 20 Jan 2015, at 17:34, Neil Girdhar wrote:
>
> > My question first:
> > test_ast is mostly generated code, but I can't find where it is being
> > generated.  I am pretty sure I know how to fix most of the introduced
> > problems.  Who is generating test_ast??
> >
> > Update:
> >
> > So far, I've done the following:
> >
> > Updated the patch to 3.5
> > Fixed the grammar to accept final commas in argument lists always, and to
> > work with the already implemented code.
> > Fixed the ast to accept what it needs to accept and reject according to
> the
> > limitation laid down by Guido.
> > Fixed the parsing library.
> >
> > Fixed these tests:
> > test_ast.py
> > test_extcall.py
> > test_grammar.py
> > test_syntax.py
> > test_unpack_ex.py
> >
> > As far as I can tell, all I have left is to fix test_ast and possibly
> write
> > some more tests (there are already some new tests and some of the old
> > negative tests expecting SyntaxError are now positive tests).
>
> inspect.signature might need an update.
>
> Servus,
>    Walter
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150122/c0b420c0/attachment.html>

From greg at krypto.org  Fri Jan 23 07:15:00 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 23 Jan 2015 06:15:00 +0000
Subject: [Python-Dev] [Python-checkins] cpython (3.4): Remove the
 unimplemented but ignored without='-E' parameters being passed to
References: <20150123060512.87133.76800@psf.io>
Message-ID: <CAGE7PN+Oxn2hZyHRA=VPz1C_QrSHfsY0xx6NDfE2Wb7+y=0-7g@mail.gmail.com>

I misinterpreted the side effect this code was triggering. It turns out
that any kwarg would have the desired effect as a side effect (as seen in
dummyvar='' being passed later in this file for the same purpose).  I'm
looking at refactoring various uses of -E that interfere with running the
test suite in an environment where you need PYTHONHOME set. I will revert
this change or otherwise do the equivalent of reinstating its behavior in a
followup check-in as appropriate.

-gregory.p.smith

On Thu Jan 22 2015 at 10:05:56 PM gregory.p.smith <
python-checkins at python.org> wrote:

> https://hg.python.org/cpython/rev/7b833bd1f509
> changeset:   94249:7b833bd1f509
> branch:      3.4
> parent:      94247:7f3ac2ce24ed
> user:        Gregory P. Smith <greg at krypto.org>
> date:        Thu Jan 22 22:04:16 2015 -0800
> summary:
>   Remove the unimplemented but ignored without='-E' parameters being
> passed to
> script_helper.assert_python_failure().  No such feature has ever existed,
> thus it doesn't do what the comment claims.  (It does add a 'without'
> variable to the environment of the child process but that was not intended)
>
> files:
>   Lib/test/test_cmd_line.py |  6 ++----
>   1 files changed, 2 insertions(+), 4 deletions(-)
>
>
> diff --git a/Lib/test/test_cmd_line.py b/Lib/test/test_cmd_line.py
> --- a/Lib/test/test_cmd_line.py
> +++ b/Lib/test/test_cmd_line.py
> @@ -426,13 +426,11 @@
>          self.assertIn(b'Unknown option: -z', err)
>          self.assertEqual(err.splitlines().count(b'Unknown option: -z'),
> 1)
>          self.assertEqual(b'', out)
> -        # Add "without='-E'" to prevent _assert_python to append -E
> -        # to env_vars and change the output of stderr
> -        rc, out, err = assert_python_failure('-z', without='-E')
> +        rc, out, err = assert_python_failure('-z')
>          self.assertIn(b'Unknown option: -z', err)
>          self.assertEqual(err.splitlines().count(b'Unknown option: -z'),
> 1)
>          self.assertEqual(b'', out)
> -        rc, out, err = assert_python_failure('-a', '-z', without='-E')
> +        rc, out, err = assert_python_failure('-a', '-z')
>          self.assertIn(b'Unknown option: -a', err)
>          # only the first unknown option is reported
>          self.assertNotIn(b'Unknown option: -z', err)
>
> --
> Repository URL: https://hg.python.org/cpython
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> https://mail.python.org/mailman/listinfo/python-checkins
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/bfdda77f/attachment.html>

From walter at livinglogic.de  Fri Jan 23 11:00:11 2015
From: walter at livinglogic.de (Walter =?utf-8?q?D=C3=B6rwald?=)
Date: Fri, 23 Jan 2015 11:00:11 +0100
Subject: [Python-Dev]
 =?utf-8?q?PEP_448_=28almost_finished!=29_=E2=80=94__?=
 =?utf-8?q?Question_regarding_test=5Fast?=
In-Reply-To: <CAA68w_mgThXs=6cieTPAiTnsPRN5eertbEmboopguNmONrVXzg@mail.gmail.com>
References: <CAA68w_=3=eUy=gxxTGg3DHrZr-mzMVmsB2Z6wddn16mLOT2xzQ@mail.gmail.com>
 <C75ED11D-D1D9-4A91-ACA3-D3D6CF121D33@livinglogic.de>
 <CAA68w_mgThXs=6cieTPAiTnsPRN5eertbEmboopguNmONrVXzg@mail.gmail.com>
Message-ID: <481CE05A-496D-4253-8BBB-4278D5A47218@livinglogic.de>

On 22 Jan 2015, at 23:03, Neil Girdhar wrote:

> Thanks for taking a look.  I looked at inspect and I can't see 
> anything
> that needs to change since it's the caller rather than the receiver 
> who has
> more options after this PEP.

You are probably right. And for calling via Signature.bind() your patch 
takes care of business.

> Did you see anything in particular?

No, I was just using inspect.signature lately and reading the PEP 
reminded me of it.

> Best,
>
> Neil

Servus,
    Walter

> On Thu, Jan 22, 2015 at 12:23 PM, Walter D?rwald 
> <walter at livinglogic.de>
> wrote:
>
>> On 20 Jan 2015, at 17:34, Neil Girdhar wrote:
>>
>>> My question first:
>>> test_ast is mostly generated code, but I can't find where it is 
>>> being
>>> generated.  I am pretty sure I know how to fix most of the 
>>> introduced
>>> problems.  Who is generating test_ast??
>>>
>>> Update:
>>>
>>> So far, I've done the following:
>>>
>>> Updated the patch to 3.5
>>> Fixed the grammar to accept final commas in argument lists always, 
>>> and to
>>> work with the already implemented code.
>>> Fixed the ast to accept what it needs to accept and reject according 
>>> to
>> the
>>> limitation laid down by Guido.
>>> Fixed the parsing library.
>>>
>>> Fixed these tests:
>>> test_ast.py
>>> test_extcall.py
>>> test_grammar.py
>>> test_syntax.py
>>> test_unpack_ex.py
>>>
>>> As far as I can tell, all I have left is to fix test_ast and 
>>> possibly
>> write
>>> some more tests (there are already some new tests and some of the 
>>> old
>>> negative tests expecting SyntaxError are now positive tests).
>>
>> inspect.signature might need an update.
>>
>> Servus,
>>  Walter
>>

From chaselton at gmail.com  Fri Jan 23 17:50:26 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 23 Jan 2015 10:50:26 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
 <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>
 <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>
Message-ID: <CAHu5PrY23Z5svNdxDOqJLmiR0f7TiCijg2-m4ZtsyF75jJxnqw@mail.gmail.com>

I guess I'll keep waiting...given the zero responses I've gotten from
the android side. :-(.

In the meantime...in case anyone is interested... since I have the
working binary, I executed it and went through each command in
setup.py to see what throws the 'undefined reference to dlopen' error.
Turns out that the distutils module is the culprit...specifically
distutils.core.


On Thu, Jan 22, 2015 at 8:33 AM, Brett Cannon <brett at python.org> wrote:
> A mobile SIG is being formed, but it doesn't have a mailing list yet, else
> that would be a good place to ask this question.
>
> On Wed Jan 21 2015 at 5:54:39 PM Guido van Rossum <guido at python.org> wrote:
>>
>> Maybe try a list focused on Android development? Few people in the Python
>> core development community have any Android experience. But the issues and
>> context you offer seem to be related to the Android world, not to Python.
>> (dlopen is used by a lot of systems, not just Python.)
>>
>> On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com>
>>> wrote:
>>> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton <chaselton at gmail.com>
>>> > wrote:
>>> >> Hello,
>>> >> I'm struggling with a build issue on Android; I've posted to the
>>> >> general python list with no result, so I'm re-posting here in hopes
>>> >> that someone can help.  If this is the wrong place feel free to let me
>>> >> know.
>>> >>
>>> >> I'm attempting to build Python 2.7.8 on my Android device; I'm using
>>> >> an environment that simulates a Linux filesystem within the Android
>>> >> terminal using a port of fakechroot.  Within that environment I've
>>> >> ported and/or bootstrapped a number of Linux utilities (curl, git,
>>> >> openssl, gcc)
>>> >>
>>> >> I run ./configure, then make, and the executable and library are
>>> >> built.  The problem occurs when build_ext is run; the newly built
>>> >> python executable builds, then links _struct, and immediately
>>> >> afterwards I get an 'undefined reference to dlopen' error.
>>> >>
>>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>>> >> --libraries='c dl m' -f, the 'undefined reference to dlopen' error is
>>> >> thrown again.
>>> >>
>>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>>> >> --libraries='-lc -ldl -lm' -f the build continues past _struct...even
>>> >> though ld throws the expected 'unable to find -l-lc' and other errors.
>>> >>
>>> >> Let me know if you need me to provide additional information.  Any
>>> >> help would be greatly appreciated.
>>> >>
>>> >> Cyd
>>> >
>>> >
>>> > Additionally I took a strace of the error producing command. The
>>> > following is (hopefully) a relevant portion minus the various 'no such
>>> > file' lines before the correct lib is found (which it always is)
>>> >
>>> > 16513
>>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.py",
>>> > O_RDONLY|O_LARGEFILE) = 3           16513
>>> >
>>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-2.7.8/Lib/distutils/unixccompiler.pyc",
>>> > O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
>>> >  = 16525
>>> > 16513 wait4(16525,  <unfinished ...>
>>> > 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
>>> > O_RDONLY|O_LARGEFILE) = 3
>>> > 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
>>> > "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
>>> > build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars */])
>>> > = 0
>>> >
>>> > *snip call to libc intercepted by libfakechroot*
>>> >
>>> > 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
>>> > 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) = 3
>>> > 16525 open("/dev/__properties__",
>>> > O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
>>> > 16525 open("build/temp.linux-armv7l-2.7/multiarch",
>>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>>> > 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
>>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>>> > 16525 fork()                            = 16526
>>> > 16525 wait4(-1,  <unfinished ...>
>>> > 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE, 0666)
>>> > = -1 EACCES (Permission denied)
>>> >
>>> > See attached for remainder
>>>
>>> Should I be posting this issue elsewhere?
>>> Is more information needed?
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>>
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/brett%40python.org

From status at bugs.python.org  Fri Jan 23 18:08:11 2015
From: status at bugs.python.org (Python tracker)
Date: Fri, 23 Jan 2015 18:08:11 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20150123170811.61129562C7@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2015-01-16 - 2015-01-23)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4760 (+24)
  closed 30307 (+30)
  total  35067 (+54)

Open issues with patches: 2228 


Issues opened (38)
==================

#23095: [Windows] asyncio: race condition when cancelling a _WaitHandl
http://bugs.python.org/issue23095  reopened by haypo

#23252: Add support of writing to unseekable file in zipfile
http://bugs.python.org/issue23252  opened by serhiy.storchaka

#23253: Delay-load ShellExecute
http://bugs.python.org/issue23253  opened by steve.dower

#23254: Document how to close the TCPServer listening socket
http://bugs.python.org/issue23254  opened by vadmium

#23255: SimpleHTTPRequestHandler refactor for more extensible usage.
http://bugs.python.org/issue23255  opened by last-ent

#23257: Update Windows build/setup instructions in devguide
http://bugs.python.org/issue23257  opened by steve.dower

#23258: Cannot Install Python 3.4.2 on Windows 7 64 bit / screen print
http://bugs.python.org/issue23258  opened by 20scanman

#23260: Update Windows installer
http://bugs.python.org/issue23260  opened by steve.dower

#23262: webbrowser module broken with Firefox 36+
http://bugs.python.org/issue23262  opened by ssokolow

#23263: Python 3 gives misleading errors when validating unicode ident
http://bugs.python.org/issue23263  opened by Matt.Bachmann

#23264: Add pickle support of dict views
http://bugs.python.org/issue23264  opened by serhiy.storchaka

#23267: multiprocessing pool.py doesn't close inqueue and outqueue pip
http://bugs.python.org/issue23267  opened by shanip

#23268: Fix comparison of ipaddress classes
http://bugs.python.org/issue23268  opened by serhiy.storchaka

#23270: Use the new __builtin_mul_overflow() of Clang and GCC 5 to che
http://bugs.python.org/issue23270  opened by haypo

#23273: traceback: formatting a traceback stats the filesystem
http://bugs.python.org/issue23273  opened by rbcollins

#23274: make_ssl_data.py in Python 2.7.9 needs Python 3 to run
http://bugs.python.org/issue23274  opened by schlenk

#23275: Can assign [] = (), but not () = []
http://bugs.python.org/issue23275  opened by Devin Jeanpierre

#23276: hackcheck is broken in association with __setattr__
http://bugs.python.org/issue23276  opened by devkid

#23277: Cleanup unused and duplicate imports in tests
http://bugs.python.org/issue23277  opened by jdufresne

#23278: multiprocessing maxtasksperchild=1 + logging = task loss
http://bugs.python.org/issue23278  opened by nelson

#23279: test_site/test_startup_imports fails when mpl_toolkit or logil
http://bugs.python.org/issue23279  opened by drudd

#23282: Slightly faster set lookup
http://bugs.python.org/issue23282  opened by serhiy.storchaka

#23283: Backport Tools/clinic to 3.4
http://bugs.python.org/issue23283  opened by zach.ware

#23284: curses, readline, tinfo, and also --prefix, dbm, CPPFLAGS
http://bugs.python.org/issue23284  opened by pooryorick

#23285: PEP 475 - EINTR handling
http://bugs.python.org/issue23285  opened by neologix

#23286: A typo in the tutorial
http://bugs.python.org/issue23286  opened by aruseni

#23287: ctypes.util.find_library needlessly call crle on Solaris
http://bugs.python.org/issue23287  opened by jbeck

#23289: concurrent.futures.Executor.map is not equivalent to map.
http://bugs.python.org/issue23289  opened by majkrzak

#23290: Faster set copying
http://bugs.python.org/issue23290  opened by serhiy.storchaka

#23291: Documentation about Py_Finalize(): Freeing objects
http://bugs.python.org/issue23291  opened by Albert.Zeyer

#23292: Enum doc suggestion
http://bugs.python.org/issue23292  opened by mark

#23295: [Windows] asyncio: add UDP support to ProactorEventLoop
http://bugs.python.org/issue23295  opened by haypo

#23297: ???tokenize.detect_encoding??? is confused between text and by
http://bugs.python.org/issue23297  opened by bignose

#23298: Add ArgumentParser.add_mutually_dependence_group
http://bugs.python.org/issue23298  opened by dongwm

#23300: httplib is using a method that doesn't exist
http://bugs.python.org/issue23300  opened by guohua

#23303: test_license_exists_at_url() of test_site fails on "x86 XP-4 3
http://bugs.python.org/issue23303  opened by haypo

#23304: Unused Superclass in calendar.py
http://bugs.python.org/issue23304  opened by CliffM

#23305: RotatingFileHandler does not rotate if backupCount is 0
http://bugs.python.org/issue23305  opened by Juha.Lemmetti



Most recent 15 issues with no replies (15)
==========================================

#23305: RotatingFileHandler does not rotate if backupCount is 0
http://bugs.python.org/issue23305

#23304: Unused Superclass in calendar.py
http://bugs.python.org/issue23304

#23291: Documentation about Py_Finalize(): Freeing objects
http://bugs.python.org/issue23291

#23289: concurrent.futures.Executor.map is not equivalent to map.
http://bugs.python.org/issue23289

#23287: ctypes.util.find_library needlessly call crle on Solaris
http://bugs.python.org/issue23287

#23286: A typo in the tutorial
http://bugs.python.org/issue23286

#23284: curses, readline, tinfo, and also --prefix, dbm, CPPFLAGS
http://bugs.python.org/issue23284

#23279: test_site/test_startup_imports fails when mpl_toolkit or logil
http://bugs.python.org/issue23279

#23273: traceback: formatting a traceback stats the filesystem
http://bugs.python.org/issue23273

#23270: Use the new __builtin_mul_overflow() of Clang and GCC 5 to che
http://bugs.python.org/issue23270

#23268: Fix comparison of ipaddress classes
http://bugs.python.org/issue23268

#23263: Python 3 gives misleading errors when validating unicode ident
http://bugs.python.org/issue23263

#23254: Document how to close the TCPServer listening socket
http://bugs.python.org/issue23254

#23252: Add support of writing to unseekable file in zipfile
http://bugs.python.org/issue23252

#23247: Multibyte codec StreamWriter.reset() crashes
http://bugs.python.org/issue23247



Most recent 15 issues waiting for review (15)
=============================================

#23300: httplib is using a method that doesn't exist
http://bugs.python.org/issue23300

#23298: Add ArgumentParser.add_mutually_dependence_group
http://bugs.python.org/issue23298

#23290: Faster set copying
http://bugs.python.org/issue23290

#23287: ctypes.util.find_library needlessly call crle on Solaris
http://bugs.python.org/issue23287

#23286: A typo in the tutorial
http://bugs.python.org/issue23286

#23285: PEP 475 - EINTR handling
http://bugs.python.org/issue23285

#23284: curses, readline, tinfo, and also --prefix, dbm, CPPFLAGS
http://bugs.python.org/issue23284

#23282: Slightly faster set lookup
http://bugs.python.org/issue23282

#23277: Cleanup unused and duplicate imports in tests
http://bugs.python.org/issue23277

#23268: Fix comparison of ipaddress classes
http://bugs.python.org/issue23268

#23264: Add pickle support of dict views
http://bugs.python.org/issue23264

#23263: Python 3 gives misleading errors when validating unicode ident
http://bugs.python.org/issue23263

#23260: Update Windows installer
http://bugs.python.org/issue23260

#23257: Update Windows build/setup instructions in devguide
http://bugs.python.org/issue23257

#23255: SimpleHTTPRequestHandler refactor for more extensible usage.
http://bugs.python.org/issue23255



Top 10 most discussed issues (10)
=================================

#2292: Missing *-unpacking generalizations
http://bugs.python.org/issue2292  65 msgs

#3566: httplib persistent connections violate MUST in RFC2616 sec 8.1
http://bugs.python.org/issue3566  14 msgs

#23251: mention in time.sleep() docs that it does not block other Pyth
http://bugs.python.org/issue23251  14 msgs

#23300: httplib is using a method that doesn't exist
http://bugs.python.org/issue23300  11 msgs

#23095: [Windows] asyncio: race condition when cancelling a _WaitHandl
http://bugs.python.org/issue23095   9 msgs

#23262: webbrowser module broken with Firefox 36+
http://bugs.python.org/issue23262   9 msgs

#9393: shelve.open/bsddb.hashopen exception with unicode paths
http://bugs.python.org/issue9393   5 msgs

#17911: traceback: add a new thin class storing a traceback without st
http://bugs.python.org/issue17911   5 msgs

#22885: Arbitrary code execution vulnerability due to unchecked eval()
http://bugs.python.org/issue22885   5 msgs

#23255: SimpleHTTPRequestHandler refactor for more extensible usage.
http://bugs.python.org/issue23255   5 msgs



Issues closed (29)
==================

#11024: imaplib: Time2Internaldate() returns localized strings
http://bugs.python.org/issue11024  closed by r.david.murray

#20702: warning in cmdline.rst
http://bugs.python.org/issue20702  closed by berker.peksag

#20898: Missing 507 response description
http://bugs.python.org/issue20898  closed by berker.peksag

#21817: `concurrent.futures.ProcessPoolExecutor` swallows tracebacks
http://bugs.python.org/issue21817  closed by pitrou

#22317: action argument is not documented in argparse add_subparser() 
http://bugs.python.org/issue22317  closed by berker.peksag

#22992: Adding a git developer's guide to Mercurial to devguide
http://bugs.python.org/issue22992  closed by brett.cannon

#23098: mknod devices can be >32 bits
http://bugs.python.org/issue23098  closed by serhiy.storchaka

#23133: Pickling of ipaddress classes
http://bugs.python.org/issue23133  closed by serhiy.storchaka

#23180: Rename IDLE's "Windows" menu item to "Window"
http://bugs.python.org/issue23180  closed by ned.deily

#23211: test.test_logging.SMTPHandlerTest failing on Snow Leopard
http://bugs.python.org/issue23211  closed by ned.deily

#23248: Update ssl data
http://bugs.python.org/issue23248  closed by pitrou

#23250: http.cookies HttpOnly attribute does not use suggested case-st
http://bugs.python.org/issue23250  closed by python-dev

#23256: test_ctypes crashes on Windows on debug builds
http://bugs.python.org/issue23256  closed by python-dev

#23259: Remove dummy reuse optimization from sets
http://bugs.python.org/issue23259  closed by rhettinger

#23261: Clean-up set.pop() search finger logic
http://bugs.python.org/issue23261  closed by rhettinger

#23265: tar.xz support for sdist
http://bugs.python.org/issue23265  closed by ned.deily

#23266: Faster implementation to collapse non-consecutive ip-addresses
http://bugs.python.org/issue23266  closed by pitrou

#23269: Tighten-up search loops in sets
http://bugs.python.org/issue23269  closed by rhettinger

#23271: Unicode HOWTO documentation error
http://bugs.python.org/issue23271  closed by eric.smith

#23272: Python built-in comparison problem
http://bugs.python.org/issue23272  closed by Luk????.N??mec

#23280: Convert binascii.{un}hexlify to Argument Clinic (fix docstring
http://bugs.python.org/issue23280  closed by python-dev

#23281: Access violation - pyc file
http://bugs.python.org/issue23281  closed by brett.cannon

#23288: subprocess.Popen close_fds behaviour differs between 3.2 and 3
http://bugs.python.org/issue23288  closed by mfs

#23293: [Windows] asyncio: race condition related to IocpProactor.conn
http://bugs.python.org/issue23293  closed by haypo

#23294: A typo in the tutorial
http://bugs.python.org/issue23294  closed by aruseni

#23296: ???tokenize.detect_encoding??? is confused between text and by
http://bugs.python.org/issue23296  closed by benjamin.peterson

#23299: Documentation correction - 5.1.4. List Comprehensions
http://bugs.python.org/issue23299  closed by r.david.murray

#23301: ConfigParser does not handle square brackets in section name
http://bugs.python.org/issue23301  closed by r.david.murray

#23302: Small fixes around the use of TCP MSS in http.client
http://bugs.python.org/issue23302  closed by python-dev

From chaselton at gmail.com  Fri Jan 23 18:30:54 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 23 Jan 2015 11:30:54 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
Message-ID: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>

Related to my earlier question regarding building Python on Android
and an undefined reference to dlopen error...I have the following
question:  Is it possible to build and install Python without having
to build and install...or use...distutils?

Some background:
I can build the python interpreter on my device, and I can build a
bunch of modules.  The problem appears when make reaches the part
where setup.py is used to build and import modules...specifically when
setup.py attempts to import distutils.core.

Thanks,
Cyd

From doko at ubuntu.com  Fri Jan 23 19:48:36 2015
From: doko at ubuntu.com (Matthias Klose)
Date: Fri, 23 Jan 2015 19:48:36 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
Message-ID: <54C29784.2080908@ubuntu.com>

On 01/23/2015 06:30 PM, Cyd Haselton wrote:
> Related to my earlier question regarding building Python on Android
> and an undefined reference to dlopen error...I have the following
> question:  Is it possible to build and install Python without having
> to build and install...or use...distutils?
> 
> Some background:
> I can build the python interpreter on my device, and I can build a
> bunch of modules.  The problem appears when make reaches the part
> where setup.py is used to build and import modules...specifically when
> setup.py attempts to import distutils.core.

you can do this using Setup.local. This works for me building additional
extensions as builtins.  It might require some tweaking to build everything.
Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).

  Matthias


From chaselton at gmail.com  Fri Jan 23 19:54:51 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 23 Jan 2015 12:54:51 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C29784.2080908@ubuntu.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com>
Message-ID: <CAHu5Prar9reDVN6v9OW1BEHU_rLd3DLcFFi1pTf3eRHsFrCZVw@mail.gmail.com>

On Fri, Jan 23, 2015 at 12:48 PM, Matthias Klose <doko at ubuntu.com> wrote:
> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> Related to my earlier question regarding building Python on Android
>> and an undefined reference to dlopen error...I have the following
>> question:  Is it possible to build and install Python without having
>> to build and install...or use...distutils?
>>
>> Some background:
>> I can build the python interpreter on my device, and I can build a
>> bunch of modules.  The problem appears when make reaches the part
>> where setup.py is used to build and import modules...specifically when
>> setup.py attempts to import distutils.core.
>
> you can do this using Setup.local. This works for me building additional
> extensions as builtins.  It might require some tweaking to build everything.
> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>
>   Matthias
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chaselton%40gmail.com

Thanks!  I'll give this a shot.  Any documentation on the non-distutils build?

From mal at egenix.com  Fri Jan 23 20:19:46 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 23 Jan 2015 20:19:46 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C29784.2080908@ubuntu.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com>
Message-ID: <54C29ED2.2050508@egenix.com>

On 23.01.2015 19:48, Matthias Klose wrote:
> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> Related to my earlier question regarding building Python on Android
>> and an undefined reference to dlopen error...I have the following
>> question:  Is it possible to build and install Python without having
>> to build and install...or use...distutils?
>>
>> Some background:
>> I can build the python interpreter on my device, and I can build a
>> bunch of modules.  The problem appears when make reaches the part
>> where setup.py is used to build and import modules...specifically when
>> setup.py attempts to import distutils.core.
> 
> you can do this using Setup.local. This works for me building additional
> extensions as builtins.  It might require some tweaking to build everything.

You may want to have a look at the Setup files we're using
in eGenix PyRun, which uses them to force static builds of the
various built-in extensions.

Look for these files:

PyRun/Runtime/Setup.PyRun-2.7
PyRun/Runtime/Setup.PyRun-3.4

in the source archives:

http://www.egenix.com/products/python/PyRun/

> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).

Why ? It's great for finding stuff on your system and configuring
everything without user intervention (well, most of the time :-)).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 23 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From mal at egenix.com  Fri Jan 23 22:01:56 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 23 Jan 2015 22:01:56 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAHu5PrZ_pCG_Jvfq9qE71cBj2376uCNRXmhxUiWGL4S-QQiaVg@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>	<54C29784.2080908@ubuntu.com>
 <54C29ED2.2050508@egenix.com>
 <CAHu5PrZ_pCG_Jvfq9qE71cBj2376uCNRXmhxUiWGL4S-QQiaVg@mail.gmail.com>
Message-ID: <54C2B6C4.9040706@egenix.com>

On 23.01.2015 21:56, Cyd Haselton wrote:
> On Fri, Jan 23, 2015 at 1:19 PM, M.-A. Lemburg <mal at egenix.com> wrote:
>> On 23.01.2015 19:48, Matthias Klose wrote:
>>> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>>> Related to my earlier question regarding building Python on Android
>>>> and an undefined reference to dlopen error...I have the following
>>>> question:  Is it possible to build and install Python without having
>>>> to build and install...or use...distutils?
>>>>
>>>> Some background:
>>>> I can build the python interpreter on my device, and I can build a
>>>> bunch of modules.  The problem appears when make reaches the part
>>>> where setup.py is used to build and import modules...specifically when
>>>> setup.py attempts to import distutils.core.
>>>
>>> you can do this using Setup.local. This works for me building additional
>>> extensions as builtins.  It might require some tweaking to build everything.
>>
>> You may want to have a look at the Setup files we're using
>> in eGenix PyRun, which uses them to force static builds of the
>> various built-in extensions.
>>
>> Look for these files:
>>
>> PyRun/Runtime/Setup.PyRun-2.7
>> PyRun/Runtime/Setup.PyRun-3.4
>>
>> in the source archives:
>>
>> http://www.egenix.com/products/python/PyRun/
> 
> Do you have a browseable git repo?

No, but we're thinking of mirroring our open-source packages on
Github or Bitbucket.

>>> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>>
>> Why ? It's great for finding stuff on your system and configuring
>> everything without user intervention (well, most of the time :-)).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 23 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From chaselton at gmail.com  Fri Jan 23 21:56:17 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 23 Jan 2015 14:56:17 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C29ED2.2050508@egenix.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
Message-ID: <CAHu5PrZ_pCG_Jvfq9qE71cBj2376uCNRXmhxUiWGL4S-QQiaVg@mail.gmail.com>

On Fri, Jan 23, 2015 at 1:19 PM, M.-A. Lemburg <mal at egenix.com> wrote:
> On 23.01.2015 19:48, Matthias Klose wrote:
>> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>> Related to my earlier question regarding building Python on Android
>>> and an undefined reference to dlopen error...I have the following
>>> question:  Is it possible to build and install Python without having
>>> to build and install...or use...distutils?
>>>
>>> Some background:
>>> I can build the python interpreter on my device, and I can build a
>>> bunch of modules.  The problem appears when make reaches the part
>>> where setup.py is used to build and import modules...specifically when
>>> setup.py attempts to import distutils.core.
>>
>> you can do this using Setup.local. This works for me building additional
>> extensions as builtins.  It might require some tweaking to build everything.
>
> You may want to have a look at the Setup files we're using
> in eGenix PyRun, which uses them to force static builds of the
> various built-in extensions.
>
> Look for these files:
>
> PyRun/Runtime/Setup.PyRun-2.7
> PyRun/Runtime/Setup.PyRun-3.4
>
> in the source archives:
>
> http://www.egenix.com/products/python/PyRun/

Do you have a browseable git repo?
>
>> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>
> Why ? It's great for finding stuff on your system and configuring
> everything without user intervention (well, most of the time :-)).
>
> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 23 2015)
>>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/

From greg at krypto.org  Fri Jan 23 23:45:11 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 23 Jan 2015 22:45:11 +0000
Subject: [Python-Dev] Can Python Be Built Without Distutils
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
Message-ID: <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>

On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com> wrote:

> On 23.01.2015 19:48, Matthias Klose wrote:
> > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
> >> Related to my earlier question regarding building Python on Android
> >> and an undefined reference to dlopen error...I have the following
> >> question:  Is it possible to build and install Python without having
> >> to build and install...or use...distutils?
> >>
> >> Some background:
> >> I can build the python interpreter on my device, and I can build a
> >> bunch of modules.  The problem appears when make reaches the part
> >> where setup.py is used to build and import modules...specifically when
> >> setup.py attempts to import distutils.core.
> >
> > you can do this using Setup.local. This works for me building additional
> > extensions as builtins.  It might require some tweaking to build
> everything.
>
> You may want to have a look at the Setup files we're using
> in eGenix PyRun, which uses them to force static builds of the
> various built-in extensions.
>
> Look for these files:
>
> PyRun/Runtime/Setup.PyRun-2.7
> PyRun/Runtime/Setup.PyRun-3.4
>
> in the source archives:
>
> http://www.egenix.com/products/python/PyRun/
>
> > Otoh, I would like to get rid off the setup.py altogether (/me ducks
> ...).
>
> Why ? It's great for finding stuff on your system and configuring
> everything without user intervention (well, most of the time :-)).
>

Because our setup.py is a nightmare of arbitrary code run in a linear
fashion with ad-hoc checks for things that are unlike how any other project
on the planet determines what is available on your system.  It may have
seemed "great" when it was created in 2001.  It really shows its age now.

It defeats build parallelism and dependency declaration.
It also prevents cross compilation.

Building an interpreter with a limited standard library on your build host
so that you can run said interpreter to have it drive the remainder of your
build is way more self hosting that we rightfully need to be for CPython.

-gps


> --
> Marc-Andre Lemburg
> eGenix.com
>
> Professional Python Services directly from the Source  (#1, Jan 23 2015)
> >>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
> >>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
> >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
> ________________________________________________________________________
>
> ::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::
>
>    eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
>     D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
>            Registered at Amtsgericht Duesseldorf: HRB 46611
>                http://www.egenix.com/company/contact/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/2696d2a7/attachment-0001.html>

From marfig at gmail.com  Sat Jan 24 00:05:00 2015
From: marfig at gmail.com (Mario Figueiredo)
Date: Sat, 24 Jan 2015 00:05:00 +0100
Subject: [Python-Dev] PEP 484
Message-ID: <CAJcoZmqUSKvNhfAvCdNqFz9DCDNSBDAjnYS4Uk5mVYNDMCUS5A@mail.gmail.com>

Please don't clutter executable code. We need to read it without growing an
headache.

Much better is:

      def myfunction(arg1, arg2):
      """
      Normal docstring...
      @hint: (str, int) -> bool
      """
          return True

While I agree type hinting, for the purposes of static analysis, has no
place in __doc__, I think that we could live with that. Otherwise:

      def myfunction(arg1, arg2):
      """
      Normal docstring...
      """
      "@hint: (str, int) -> bool"
          return True
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/3a71b09b/attachment.html>

From ethan at stoneleaf.us  Sat Jan 24 00:43:00 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 23 Jan 2015 15:43:00 -0800
Subject: [Python-Dev] rst files
Message-ID: <54C2DC84.10304@stoneleaf.us>

Can somebody please explain this?

.. index::
   single: formatting, string (%)
   single: interpolation, string (%)
   single: string; formatting
   single: string; interpolation
   single: printf-style formatting
   single: sprintf-style formatting
   single: % formatting
   single: % interpolation

Specifically, what does index mean?  What does single vs double vs triple mean?  Is there a reference somewhere I can read?

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/39463183/attachment.sig>

From guido at python.org  Sat Jan 24 00:55:29 2015
From: guido at python.org (Guido van Rossum)
Date: Fri, 23 Jan 2015 15:55:29 -0800
Subject: [Python-Dev] rst files
In-Reply-To: <54C2DC84.10304@stoneleaf.us>
References: <54C2DC84.10304@stoneleaf.us>
Message-ID: <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>

This adds entries to the index of the document -- similar to the index at
the end of a book. I think single vs. double refers to different types of
entries. Check out this page: https://docs.python.org/3/genindex.html

On Fri, Jan 23, 2015 at 3:43 PM, Ethan Furman <ethan at stoneleaf.us> wrote:

> Can somebody please explain this?
>
> .. index::
>    single: formatting, string (%)
>    single: interpolation, string (%)
>    single: string; formatting
>    single: string; interpolation
>    single: printf-style formatting
>    single: sprintf-style formatting
>    single: % formatting
>    single: % interpolation
>
> Specifically, what does index mean?  What does single vs double vs triple
> mean?  Is there a reference somewhere I can read?
>
> --
> ~Ethan~
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/c86a408e/attachment.html>

From rdmurray at bitdance.com  Sat Jan 24 01:15:59 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Fri, 23 Jan 2015 19:15:59 -0500
Subject: [Python-Dev] rst files
In-Reply-To: <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>
References: <54C2DC84.10304@stoneleaf.us>
 <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>
Message-ID: <20150124001600.0163A250EEC@webabinitio.net>

On Fri, 23 Jan 2015 15:55:29 -0800, Guido van Rossum <guido at python.org> wrote:
> This adds entries to the index of the document -- similar to the index at
> the end of a book. I think single vs. double refers to different types of
> entries. Check out this page: https://docs.python.org/3/genindex.html
> 
> On Fri, Jan 23, 2015 at 3:43 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> 
> > Can somebody please explain this?
> >
> > .. index::
> >    single: formatting, string (%)
> >    single: interpolation, string (%)
> >    single: string; formatting
> >    single: string; interpolation
> >    single: printf-style formatting
> >    single: sprintf-style formatting
> >    single: % formatting
> >    single: % interpolation
> >
> > Specifically, what does index mean?  What does single vs double vs triple
> > mean?  Is there a reference somewhere I can read?

It is explained in the Sphinx documentation:
http://sphinx-doc.org/contents.html

Specifically:

http://sphinx-doc.org/markup/misc.html#directive-index

--David

From gherron at digipen.edu  Sat Jan 24 00:56:54 2015
From: gherron at digipen.edu (Gary Herron)
Date: Fri, 23 Jan 2015 15:56:54 -0800
Subject: [Python-Dev] rst files
In-Reply-To: <54C2DC84.10304@stoneleaf.us>
References: <54C2DC84.10304@stoneleaf.us>
Message-ID: <54C2DFC6.7000202@digipen.edu>

On 01/23/2015 03:43 PM, Ethan Furman wrote:
> Can somebody please explain this?
>
> .. index::
>     single: formatting, string (%)
>     single: interpolation, string (%)
>     single: string; formatting
>     single: string; interpolation
>     single: printf-style formatting
>     single: sprintf-style formatting
>     single: % formatting
>     single: % interpolation
>
> Specifically, what does index mean?  What does single vs double vs triple mean?  Is there a reference somewhere I can read?

In restructured text (rst), the .. syntax starts a directive.  There are 
a number of built-in directives, and there is the ability to extend the 
syntax with domain specific directives.

That must be the case here (since there doesn't appear to be a built-in 
directive named index), so we'll need more information about the source 
of your rst file before we can answer anything about that particular add 
on directive.

Gary Herron

-- 
Dr. Gary Herron
Department of Computer Science
DigiPen Institute of Technology
(425) 895-4418


From willingc at willingconsulting.com  Sat Jan 24 01:13:03 2015
From: willingc at willingconsulting.com (Carol Willing)
Date: Fri, 23 Jan 2015 16:13:03 -0800
Subject: [Python-Dev] rst files
In-Reply-To: <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>
References: <54C2DC84.10304@stoneleaf.us>
 <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>
Message-ID: <54C2E38F.1070006@willingconsulting.com>

Hello Ethan,

In addition to Guido's suggestion, the Python Developer Guide's Section 
7.4.9 should be helpful. Here's a link to Section 7 on Restructured Text 
https://docs.python.org/devguide/documenting.html#restructuredtext-primer
You may also find the Sphinx project documentation helpful if you need 
additional details.

Carol

On 23/01/2015 15:55, Guido van Rossum wrote:
> This adds entries to the index of the document -- similar to the index 
> at the end of a book. I think single vs. double refers to different 
> types of entries. Check out this page: 
> https://docs.python.org/3/genindex.html
>
> On Fri, Jan 23, 2015 at 3:43 PM, Ethan Furman <ethan at stoneleaf.us 
> <mailto:ethan at stoneleaf.us>> wrote:
>
>     Can somebody please explain this?
>
>     .. index::
>        single: formatting, string (%)
>        single: interpolation, string (%)
>        single: string; formatting
>        single: string; interpolation
>        single: printf-style formatting
>        single: sprintf-style formatting
>        single: % formatting
>        single: % interpolation
>
>     Specifically, what does index mean?  What does single vs double vs
>     triple mean?  Is there a reference somewhere I can read?
>
>     --
>     ~Ethan~
>
>
>     _______________________________________________
>     Python-Dev mailing list
>     Python-Dev at python.org <mailto:Python-Dev at python.org>
>     https://mail.python.org/mailman/listinfo/python-dev
>     Unsubscribe:
>     https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> -- 
> --Guido van Rossum (python.org/~guido <http://python.org/%7Eguido>)
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/willingc%40willingconsulting.com


-- 
*Carol Willing*
Developer | Willing Consulting
https://willingconsulting.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/5af69519/attachment.html>

From tjreedy at udel.edu  Sat Jan 24 02:07:00 2015
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 23 Jan 2015 20:07:00 -0500
Subject: [Python-Dev] rst files
In-Reply-To: <20150124001600.0163A250EEC@webabinitio.net>
References: <54C2DC84.10304@stoneleaf.us>
 <CAP7+vJLHYrTTebpnXRMEt8gCrr8n1XoHaPdJMOQY1cR6Q05t9w@mail.gmail.com>
 <20150124001600.0163A250EEC@webabinitio.net>
Message-ID: <m9ur7l$4l1$1@ger.gmane.org>

On 1/23/2015 7:15 PM, R. David Murray wrote:
> On Fri, 23 Jan 2015 15:55:29 -0800, Guido van Rossum <guido at python.org> wrote:
>> This adds entries to the index of the document -- similar to the index at
>> the end of a book. I think single vs. double refers to different types of
>> entries. Check out this page: https://docs.python.org/3/genindex.html
>>
>> On Fri, Jan 23, 2015 at 3:43 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
>>
>>> Can somebody please explain this?
>>>
>>> .. index::
>>>     single: formatting, string (%)
>>>     single: interpolation, string (%)
>>>     single: string; formatting
>>>     single: string; interpolation
>>>     single: printf-style formatting
>>>     single: sprintf-style formatting
>>>     single: % formatting
>>>     single: % interpolation
>>>
>>> Specifically, what does index mean?  What does single vs double vs triple
>>> mean?  Is there a reference somewhere I can read?
>
> It is explained in the Sphinx documentation:
> http://sphinx-doc.org/contents.html
>
> Specifically:
>
> http://sphinx-doc.org/markup/misc.html#directive-index

The devguide has a cpython doc oriented intro to .rst that includes
https://docs.python.org/devguide/documenting.html#index-generating-markup


-- 
Terry Jan Reedy


From ethan at stoneleaf.us  Sat Jan 24 02:11:29 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Fri, 23 Jan 2015 17:11:29 -0800
Subject: [Python-Dev] rst files
In-Reply-To: <54C2DC84.10304@stoneleaf.us>
References: <54C2DC84.10304@stoneleaf.us>
Message-ID: <54C2F141.3040004@stoneleaf.us>

Many thanks to all responders!

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150123/c2b40232/attachment.sig>

From ncoghlan at gmail.com  Sat Jan 24 08:43:33 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Jan 2015 17:43:33 +1000
Subject: [Python-Dev] New Windows installer for Python 3.5
In-Reply-To: <DM2PR0301MB0734FA9464A37199595DFA3FF5430@DM2PR0301MB0734.namprd03.prod.outlook.com>
References: <BAY407-EAS4050D306D18EF1DC63CF8F487440@phx.gbl>
 <DM2PR0301MB073481792A06799B3119D134F5450@DM2PR0301MB0734.namprd03.prod.outlook.com>
 <BAY407-EAS4025AABEB1F5D2537407F2387450@phx.gbl>
 <54B0807F.4090303@stoneleaf.us>
 <CADiSq7cdcNxGuo1gEQLB53HxOPV0CMz-ibAahZJUKE-nGAhUxg@mail.gmail.com>
 <CALGmxE+u6LRj=2WJsr6FJ0Bf7hiQO_mGuNuXCbchbS8oducksQ@mail.gmail.com>
 <DM2PR0301MB0734FA9464A37199595DFA3FF5430@DM2PR0301MB0734.namprd03.prod.outlook.com>
Message-ID: <CADiSq7cd_g9h=FsRKmrMGwzzUaVf8CMKP+w_F6ivqX-LfEhGow@mail.gmail.com>

On 13 January 2015 at 06:02, Steve Dower <Steve.Dower at microsoft.com> wrote:
>> On Sat, Jan 10, 2015 at 3:28 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> For the time being, things like PyInstaller, PyRun, Portable Python,
>>> etc are going to offer a better solution than anything we provide in
>>> the standard installers.
>>
>> See also Anaconda and Enthought Canopy. I think miniconda, for instance, may
>> give you just what you need, if you don't want to go the py2exe/PyInstaller
>> approach (though you probably do want to go that way, as far as I can tell from
>> your description of your use-case.
>
> Actually, both Anaconda and Canopy suffer from the initialization process issues. You can't really use virtualenv with either of them unless they set the registry PythonPath value (since they won't find the libraries they need to launch), and if you install both then they'll fight over the registry key and you'll get Canopy launching with Anaconda's path or vice-versa.
>
>> I'm inclined to think that this does not belong as part of the standard
>> installer.
>
> The problems are inherent to the standard python.exe, and are likely part of the standard pythonXY.dll. Nick's scared of fixing it, so I'm absolutely petrified :)

Technically, I'm not scared of fixing it per se, I'm just wary of
trying to fix it without first refactoring it and writing some more
tests :)

Finding the roundtuits for PEP 432 is an eternal struggle though - so
many things end up edging ahead of it on the personal priority list :(

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Jan 24 08:49:30 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Jan 2015 17:49:30 +1000
Subject: [Python-Dev] C ABI and #ifndef Py_LIMITED_API
In-Reply-To: <54B6982F.5080403@stoneleaf.us>
References: <54B61B83.4020105@stoneleaf.us> <m958fa$ngc$1@ger.gmane.org>
 <54B6982F.5080403@stoneleaf.us>
Message-ID: <CADiSq7fm08-KZ4yzMbT6N4kMd1AHycp1U6=YgumFh=sS7vOdyg@mail.gmail.com>

On 15 January 2015 at 02:24, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 01/14/2015 12:13 AM, Georg Brandl wrote:
>> On 01/14/2015 08:32 AM, Ethan Furman wrote:
>>>
>>> In the CPython source code I see
>>>
>>> #ifndef Py_LIMITED_API
>>>
>>> Is there a section in the docs that explains the purpose?  If not, can
>>> someone give me the cliff notes version?
>>
>> PEP 384, and in particular [1] should get you started.
>>
>> [1]
>> https://www.python.org/dev/peps/pep-0384/#header-files-and-preprocessor-definitions
>
> Nice, that answered other questions as well!  :)

It's worth noting that https://docs.python.org/3/c-api/stable.html is
the relevant reference in the main C API docs.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Jan 24 08:57:18 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Jan 2015 17:57:18 +1000
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
 <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>
Message-ID: <CADiSq7f__+JEA2HxVdkoNJ04U+wGucayMQLRJxfZRYOrgG6XGg@mail.gmail.com>

On 15 January 2015 at 07:35, Donald Stufft <donald at stufft.io> wrote:
>
> On Jan 14, 2015, at 4:19 PM, Brett Cannon <brett at python.org> wrote:
>
> But as Guido pointed out, we _like_ it being difficult to do because we
> don't want this kind of substitution happening as code ends up depending on
> bugs and quirks that you may fix.
>
> Not all of us, I hate the default order of sys.path.

It's mostly an opinion that arises from debugging other people's
problems after they've managed to import the wrong thing without
realising it (cf. the "don't use 'socket.py' as the name of your
script for learning about how TCP sockets work" problem). We're aware
that annoys power users, but they're far better equipped to handle the
problem than if we inverted the situation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Jan 24 09:00:46 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 24 Jan 2015 18:00:46 +1000
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
Message-ID: <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>

On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org> wrote:
>
>
> On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
>> I was also wondering why files like Python/graminit.c are in the
>> respository?  They generate spurious merge conflicts.
>
> Convenience mostly.

It also gets us a round a couple of bootstrapping problems, where
generating some of those files requires a working Python interpreter,
which you may not have if you just cloned the source tree or unpacked
the tarball.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From fijall at gmail.com  Sat Jan 24 11:50:26 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 24 Jan 2015 12:50:26 +0200
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
Message-ID: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>

Hi

I would like to point out that we implemented rhettingers idea in PyPy
that makes all the dicts ordered by default and we don't have any
adverse performance effects (in fact, there is quite significant
memory saving coming from it). The measurments on CPython could be
different, but in principle OrderedDict can be implemented as
efficiently as normal dict.

Writeup: http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html

Previous discussion:
https://mail.python.org/pipermail/python-dev/2012-December/123028.html

Cheers,
fijal

From fijall at gmail.com  Sat Jan 24 11:51:11 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 24 Jan 2015 12:51:11 +0200
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
Message-ID: <CAK5idxSRbM980YZaeU8MCN6+46O-arUs9Dm27h2XdX=9B3hUqg@mail.gmail.com>

On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> Hi
>
> I would like to point out that we implemented rhettingers idea in PyPy
> that makes all the dicts ordered by default and we don't have any
> adverse performance effects (in fact, there is quite significant
> memory saving coming from it). The measurments on CPython could be
> different, but in principle OrderedDict can be implemented as
> efficiently as normal dict.
>
> Writeup: http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
>
> Previous discussion:
> https://mail.python.org/pipermail/python-dev/2012-December/123028.html
>
> Cheers,
> fijal

also as a sidenote: PEP should maybe mention that PyPy is already
supporting it, a bit by chance

From chaselton at gmail.com  Sat Jan 24 13:43:26 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 24 Jan 2015 06:43:26 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C29784.2080908@ubuntu.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com>
Message-ID: <CAHu5PraCZ=rABvnCkyZH1C-t6-5gvS6gOze4hRoKcm_1kL7wwQ@mail.gmail.com>

On Fri, Jan 23, 2015 at 12:48 PM, Matthias Klose <doko at ubuntu.com> wrote:
> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> Related to my earlier question regarding building Python on Android
>> and an undefined reference to dlopen error...I have the following
>> question:  Is it possible to build and install Python without having
>> to build and install...or use...distutils?
>>
>> Some background:
>> I can build the python interpreter on my device, and I can build a
>> bunch of modules.  The problem appears when make reaches the part
>> where setup.py is used to build and import modules...specifically when
>> setup.py attempts to import distutils.core.
>
> you can do this using Setup.local. This works for me building additional
> extensions as builtins.  It might require some tweaking to build everything.
> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>
>   Matthias
>
Was that Setup.local or Setup? Setup.local is mostly empty...

From chaselton at gmail.com  Sat Jan 24 14:06:42 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 24 Jan 2015 07:06:42 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C29784.2080908@ubuntu.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com>
Message-ID: <CAHu5Prb1A073przh12kZJoS9+GQXnPeKmY-EpwCfX3CROZgrog@mail.gmail.com>

On Fri, Jan 23, 2015 at 12:48 PM, Matthias Klose <doko at ubuntu.com> wrote:
> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> Related to my earlier question regarding building Python on Android
>> and an undefined reference to dlopen error...I have the following
>> question:  Is it possible to build and install Python without having
>> to build and install...or use...distutils?
>>
>> Some background:
>> I can build the python interpreter on my device, and I can build a
>> bunch of modules.  The problem appears when make reaches the part
>> where setup.py is used to build and import modules...specifically when
>> setup.py attempts to import distutils.core.
>
> you can do this using Setup.local. This works for me building additional
> extensions as builtins.  It might require some tweaking to build everything.
> Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>
>   Matthias

Regarding previous question, build completes when using Setup, and
changing *shared* to *static* but 'make install' fails.

If I run 'make altbininstall' or 'make altmaninstall' thst
works...actually they all work except any of the libinstall targets,
which is probably because in the makefile all lib install targets run
setup.py.  Is there a special way to install python without setup.py

From benjamin at python.org  Sat Jan 24 16:09:19 2015
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 24 Jan 2015 10:09:19 -0500
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
Message-ID: <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>



On Sat, Jan 24, 2015, at 03:00, Nick Coghlan wrote:
> On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org>
> wrote:
> >
> >
> > On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
> >> I was also wondering why files like Python/graminit.c are in the
> >> respository?  They generate spurious merge conflicts.
> >
> > Convenience mostly.
> 
> It also gets us a round a couple of bootstrapping problems, where
> generating some of those files requires a working Python interpreter,
> which you may not have if you just cloned the source tree or unpacked
> the tarball.

We could distribute the generated files in tarballs as part of the
release process.

From donald at stufft.io  Sat Jan 24 16:17:29 2015
From: donald at stufft.io (Donald Stufft)
Date: Sat, 24 Jan 2015 10:17:29 -0500
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <CADiSq7f__+JEA2HxVdkoNJ04U+wGucayMQLRJxfZRYOrgG6XGg@mail.gmail.com>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
 <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>
 <CADiSq7f__+JEA2HxVdkoNJ04U+wGucayMQLRJxfZRYOrgG6XGg@mail.gmail.com>
Message-ID: <E36B35D7-AA9F-4F2B-B8CA-5FECE22164AE@stufft.io>


> On Jan 24, 2015, at 2:57 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> On 15 January 2015 at 07:35, Donald Stufft <donald at stufft.io> wrote:
>> 
>> On Jan 14, 2015, at 4:19 PM, Brett Cannon <brett at python.org> wrote:
>> 
>> But as Guido pointed out, we _like_ it being difficult to do because we
>> don't want this kind of substitution happening as code ends up depending on
>> bugs and quirks that you may fix.
>> 
>> Not all of us, I hate the default order of sys.path.
> 
> It's mostly an opinion that arises from debugging other people's
> problems after they've managed to import the wrong thing without
> realising it (cf. the "don't use 'socket.py' as the name of your
> script for learning about how TCP sockets work" problem). We're aware
> that annoys power users, but they're far better equipped to handle the
> problem than if we inverted the situation.


It?s not just power users that it?s good for, it makes it harder for
even beginners to use things like backports of modules. For example
unittest2 and explaining to people the difference between unittest and
unittest2 and that unittest2 isn?t actually any different than unittest
on newer versions of Python.

Or, for example, PEP 453 could have been like 100x better if it would
have been reasonable to just add pip to the stdlib but still enabling
the ability to install an upgraded version of it that would take precedence.

Or you have things like pdb++ which needs to replace the pdb import
because a lot of tools only have a flag like ?pdb and do not provide
a way to switch it to a different import. The sys.path ordering means
that pdb++ has to do hacks in its setup.py[1] which means it won?t be
compatible with Wheel files or with a world where sdists don?t use
a setup.py.

The current situation is that if you install something as an egg (which
setuptools does by default anyways) then setuptools will put it before
the stdlib and it?ll take precedence. This is a nice situation because
it means that if you do run into a problem then it?s easier to debug
because ``python -c import module; print(module.__file__)`` will always
return the same answer in the ?broken? environment. The alternative is
often either a different name (which confuses people as to the relation)
or monkey patching which means that module.__file__ might either be wrong
if they just monkey patched the file and it always means that the behavior
is going to change depending on what you?ve imported which is way more
confusing then being able to override the stdlib.

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From donald at stufft.io  Sat Jan 24 16:21:19 2015
From: donald at stufft.io (Donald Stufft)
Date: Sat, 24 Jan 2015 10:21:19 -0500
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <E36B35D7-AA9F-4F2B-B8CA-5FECE22164AE@stufft.io>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
 <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>
 <CADiSq7f__+JEA2HxVdkoNJ04U+wGucayMQLRJxfZRYOrgG6XGg@mail.gmail.com>
 <E36B35D7-AA9F-4F2B-B8CA-5FECE22164AE@stufft.io>
Message-ID: <61967DB7-9AF3-4C05-BAF3-CAE850E9EC81@stufft.io>


> On Jan 24, 2015, at 10:17 AM, Donald Stufft <donald at stufft.io> wrote:
> 
> Or you have things like pdb++ which needs to replace the pdb import
> because a lot of tools only have a flag like ?pdb and do not provide
> a way to switch it to a different import. The sys.path ordering means
> that pdb++ has to do hacks in its setup.py[1] which means it won?t be
> compatible with Wheel files or with a world where sdists don?t use
> a setup.py.

Sorry, forgot to link this https://bitbucket.org/antocuni/pdb/src/4669c3747a396e3766173feb40ebece32ab08aad/setup.py?at=default#cl-7

---
Donald Stufft
PGP: 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA


From guido at python.org  Sat Jan 24 18:39:13 2015
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Jan 2015 09:39:13 -0800
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAK5idxSRbM980YZaeU8MCN6+46O-arUs9Dm27h2XdX=9B3hUqg@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
 <CAK5idxSRbM980YZaeU8MCN6+46O-arUs9Dm27h2XdX=9B3hUqg@mail.gmail.com>
Message-ID: <CAP7+vJJbbrbX+AnuA3o9HyBxvMZVsVchN6fb8EqA50N9Fyen_g@mail.gmail.com>

Wow, very cool. When I implemented the very first Python dict (cribbing
from an algorithm in Knuth) I had no idea that 25 years later there would
still be ways to improve upon it! I've got a feeling Knuth probably didn't
expect this either...

On Sat, Jan 24, 2015 at 2:51 AM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
> > Hi
> >
> > I would like to point out that we implemented rhettingers idea in PyPy
> > that makes all the dicts ordered by default and we don't have any
> > adverse performance effects (in fact, there is quite significant
> > memory saving coming from it). The measurments on CPython could be
> > different, but in principle OrderedDict can be implemented as
> > efficiently as normal dict.
> >
> > Writeup:
> http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
> >
> > Previous discussion:
> > https://mail.python.org/pipermail/python-dev/2012-December/123028.html
> >
> > Cheers,
> > fijal
>
> also as a sidenote: PEP should maybe mention that PyPy is already
> supporting it, a bit by chance
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/95f10bd9/attachment.html>

From brett at python.org  Sat Jan 24 17:49:31 2015
From: brett at python.org (Brett Cannon)
Date: Sat, 24 Jan 2015 16:49:31 +0000
Subject: [Python-Dev] Can Python Be Built Without Distutils
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
Message-ID: <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>

On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org> wrote:

> On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com> wrote:
>
>> On 23.01.2015 19:48, Matthias Klose wrote:
>> > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>> >> Related to my earlier question regarding building Python on Android
>> >> and an undefined reference to dlopen error...I have the following
>> >> question:  Is it possible to build and install Python without having
>> >> to build and install...or use...distutils?
>> >>
>> >> Some background:
>> >> I can build the python interpreter on my device, and I can build a
>> >> bunch of modules.  The problem appears when make reaches the part
>> >> where setup.py is used to build and import modules...specifically when
>> >> setup.py attempts to import distutils.core.
>> >
>> > you can do this using Setup.local. This works for me building additional
>> > extensions as builtins.  It might require some tweaking to build
>> everything.
>>
>> You may want to have a look at the Setup files we're using
>> in eGenix PyRun, which uses them to force static builds of the
>> various built-in extensions.
>>
>> Look for these files:
>>
>> PyRun/Runtime/Setup.PyRun-2.7
>> PyRun/Runtime/Setup.PyRun-3.4
>>
>> in the source archives:
>>
>> http://www.egenix.com/products/python/PyRun/
>>
>> > Otoh, I would like to get rid off the setup.py altogether (/me ducks
>> ...).
>>
>> Why ? It's great for finding stuff on your system and configuring
>> everything without user intervention (well, most of the time :-)).
>>
>
> Because our setup.py is a nightmare of arbitrary code run in a linear
> fashion with ad-hoc checks for things that are unlike how any other project
> on the planet determines what is available on your system.  It may have
> seemed "great" when it was created in 2001.  It really shows its age now.
>
> It defeats build parallelism and dependency declaration.
> It also prevents cross compilation.
>
> Building an interpreter with a limited standard library on your build host
> so that you can run said interpreter to have it drive the remainder of your
> build is way more self hosting that we rightfully need to be for CPython.
>

So are you suggesting to add the build rules to configure and the Makefile
-- and Windows build file -- in order to drop setup.py?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/c0b54c12/attachment.html>

From ericsnowcurrently at gmail.com  Sat Jan 24 19:05:00 2015
From: ericsnowcurrently at gmail.com (Eric Snow)
Date: Sat, 24 Jan 2015 11:05:00 -0700
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
Message-ID: <CALFfu7BXjDWLBG7zuRkyW4Gqm+fO_kh-kdvgLWwRBzKCXY1HWQ@mail.gmail.com>

On Sat, Jan 24, 2015 at 3:50 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> I would like to point out that we implemented rhettingers idea in PyPy
> that makes all the dicts ordered by default and we don't have any
> adverse performance effects (in fact, there is quite significant
> memory saving coming from it). The measurments on CPython could be
> different, but in principle OrderedDict can be implemented as
> efficiently as normal dict.
>
> Writeup: http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
>
> Previous discussion:
> https://mail.python.org/pipermail/python-dev/2012-December/123028.html

Cool.  I'll add a note to the PEP.

-eric

From fijall at gmail.com  Sat Jan 24 20:11:07 2015
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 24 Jan 2015 21:11:07 +0200
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAP7+vJJbbrbX+AnuA3o9HyBxvMZVsVchN6fb8EqA50N9Fyen_g@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
 <CAK5idxSRbM980YZaeU8MCN6+46O-arUs9Dm27h2XdX=9B3hUqg@mail.gmail.com>
 <CAP7+vJJbbrbX+AnuA3o9HyBxvMZVsVchN6fb8EqA50N9Fyen_g@mail.gmail.com>
Message-ID: <CAK5idxToqB=aJ14d8A7UQsv0hWjrEfRdqjd16qY95JnbeGBVew@mail.gmail.com>

Hi Guido.

I *think* part of the reason why our implementation works is that
machines are significantly different than at the times of Knuth.
Avoiding cache misses is a very effective way to improve performance
these days.

Cheers,
fijal

On Sat, Jan 24, 2015 at 7:39 PM, Guido van Rossum <guido at python.org> wrote:
> Wow, very cool. When I implemented the very first Python dict (cribbing from
> an algorithm in Knuth) I had no idea that 25 years later there would still
> be ways to improve upon it! I've got a feeling Knuth probably didn't expect
> this either...
>
> On Sat, Jan 24, 2015 at 2:51 AM, Maciej Fijalkowski <fijall at gmail.com>
> wrote:
>>
>> On Sat, Jan 24, 2015 at 12:50 PM, Maciej Fijalkowski <fijall at gmail.com>
>> wrote:
>> > Hi
>> >
>> > I would like to point out that we implemented rhettingers idea in PyPy
>> > that makes all the dicts ordered by default and we don't have any
>> > adverse performance effects (in fact, there is quite significant
>> > memory saving coming from it). The measurments on CPython could be
>> > different, but in principle OrderedDict can be implemented as
>> > efficiently as normal dict.
>> >
>> > Writeup:
>> > http://morepypy.blogspot.com/2015/01/faster-more-memory-efficient-and-more.html
>> >
>> > Previous discussion:
>> > https://mail.python.org/pipermail/python-dev/2012-December/123028.html
>> >
>> > Cheers,
>> > fijal
>>
>> also as a sidenote: PEP should maybe mention that PyPy is already
>> supporting it, a bit by chance
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)

From guido at python.org  Sat Jan 24 20:42:01 2015
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Jan 2015 11:42:01 -0800
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAK5idxToqB=aJ14d8A7UQsv0hWjrEfRdqjd16qY95JnbeGBVew@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
 <CAK5idxSRbM980YZaeU8MCN6+46O-arUs9Dm27h2XdX=9B3hUqg@mail.gmail.com>
 <CAP7+vJJbbrbX+AnuA3o9HyBxvMZVsVchN6fb8EqA50N9Fyen_g@mail.gmail.com>
 <CAK5idxToqB=aJ14d8A7UQsv0hWjrEfRdqjd16qY95JnbeGBVew@mail.gmail.com>
Message-ID: <CAP7+vJKYU1g5emsS2=ANdntaorHvxoB6ZD=9-3EHa-UomurRfQ@mail.gmail.com>

On Sat, Jan 24, 2015 at 11:11 AM, Maciej Fijalkowski <fijall at gmail.com>
wrote:

> Hi Guido.
>
> I *think* part of the reason why our implementation works is that
> machines are significantly different than at the times of Knuth.
> Avoiding cache misses is a very effective way to improve performance
> these days.
>

Right. We might look what Knuth has to say about data structures that are
stored on disk and loaded into memory piecemeal. :-)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/f64efd46/attachment-0001.html>

From mal at egenix.com  Sat Jan 24 21:53:06 2015
From: mal at egenix.com (M.-A. Lemburg)
Date: Sat, 24 Jan 2015 21:53:06 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>	<54C29784.2080908@ubuntu.com>
 <54C29ED2.2050508@egenix.com>	<CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>	<CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
Message-ID: <54C40632.4020301@egenix.com>

On 24.01.2015 21:23, Zachary Ware wrote:
> On Saturday, January 24, 2015, Brett Cannon <brett at python.org> wrote:
> 
>> On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org
>> <javascript:_e(%7B%7D,'cvml','greg at krypto.org');>> wrote:
>>
>>> On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com
>>> <javascript:_e(%7B%7D,'cvml','mal at egenix.com');>> wrote:
>>>
>>>> On 23.01.2015 19:48, Matthias Klose wrote:
>>>>> On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>>>>> Related to my earlier question regarding building Python on Android
>>>>>> and an undefined reference to dlopen error...I have the following
>>>>>> question:  Is it possible to build and install Python without having
>>>>>> to build and install...or use...distutils?
>>>>>>
>>>>>> Some background:
>>>>>> I can build the python interpreter on my device, and I can build a
>>>>>> bunch of modules.  The problem appears when make reaches the part
>>>>>> where setup.py is used to build and import modules...specifically when
>>>>>> setup.py attempts to import distutils.core.
>>>>>
>>>>> you can do this using Setup.local. This works for me building
>>>> additional
>>>>> extensions as builtins.  It might require some tweaking to build
>>>> everything.
>>>>
>>>> You may want to have a look at the Setup files we're using
>>>> in eGenix PyRun, which uses them to force static builds of the
>>>> various built-in extensions.
>>>>
>>>> Look for these files:
>>>>
>>>> PyRun/Runtime/Setup.PyRun-2.7
>>>> PyRun/Runtime/Setup.PyRun-3.4
>>>>
>>>> in the source archives:
>>>>
>>>> http://www.egenix.com/products/python/PyRun/
>>>>
>>>>> Otoh, I would like to get rid off the setup.py altogether (/me ducks
>>>> ...).
>>>>
>>>> Why ? It's great for finding stuff on your system and configuring
>>>> everything without user intervention (well, most of the time :-)).
>>>>
>>>
>>> Because our setup.py is a nightmare of arbitrary code run in a linear
>>> fashion with ad-hoc checks for things that are unlike how any other project
>>> on the planet determines what is available on your system.  It may have
>>> seemed "great" when it was created in 2001.  It really shows its age now.
>>>
>>> It defeats build parallelism and dependency declaration.
>>> It also prevents cross compilation.
>>>
>>> Building an interpreter with a limited standard library on your build
>>> host so that you can run said interpreter to have it drive the remainder of
>>> your build is way more self hosting that we rightfully need to be for
>>> CPython.
>>>
>>
>> So are you suggesting to add the build rules to configure and the Makefile
>> -- and Windows build file -- in order to drop setup.py?
>>
> 
> Windows already doesn't use setup.py. There are a lot more modules built-in
> on Windows, and others have their own project files; distutils isn't used
> at all.

The Windows installers comes with all dependencies included.

On other systems, this is not the case and so determining the
various settings is done using Python, which is a lot easier to
do than e.g. relying on autoconf finding everything and setting up
the correct compile options for each built-in.

All that said: if you can come up with a better system that's
both easy to maintain and provides the same level of user
friendliness, I'm sure this would be considered.

BTW: Parallel execution, cross compilation can be added
to setup.py. I don't think parallel execution is all
that important, but cross compilation would certainly be an
interesting feature to have (this currently has to be done
using the Setup.in approach) and an option to build the
modules statically linked to the interpreter would also be
nice.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 24 2015)
>>> Python Projects, Coaching and Consulting ...  http://www.egenix.com/
>>> mxODBC Plone/Zope Database Adapter ...       http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::::: Try our mxODBC.Connect Python Database Interface for free ! ::::::

   eGenix.com Software, Skills and Services GmbH  Pastor-Loeh-Str.48
    D-40764 Langenfeld, Germany. CEO Dipl.-Math. Marc-Andre Lemburg
           Registered at Amtsgericht Duesseldorf: HRB 46611
               http://www.egenix.com/company/contact/

From zachary.ware at gmail.com  Sat Jan 24 21:23:05 2015
From: zachary.ware at gmail.com (Zachary Ware)
Date: Sat, 24 Jan 2015 14:23:05 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
Message-ID: <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>

On Saturday, January 24, 2015, Brett Cannon <brett at python.org> wrote:

> On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org
> <javascript:_e(%7B%7D,'cvml','greg at krypto.org');>> wrote:
>
>> On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com
>> <javascript:_e(%7B%7D,'cvml','mal at egenix.com');>> wrote:
>>
>>> On 23.01.2015 19:48, Matthias Klose wrote:
>>> > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>> >> Related to my earlier question regarding building Python on Android
>>> >> and an undefined reference to dlopen error...I have the following
>>> >> question:  Is it possible to build and install Python without having
>>> >> to build and install...or use...distutils?
>>> >>
>>> >> Some background:
>>> >> I can build the python interpreter on my device, and I can build a
>>> >> bunch of modules.  The problem appears when make reaches the part
>>> >> where setup.py is used to build and import modules...specifically when
>>> >> setup.py attempts to import distutils.core.
>>> >
>>> > you can do this using Setup.local. This works for me building
>>> additional
>>> > extensions as builtins.  It might require some tweaking to build
>>> everything.
>>>
>>> You may want to have a look at the Setup files we're using
>>> in eGenix PyRun, which uses them to force static builds of the
>>> various built-in extensions.
>>>
>>> Look for these files:
>>>
>>> PyRun/Runtime/Setup.PyRun-2.7
>>> PyRun/Runtime/Setup.PyRun-3.4
>>>
>>> in the source archives:
>>>
>>> http://www.egenix.com/products/python/PyRun/
>>>
>>> > Otoh, I would like to get rid off the setup.py altogether (/me ducks
>>> ...).
>>>
>>> Why ? It's great for finding stuff on your system and configuring
>>> everything without user intervention (well, most of the time :-)).
>>>
>>
>> Because our setup.py is a nightmare of arbitrary code run in a linear
>> fashion with ad-hoc checks for things that are unlike how any other project
>> on the planet determines what is available on your system.  It may have
>> seemed "great" when it was created in 2001.  It really shows its age now.
>>
>> It defeats build parallelism and dependency declaration.
>> It also prevents cross compilation.
>>
>> Building an interpreter with a limited standard library on your build
>> host so that you can run said interpreter to have it drive the remainder of
>> your build is way more self hosting that we rightfully need to be for
>> CPython.
>>
>
> So are you suggesting to add the build rules to configure and the Makefile
> -- and Windows build file -- in order to drop setup.py?
>

Windows already doesn't use setup.py. There are a lot more modules built-in
on Windows, and others have their own project files; distutils isn't used
at all.


-- 
Sent from Gmail Mobile
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/b23bbd2a/attachment.html>

From chaselton at gmail.com  Sat Jan 24 23:01:25 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 24 Jan 2015 16:01:25 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
Message-ID: <CAHu5PrY-30cuTpSgzGMrUQzBw_OXsa=v7CQ-dE4WwhH2U3vPgQ@mail.gmail.com>

On Sat, Jan 24, 2015 at 2:23 PM, Zachary Ware <zachary.ware at gmail.com> wrote:
> On Saturday, January 24, 2015, Brett Cannon <brett at python.org> wrote:
>>
>> On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org> wrote:
>>>
>>> On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com> wrote:
>>>>
>>>> On 23.01.2015 19:48, Matthias Klose wrote:
>>>> > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>>> >> Related to my earlier question regarding building Python on Android
>>>> >> and an undefined reference to dlopen error...I have the following
>>>> >> question:  Is it possible to build and install Python without having
>>>> >> to build and install...or use...distutils?
>>>> >>
>>>> >> Some background:
>>>> >> I can build the python interpreter on my device, and I can build a
>>>> >> bunch of modules.  The problem appears when make reaches the part
>>>> >> where setup.py is used to build and import modules...specifically
>>>> >> when
>>>> >> setup.py attempts to import distutils.core.
>>>> >
>>>> > you can do this using Setup.local. This works for me building
>>>> > additional
>>>> > extensions as builtins.  It might require some tweaking to build
>>>> > everything.
>>>>
>>>> You may want to have a look at the Setup files we're using
>>>> in eGenix PyRun, which uses them to force static builds of the
>>>> various built-in extensions.
>>>>
>>>> Look for these files:
>>>>
>>>> PyRun/Runtime/Setup.PyRun-2.7
>>>> PyRun/Runtime/Setup.PyRun-3.4
>>>>
>>>> in the source archives:
>>>>
>>>> http://www.egenix.com/products/python/PyRun/
>>>>
>>>> > Otoh, I would like to get rid off the setup.py altogether (/me ducks
>>>> > ...).
>>>>
>>>> Why ? It's great for finding stuff on your system and configuring
>>>> everything without user intervention (well, most of the time :-)).
>>>
>>>
>>> Because our setup.py is a nightmare of arbitrary code run in a linear
>>> fashion with ad-hoc checks for things that are unlike how any other project
>>> on the planet determines what is available on your system.  It may have
>>> seemed "great" when it was created in 2001.  It really shows its age now.
>>>
>>> It defeats build parallelism and dependency declaration.
>>> It also prevents cross compilation.
>>>
>>> Building an interpreter with a limited standard library on your build
>>> host so that you can run said interpreter to have it drive the remainder of
>>> your build is way more self hosting that we rightfully need to be for
>>> CPython.
>>
>>
>> So are you suggesting to add the build rules to configure and the Makefile
>> -- and Windows build file -- in order to drop setup.py?
>
>
> Windows already doesn't use setup.py. There are a lot more modules built-in
> on Windows, and others have their own project files; distutils isn't used at
> all.
>
>
Is there a way to either a) remove setup.py from the Makefile install
targets or b) remove it entirely?  Regarding b) is it Setup.local that
should be configured or Setup?

From solipsis at pitrou.net  Sat Jan 24 23:24:30 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 24 Jan 2015 23:24:30 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C40632.4020301@egenix.com>
Message-ID: <20150124232430.49c859ca@fsol>

On Sat, 24 Jan 2015 21:53:06 +0100
"M.-A. Lemburg" <mal at egenix.com> wrote:
> 
> BTW: Parallel execution, cross compilation can be added
> to setup.py. I don't think parallel execution is all
> that important,

$ ./python setup.py build_ext --help
[...]
  --parallel (-j)      number of parallel build jobs


Regards

Antoine.



From matthew.i.frank at intel.com  Sun Jan 25 00:08:39 2015
From: matthew.i.frank at intel.com (Frank, Matthew I)
Date: Sat, 24 Jan 2015 23:08:39 +0000
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <CAHu5PrY23Z5svNdxDOqJLmiR0f7TiCijg2-m4ZtsyF75jJxnqw@mail.gmail.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
 <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>
 <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>
 <CAHu5PrY23Z5svNdxDOqJLmiR0f7TiCijg2-m4ZtsyF75jJxnqw@mail.gmail.com>
Message-ID: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E12A12E@FMSMSX106.amr.corp.intel.com>

Android's dlopen() works slightly differently than the normal Unix dlopen() in at least two different ways.  I haven't seen your particular problem, but that's probably because I'm cross-building CPython (building everything I need on a Linux machine, and then copying the install directory to the Android machine.)

(1) When building for Android you need to explicitly include the "-ldl" flag on the command line for the link step.  (In Linux the dlopen() routine is included in libc, so "-ldl" is not necessary).  I suspect that some part of your distutils was not linked correctly.  (Since I'm cross-building I've never run distutils on the Android side, which is probably why I've not seen this.)  Your best bet would be to run under a debugger and figure out which line of C code in which .so file is throwing the error.  Then going back to the build scripts and looking at how that .so file is getting linked.  (For an example of someone else having a similar problem see for example, http://stackoverflow.com/questions/25846927/git-built-on-android-throws-undefined-reference-to-dlopen-error).

(2) The other possibility has to do with a quirk in Android's dlopen() implementation.  On most legacy Unix systems (including Linux) when you dlopen() a library Z then all the already loaded libraries A, B, C, ... are searched for any dependences of Z (even if Z was not explicitly linked against any of A, B, C,...).  On Android (perhaps for security reasons) that's not true, if Z depends on A, then you need to have "-lA" when you link Z.  An example (and patch) for this problem is http://bugs.python.org/issue21668.

-Matt

> -----Original Message-----
> From: Cyd Haselton [mailto:chaselton at gmail.com]
> Sent: Friday, January 23, 2015 10:50 AM
> To: Brett Cannon
> Cc: Guido van Rossum; Python-Dev
> Subject: Re: [Python-Dev] Undefined dlopen When Building Module On Android
> 
> I guess I'll keep waiting...given the zero responses I've gotten from the android side.
> :-(.
> 
> In the meantime...in case anyone is interested... since I have the working binary, I
> executed it and went through each command in setup.py to see what throws the
> 'undefined reference to dlopen' error.
> Turns out that the distutils module is the culprit...specifically distutils.core.
> 
> 
> On Thu, Jan 22, 2015 at 8:33 AM, Brett Cannon <brett at python.org> wrote:
> > A mobile SIG is being formed, but it doesn't have a mailing list yet,
> > else that would be a good place to ask this question.
> >
> > On Wed Jan 21 2015 at 5:54:39 PM Guido van Rossum <guido at python.org> wrote:
> >>
> >> Maybe try a list focused on Android development? Few people in the
> >> Python core development community have any Android experience. But
> >> the issues and context you offer seem to be related to the Android world, not to
> Python.
> >> (dlopen is used by a lot of systems, not just Python.)
> >>
> >> On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton <chaselton at gmail.com> wrote:
> >>>
> >>> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com>
> >>> wrote:
> >>> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton
> >>> > <chaselton at gmail.com>
> >>> > wrote:
> >>> >> Hello,
> >>> >> I'm struggling with a build issue on Android; I've posted to the
> >>> >> general python list with no result, so I'm re-posting here in
> >>> >> hopes that someone can help.  If this is the wrong place feel
> >>> >> free to let me know.
> >>> >>
> >>> >> I'm attempting to build Python 2.7.8 on my Android device; I'm
> >>> >> using an environment that simulates a Linux filesystem within the
> >>> >> Android terminal using a port of fakechroot.  Within that
> >>> >> environment I've ported and/or bootstrapped a number of Linux
> >>> >> utilities (curl, git, openssl, gcc)
> >>> >>
> >>> >> I run ./configure, then make, and the executable and library are
> >>> >> built.  The problem occurs when build_ext is run; the newly built
> >>> >> python executable builds, then links _struct, and immediately
> >>> >> afterwards I get an 'undefined reference to dlopen' error.
> >>> >>
> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >>> >> --libraries='c dl m' -f, the 'undefined reference to dlopen'
> >>> >> error is thrown again.
> >>> >>
> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
> >>> >> --libraries='-lc -ldl -lm' -f the build continues past
> >>> >> _struct...even though ld throws the expected 'unable to find -l-lc' and other
> errors.
> >>> >>
> >>> >> Let me know if you need me to provide additional information.
> >>> >> Any help would be greatly appreciated.
> >>> >>
> >>> >> Cyd
> >>> >
> >>> >
> >>> > Additionally I took a strace of the error producing command. The
> >>> > following is (hopefully) a relevant portion minus the various 'no
> >>> > such file' lines before the correct lib is found (which it always
> >>> > is)
> >>> >
> >>> > 16513
> >>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-
> 2.7.8/Lib/distutils/unixccompiler.py",
> >>> > O_RDONLY|O_LARGEFILE) = 3           16513
> >>> >
> >>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-
> 2.7.8/Lib/distutils/unixccompiler.pyc",
> >>> > O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
> >>> >  = 16525
> >>> > 16513 wait4(16525,  <unfinished ...>
> >>> > 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
> >>> > O_RDONLY|O_LARGEFILE) = 3
> >>> > 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
> >>> > "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
> >>> > build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars
> >>> > */]) = 0
> >>> >
> >>> > *snip call to libc intercepted by libfakechroot*
> >>> >
> >>> > 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) =
> >>> > 3
> >>> > 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) =
> >>> > 3
> >>> > 16525 open("/dev/__properties__",
> >>> > O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
> >>> > 16525 open("build/temp.linux-armv7l-2.7/multiarch",
> >>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> >>> > 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
> >>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
> >>> > 16525 fork()                            = 16526
> >>> > 16525 wait4(-1,  <unfinished ...>
> >>> > 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE,
> >>> > 0666) = -1 EACCES (Permission denied)
> >>> >
> >>> > See attached for remainder
> >>>
> >>> Should I be posting this issue elsewhere?
> >>> Is more information needed?
> >>>
> >>> _______________________________________________
> >>> Python-Dev mailing list
> >>> Python-Dev at python.org
> >>> https://mail.python.org/mailman/listinfo/python-dev
> >>>
> >>> Unsubscribe:
> >>> https://mail.python.org/mailman/options/python-dev/guido%40python.or
> >>> g
> >>
> >>
> >>
> >>
> >> --
> >> --Guido van Rossum (python.org/~guido)
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/brett%40python.org


From chaselton at gmail.com  Sun Jan 25 00:37:24 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 24 Jan 2015 17:37:24 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E12A12E@FMSMSX106.amr.corp.intel.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
 <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>
 <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>
 <CAHu5PrY23Z5svNdxDOqJLmiR0f7TiCijg2-m4ZtsyF75jJxnqw@mail.gmail.com>
 <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E12A12E@FMSMSX106.amr.corp.intel.com>
Message-ID: <CAHu5PrZMDHb-JzD4LdqfRYC4V2UJR7fzvP1oCddKB3Ui3TH2bA@mail.gmail.com>

Replies in body of message for clarity:

On Sat, Jan 24, 2015 at 5:08 PM, Frank, Matthew I
<matthew.i.frank at intel.com> wrote:
> Android's dlopen() works slightly differently than the normal Unix dlopen() in at least two different ways.  I haven't seen your particular problem, but that's probably because I'm cross-building CPython (building everything I need on a Linux machine, and then copying the install directory to the Android machine.)

Yeah...I have a Linux machine but it currently resides as a gunzipped
tarball on an external drive...I lack the space to decompress and use
it for building purposes.

> (1) When building for Android you need to explicitly include the "-ldl" flag on the command line for the link step.  (In Linux the dlopen() routine is included in libc, so "-ldl" is not necessary).  I suspect that some part of your distutils was not linked correctly.  (Since I'm cross-building I've never run distutils on the Android side, which is probably why I've not seen this.)  Your best bet would be to run under a debugger and figure out which line of C code in which .so file is throwing the error.  Then going back to the build scripts and looking at how that .so file is getting linked.  (For an example of someone else having a similar problem see for example, http://stackoverflow.com/questions/25846927/git-built-on-android-throws-undefined-reference-to-dlopen-error).

Funny you should reference that post as I'm the one who started it; I
ported git, which required curl and openssl ports, a while ago
and...as I think I mentioned in that post...I had to go back and
re-port openssl, making sure to throw in -ldl wherever possible.

Python (which I'm building in the same environment in which I built
openssl, git and curl) is a different beast altogether.  Environment
variables that make it to the Makefile don't make it into the modules
built by setup.py, which is run by distutils, which I wasn't aware was
built alongside Python until you mentioned it just now.  Then there
are the modules in Setup...which are built or not...seemingly
independently of whether or not the line specifying the module is
commented out or not; the hack workaround is to put every module that
should be excluded in setup.py...even if it is commented out in Setup.

Obviously I'm still trying to get a handle on the Python build process.


> (2) The other possibility has to do with a quirk in Android's dlopen() implementation.  On most legacy Unix systems (including Linux) when you dlopen() a library Z then all the already loaded libraries A, B, C, ... are searched for any dependences of Z (even if Z was not explicitly linked against any of A, B, C,...).  On Android (perhaps for security reasons) that's not true, if Z depends on A, then you need to have "-lA" when you link Z.  An example (and patch) for this problem is http://bugs.python.org/issue21668.

Thanks for the bit of info above...very useful.  I've included -lc
-ldl in the requisite places in Setup, and setup.py...although it's
entirely possible I've not added it to places that need it given that
I'm still struggling to understand the build process.

Right now I'm at the point where running configure && make finishes
successfully but throws warnings during the module build and import
process.  Running make install fails completely with the same
'undefined reference to dlopen' because, for some reason, make install
requires the sharedmods target to be rebuilt again, which fails
because setup.py fails when importing distutils.core.  Why it doesn't
do the same when running make is beyond me.

Cyd

From mistersheik at gmail.com  Sun Jan 25 03:10:51 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sat, 24 Jan 2015 21:10:51 -0500
Subject: [Python-Dev] Any grammar experts?
Message-ID: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>

To finish PEP 448, I need to update the grammar for syntax such as

{**x for x in it}

and

{1:2, 3:4, **a}

It's been a long time since I've looked at grammars and I could really use
the advice of an expert.  I'm considering replacing:

dictorsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) |
                  (test (comp_for | (',' test)* [','])) )

with:

dictpopulator: test ':' test | '**' test
dictorsetmaker: ( (dictpopulator (comp_for | (',' dictpopulator)* [','])) |
                   (test (comp_for | (',' test)* [','])) )

Am I headed in the right direction?  Of course I will need to edit
parsermodule.c and ast.c.

Best,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/af6b7677/attachment-0001.html>

From chaselton at gmail.com  Sun Jan 25 03:54:57 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 24 Jan 2015 20:54:57 -0600
Subject: [Python-Dev] Undefined dlopen When Building Module On Android
In-Reply-To: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E12A12E@FMSMSX106.amr.corp.intel.com>
References: <CAHu5PrbbatOyRFcgsxw-cUKNm2e5L-q5FCkT5MZPqMLqbo_8ZQ@mail.gmail.com>
 <CAHu5PraZxdB6eoq06WSDkN_89WZg4LCTM6Zcw8AESBeSbYTRyg@mail.gmail.com>
 <CAHu5PrYg0tWvO0HZNJqiXDaWhrAwx8KNkCif8kv3mnswkk7hNA@mail.gmail.com>
 <CAP7+vJLj5BqXEnzrzqPSeHCRMUFfOdx44n5ekAJN=uUHDC8-xw@mail.gmail.com>
 <CAP1=2W5jx+WRN=zYKdNMs=5ug+EwZPG_MKy9xZ+ZugzTQWB34A@mail.gmail.com>
 <CAHu5PrY23Z5svNdxDOqJLmiR0f7TiCijg2-m4ZtsyF75jJxnqw@mail.gmail.com>
 <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E12A12E@FMSMSX106.amr.corp.intel.com>
Message-ID: <CAHu5Pra0Cc-BMZO1evKw0waf7mzkUbM3JPisV1q_A+kDXOmrWw@mail.gmail.com>

Closer.

make && make install ran successfully; however running the newly
installed binary results in the same error.

On Sat, Jan 24, 2015 at 5:08 PM, Frank, Matthew I
<matthew.i.frank at intel.com> wrote:
> Android's dlopen() works slightly differently than the normal Unix dlopen() in at least two different ways.  I haven't seen your particular problem, but that's probably because I'm cross-building CPython (building everything I need on a Linux machine, and then copying the install directory to the Android machine.)
>
> (1) When building for Android you need to explicitly include the "-ldl" flag on the command line for the link step.  (In Linux the dlopen() routine is included in libc, so "-ldl" is not necessary).  I suspect that some part of your distutils was not linked correctly.  (Since I'm cross-building I've never run distutils on the Android side, which is probably why I've not seen this.)  Your best bet would be to run under a debugger and figure out which line of C code in which .so file is throwing the error.  Then going back to the build scripts and looking at how that .so file is getting linked.  (For an example of someone else having a similar problem see for example, http://stackoverflow.com/questions/25846927/git-built-on-android-throws-undefined-reference-to-dlopen-error).
>
> (2) The other possibility has to do with a quirk in Android's dlopen() implementation.  On most legacy Unix systems (including Linux) when you dlopen() a library Z then all the already loaded libraries A, B, C, ... are searched for any dependences of Z (even if Z was not explicitly linked against any of A, B, C,...).  On Android (perhaps for security reasons) that's not true, if Z depends on A, then you need to have "-lA" when you link Z.  An example (and patch) for this problem is http://bugs.python.org/issue21668.
>
> -Matt
>
>> -----Original Message-----
>> From: Cyd Haselton [mailto:chaselton at gmail.com]
>> Sent: Friday, January 23, 2015 10:50 AM
>> To: Brett Cannon
>> Cc: Guido van Rossum; Python-Dev
>> Subject: Re: [Python-Dev] Undefined dlopen When Building Module On Android
>>
>> I guess I'll keep waiting...given the zero responses I've gotten from the android side.
>> :-(.
>>
>> In the meantime...in case anyone is interested... since I have the working binary, I
>> executed it and went through each command in setup.py to see what throws the
>> 'undefined reference to dlopen' error.
>> Turns out that the distutils module is the culprit...specifically distutils.core.
>>
>>
>> On Thu, Jan 22, 2015 at 8:33 AM, Brett Cannon <brett at python.org> wrote:
>> > A mobile SIG is being formed, but it doesn't have a mailing list yet,
>> > else that would be a good place to ask this question.
>> >
>> > On Wed Jan 21 2015 at 5:54:39 PM Guido van Rossum <guido at python.org> wrote:
>> >>
>> >> Maybe try a list focused on Android development? Few people in the
>> >> Python core development community have any Android experience. But
>> >> the issues and context you offer seem to be related to the Android world, not to
>> Python.
>> >> (dlopen is used by a lot of systems, not just Python.)
>> >>
>> >> On Wed, Jan 21, 2015 at 2:43 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>> >>>
>> >>> On Mon, Jan 19, 2015 at 5:23 PM, Cyd Haselton <chaselton at gmail.com>
>> >>> wrote:
>> >>> > On Mon, Jan 19, 2015 at 8:51 AM, Cyd Haselton
>> >>> > <chaselton at gmail.com>
>> >>> > wrote:
>> >>> >> Hello,
>> >>> >> I'm struggling with a build issue on Android; I've posted to the
>> >>> >> general python list with no result, so I'm re-posting here in
>> >>> >> hopes that someone can help.  If this is the wrong place feel
>> >>> >> free to let me know.
>> >>> >>
>> >>> >> I'm attempting to build Python 2.7.8 on my Android device; I'm
>> >>> >> using an environment that simulates a Linux filesystem within the
>> >>> >> Android terminal using a port of fakechroot.  Within that
>> >>> >> environment I've ported and/or bootstrapped a number of Linux
>> >>> >> utilities (curl, git, openssl, gcc)
>> >>> >>
>> >>> >> I run ./configure, then make, and the executable and library are
>> >>> >> built.  The problem occurs when build_ext is run; the newly built
>> >>> >> python executable builds, then links _struct, and immediately
>> >>> >> afterwards I get an 'undefined reference to dlopen' error.
>> >>> >>
>> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> >>> >> --libraries='c dl m' -f, the 'undefined reference to dlopen'
>> >>> >> error is thrown again.
>> >>> >>
>> >>> >> If I run ./python setup.py --verbose -library-dirs /path/to/lib
>> >>> >> --libraries='-lc -ldl -lm' -f the build continues past
>> >>> >> _struct...even though ld throws the expected 'unable to find -l-lc' and other
>> errors.
>> >>> >>
>> >>> >> Let me know if you need me to provide additional information.
>> >>> >> Any help would be greatly appreciated.
>> >>> >>
>> >>> >> Cyd
>> >>> >
>> >>> >
>> >>> > Additionally I took a strace of the error producing command. The
>> >>> > following is (hopefully) a relevant portion minus the various 'no
>> >>> > such file' lines before the correct lib is found (which it always
>> >>> > is)
>> >>> >
>> >>> > 16513
>> >>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-
>> 2.7.8/Lib/distutils/unixccompiler.py",
>> >>> > O_RDONLY|O_LARGEFILE) = 3           16513
>> >>> >
>> >>> > open("/data/data/jackpal.androidterm/kbox2/bld/python/Python-
>> 2.7.8/Lib/distutils/unixccompiler.pyc",
>> >>> > O_RDONLY|O_LARGEFILE) = 4       16513 vfork()
>> >>> >  = 16525
>> >>> > 16513 wait4(16525,  <unfinished ...>
>> >>> > 16525 open("/data/data/jackpal.androidterm/kbox2/bin/sh",
>> >>> > O_RDONLY|O_LARGEFILE) = 3
>> >>> > 16525 execve("/data/data/jackpal.androidterm/kbox2/bin/sh", ["sh",
>> >>> > "-c", "gcc --sysroot=/usr/gcc-4.9.2/sysroot -print-multiarch >
>> >>> > build/temp.linux-armv7l-2.7/multiarch 2> /dev/null"], [/* 58 vars
>> >>> > */]) = 0
>> >>> >
>> >>> > *snip call to libc intercepted by libfakechroot*
>> >>> >
>> >>> > 16525 open("/system/lib/libc.so", O_RDONLY|O_LARGEFILE|0x80000) =
>> >>> > 3
>> >>> > 16525 open("/system/lib/libm.so", O_RDONLY|O_LARGEFILE|0x80000) =
>> >>> > 3
>> >>> > 16525 open("/dev/__properties__",
>> >>> > O_RDONLY|O_LARGEFILE|O_NOFOLLOW|0x80000) = 3
>> >>> > 16525 open("build/temp.linux-armv7l-2.7/multiarch",
>> >>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>> >>> > 16525 open("/data/data/jackpal.androidterm/kbox2/dev/null",
>> >>> > O_WRONLY|O_CREAT|O_TRUNC|O_LARGEFILE, 0666) = 3
>> >>> > 16525 fork()                            = 16526
>> >>> > 16525 wait4(-1,  <unfinished ...>
>> >>> > 16526 open("/acct/uid/10186/tasks", O_RDWR|O_CREAT|O_LARGEFILE,
>> >>> > 0666) = -1 EACCES (Permission denied)
>> >>> >
>> >>> > See attached for remainder
>> >>>
>> >>> Should I be posting this issue elsewhere?
>> >>> Is more information needed?
>> >>>
>> >>> _______________________________________________
>> >>> Python-Dev mailing list
>> >>> Python-Dev at python.org
>> >>> https://mail.python.org/mailman/listinfo/python-dev
>> >>>
>> >>> Unsubscribe:
>> >>> https://mail.python.org/mailman/options/python-dev/guido%40python.or
>> >>> g
>> >>
>> >>
>> >>
>> >>
>> >> --
>> >> --Guido van Rossum (python.org/~guido)
>> >> _______________________________________________
>> >> Python-Dev mailing list
>> >> Python-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/python-dev
>> >> Unsubscribe:
>> >> https://mail.python.org/mailman/options/python-dev/brett%40python.org
>

From guido at python.org  Sun Jan 25 04:29:03 2015
From: guido at python.org (Guido van Rossum)
Date: Sat, 24 Jan 2015 19:29:03 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
Message-ID: <CAP7+vJ+yk52i6h9oNc0=isiUiOrJgYEZ=+PKjP7uufqqzUQQjg@mail.gmail.com>

Have you tried it yet?

I think you have to inline dictpopulator, because dictpopulator can start
with the same tokens as test, and the parser doesn't backtrack. So it
wouldn't know how to decide whether to take the dictpopulator branch or the
set branch. If you inline it, the parser will know, because it does
something clever within the rule.

As-is, I get a lot of errors from pgen about ambiguity. This one seems to
work (but you still have to adjust the code generator of course):

dictorsetmaker: ( ((test ':' test | '**' test) (comp_for | (',' (test ':'
test | '**' test))* [','])) |
                   (test (comp_for | (',' test)* [','])) )

Also I presume you want a similar treatment for the set branch (replace
both tests with (test | '*' test).

Good luck! There's plenty of code to crib from for the code generation.

--Guido

On Sat, Jan 24, 2015 at 6:10 PM, Neil Girdhar <mistersheik at gmail.com> wrote:

> To finish PEP 448, I need to update the grammar for syntax such as
>
> {**x for x in it}
>
> and
>
> {1:2, 3:4, **a}
>
> It's been a long time since I've looked at grammars and I could really use
> the advice of an expert.  I'm considering replacing:
>
> dictorsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [','])) |
>                   (test (comp_for | (',' test)* [','])) )
>
> with:
>
> dictpopulator: test ':' test | '**' test
> dictorsetmaker: ( (dictpopulator (comp_for | (',' dictpopulator)* [','])) |
>                    (test (comp_for | (',' test)* [','])) )
>
> Am I headed in the right direction?  Of course I will need to edit
> parsermodule.c and ast.c.
>
> Best,
>
> Neil
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/f7b04d97/attachment.html>

From mistersheik at gmail.com  Sun Jan 25 04:31:26 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sat, 24 Jan 2015 22:31:26 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CAP7+vJ+yk52i6h9oNc0=isiUiOrJgYEZ=+PKjP7uufqqzUQQjg@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <CAP7+vJ+yk52i6h9oNc0=isiUiOrJgYEZ=+PKjP7uufqqzUQQjg@mail.gmail.com>
Message-ID: <CAA68w_m31zZPtZB5YowKb1v4QQnb8u8=WTxfZC4O7XQM0eKq2g@mail.gmail.com>

Thanks, I had tried it and got the ambiguities, but I wasn't sure if those
would disappear with editing some peripheral file.

Yes, you're right about the set branch.

Thank you,

Neil

On Sat, Jan 24, 2015 at 10:29 PM, Guido van Rossum <guido at python.org> wrote:

> Have you tried it yet?
>
> I think you have to inline dictpopulator, because dictpopulator can start
> with the same tokens as test, and the parser doesn't backtrack. So it
> wouldn't know how to decide whether to take the dictpopulator branch or the
> set branch. If you inline it, the parser will know, because it does
> something clever within the rule.
>
> As-is, I get a lot of errors from pgen about ambiguity. This one seems to
> work (but you still have to adjust the code generator of course):
>
> dictorsetmaker: ( ((test ':' test | '**' test) (comp_for | (',' (test ':'
> test | '**' test))* [','])) |
>                    (test (comp_for | (',' test)* [','])) )
>
> Also I presume you want a similar treatment for the set branch (replace
> both tests with (test | '*' test).
>
> Good luck! There's plenty of code to crib from for the code generation.
>
> --Guido
>
> On Sat, Jan 24, 2015 at 6:10 PM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
>
>> To finish PEP 448, I need to update the grammar for syntax such as
>>
>> {**x for x in it}
>>
>> and
>>
>> {1:2, 3:4, **a}
>>
>> It's been a long time since I've looked at grammars and I could really
>> use the advice of an expert.  I'm considering replacing:
>>
>> dictorsetmaker: ( (test ':' test (comp_for | (',' test ':' test)* [',']))
>> |
>>                   (test (comp_for | (',' test)* [','])) )
>>
>> with:
>>
>> dictpopulator: test ':' test | '**' test
>> dictorsetmaker: ( (dictpopulator (comp_for | (',' dictpopulator)* [',']))
>> |
>>                    (test (comp_for | (',' test)* [','])) )
>>
>> Am I headed in the right direction?  Of course I will need to edit
>> parsermodule.c and ast.c.
>>
>> Best,
>>
>> Neil
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>
>>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/64105c70/attachment.html>

From ncoghlan at gmail.com  Sun Jan 25 05:00:57 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 25 Jan 2015 14:00:57 +1000
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
Message-ID: <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>

On 25 Jan 2015 01:09, "Benjamin Peterson" <benjamin at python.org> wrote:
>
>
>
> On Sat, Jan 24, 2015, at 03:00, Nick Coghlan wrote:
> > On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org>
> > wrote:
> > >
> > >
> > > On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
> > >> I was also wondering why files like Python/graminit.c are in the
> > >> respository?  They generate spurious merge conflicts.
> > >
> > > Convenience mostly.
> >
> > It also gets us a round a couple of bootstrapping problems, where
> > generating some of those files requires a working Python interpreter,
> > which you may not have if you just cloned the source tree or unpacked
> > the tarball.
>
> We could distribute the generated files in tarballs as part of the
> release process.

It's far more developer friendly to aim to have builds from a source
check-out "just work" if we can. That's pretty much where we are today
(getting external dependencies for the optional parts on *nix can still be
a bit fiddly - it may be worth maintaining instructions for at least apt
and yum in the developer guide that cover that)

Cheers,
Nick.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/34408ea9/attachment.html>

From mistersheik at gmail.com  Sun Jan 25 05:05:18 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sat, 24 Jan 2015 23:05:18 -0500
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
 <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
Message-ID: <CAA68w_=YN9eZx6yX2EW4-K76wvJo=jd+mOpQWQSfDCvio2bnow@mail.gmail.com>

But you can remove Python/graminit.c and "make clean && make" works, right?

On Sat, Jan 24, 2015 at 11:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

>
> On 25 Jan 2015 01:09, "Benjamin Peterson" <benjamin at python.org> wrote:
> >
> >
> >
> > On Sat, Jan 24, 2015, at 03:00, Nick Coghlan wrote:
> > > On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org>
> > > wrote:
> > > >
> > > >
> > > > On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
> > > >> I was also wondering why files like Python/graminit.c are in the
> > > >> respository?  They generate spurious merge conflicts.
> > > >
> > > > Convenience mostly.
> > >
> > > It also gets us a round a couple of bootstrapping problems, where
> > > generating some of those files requires a working Python interpreter,
> > > which you may not have if you just cloned the source tree or unpacked
> > > the tarball.
> >
> > We could distribute the generated files in tarballs as part of the
> > release process.
>
> It's far more developer friendly to aim to have builds from a source
> check-out "just work" if we can. That's pretty much where we are today
> (getting external dependencies for the optional parts on *nix can still be
> a bit fiddly - it may be worth maintaining instructions for at least apt
> and yum in the developer guide that cover that)
>
> Cheers,
> Nick.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150124/88e5ce71/attachment.html>

From greg at krypto.org  Sun Jan 25 06:22:48 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Sun, 25 Jan 2015 05:22:48 +0000
Subject: [Python-Dev] Can Python Be Built Without Distutils
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C40632.4020301@egenix.com> <20150124232430.49c859ca@fsol>
Message-ID: <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>

Why doesn't our Makefile supply that flag with the make parallelism level
in the sharedmods step?

On Sat Jan 24 2015 at 2:25:45 PM Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat, 24 Jan 2015 21:53:06 +0100
> "M.-A. Lemburg" <mal at egenix.com> wrote:
> >
> > BTW: Parallel execution, cross compilation can be added
> > to setup.py. I don't think parallel execution is all
> > that important,
>
> $ ./python setup.py build_ext --help
> [...]
>   --parallel (-j)      number of parallel build jobs
>
>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/93a4d5d0/attachment.html>

From thomas at python.org  Sun Jan 25 10:55:11 2015
From: thomas at python.org (Thomas Wouters)
Date: Sun, 25 Jan 2015 10:55:11 +0100
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CAA68w_=YN9eZx6yX2EW4-K76wvJo=jd+mOpQWQSfDCvio2bnow@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
 <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
 <CAA68w_=YN9eZx6yX2EW4-K76wvJo=jd+mOpQWQSfDCvio2bnow@mail.gmail.com>
Message-ID: <CAPdQG2rvcKANmG0_Sgghqxnd-WSpX9DOCpeYBq+5O7t1fgkCfw@mail.gmail.com>

On Sun, Jan 25, 2015 at 5:05 AM, Neil Girdhar <mistersheik at gmail.com> wrote:

> But you can remove Python/graminit.c and "make clean && make" works, right?
>

If you can write to the directory, yes. Except if you build in a way that
you can't run pgen on the host system, like in a cross build (this may have
been fixed with the last few rounds of cross build fixes) or when
instrumenting Python. Checking these files in trades very minor "committer
pain" (tossing merge conflicts and regenerating the files) for equally
minor pain in the much more diverse group of people compiling CPython.


>
> On Sat, Jan 24, 2015 at 11:00 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>>
>> On 25 Jan 2015 01:09, "Benjamin Peterson" <benjamin at python.org> wrote:
>> >
>> >
>> >
>> > On Sat, Jan 24, 2015, at 03:00, Nick Coghlan wrote:
>> > > On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org>
>> > > wrote:
>> > > >
>> > > >
>> > > > On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
>> > > >> I was also wondering why files like Python/graminit.c are in the
>> > > >> respository?  They generate spurious merge conflicts.
>> > > >
>> > > > Convenience mostly.
>> > >
>> > > It also gets us a round a couple of bootstrapping problems, where
>> > > generating some of those files requires a working Python interpreter,
>> > > which you may not have if you just cloned the source tree or unpacked
>> > > the tarball.
>> >
>> > We could distribute the generated files in tarballs as part of the
>> > release process.
>>
>> It's far more developer friendly to aim to have builds from a source
>> check-out "just work" if we can. That's pretty much where we are today
>> (getting external dependencies for the optional parts on *nix can still be
>> a bit fiddly - it may be worth maintaining instructions for at least apt
>> and yum in the developer guide that cover that)
>>
>> Cheers,
>> Nick.
>>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>
>


-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/958c9d93/attachment-0001.html>

From a.cavallo at cavallinux.eu  Sun Jan 25 11:08:35 2015
From: a.cavallo at cavallinux.eu (Antonio Cavallo)
Date: Sun, 25 Jan 2015 10:08:35 +0000
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
Message-ID: <54C4C0A3.7060701@cavallinux.eu>

I tried a Makefile based build of python (+ some module) in the past for Android (and macos):

    https://bitbucket.org/cavallo71/android

There was no particular problem in dropping autoconfigure+setup.py in the process: the only real problem was the pgen must be 
compiled on the host machine (but that could have changed since).

In general python was a plain compile and link stuff with not much magic involved and that is (very) good in my opinion: magic 
detection tends to fail, autoconfigure need to "execute" stuff to detect things (reason why is quite useless in cross-compile 
mode) etc.

I would've tried also cmake and qmake builds as well, but simply I didn't have time to spend on it.

I hope this helps,
Antonio



Zachary Ware wrote:
> On Saturday, January 24, 2015, Brett Cannon <brett at python.org <mailto:brett at python.org>> wrote:
>
>     On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org <javascript:_e(%7B%7D,'cvml','greg at krypto.org');>> wrote:
>
>         On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com <javascript:_e(%7B%7D,'cvml','mal at egenix.com');>> wrote:
>
>             On 23.01.2015 19:48, Matthias Klose wrote:
>              > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>              >> Related to my earlier question regarding building Python on Android
>              >> and an undefined reference to dlopen error...I have the following
>              >> question:  Is it possible to build and install Python without having
>              >> to build and install...or use...distutils?
>              >>
>              >> Some background:
>              >> I can build the python interpreter on my device, and I can build a
>              >> bunch of modules.  The problem appears when make reaches the part
>              >> where setup.py is used to build and import modules...specifically when
>              >> setup.py attempts to import distutils.core.
>              >
>              > you can do this using Setup.local. This works for me building additional
>              > extensions as builtins.  It might require some tweaking to build everything.
>
>             You may want to have a look at the Setup files we're using
>             in eGenix PyRun, which uses them to force static builds of the
>             various built-in extensions.
>
>             Look for these files:
>
>             PyRun/Runtime/Setup.PyRun-2.7
>             PyRun/Runtime/Setup.PyRun-3.4
>
>             in the source archives:
>
>             http://www.egenix.com/__products/python/PyRun/ <http://www.egenix.com/products/python/PyRun/>
>
>              > Otoh, I would like to get rid off the setup.py altogether (/me ducks ...).
>
>             Why ? It's great for finding stuff on your system and configuring
>             everything without user intervention (well, most of the time :-)).
>
>
>         Because our setup.py is a nightmare of arbitrary code run in a linear fashion with ad-hoc checks for things that are
>         unlike how any other project on the planet determines what is available on your system.  It may have seemed "great" when
>         it was created in 2001.  It really shows its age now.
>
>         It defeats build parallelism and dependency declaration.
>         It also prevents cross compilation.
>
>         Building an interpreter with a limited standard library on your build host so that you can run said interpreter to have
>         it drive the remainder of your build is way more self hosting that we rightfully need to be for CPython.
>
>
>     So are you suggesting to add the build rules to configure and the Makefile -- and Windows build file -- in order to drop
>     setup.py?
>
>
> Windows already doesn't use setup.py. There are a lot more modules built-in on Windows, and others have their own project files;
> distutils isn't used at all.
>
>
> --
> Sent from Gmail Mobile
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu

From mistersheik at gmail.com  Sun Jan 25 12:54:06 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 25 Jan 2015 06:54:06 -0500
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CAPdQG2rvcKANmG0_Sgghqxnd-WSpX9DOCpeYBq+5O7t1fgkCfw@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
 <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
 <CAA68w_=YN9eZx6yX2EW4-K76wvJo=jd+mOpQWQSfDCvio2bnow@mail.gmail.com>
 <CAPdQG2rvcKANmG0_Sgghqxnd-WSpX9DOCpeYBq+5O7t1fgkCfw@mail.gmail.com>
Message-ID: <CAA68w_k2Kao+YTW03A_5GH1BSPxR5Lhvo26RR0eM41Ng73XVaA@mail.gmail.com>

That makes sense.  Thanks for explaining.

On Sun, Jan 25, 2015 at 4:55 AM, Thomas Wouters <thomas at python.org> wrote:

>
>
> On Sun, Jan 25, 2015 at 5:05 AM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
>
>> But you can remove Python/graminit.c and "make clean && make" works,
>> right?
>>
>
> If you can write to the directory, yes. Except if you build in a way that
> you can't run pgen on the host system, like in a cross build (this may have
> been fixed with the last few rounds of cross build fixes) or when
> instrumenting Python. Checking these files in trades very minor "committer
> pain" (tossing merge conflicts and regenerating the files) for equally
> minor pain in the much more diverse group of people compiling CPython.
>
>
>>
>> On Sat, Jan 24, 2015 at 11:00 PM, Nick Coghlan <ncoghlan at gmail.com>
>> wrote:
>>
>>>
>>> On 25 Jan 2015 01:09, "Benjamin Peterson" <benjamin at python.org> wrote:
>>> >
>>> >
>>> >
>>> > On Sat, Jan 24, 2015, at 03:00, Nick Coghlan wrote:
>>> > > On 20 January 2015 at 10:53, Benjamin Peterson <benjamin at python.org>
>>> > > wrote:
>>> > > >
>>> > > >
>>> > > > On Mon, Jan 19, 2015, at 19:40, Neil Girdhar wrote:
>>> > > >> I was also wondering why files like Python/graminit.c are in the
>>> > > >> respository?  They generate spurious merge conflicts.
>>> > > >
>>> > > > Convenience mostly.
>>> > >
>>> > > It also gets us a round a couple of bootstrapping problems, where
>>> > > generating some of those files requires a working Python interpreter,
>>> > > which you may not have if you just cloned the source tree or unpacked
>>> > > the tarball.
>>> >
>>> > We could distribute the generated files in tarballs as part of the
>>> > release process.
>>>
>>> It's far more developer friendly to aim to have builds from a source
>>> check-out "just work" if we can. That's pretty much where we are today
>>> (getting external dependencies for the optional parts on *nix can still be
>>> a bit fiddly - it may be worth maintaining instructions for at least apt
>>> and yum in the developer guide that cover that)
>>>
>>> Cheers,
>>> Nick.
>>>
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/thomas%40python.org
>>
>>
>
>
> --
> Thomas Wouters <thomas at python.org>
>
> Hi! I'm an email virus! Think twice before sending your email to help me
> spread!
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/75b3ef11/attachment.html>

From mistersheik at gmail.com  Sun Jan 25 12:55:52 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 25 Jan 2015 06:55:52 -0500
Subject: [Python-Dev] Disassembly of generated comprehensions
Message-ID: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>

How do I disassemble a generated comprehension?

For example, I am trying to debug the following:

>>> dis.dis('{**{} for x in [{1:2}]}')
  1           0 LOAD_CONST               0 (<code object <dictcomp> at
0x10160b7c0, file "<dis>", line 1>)
              3 LOAD_CONST               1 ('<dictcomp>')
              6 MAKE_FUNCTION            0
              9 LOAD_CONST               2 (2)
             12 LOAD_CONST               3 (1)
             15 BUILD_MAP                1
             18 BUILD_LIST               1
             21 GET_ITER
             22 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             25 RETURN_VALUE

(This requires the new patch in issue 2292.)

The code here looks fine to me, so I need to look into the code object
<dictcomp>.  How do I do that?

Thanks,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/64d60caf/attachment.html>

From mistersheik at gmail.com  Sun Jan 25 13:12:02 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Sun, 25 Jan 2015 07:12:02 -0500
Subject: [Python-Dev] Disassembly of generated comprehensions
In-Reply-To: <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>
References: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>
 <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>
Message-ID: <CAA68w_=iKiz38mwB0dqOqLOvno9K7X1xLsJzmiKsdMmgaNK-5g@mail.gmail.com>

Perfect, thanks!

On Sun, Jan 25, 2015 at 7:08 AM, Petr Viktorin <encukou at gmail.com> wrote:

> On Sun, Jan 25, 2015 at 12:55 PM, Neil Girdhar <mistersheik at gmail.com>
> wrote:
> > How do I disassemble a generated comprehension?
> >
> > For example, I am trying to debug the following:
> >
> >>>> dis.dis('{**{} for x in [{1:2}]}')
> >   1           0 LOAD_CONST               0 (<code object <dictcomp> at
> > 0x10160b7c0, file "<dis>", line 1>)
> >               3 LOAD_CONST               1 ('<dictcomp>')
> >               6 MAKE_FUNCTION            0
> >               9 LOAD_CONST               2 (2)
> >              12 LOAD_CONST               3 (1)
> >              15 BUILD_MAP                1
> >              18 BUILD_LIST               1
> >              21 GET_ITER
> >              22 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
> >              25 RETURN_VALUE
> >
> > (This requires the new patch in issue 2292.)
> >
> > The code here looks fine to me, so I need to look into the code object
> > <dictcomp>.  How do I do that?
>
> Put it in a function, then get it from the function's code's constants.
> I don't have the patch applied but it should work like this even for
> the new syntax:
>
> >>> import dis
> >>> def f(): return {{} for x in [{1:2}]}
> ...
> >>> dis.dis(f)
>   1           0 LOAD_CONST               1 (<code object <setcomp> at
> 0x7ff2c0647420, file "<stdin>", line 1>)
>               3 LOAD_CONST               2 ('f.<locals>.<setcomp>')
>               6 MAKE_FUNCTION            0
>               9 BUILD_MAP                1
>              12 LOAD_CONST               3 (2)
>              15 LOAD_CONST               4 (1)
>              18 STORE_MAP
>              19 BUILD_LIST               1
>              22 GET_ITER
>              23 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
>              26 RETURN_VALUE
> >>> f.__code__.co_consts[1]  # from "LOAD_CONST 1"
> <code object <setcomp> at 0x7ff2c0647420, file "<stdin>", line 1>
> >>> dis.dis(f.__code__.co_consts[1])
>   1           0 BUILD_SET                0
>               3 LOAD_FAST                0 (.0)
>         >>    6 FOR_ITER                12 (to 21)
>               9 STORE_FAST               1 (x)
>              12 BUILD_MAP                0
>              15 SET_ADD                  2
>              18 JUMP_ABSOLUTE            6
>         >>   21 RETURN_VALUE
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/7e17c44c/attachment.html>

From solipsis at pitrou.net  Sun Jan 25 16:07:53 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Jan 2015 16:07:53 +0100
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C40632.4020301@egenix.com> <20150124232430.49c859ca@fsol>
 <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>
Message-ID: <20150125160753.53b98d7b@fsol>

On Sun, 25 Jan 2015 05:22:48 +0000
"Gregory P. Smith" <greg at krypto.org> wrote:
> Why doesn't our Makefile supply that flag with the make parallelism level
> in the sharedmods step?

If I run "make -j4" here, it works (on the default branch).

Regards

Antoine.

From solipsis at pitrou.net  Sun Jan 25 16:08:54 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 25 Jan 2015 16:08:54 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
Message-ID: <20150125160854.36e785e0@fsol>

On Sat, 24 Jan 2015 21:10:51 -0500
Neil Girdhar <mistersheik at gmail.com> wrote:
> To finish PEP 448, I need to update the grammar for syntax such as
> 
> {**x for x in it}

Is this seriously allowed by the PEP? What does it mean exactly?

Regards

Antoine.



From encukou at gmail.com  Sun Jan 25 13:08:57 2015
From: encukou at gmail.com (Petr Viktorin)
Date: Sun, 25 Jan 2015 13:08:57 +0100
Subject: [Python-Dev] Disassembly of generated comprehensions
In-Reply-To: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>
References: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>
Message-ID: <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>

On Sun, Jan 25, 2015 at 12:55 PM, Neil Girdhar <mistersheik at gmail.com> wrote:
> How do I disassemble a generated comprehension?
>
> For example, I am trying to debug the following:
>
>>>> dis.dis('{**{} for x in [{1:2}]}')
>   1           0 LOAD_CONST               0 (<code object <dictcomp> at
> 0x10160b7c0, file "<dis>", line 1>)
>               3 LOAD_CONST               1 ('<dictcomp>')
>               6 MAKE_FUNCTION            0
>               9 LOAD_CONST               2 (2)
>              12 LOAD_CONST               3 (1)
>              15 BUILD_MAP                1
>              18 BUILD_LIST               1
>              21 GET_ITER
>              22 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
>              25 RETURN_VALUE
>
> (This requires the new patch in issue 2292.)
>
> The code here looks fine to me, so I need to look into the code object
> <dictcomp>.  How do I do that?

Put it in a function, then get it from the function's code's constants.
I don't have the patch applied but it should work like this even for
the new syntax:

>>> import dis
>>> def f(): return {{} for x in [{1:2}]}
...
>>> dis.dis(f)
  1           0 LOAD_CONST               1 (<code object <setcomp> at
0x7ff2c0647420, file "<stdin>", line 1>)
              3 LOAD_CONST               2 ('f.<locals>.<setcomp>')
              6 MAKE_FUNCTION            0
              9 BUILD_MAP                1
             12 LOAD_CONST               3 (2)
             15 LOAD_CONST               4 (1)
             18 STORE_MAP
             19 BUILD_LIST               1
             22 GET_ITER
             23 CALL_FUNCTION            1 (1 positional, 0 keyword pair)
             26 RETURN_VALUE
>>> f.__code__.co_consts[1]  # from "LOAD_CONST 1"
<code object <setcomp> at 0x7ff2c0647420, file "<stdin>", line 1>
>>> dis.dis(f.__code__.co_consts[1])
  1           0 BUILD_SET                0
              3 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER                12 (to 21)
              9 STORE_FAST               1 (x)
             12 BUILD_MAP                0
             15 SET_ADD                  2
             18 JUMP_ABSOLUTE            6
        >>   21 RETURN_VALUE

From g.brandl at gmx.net  Sun Jan 25 16:32:01 2015
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 25 Jan 2015 16:32:01 +0100
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150125160854.36e785e0@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol>
Message-ID: <ma329h$utg$1@ger.gmane.org>

On 01/25/2015 04:08 PM, Antoine Pitrou wrote:
> On Sat, 24 Jan 2015 21:10:51 -0500
> Neil Girdhar <mistersheik at gmail.com> wrote:
>> To finish PEP 448, I need to update the grammar for syntax such as
>> 
>> {**x for x in it}
> 
> Is this seriously allowed by the PEP? What does it mean exactly?

It appears to go a bit far.  Especially since you also would have to allow

{*x for x in it}

which is a set comprehension, while the other is a dict comprehension :)

Georg


From chaselton at gmail.com  Sun Jan 25 16:32:41 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sun, 25 Jan 2015 09:32:41 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C40632.4020301@egenix.com> <20150124232430.49c859ca@fsol>
 <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>
Message-ID: <CAHu5PrZcCn=mzx-hoB351KiG0TTAo3pQ89CoyZ5YduZfjrHNFw@mail.gmail.com>

Since (judging from the lack of responses) setup.py can't be removed
from the Makefile, I kept troubleshooting.I've managed to get the
build to complete and make install runs instead of throwing an
undefined reference right off the bat, unfortunately I've run into the
following:

ImportError: No module named _struct

This also happens when make test is run.  I;ve checked...the _struct
module is there.

On Sat, Jan 24, 2015 at 11:22 PM, Gregory P. Smith <greg at krypto.org> wrote:
> Why doesn't our Makefile supply that flag with the make parallelism level in
> the sharedmods step?
>
> On Sat Jan 24 2015 at 2:25:45 PM Antoine Pitrou <solipsis at pitrou.net> wrote:
>>
>> On Sat, 24 Jan 2015 21:53:06 +0100
>> "M.-A. Lemburg" <mal at egenix.com> wrote:
>> >
>> > BTW: Parallel execution, cross compilation can be added
>> > to setup.py. I don't think parallel execution is all
>> > that important,
>>
>> $ ./python setup.py build_ext --help
>> [...]
>>   --parallel (-j)      number of parallel build jobs
>>
>>
>> Regards
>>
>> Antoine.
>>
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chaselton%40gmail.com
>

From rdmurray at bitdance.com  Sun Jan 25 17:32:07 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 25 Jan 2015 11:32:07 -0500
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
 <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
Message-ID: <20150125163207.8DCEC250F55@webabinitio.net>

On Sun, 25 Jan 2015 14:00:57 +1000, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It's far more developer friendly to aim to have builds from a source
> check-out "just work" if we can. That's pretty much where we are today
> (getting external dependencies for the optional parts on *nix can still be
> a bit fiddly - it may be worth maintaining instructions for at least apt
> and yum in the developer guide that cover that)

https://docs.python.org/devguide/setup.html#build-dependencies

From chaselton at gmail.com  Sun Jan 25 20:51:22 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sun, 25 Jan 2015 13:51:22 -0600
Subject: [Python-Dev] Can Python Be Built Without Distutils
In-Reply-To: <54C4C0A3.7060701@cavallinux.eu>
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C4C0A3.7060701@cavallinux.eu>
Message-ID: <CAHu5Prata158zmULkbExozXnvD7YBhngZf98-aBM6rRotcdr7A@mail.gmail.com>

Thanks...this looks interesting Antonio.  I'm not familiar enough with
Makefile syntax and creation to know if this can help in my case and
(fingers crossed) I believe I've just about solved my original
undefined reference to dlopen problem, but I'll be bookmarking it for
future reference (and in case I'm wrong about being close to solving
my original Android build issues).

On Sun, Jan 25, 2015 at 4:08 AM, Antonio Cavallo
<a.cavallo at cavallinux.eu> wrote:
> I tried a Makefile based build of python (+ some module) in the past for
> Android (and macos):
>
>    https://bitbucket.org/cavallo71/android
>
> There was no particular problem in dropping autoconfigure+setup.py in the
> process: the only real problem was the pgen must be compiled on the host
> machine (but that could have changed since).
>
> In general python was a plain compile and link stuff with not much magic
> involved and that is (very) good in my opinion: magic detection tends to
> fail, autoconfigure need to "execute" stuff to detect things (reason why is
> quite useless in cross-compile mode) etc.
>
> I would've tried also cmake and qmake builds as well, but simply I didn't
> have time to spend on it.
>
> I hope this helps,
> Antonio
>
>
>
> Zachary Ware wrote:
>>
>> On Saturday, January 24, 2015, Brett Cannon <brett at python.org
>> <mailto:brett at python.org>> wrote:
>>
>>     On Fri Jan 23 2015 at 5:45:28 PM Gregory P. Smith <greg at krypto.org
>> <javascript:_e(%7B%7D,'cvml','greg at krypto.org');>> wrote:
>>
>>         On Fri Jan 23 2015 at 11:20:02 AM M.-A. Lemburg <mal at egenix.com
>> <javascript:_e(%7B%7D,'cvml','mal at egenix.com');>> wrote:
>>
>>             On 23.01.2015 19:48, Matthias Klose wrote:
>>              > On 01/23/2015 06:30 PM, Cyd Haselton wrote:
>>              >> Related to my earlier question regarding building Python
>> on Android
>>              >> and an undefined reference to dlopen error...I have the
>> following
>>              >> question:  Is it possible to build and install Python
>> without having
>>              >> to build and install...or use...distutils?
>>              >>
>>              >> Some background:
>>              >> I can build the python interpreter on my device, and I can
>> build a
>>              >> bunch of modules.  The problem appears when make reaches
>> the part
>>              >> where setup.py is used to build and import
>> modules...specifically when
>>              >> setup.py attempts to import distutils.core.
>>              >
>>              > you can do this using Setup.local. This works for me
>> building additional
>>              > extensions as builtins.  It might require some tweaking to
>> build everything.
>>
>>             You may want to have a look at the Setup files we're using
>>             in eGenix PyRun, which uses them to force static builds of the
>>             various built-in extensions.
>>
>>             Look for these files:
>>
>>             PyRun/Runtime/Setup.PyRun-2.7
>>             PyRun/Runtime/Setup.PyRun-3.4
>>
>>             in the source archives:
>>
>>             http://www.egenix.com/__products/python/PyRun/
>> <http://www.egenix.com/products/python/PyRun/>
>>
>>              > Otoh, I would like to get rid off the setup.py altogether
>> (/me ducks ...).
>>
>>             Why ? It's great for finding stuff on your system and
>> configuring
>>             everything without user intervention (well, most of the time
>> :-)).
>>
>>
>>         Because our setup.py is a nightmare of arbitrary code run in a
>> linear fashion with ad-hoc checks for things that are
>>         unlike how any other project on the planet determines what is
>> available on your system.  It may have seemed "great" when
>>         it was created in 2001.  It really shows its age now.
>>
>>         It defeats build parallelism and dependency declaration.
>>         It also prevents cross compilation.
>>
>>         Building an interpreter with a limited standard library on your
>> build host so that you can run said interpreter to have
>>         it drive the remainder of your build is way more self hosting that
>> we rightfully need to be for CPython.
>>
>>
>>     So are you suggesting to add the build rules to configure and the
>> Makefile -- and Windows build file -- in order to drop
>>     setup.py?
>>
>>
>> Windows already doesn't use setup.py. There are a lot more modules
>> built-in on Windows, and others have their own project files;
>> distutils isn't used at all.
>>
>>
>> --
>> Sent from Gmail Mobile
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/a.cavallo%40cavallinux.eu
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/chaselton%40gmail.com

From greg.ewing at canterbury.ac.nz  Sun Jan 25 22:56:17 2015
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Mon, 26 Jan 2015 10:56:17 +1300
Subject: [Python-Dev] Disassembly of generated comprehensions
In-Reply-To: <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>
References: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>
 <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>
Message-ID: <54C56681.9060607@canterbury.ac.nz>

Petr Viktorin wrote:
> On Sun, Jan 25, 2015 at 12:55 PM, Neil Girdhar <mistersheik at gmail.com> wrote:
> 
>>How do I disassemble a generated comprehension?
>>
> Put it in a function, then get it from the function's code's constants.

It would be handy if dis had an option to disassemble nested
functions like this automatically.

-- 
Greg

From ncoghlan at gmail.com  Mon Jan 26 00:05:19 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Jan 2015 09:05:19 +1000
Subject: [Python-Dev] Why are generated files in the repository?
In-Reply-To: <20150125163207.8DCEC250F55@webabinitio.net>
References: <CAA68w_kPy5R7by_T_nE5Ta9xz8C=o+=6-WrM=qO3BpO-uLRHXw@mail.gmail.com>
 <1421715205.3228470.216025885.783BAA35@webmail.messagingengine.com>
 <CADiSq7cTZsbNDeRsek6Fv+Jb6hLNVvZfXbjzT2nY4Bf=sX66Lg@mail.gmail.com>
 <1422112159.1282007.218271121.1981C99D@webmail.messagingengine.com>
 <CADiSq7ffF4OVariD-6CQrLSTrkZZngZrEn+cwFeDKwz+T8M5yQ@mail.gmail.com>
 <20150125163207.8DCEC250F55@webabinitio.net>
Message-ID: <CADiSq7ckiUSgKVmrtnBT__nCddA1KmSGWEUQogx9y0yfJSxR_A@mail.gmail.com>

On 26 Jan 2015 02:33, "R. David Murray" <rdmurray at bitdance.com> wrote:
>
> On Sun, 25 Jan 2015 14:00:57 +1000, Nick Coghlan <ncoghlan at gmail.com>
wrote:
> > It's far more developer friendly to aim to have builds from a source
> > check-out "just work" if we can. That's pretty much where we are today
> > (getting external dependencies for the optional parts on *nix can still
be
> > a bit fiddly - it may be worth maintaining instructions for at least apt
> > and yum in the developer guide that cover that)
>
> https://docs.python.org/devguide/setup.html#build-dependencies

Heh, as I was writing that I was thinking, "Did we do that already?". I
should have gone and checked :)

Cheers,
Nick.

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/7efd59b5/attachment.html>

From guido at python.org  Sun Jan 25 23:59:42 2015
From: guido at python.org (Guido van Rossum)
Date: Sun, 25 Jan 2015 14:59:42 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <ma329h$utg$1@ger.gmane.org>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
Message-ID: <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>

On Sun, Jan 25, 2015 at 7:32 AM, Georg Brandl <g.brandl at gmx.net> wrote:

> On 01/25/2015 04:08 PM, Antoine Pitrou wrote:
> > On Sat, 24 Jan 2015 21:10:51 -0500
> > Neil Girdhar <mistersheik at gmail.com> wrote:
> >> To finish PEP 448, I need to update the grammar for syntax such as
> >>
> >> {**x for x in it}
> >
> > Is this seriously allowed by the PEP? What does it mean exactly?
>
> It appears to go a bit far.  Especially since you also would have to allow
>
> {*x for x in it}
>
> which is a set comprehension, while the other is a dict comprehension :)
>

That distinction doesn't bother me -- you might as well claim it's
confusing that f(*x) passes positional args from x while f(**x) passes
keyword args.

And the varargs set comprehension is similar to the varargs list
comprehension:

[*x for x in it]

If `it` were a list of three items, this would be the same as

[*it[0], *it[1], *it[2]]

so the original is a flattening of `it`: [*it[0], *it[1], ...]. (The type
of `it` is wider than list of lists -- it could be an iterable of
iterables. But the thing is still a flattening.)

The set flattening follows from a direct analogy. And `it` doesn't have to
be a set of sets, it still  just needs to be an iterable of iterables --
it's only the flattened result that's turned into a set.

The dict comprehension follows from there -- the input must be an iterable
of iterables of (key, value) pairs.

I like the example from the PEP, a dict combining globals() and locals():

    {**dictionary for dictionary in (globals(), locals())}

This could be written as

    {**globals(), **locals()}

but the general case (a variable number of dicts to combine) requires the
comprehension.

The PEP also supports generator expressions that are flattenings, and again
that follows directly from analogy.

Interestingly, the non-dict versions can all be written today using a
double-nested comprehension, e.g. {**x for x in it} can be written as:

    {x for x in xs for xs in it}

But it's not so straightforward for dict comprehensions -- you'd have to
switch to calling dict():

    dict(x for x in xs for xs in it)

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150125/f2316d03/attachment.html>

From ncoghlan at gmail.com  Mon Jan 26 00:11:20 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 26 Jan 2015 09:11:20 +1000
Subject: [Python-Dev] Disassembly of generated comprehensions
In-Reply-To: <54C56681.9060607@canterbury.ac.nz>
References: <CAA68w_=AEXt+F1Ei2z0ui=jJ7++Ad9KOuwKWYL0HyxyKS0c3Hg@mail.gmail.com>
 <CA+=+wqD7cSXSdMOvB37LEdAA2uq9FT6s9o0ub6xx44kNsD1u8Q@mail.gmail.com>
 <54C56681.9060607@canterbury.ac.nz>
Message-ID: <CADiSq7c8XotZo0_rQ1uvvttDdETdjA5SuoaPxPEPiFOxVVf_rw@mail.gmail.com>

On 26 Jan 2015 07:58, "Greg Ewing" <greg.ewing at canterbury.ac.nz> wrote:
>
> Petr Viktorin wrote:
>>
>> On Sun, Jan 25, 2015 at 12:55 PM, Neil Girdhar <mistersheik at gmail.com>
wrote:
>>
>>> How do I disassemble a generated comprehension?
>>>
>> Put it in a function, then get it from the function's code's constants.
>
>
> It would be handy if dis had an option to disassemble nested
> functions like this automatically.

The code for that already exists, but it needs someone with the roundtuits
to put together new tests and docs: http://bugs.python.org/issue11822

Cheers,
Nick.

>
> --
> Greg
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
https://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/a0788114/attachment.html>

From solipsis at pitrou.net  Mon Jan 26 01:21:24 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Jan 2015 01:21:24 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
Message-ID: <20150126012124.23b6ce31@fsol>

On Sun, 25 Jan 2015 14:59:42 -0800
Guido van Rossum <guido at python.org> wrote:
> On Sun, Jan 25, 2015 at 7:32 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> 
> > On 01/25/2015 04:08 PM, Antoine Pitrou wrote:
> > > On Sat, 24 Jan 2015 21:10:51 -0500
> > > Neil Girdhar <mistersheik at gmail.com> wrote:
> > >> To finish PEP 448, I need to update the grammar for syntax such as
> > >>
> > >> {**x for x in it}
> > >
> > > Is this seriously allowed by the PEP? What does it mean exactly?
> >
> > It appears to go a bit far.  Especially since you also would have to allow
> >
> > {*x for x in it}
> >
> > which is a set comprehension, while the other is a dict comprehension :)
> >
> 
> That distinction doesn't bother me -- you might as well claim it's
> confusing that f(*x) passes positional args from x while f(**x) passes
> keyword args.
> 
> And the varargs set comprehension is similar to the varargs list
> comprehension:
> 
> [*x for x in it]
> 
> If `it` were a list of three items, this would be the same as
> 
> [*it[0], *it[1], *it[2]]

I find all this unreadable and difficult to understand.

Regards

Antoine.



From rdmurray at bitdance.com  Mon Jan 26 03:31:36 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 25 Jan 2015 21:31:36 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126012124.23b6ce31@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
Message-ID: <20150126023137.0475C250DAC@webabinitio.net>

On Mon, 26 Jan 2015 01:21:24 +0100, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sun, 25 Jan 2015 14:59:42 -0800
> Guido van Rossum <guido at python.org> wrote:
> > On Sun, Jan 25, 2015 at 7:32 AM, Georg Brandl <g.brandl at gmx.net> wrote:
> > 
> > > On 01/25/2015 04:08 PM, Antoine Pitrou wrote:
> > > > On Sat, 24 Jan 2015 21:10:51 -0500
> > > > Neil Girdhar <mistersheik at gmail.com> wrote:
> > > >> To finish PEP 448, I need to update the grammar for syntax such as
> > > >>
> > > >> {**x for x in it}
> > > >
> > > > Is this seriously allowed by the PEP? What does it mean exactly?
> > >
> > > It appears to go a bit far.  Especially since you also would have to allow
> > >
> > > {*x for x in it}
> > >
> > > which is a set comprehension, while the other is a dict comprehension :)
> > >
> > 
> > That distinction doesn't bother me -- you might as well claim it's
> > confusing that f(*x) passes positional args from x while f(**x) passes
> > keyword args.
> > 
> > And the varargs set comprehension is similar to the varargs list
> > comprehension:
> > 
> > [*x for x in it]
> > 
> > If `it` were a list of three items, this would be the same as
> > 
> > [*it[0], *it[1], *it[2]]
> 
> I find all this unreadable and difficult to understand.

I did too, before reading the PEP.

After reading the PEP, it makes perfect sense to me.  Nor is the PEP
complicated...it's just a matter of wrapping your head around the
generalization[*] of what are currently special cases that is going on
here.

--David

[*] The *further* generalization...we've already had, for example,
the generalization that allows:

    a, *b = (1, 3, 4)

to work, and that seems very clear to us....now.

From barry at python.org  Mon Jan 26 15:43:26 2015
From: barry at python.org (Barry Warsaw)
Date: Mon, 26 Jan 2015 09:43:26 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126023137.0475C250DAC@webabinitio.net>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
Message-ID: <20150126094326.7ee63001@limelight.wooz.org>

On Jan 25, 2015, at 09:31 PM, R. David Murray wrote:

>> > > {*x for x in it}
>> > >
>> > > which is a set comprehension, while the other is a dict comprehension :)
>> > >
>> > 
>> > That distinction doesn't bother me -- you might as well claim it's
>> > confusing that f(*x) passes positional args from x while f(**x) passes
>> > keyword args.
>> > 
>> > And the varargs set comprehension is similar to the varargs list
>> > comprehension:
>> > 
>> > [*x for x in it]
>> > 
>> > If `it` were a list of three items, this would be the same as
>> > 
>> > [*it[0], *it[1], *it[2]]
>> 
>> I find all this unreadable and difficult to understand.
>
>I did too, before reading the PEP.
>
>After reading the PEP, it makes perfect sense to me.  Nor is the PEP
>complicated...it's just a matter of wrapping your head around the
>generalization[*] of what are currently special cases that is going on
>here.

It does make sense after reading the PEP but it also reduces the readability
and instant understanding of any such code.  This is head-scratcher code that
I'm sure I'd get asked about from folks who aren't steeped in all the dark
corners of Python.  I don't know if that's an argument not to adopt the PEP,
but it I think it would be a good reason to recommend against using such
obscure syntax, unless there's a good reason (and good comments!).

Cheers,
-Barry

From rdmurray at bitdance.com  Mon Jan 26 16:46:02 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 26 Jan 2015 10:46:02 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126094326.7ee63001@limelight.wooz.org>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
Message-ID: <20150126154602.BEC80250F10@webabinitio.net>

On Mon, 26 Jan 2015 09:43:26 -0500, Barry Warsaw <barry at python.org> wrote:
> On Jan 25, 2015, at 09:31 PM, R. David Murray wrote:
> 
> >> > > {*x for x in it}
> >> > >
> >> > > which is a set comprehension, while the other is a dict comprehension :)
> >> > >
> >> > 
> >> > That distinction doesn't bother me -- you might as well claim it's
> >> > confusing that f(*x) passes positional args from x while f(**x) passes
> >> > keyword args.
> >> > 
> >> > And the varargs set comprehension is similar to the varargs list
> >> > comprehension:
> >> > 
> >> > [*x for x in it]
> >> > 
> >> > If `it` were a list of three items, this would be the same as
> >> > 
> >> > [*it[0], *it[1], *it[2]]
> >> 
> >> I find all this unreadable and difficult to understand.
> >
> >I did too, before reading the PEP.
> >
> >After reading the PEP, it makes perfect sense to me.  Nor is the PEP
> >complicated...it's just a matter of wrapping your head around the
> >generalization[*] of what are currently special cases that is going on
> >here.
> 
> It does make sense after reading the PEP but it also reduces the readability
> and instant understanding of any such code.  This is head-scratcher code that
> I'm sure I'd get asked about from folks who aren't steeped in all the dark
> corners of Python.  I don't know if that's an argument not to adopt the PEP,
> but it I think it would be a good reason to recommend against using such
> obscure syntax, unless there's a good reason (and good comments!).

But it is only obscure because we are not used to it yet.  It is a
logical extension of Python's existing conventions once you think about
it.  It will become "obvious" quickly, IMO.

--David

From greg at krypto.org  Mon Jan 26 18:24:31 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 26 Jan 2015 17:24:31 +0000
Subject: [Python-Dev] is a new buildbot gps-debian-profile-opt possible?
References: <CAGE7PNJiSHsms1Ee4P==E58YeqkF9O1JxU2AANSPmR=EF5+7+w@mail.gmail.com>
Message-ID: <CAGE7PNKk4aq7yXvDpuJonEHfJnM+8sDBeODA2gM-81tb6DNTMw@mail.gmail.com>

I want this buildbot to run "make profile-opt" instead of "make all" and
it'd be best if it could _not_ have --with-pydebug on the ./configure
command line.

I've got a small VM at the moment with one CPU and 600mb ram so just give
it a make -j2 at most to start with.

If that is possible, please assign the new buildbot name and password for
me and i'll get it up and running.

motivation for such a buildbot: https://bugs.python.org/issue22904 and any
potential similar issues.  fwiw, the major linux distros ship with
profile-opt interpreter builds as they are significantly faster.

thanks,
-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/86247c08/attachment.html>

From solipsis at pitrou.net  Mon Jan 26 19:12:09 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Jan 2015 19:12:09 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
Message-ID: <20150126191209.11233ab7@fsol>

On Mon, 26 Jan 2015 10:46:02 -0500
"R. David Murray" <rdmurray at bitdance.com> wrote:

> On Mon, 26 Jan 2015 09:43:26 -0500, Barry Warsaw <barry at python.org> wrote:
> > On Jan 25, 2015, at 09:31 PM, R. David Murray wrote:
> > 
> > >> > > {*x for x in it}
> > >> > >
> > >> > > which is a set comprehension, while the other is a dict comprehension :)
> > >> > >
> > >> > 
> > >> > That distinction doesn't bother me -- you might as well claim it's
> > >> > confusing that f(*x) passes positional args from x while f(**x) passes
> > >> > keyword args.
> > >> > 
> > >> > And the varargs set comprehension is similar to the varargs list
> > >> > comprehension:
> > >> > 
> > >> > [*x for x in it]
> > >> > 
> > >> > If `it` were a list of three items, this would be the same as
> > >> > 
> > >> > [*it[0], *it[1], *it[2]]
> > >> 
> > >> I find all this unreadable and difficult to understand.
> > >
> > >I did too, before reading the PEP.
> > >
> > >After reading the PEP, it makes perfect sense to me.  Nor is the PEP
> > >complicated...it's just a matter of wrapping your head around the
> > >generalization[*] of what are currently special cases that is going on
> > >here.
> > 
> > It does make sense after reading the PEP but it also reduces the readability
> > and instant understanding of any such code.  This is head-scratcher code that
> > I'm sure I'd get asked about from folks who aren't steeped in all the dark
> > corners of Python.  I don't know if that's an argument not to adopt the PEP,
> > but it I think it would be a good reason to recommend against using such
> > obscure syntax, unless there's a good reason (and good comments!).
> 
> But it is only obscure because we are not used to it yet.  It is a
> logical extension of Python's existing conventions once you think about
> it.  It will become "obvious" quickly, IMO.

I have to agree with Barry. While the following is obvious even without
having ever used it:

  a, b, *c = range(5)

the following makes me scratch my head and I can't seem to guess what
the *intent* is - which is very problematic when e.g. reviewing code:

  [*x for x in it]
  {**x for x in it}

  (and other similar things)

I also think the multiple-starargs function calls are completely
overboard:

  f(**someargs, **someotherargs)

(I might add I've never felt any need for those)

These generalizations look to me like a case of consistency taken too
far. When some construct needs commenting to be understandable, then
probably that construct shouldn't exist at all.

(yes, I should have reacted earlier. I hadn't realized the PEP
went that far. It seemed short and simple so I didn't look
carefully :-))

By the way, when reading https://www.python.org/dev/peps/pep-0448/, I
can't find a reference to the final pronouncement.

Regards

Antoine.



From skip.montanaro at gmail.com  Mon Jan 26 19:25:10 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 26 Jan 2015 12:25:10 -0600
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126191209.11233ab7@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
Message-ID: <CANc-5UzH7=rnuapgvhogb51cix-c3qnHtN44VXzdBMhRJo-sxg@mail.gmail.com>

On Mon, Jan 26, 2015 at 12:12 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I also think the multiple-starargs function calls are completely
> overboard:
>
>   f(**someargs, **someotherargs)
>
> (I might add I've never felt any need for those)

This makes sense to me, but I wonder how you resolve the case of
overlapping keys.

I will note that pylint complains about any use of *args or **kwds
(calling it "magic"), which seems a bit overboard to me. There's
nothing magic in the current implementation as far as I can see. I
wonder if it makes sense for pylint to allow simple use (basically,
the status quo) to fly by silently, but start to chirp when people use
the facility in all its baroque glory as enabled by the PEP.

Skip

From p.f.moore at gmail.com  Mon Jan 26 19:40:12 2015
From: p.f.moore at gmail.com (Paul Moore)
Date: Mon, 26 Jan 2015 18:40:12 +0000
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126191209.11233ab7@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
Message-ID: <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>

On 26 January 2015 at 18:12, Antoine Pitrou <solipsis at pitrou.net> wrote:
> I have to agree with Barry. While the following is obvious even without
> having ever used it:
>
>   a, b, *c = range(5)
>
> the following makes me scratch my head and I can't seem to guess what
> the *intent* is - which is very problematic when e.g. reviewing code:
>
>   [*x for x in it]
>   {**x for x in it}
>
>   (and other similar things)
>
> I also think the multiple-starargs function calls are completely
> overboard:
>
>   f(**someargs, **someotherargs)
>
> (I might add I've never felt any need for those)

These cases made little sense to me when I first saw them here, but a
quick read of the PEP made them feel reasonably straightforward - *x
is essentially splicing in a list, and a sequence of **x is a sort of
"if you don't find it here, look here" fallback mechanism. Also, real
examples would presumably have clearer variable names, and maybe even
an explanatory comment. And if they don't, they should do (and a code
review that said "this is confusing, add a comment" would seem
entirely appropriate to me).

There *are* some nastily non-intuitive corner cases (for example, if
from_env={'a':12} and from_config={'a':13}, I don't have any sort of
intuition as to what a would be in f(**from_env, **from_config). I'd
go with 12 because the PEP links multiple **-unpackings with
collections.ChainMap, but I wouldn't dare rely on that guess).

My feeling is that the PEP is essentially fine, but the
"Disadvantages" section needs expansion to note (in a reasonable
amount of detail) that it's possible to write very obfuscated code
with these constructs. It should also call out the corner cases and
note that the behaviour, although following from the rules, isn't
obvious. Personally, I don't think the resulting disadvantages are so
bad that the PEP needs to be rejected, it's just a matter of being
open about the potential for unclear code.

Paul

From solipsis at pitrou.net  Mon Jan 26 19:55:52 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Jan 2015 19:55:52 +0100
Subject: [Python-Dev] is a new buildbot gps-debian-profile-opt possible?
References: <CAGE7PNJiSHsms1Ee4P==E58YeqkF9O1JxU2AANSPmR=EF5+7+w@mail.gmail.com>
 <CAGE7PNKk4aq7yXvDpuJonEHfJnM+8sDBeODA2gM-81tb6DNTMw@mail.gmail.com>
Message-ID: <20150126195552.09642df2@fsol>


Of course. I hadn't had time to answer your private e-mail but will do
so soon :-)

Regards

Antoine.


On Mon, 26 Jan 2015 17:24:31 +0000
"Gregory P. Smith" <greg at krypto.org> wrote:
> I want this buildbot to run "make profile-opt" instead of "make all" and
> it'd be best if it could _not_ have --with-pydebug on the ./configure
> command line.
> 
> I've got a small VM at the moment with one CPU and 600mb ram so just give
> it a make -j2 at most to start with.
> 
> If that is possible, please assign the new buildbot name and password for
> me and i'll get it up and running.
> 
> motivation for such a buildbot: https://bugs.python.org/issue22904 and any
> potential similar issues.  fwiw, the major linux distros ship with
> profile-opt interpreter builds as they are significantly faster.
> 
> thanks,
> -gps
> 



From ethan at stoneleaf.us  Mon Jan 26 19:55:58 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 10:55:58 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
Message-ID: <54C68DBE.2020404@stoneleaf.us>

On 01/26/2015 10:40 AM, Paul Moore wrote:

> There *are* some nastily non-intuitive corner cases (for example, if
> from_env={'a':12} and from_config={'a':13}, I don't have any sort of
> intuition as to what a would be in f(**from_env, **from_config). I'd
> go with 12 because the PEP links multiple **-unpackings with
> collections.ChainMap, but I wouldn't dare rely on that guess).

In the your example

  from_env = {'a': 12}
  from_config = {'a': 13}

  f(**from_env, **from_config)

I would think 'a' should be 13, as from_config is processed /after/ from_env.

So which is it?

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/eb99480e/attachment.sig>

From skip.montanaro at gmail.com  Mon Jan 26 20:04:38 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 26 Jan 2015 13:04:38 -0600
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <54C68DBE.2020404@stoneleaf.us>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
Message-ID: <CANc-5UzgBN2Vea526o4pcf=5bp9B2SE7kNFDAQbueDvb8taO1Q@mail.gmail.com>

On Mon, Jan 26, 2015 at 12:55 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> So which is it?

Precisely...

S

From ethan at stoneleaf.us  Mon Jan 26 20:04:25 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 11:04:25 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <54C68DBE.2020404@stoneleaf.us>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
Message-ID: <54C68FB9.602@stoneleaf.us>

On 01/26/2015 10:55 AM, Ethan Furman wrote:
> On 01/26/2015 10:40 AM, Paul Moore wrote:
> 
>> There *are* some nastily non-intuitive corner cases (for example, if
>> from_env={'a':12} and from_config={'a':13}, I don't have any sort of
>> intuition as to what a would be in f(**from_env, **from_config). I'd
>> go with 12 because the PEP links multiple **-unpackings with
>> collections.ChainMap, but I wouldn't dare rely on that guess).
> 
> In the your example
> 
>   from_env = {'a': 12}
>   from_config = {'a': 13}
> 
>   f(**from_env, **from_config)
> 
> I would think 'a' should be 13, as from_config is processed /after/ from_env.
> 
> So which is it?

Going to the PEP:

  kwargs = dict(kw_arguments)
  kwargs.update(more_arguments)
  function(**kwargs)

  or, if you know to do so:

  from collections import ChainMap
  function(**ChainMap(more_arguments, arguments))

I see the arguments to ChainMap are reversed (more_arguments is first), so in Paul's example 'a' would be 13.

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/01a6b1a5/attachment.sig>

From barry at python.org  Mon Jan 26 20:24:59 2015
From: barry at python.org (Barry Warsaw)
Date: Mon, 26 Jan 2015 14:24:59 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <54C68DBE.2020404@stoneleaf.us>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
Message-ID: <20150126142459.74c4d7d4@limelight.wooz.org>

On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:

>In the your example
>
>  from_env = {'a': 12}
>  from_config = {'a': 13}
>
>  f(**from_env, **from_config)
>
>I would think 'a' should be 13, as from_config is processed /after/ from_env.
>
>So which is it?

In the face of ambiguity, refuse the temptation to guess.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/96b75663/attachment.sig>

From ethan at stoneleaf.us  Mon Jan 26 20:29:20 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 11:29:20 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126142459.74c4d7d4@limelight.wooz.org>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
Message-ID: <54C69590.7070409@stoneleaf.us>

On 01/26/2015 11:24 AM, Barry Warsaw wrote:
> On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:
> 
>> In the your example
>>
>>  from_env = {'a': 12}
>>  from_config = {'a': 13}
>>
>>  f(**from_env, **from_config)
>>
>> I would think 'a' should be 13, as from_config is processed /after/ from_env.
>>
>> So which is it?
> 
> In the face of ambiguity, refuse the temptation to guess.

Lots of things are ambiguous until one learns the rules.  ;)

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/4aa156ab/attachment-0001.sig>

From demianbrecht at gmail.com  Mon Jan 26 20:49:22 2015
From: demianbrecht at gmail.com (Demian Brecht)
Date: Mon, 26 Jan 2015 11:49:22 -0800
Subject: [Python-Dev] Overriding stdlib http package
In-Reply-To: <E36B35D7-AA9F-4F2B-B8CA-5FECE22164AE@stufft.io>
References: <54B69A17.9060209@gmail.com>
 <CANkHFr8ErURQf9Yd-+JYtB78mCZWhb03BXx1Xzf+dFsao2xiOg@mail.gmail.com>
 <CAP7+vJ+jrbJSLadBweLSQGx64x07Df5uKC8JQd1LPAtw7PH+aw@mail.gmail.com>
 <54B6CC99.2040501@gmail.com>
 <CAP7+vJ+xHW0pOA7PgyY0qzxjiLTnNL3aqM4oWRHkRVFDNb8Saw@mail.gmail.com>
 <54B6D8D3.4090706@gmail.com>
 <CAP1=2W5TWMx_qSSQ9sYTvnd8+kuNKRJzrigyJB5k=pESpmUM6A@mail.gmail.com>
 <C8C82150-FA63-4A92-8614-DA658D7A8131@stufft.io>
 <CADiSq7f__+JEA2HxVdkoNJ04U+wGucayMQLRJxfZRYOrgG6XGg@mail.gmail.com>
 <E36B35D7-AA9F-4F2B-B8CA-5FECE22164AE@stufft.io>
Message-ID: <54C69A42.6010700@gmail.com>

On 2015-01-24 7:17 AM, Donald Stufft wrote:
> It?s not just power users that it?s good for, it makes it harder for
> even beginners to use things like backports of modules.

What about cases where new module versions are put in as dependencies of
other packages and they stomp standard library packages unbeknownst to
the user installing the higher level package?

For example, let's say packageB overrides stdlib's packageA. packageC
requires packageB, which stomps packageA at import time. Now, author of
packageD requires packageC but is unaware of the fact that packageB
overrides packageA, but heavily uses packageA directly and expects the
stdlib behavior, not the modified behavior in packageB.

(Hope I got the hierarchy right in that description ;))

This would likely cause unexpected behavior and I can only assume that
it would likely be quite difficult to track down, even for a power user.
The same logic applies to unrelated stdlib modules that depend on the
stdlib behavior of packageA as Brett pointed out.

As someone who's recently faced the problem, while making it easier
would have been immediately beneficial to me as the module author, I can
understand the reasoning behind making this a difficult thing to do.

I /do/ think that it might be worthwhile to invest some time in making
it easier to do while still satisfying the safety of other packages, but
I would venture to say it would definitely be non-trivial.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/b3a245eb/attachment.sig>

From chaselton at gmail.com  Mon Jan 26 19:49:01 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Mon, 26 Jan 2015 12:49:01 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
Message-ID: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>

Hello,
I've finally managed to build a (somewhat) working Python port for the
Android tablet I'm using.  Unfortunately, as I quickly found out,
Python's built-in help function requires tkinter, which requires
tcl/tk.

I did download the sources for tcl/tk and built tcl, but found out
that tk requires XWindows libraries and includes.

Long story short:  Is there an alternative documentation system (i.e.
epydoc) that does not have tkinter dependencies?  If so, is there a
parameter or env variable that would allow me to use it instead of
pydoc?

Thanks in advance,
Cyd

From encukou at gmail.com  Mon Jan 26 20:39:00 2015
From: encukou at gmail.com (Petr Viktorin)
Date: Mon, 26 Jan 2015 20:39:00 +0100
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <54C69590.7070409@stoneleaf.us>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
Message-ID: <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>

On Mon, Jan 26, 2015 at 8:29 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> On 01/26/2015 11:24 AM, Barry Warsaw wrote:
>> On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:
>>
>>> In the your example
>>>
>>>  from_env = {'a': 12}
>>>  from_config = {'a': 13}
>>>
>>>  f(**from_env, **from_config)
>>>
>>> I would think 'a' should be 13, as from_config is processed /after/ from_env.
>>>
>>> So which is it?
>>
>> In the face of ambiguity, refuse the temptation to guess.
>
> Lots of things are ambiguous until one learns the rules.  ;)

I don't see why `f(**{'a': 12}, **{'a': 13})` should not be equivalent
to `f(a=12, **{'a':13})` ? iow, raise TypeError.

From ethan at stoneleaf.us  Mon Jan 26 21:06:26 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 12:06:26 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
Message-ID: <54C69E42.4020905@stoneleaf.us>

On 01/26/2015 11:39 AM, Petr Viktorin wrote:
> On Mon, Jan 26, 2015 at 8:29 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 01/26/2015 11:24 AM, Barry Warsaw wrote:
>>> On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:
>>>
>>>> In the your example
>>>>
>>>>  from_env = {'a': 12}
>>>>  from_config = {'a': 13}
>>>>
>>>>  f(**from_env, **from_config)
>>>>
>>>> I would think 'a' should be 13, as from_config is processed /after/ from_env.
>>>>
>>>> So which is it?
>>>
>>> In the face of ambiguity, refuse the temptation to guess.
>>
>> Lots of things are ambiguous until one learns the rules.  ;)
> 
> I don't see why `f(**{'a': 12}, **{'a': 13})` should not be equivalent
> to `f(a=12, **{'a':13})` ? iow, raise TypeError.

It destroy's the chaining value and pretty much makes the improvement not an improvement.  If there's a possibility that
the same key could be in more than one of the dictionaries then you still have to do the

  dict.update(another_dict)

dance.

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/c2779497/attachment.sig>

From solipsis at pitrou.net  Mon Jan 26 21:09:44 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Jan 2015 21:09:44 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
 <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us>
Message-ID: <20150126210944.425ccad3@fsol>

On Mon, 26 Jan 2015 12:06:26 -0800
Ethan Furman <ethan at stoneleaf.us> wrote:
> It destroy's the chaining value and pretty much makes the improvement not an improvement.  If there's a possibility that
> the same key could be in more than one of the dictionaries then you still have to do the
> 
>   dict.update(another_dict)

So what? Is the situation where chaining is desirable common enough?

Not every new feature warrants a syntax addition - especially when it
raises eyebrows as here, and ends up being as obscure as Perl code.

Regards

Antoine.



From rymg19 at gmail.com  Mon Jan 26 21:08:59 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Mon, 26 Jan 2015 14:08:59 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
Message-ID: <CAO41-mPG-O4+MriAZ6g23hs5tcJQGHKUNeN4ziK5fKG=_108GQ@mail.gmail.com>

Looking at pydoc.py, it looks like the Tk is purely optional...and isn't
called from the interpreter. (I'm not a core dev, though, so take that with
a grain of salt.) However, can't you just strip out the gui function and
the one place in the file where it's called?

Again, not a main Python developer, so don't take this too seriously!

On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> Hello,
> I've finally managed to build a (somewhat) working Python port for the
> Android tablet I'm using.  Unfortunately, as I quickly found out,
> Python's built-in help function requires tkinter, which requires
> tcl/tk.
>
> I did download the sources for tcl/tk and built tcl, but found out
> that tk requires XWindows libraries and includes.
>
> Long story short:  Is there an alternative documentation system (i.e.
> epydoc) that does not have tkinter dependencies?  If so, is there a
> parameter or env variable that would allow me to use it instead of
> pydoc?
>
> Thanks in advance,
> Cyd
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/8a018466/attachment.html>

From tseaver at palladion.com  Mon Jan 26 21:17:27 2015
From: tseaver at palladion.com (Tres Seaver)
Date: Mon, 26 Jan 2015 15:17:27 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126094326.7ee63001@limelight.wooz.org>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
Message-ID: <ma67cr$u3h$1@ger.gmane.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 01/26/2015 09:43 AM, Barry Warsaw wrote:

> This is head-scratcher code that I'm sure I'd get asked about from
> folks who aren't steeped in all the dark corners of Python.  I don't
> know if that's an argument not to adopt the PEP, but it I think it
> would be a good reason to recommend against using such obscure syntax,
> unless there's a good reason

Heh, 2003 called, and they want their "list incomprehension" debate back. ;)



Tres.
- -- 
===================================================================
Tres Seaver          +1 540-429-0999          tseaver at palladion.com
Palladion Software   "Excellence by Design"    http://palladion.com
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iEYEARECAAYFAlTGoNcACgkQ+gerLs4ltQ4AWQCfQ/zjosbaC6YcPItzmxZer/oW
f/cAoLwXzzfHV2LhT4AZD/KS31TkKJyI
=Oxqm
-----END PGP SIGNATURE-----


From ethan at stoneleaf.us  Mon Jan 26 21:22:20 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 12:22:20 -0800
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126210944.425ccad3@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us> <20150126210944.425ccad3@fsol>
Message-ID: <54C6A1FC.7090307@stoneleaf.us>

On 01/26/2015 12:09 PM, Antoine Pitrou wrote:
> On Mon, 26 Jan 2015 12:06:26 -0800
> Ethan Furman <ethan at stoneleaf.us> wrote:
>> It destroy's the chaining value and pretty much makes the improvement not an improvement.  If there's a possibility that
>> the same key could be in more than one of the dictionaries then you still have to do the
>>
>>   dict.update(another_dict)
> 
> So what? Is the situation where chaining is desirable common enough?

Common enough to not break it, yes.

> Not every new feature warrants a syntax addition - especially when it
> raises eyebrows as here, and ends up being as obscure as Perl code.

Not sure what you mean here -- the new feature is a syntax addition (or more appropriately a generalization of existing
syntax).

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/7f0e618f/attachment.sig>

From ethan at stoneleaf.us  Mon Jan 26 21:25:19 2015
From: ethan at stoneleaf.us (Ethan Furman)
Date: Mon, 26 Jan 2015 12:25:19 -0800
Subject: [Python-Dev] Python bug tracker & Roundup
Message-ID: <54C6A2AF.60301@stoneleaf.us>

Do we have our own mailing list for Roundup?

I'm trying to fix the Activity Date search bug (only returns issues where the *last* activity was on a date, not *any*
activity on a date).

--
~Ethan~

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150126/75109175/attachment.sig>

From skip.montanaro at gmail.com  Mon Jan 26 21:26:52 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Mon, 26 Jan 2015 14:26:52 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
Message-ID: <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>

On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:
> Unfortunately, as I quickly found out,
> Python's built-in help function requires tkinter, which requires
> tcl/tk.

I'm a little confused. Are you using some sort of freeze system which
is deciding Tkinter is required? I use help() all the time from the
interpreter prompt and never get a GUI. IMO you should be able to
strip out the gui() function (or even just comment out the Tkinter
import).

Skip

From berker.peksag at gmail.com  Mon Jan 26 21:30:21 2015
From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=)
Date: Mon, 26 Jan 2015 22:30:21 +0200
Subject: [Python-Dev] Python bug tracker & Roundup
In-Reply-To: <54C6A2AF.60301@stoneleaf.us>
References: <54C6A2AF.60301@stoneleaf.us>
Message-ID: <CAF4280JLQRKe5sy_Uobps+1YJ=uRc4RgYnyKUSsm-eDfhBXYQw@mail.gmail.com>

On Mon, Jan 26, 2015 at 10:25 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
> Do we have our own mailing list for Roundup?

Hi,

Yes: https://mail.python.org/mailman/listinfo/tracker-discuss There is
also a tracker for bugs.p.o:
http://psf.upfronthosting.co.za/roundup/meta/

--Berker

From zachary.ware+pydev at gmail.com  Mon Jan 26 21:25:57 2015
From: zachary.ware+pydev at gmail.com (Zachary Ware)
Date: Mon, 26 Jan 2015 14:25:57 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
Message-ID: <CAKJDb-PcRHFeerHq6TGWQYN_nM+NqrqFG9v4KyT8_4LRSti=_A@mail.gmail.com>

On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:
> Hello,
> I've finally managed to build a (somewhat) working Python port for the
> Android tablet I'm using.  Unfortunately, as I quickly found out,
> Python's built-in help function requires tkinter, which requires
> tcl/tk.

What version of Python are you building, and how did you call help
that it seems to want to use tkinter?  The pydoc GUI was removed from
the 3.x series long ago, and should only be activated in 2.7 if you
call it as 'pydoc -g ...' from the command line (or use pydoc.gui()
directly).

-- 
Zach

From g.brandl at gmx.net  Mon Jan 26 21:32:33 2015
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 26 Jan 2015 21:32:33 +0100
Subject: [Python-Dev] Python bug tracker & Roundup
In-Reply-To: <54C6A2AF.60301@stoneleaf.us>
References: <54C6A2AF.60301@stoneleaf.us>
Message-ID: <ma6891$jbf$1@ger.gmane.org>

On 01/26/2015 09:25 PM, Ethan Furman wrote:
> Do we have our own mailing list for Roundup?
> 
> I'm trying to fix the Activity Date search bug (only returns issues where the
> *last* activity was on a date, not *any* activity on a date).

That would be tracker-discuss.

Georg


From g.brandl at gmx.net  Mon Jan 26 21:36:29 2015
From: g.brandl at gmx.net (Georg Brandl)
Date: Mon, 26 Jan 2015 21:36:29 +0100
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CANc-5UzH7=rnuapgvhogb51cix-c3qnHtN44VXzdBMhRJo-sxg@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CANc-5UzH7=rnuapgvhogb51cix-c3qnHtN44VXzdBMhRJo-sxg@mail.gmail.com>
Message-ID: <ma68gd$jbf$2@ger.gmane.org>

On 01/26/2015 07:25 PM, Skip Montanaro wrote:
> On Mon, Jan 26, 2015 at 12:12 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> I also think the multiple-starargs function calls are completely
>> overboard:
>>
>>   f(**someargs, **someotherargs)
>>
>> (I might add I've never felt any need for those)
> 
> This makes sense to me, but I wonder how you resolve the case of
> overlapping keys.
> 
> I will note that pylint complains about any use of *args or **kwds
> (calling it "magic"), which seems a bit overboard to me. There's
> nothing magic in the current implementation as far as I can see.

Yeah, that's one of the sillier on-by-default warnings that I always
disable immediately.

Georg


From storchaka at gmail.com  Mon Jan 26 21:47:16 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 26 Jan 2015 22:47:16 +0200
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150125160854.36e785e0@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol>
Message-ID: <ma694k$m5$1@ger.gmane.org>

On 25.01.15 17:08, Antoine Pitrou wrote:
> On Sat, 24 Jan 2015 21:10:51 -0500
> Neil Girdhar <mistersheik at gmail.com> wrote:
>> To finish PEP 448, I need to update the grammar for syntax such as
>> {**x for x in it}
> Is this seriously allowed by the PEP? What does it mean exactly?

I would understand this as

    {k: v for x in it for k, v in x.items()}



From storchaka at gmail.com  Mon Jan 26 21:50:08 2015
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 26 Jan 2015 22:50:08 +0200
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
Message-ID: <ma69a0$81i$1@ger.gmane.org>

On 26.01.15 00:59, Guido van Rossum wrote:
> Interestingly, the non-dict versions can all be written today using a
> double-nested comprehension, e.g. {**x for x in it} can be written as:
>
>      {x for x in xs for xs in it}

      {x for xs in it for x in xs}

> But it's not so straightforward for dict comprehensions -- you'd have to
> switch to calling dict():
>
>      dict(x for x in xs for xs in it)

      {k: v for xs in it for k, v in xs.items()}

So actually this is just a syntax sugar.



From solipsis at pitrou.net  Mon Jan 26 22:05:44 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 26 Jan 2015 22:05:44 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
 <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us> <20150126210944.425ccad3@fsol>
 <54C6A1FC.7090307@stoneleaf.us>
Message-ID: <20150126220544.4c069c78@fsol>

On Mon, 26 Jan 2015 12:22:20 -0800
Ethan Furman <ethan at stoneleaf.us> wrote:
> On 01/26/2015 12:09 PM, Antoine Pitrou wrote:
> > On Mon, 26 Jan 2015 12:06:26 -0800
> > Ethan Furman <ethan at stoneleaf.us> wrote:
> >> It destroy's the chaining value and pretty much makes the improvement not an improvement.  If there's a possibility that
> >> the same key could be in more than one of the dictionaries then you still have to do the
> >>
> >>   dict.update(another_dict)
> > 
> > So what? Is the situation where chaining is desirable common enough?
> 
> Common enough to not break it, yes.

Really? What are the use cases?

> > Not every new feature warrants a syntax addition - especially when it
> > raises eyebrows as here, and ends up being as obscure as Perl code.
> 
> Not sure what you mean here -- the new feature is a syntax addition

That's what I said.




From rdmurray at bitdance.com  Mon Jan 26 22:28:24 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 26 Jan 2015 16:28:24 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150126220544.4c069c78@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us> <20150126210944.425ccad3@fsol>
 <54C6A1FC.7090307@stoneleaf.us> <20150126220544.4c069c78@fsol>
Message-ID: <20150126212824.E7EF5250EE9@webabinitio.net>

On Mon, 26 Jan 2015 22:05:44 +0100, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 26 Jan 2015 12:22:20 -0800
> Ethan Furman <ethan at stoneleaf.us> wrote:
> > On 01/26/2015 12:09 PM, Antoine Pitrou wrote:
> > > On Mon, 26 Jan 2015 12:06:26 -0800
> > > Ethan Furman <ethan at stoneleaf.us> wrote:
> > >> It destroy's the chaining value and pretty much makes the improvement not an improvement.  If there's a possibility that
> > >> the same key could be in more than one of the dictionaries then you still have to do the
> > >>
> > >>   dict.update(another_dict)
> > > 
> > > So what? Is the situation where chaining is desirable common enough?
> > 
> > Common enough to not break it, yes.
> 
> Really? What are the use cases?

My use case is a configuration method that takes keyword parameters.
In tests I want to specify a bunch of default values for the
configuration, but I want individual test methods to be able
to override those values.  So I have a bunch of code that does
the equivalent of:

from test.support import default_config
[...]
    def _prep(self, config_overrides):
        config = default.config.copy()
        config.update(config_overrides)
        my_config_object.load(**config)
        ....

With the current proposal I could instead do:

    def _prep(self, config_overrides):
        my_config_object.load(**default_config, **config_overrides)

I suspect I have run into situations like this elsewhere as well, but
this is the one from one of my current projects.

That said, I must admit to being a bit ambivalent about this, since
we otherwise are careful to disallow duplicate argument names.

So, instead we could write:

    my_config_object.load(**{**default_config, **config_overrides})

since dict literals *do* allow duplicate keys.

--David

From solipsis at pitrou.net  Tue Jan 27 00:07:08 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 27 Jan 2015 00:07:08 +0100
Subject: [Python-Dev] Any grammar experts?
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol>
 <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net>
 <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us>
 <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us> <20150126210944.425ccad3@fsol>
 <54C6A1FC.7090307@stoneleaf.us> <20150126220544.4c069c78@fsol>
 <20150126212824.E7EF5250EE9@webabinitio.net>
Message-ID: <20150127000708.3cbccb39@fsol>

On Mon, 26 Jan 2015 16:28:24 -0500
"R. David Murray" <rdmurray at bitdance.com> wrote:
> 
> My use case is a configuration method that takes keyword parameters.
> In tests I want to specify a bunch of default values for the
> configuration, but I want individual test methods to be able
> to override those values.  So I have a bunch of code that does
> the equivalent of:
> 
> from test.support import default_config
> [...]
>     def _prep(self, config_overrides):
>         config = default.config.copy()
>         config.update(config_overrides)
>         my_config_object.load(**config)
>         ....
> 
> With the current proposal I could instead do:
> 
>     def _prep(self, config_overrides):
>         my_config_object.load(**default_config, **config_overrides)

It sounds like the _prep() method exists once in your code base, this
isn't an idiom you are duplicating everywhere. The incentive for a
syntactic shortcut looks pretty thin.

Regards

Antoine.



From python at mrabarnett.plus.com  Tue Jan 27 00:24:21 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Mon, 26 Jan 2015 23:24:21 +0000
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
Message-ID: <54C6CCA5.7010108@mrabarnett.plus.com>

On 2015-01-26 19:39, Petr Viktorin wrote:
> On Mon, Jan 26, 2015 at 8:29 PM, Ethan Furman <ethan at stoneleaf.us> wrote:
>> On 01/26/2015 11:24 AM, Barry Warsaw wrote:
>>> On Jan 26, 2015, at 10:55 AM, Ethan Furman wrote:
>>>
>>>> In the your example
>>>>
>>>>  from_env = {'a': 12}
>>>>  from_config = {'a': 13}
>>>>
>>>>  f(**from_env, **from_config)
>>>>
>>>> I would think 'a' should be 13, as from_config is processed /after/ from_env.
>>>>
>>>> So which is it?
>>>
>>> In the face of ambiguity, refuse the temptation to guess.
>>
>> Lots of things are ambiguous until one learns the rules.  ;)
>
> I don't see why `f(**{'a': 12}, **{'a': 13})` should not be equivalent
> to `f(a=12, **{'a':13})` ? iow, raise TypeError.
>
One the one hand we have:

 >>> foo(a=12, **{'a': 13})
Traceback (most recent call last):
   File "<stdin>", line 1, in <module>
TypeError: foo() got multiple values for keyword argument 'a'

and on the other hand we have:

 >>> foo(a=12, a=13)
   File "<stdin>", line 1
SyntaxError: keyword argument repeated

(Should this be a SyntaxError?)

But we also have:

 >>> {'a': 12, 'a': 13}
{'a': 13}

So, what should:

 >>> f(**from_env, **from_config)

do if there are common keys?

Raise an exception? Behave like dict.update? Behave like ChainMap?


From rdmurray at bitdance.com  Tue Jan 27 00:32:49 2015
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 26 Jan 2015 18:32:49 -0500
Subject: [Python-Dev] Any grammar experts?
In-Reply-To: <20150127000708.3cbccb39@fsol>
References: <CAA68w_=TZ5keMmHs6z1=Uvh4=DuUTSj0RQ_9F7YsroMmC3F4Ug@mail.gmail.com>
 <20150125160854.36e785e0@fsol> <ma329h$utg$1@ger.gmane.org>
 <CAP7+vJK7378gapFJof+w0ds2Gwc-bZoA1BOFTfJHMdJx87-9xQ@mail.gmail.com>
 <20150126012124.23b6ce31@fsol> <20150126023137.0475C250DAC@webabinitio.net>
 <20150126094326.7ee63001@limelight.wooz.org>
 <20150126154602.BEC80250F10@webabinitio.net> <20150126191209.11233ab7@fsol>
 <CACac1F8xkFKeEXp=fzJr62mrauH5s6rOZpdmHHQC=+pOvd9RVQ@mail.gmail.com>
 <54C68DBE.2020404@stoneleaf.us> <20150126142459.74c4d7d4@limelight.wooz.org>
 <54C69590.7070409@stoneleaf.us>
 <CA+=+wqDmPCEBrpU5QJ6TLQ4j9kKPc2i716XmXYuDXW2mC0-Nrw@mail.gmail.com>
 <54C69E42.4020905@stoneleaf.us> <20150126210944.425ccad3@fsol>
 <54C6A1FC.7090307@stoneleaf.us> <20150126220544.4c069c78@fsol>
 <20150126212824.E7EF5250EE9@webabinitio.net> <20150127000708.3cbccb39@fsol>
Message-ID: <20150126233249.79672250EEE@webabinitio.net>

On Tue, 27 Jan 2015 00:07:08 +0100, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 26 Jan 2015 16:28:24 -0500
> "R. David Murray" <rdmurray at bitdance.com> wrote:
> > 
> > My use case is a configuration method that takes keyword parameters.
> > In tests I want to specify a bunch of default values for the
> > configuration, but I want individual test methods to be able
> > to override those values.  So I have a bunch of code that does
> > the equivalent of:
> > 
> > from test.support import default_config
> > [...]
> >     def _prep(self, config_overrides):
> >         config = default.config.copy()
> >         config.update(config_overrides)
> >         my_config_object.load(**config)
> >         ....
> > 
> > With the current proposal I could instead do:
> > 
> >     def _prep(self, config_overrides):
> >         my_config_object.load(**default_config, **config_overrides)
> 
> It sounds like the _prep() method exists once in your code base, this
> isn't an idiom you are duplicating everywhere. The incentive for a
> syntactic shortcut looks pretty thin.

Something similar exists between five and ten times (I didn't go in and
count) in my current code base, in various specific forms for different
test classes.

--David

From skip.montanaro at gmail.com  Tue Jan 27 13:52:44 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 27 Jan 2015 06:52:44 -0600
Subject: [Python-Dev] Undefined reference to dlopen (was: Pydoc Replacement
	for Python's help()?)
Message-ID: <CANc-5Ux4-2WTBDwfkJVxw2Q95rTCLxZpA4HYexgpVPRTQbSkqg@mail.gmail.com>

On Tue, Jan 27, 2015 at 6:31 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> Additionally it appears as though some modules were not built with the
> correct links to -lc -ldl, even though I added them as dependencies in
> Setup and setup.py, as well as in the appropriate env variables.
> Importing string, tokenize, operator, inspect...and probably others I
> haven't tested...throw the 'undefined reference to dlopen' error.

Is this another topic? If so, please start another thread. People
glancing at subjects
won't recognize that you're now tackling a different problem.

The modules you mention here (as well as pydoc) are all pure Python modules.
I don't think any of them would have directly triggered a dlopen error. Do you
have a traceback?

Skip

From chaselton at gmail.com  Tue Jan 27 13:17:31 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 06:17:31 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAKJDb-PcRHFeerHq6TGWQYN_nM+NqrqFG9v4KyT8_4LRSti=_A@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CAKJDb-PcRHFeerHq6TGWQYN_nM+NqrqFG9v4KyT8_4LRSti=_A@mail.gmail.com>
Message-ID: <CAHu5PrZ61DUxfLwaX7pwjmo1Tg6cCMzo4HQF4f3YrFmPuPqrVg@mail.gmail.com>

On Mon, Jan 26, 2015 at 2:25 PM, Zachary Ware
<zachary.ware+pydev at gmail.com> wrote:
> On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>> Hello,
>> I've finally managed to build a (somewhat) working Python port for the
>> Android tablet I'm using.  Unfortunately, as I quickly found out,
>> Python's built-in help function requires tkinter, which requires
>> tcl/tk.
>
> What version of Python are you building, and how did you call help
> that it seems to want to use tkinter?  The pydoc GUI was removed from
> the 3.x series long ago, and should only be activated in 2.7 if you
> call it as 'pydoc -g ...' from the command line (or use pydoc.gui()
> directly).
>
> --
> Zach

I'm using version 2.7.8. I'm calling help() from within the python interpreter.

From chaselton at gmail.com  Tue Jan 27 13:31:44 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 06:31:44 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
Message-ID: <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>

On Mon, Jan 26, 2015 at 2:26 PM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>> Unfortunately, as I quickly found out,
>> Python's built-in help function requires tkinter, which requires
>> tcl/tk.
>
> I'm a little confused. Are you using some sort of freeze system which
> is deciding Tkinter is required? I use help() all the time from the
> interpreter prompt and never get a GUI. IMO you should be able to
> strip out the gui() function (or even just comment out the Tkinter
> import).
>
> Skip

I'm not using a freeze system, just the python I built from sources. I
may have misread the pydoc and help() documentation online.

Additionally it appears as though some modules were not built with the
correct links to -lc -ldl, even though I added them as dependencies in
Setup and setup.py, as well as in the appropriate env variables.
Importing string, tokenize, operator, inspect...and probably others I
haven't tested...throw the 'undefined reference to dlopen' error.

Looks like I'll need to run make distclean and start over.  Is there
documentation on which modules are built where and how and with which
libraries they are linked?

From chaselton at gmail.com  Tue Jan 27 13:46:38 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 06:46:38 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
Message-ID: <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>

On Tue, Jan 27, 2015 at 6:31 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> On Mon, Jan 26, 2015 at 2:26 PM, Skip Montanaro
> <skip.montanaro at gmail.com> wrote:
>> On Mon, Jan 26, 2015 at 12:49 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>> Unfortunately, as I quickly found out,
>>> Python's built-in help function requires tkinter, which requires
>>> tcl/tk.
>>
>> I'm a little confused. Are you using some sort of freeze system which
>> is deciding Tkinter is required? I use help() all the time from the
>> interpreter prompt and never get a GUI. IMO you should be able to
>> strip out the gui() function (or even just comment out the Tkinter
>> import).
>>
>> Skip
>
> I'm not using a freeze system, just the python I built from sources. I
> may have misread the pydoc and help() documentation online.
>
> Additionally it appears as though some modules were not built with the
> correct links to -lc -ldl, even though I added them as dependencies in
> Setup and setup.py, as well as in the appropriate env variables.
> Importing string, tokenize, operator, inspect...and probably others I
> haven't tested...throw the 'undefined reference to dlopen' error.
>
> Looks like I'll need to run make distclean and start over.  Is there
> documentation on which modules are built where and how and with which
> libraries they are linked?

A quick FYI: The decision to build 2.7.8 (instead of 3.x) on Android
was made after reading this article:
https://wiki.python.org/moin/Python2orPython3

From skip.montanaro at gmail.com  Tue Jan 27 16:07:35 2015
From: skip.montanaro at gmail.com (Skip Montanaro)
Date: Tue, 27 Jan 2015 09:07:35 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
 <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
Message-ID: <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>

On Tue, Jan 27, 2015 at 6:46 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> A quick FYI: The decision to build 2.7.8 (instead of 3.x) on Android
> was made after reading this article:
> https://wiki.python.org/moin/Python2orPython3

What in that document convinced you to port to Python 2 instead of
Python 3? That page is intended for people deciding which version of
the language to use for their work. I would think as someone trying to
port to a new platform your criteria would be different. Also, note
that the above page was last updated in April 2014. Look at the info
for that page:

    https://wiki.python.org/moin/Python2orPython3?action=info

and you'll see there was probably nothing of substance added to that
page in over a year.

Right at the very top of that page, I read:

    Short version: Python 2.x is legacy, Python 3.x is the present and
future of the language

I would think that would be enough to convince you to use Python 3.x
as your starting point. I say this as someone who, in his day-to-day
work uses Python 2.7 exclusively, doesn't expect his employer to ever
convert to Python 3, and has never done much more with Python 3 than
build it. With all that, if I was going to attempt a port of Python to
a new platform, I would still choose to port Python 3.

Skip

From chaselton at gmail.com  Tue Jan 27 16:56:53 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 09:56:53 -0600
Subject: [Python-Dev] Undefined reference to dlopen (was: Pydoc
 Replacement for Python's help()?)
In-Reply-To: <CANc-5Ux4-2WTBDwfkJVxw2Q95rTCLxZpA4HYexgpVPRTQbSkqg@mail.gmail.com>
References: <CANc-5Ux4-2WTBDwfkJVxw2Q95rTCLxZpA4HYexgpVPRTQbSkqg@mail.gmail.com>
Message-ID: <CAHu5PrZ85a8fwmzZb_QgciBRCEkYAEpeafcTzY5PO6e5eLn16w@mail.gmail.com>

On Tue, Jan 27, 2015 at 6:52 AM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> On Tue, Jan 27, 2015 at 6:31 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>> Additionally it appears as though some modules were not built with the
>> correct links to -lc -ldl, even though I added them as dependencies in
>> Setup and setup.py, as well as in the appropriate env variables.
>> Importing string, tokenize, operator, inspect...and probably others I
>> haven't tested...throw the 'undefined reference to dlopen' error.
>
> Is this another topic? If so, please start another thread. People
> glancing at subjects
> won't recognize that you're now tackling a different problem.
>
> The modules you mention here (as well as pydoc) are all pure Python modules.
> I don't think any of them would have directly triggered a dlopen error. Do you
> have a traceback?
>
> Skip

Apologies for the confusion; the above was not intended to be another
topic but an explanation of why the help() module failed in my port
and why I believe I'll need to start over.

Basically...between my original email and the one above, I managed to
review the pydoc.py code, test the imports in said code and discover
ones (not tkinter) that were causing errors.

From chaselton at gmail.com  Tue Jan 27 17:04:45 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 10:04:45 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
 <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
 <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>
Message-ID: <CAHu5PraCiz1hCsH5xBXApC-PAtXRjdwBP9fR8fDOts9=c9Cvjg@mail.gmail.com>

On Tue, Jan 27, 2015 at 9:07 AM, Skip Montanaro
<skip.montanaro at gmail.com> wrote:
> On Tue, Jan 27, 2015 at 6:46 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>> A quick FYI: The decision to build 2.7.8 (instead of 3.x) on Android
>> was made after reading this article:
>> https://wiki.python.org/moin/Python2orPython3
>
> What in that document convinced you to port to Python 2 instead of
> Python 3?

Among other things, the points immediately following this part:

However, there are some key issues that may require you to use Python
2 rather than Python 3.

 That page is intended for people deciding which version of
> the language to use for their work. I would think as someone trying to
> port to a new platform your criteria would be different. Also, note
> that the above page was last updated in April 2014. Look at the info
> for that page:
>
>     https://wiki.python.org/moin/Python2orPython3?action=info
>
> and you'll see there was probably nothing of substance added to that
> page in over a year.
>
Something to keep in mind is that i'm using an Android tablet for
building and troubleshooting errors...which means webpages have
different layouts.  The change date for the wiki page I referenced is
waaaay down at the bottom, in extra small, faint type.

I wasn't aware the info was for those using Python.

> Right at the very top of that page, I read:
>
>     Short version: Python 2.x is legacy, Python 3.x is the present and
> future of the language
>
> I would think that would be enough to convince you to use Python 3.x
> as your starting point.

It was, initially, until I read (most of) the rest of the article

I say this as someone who, in his day-to-day
> work uses Python 2.7 exclusively, doesn't expect his employer to ever
> convert to Python 3, and has never done much more with Python 3 than
> build it. With all that, if I was going to attempt a port of Python to
> a new platform, I would still choose to port Python 3.
>
> Skip

Noted. Will proceed with the 3.x releass.

From v+python at g.nevcal.com  Tue Jan 27 21:45:28 2015
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Tue, 27 Jan 2015 12:45:28 -0800
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5PraCiz1hCsH5xBXApC-PAtXRjdwBP9fR8fDOts9=c9Cvjg@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
 <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
 <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>
 <CAHu5PraCiz1hCsH5xBXApC-PAtXRjdwBP9fR8fDOts9=c9Cvjg@mail.gmail.com>
Message-ID: <54C7F8E8.8080708@g.nevcal.com>

On 1/27/2015 8:04 AM, Cyd Haselton wrote:
> Noted. Will proceed with the 3.x releass.
I had been excited you were working on Android Python until I realized 
you were working on 2.x. I started with Python 3, and have only dabbled 
in 2.x for a couple projects that had unported dependency needs. One of 
them, reportlab, was ported in the last year, and the other is my web 
server CGI ports which is still constrained by a couple libraries, and 
although I think they may have recently been ported, I haven't had time 
to upgrade it. Probably will by April, when I have to ditch my Google 
OpenID login system, because they have abandoned it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150127/2c4ef5db/attachment.html>

From arigo at tunes.org  Tue Jan 27 23:12:09 2015
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 27 Jan 2015 23:12:09 +0100
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
Message-ID: <CAMSv6X253ps=khg_4wKyQEvMmzSTjatqwfHn0SBR9hJhqQRmzA@mail.gmail.com>

Hi all,

On 24 January 2015 at 11:50, Maciej Fijalkowski <fijall at gmail.com> wrote:
> I would like to point out that we implemented rhettingers idea in PyPy
> that makes all the dicts ordered by default and we don't have any
> adverse performance effects (in fact, there is quite significant
> memory saving coming from it). The measurments on CPython could be
> different, but in principle OrderedDict can be implemented as
> efficiently as normal dict.

I would like to add that http://bugs.python.org/issue16991 is the same
as today's dicts with an additional doubly-linked list for the order.
I'm unsure why you would do that after the 2012 thread started by
Raymond Hettinger, but anyway, don't conclude from only this that in
the CPython case ordered dictionaries would be slower and bigger.  My
guess is that, with a simple port of what is now in PyPy, they would
not be (but of course no-one can be sure at this point).  Let's say,
if you could imagine that CPython's dictionaries, tomorrow, are always
magically fully ordered, then would it still be a bad idea?

If such a discussion would resurface (soon or "one day"), and if other
related issues are resolved (like what to do in Jython and
IronPython), and if the conclusion would tentatively turn out
positive... then, provided there would at that point still be no
"Raymond-style" implementation of dicts, I would volunteer to port
PyPy's one to CPython[1].  As you may have guessed I don't consider
this particularly likely to occur, but it is a standing offer
nevertheless :-)


A bient?t,

Armin.


[1]  Someone could also do such a port for the goal of getting an
alternate `odictobject.c`.  He would be welcome to #pypy to get some
help from the PyPy guys, including me --- but my offer above doesn't
apply in this case.  I want to remove a thorn in the foot of
python-dev discussing about the language; I'm not really interested in
contributing to the `collections.OrderedDict` type.

From chaselton at gmail.com  Wed Jan 28 00:28:19 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Tue, 27 Jan 2015 17:28:19 -0600
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <54C7F8E8.8080708@g.nevcal.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
 <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
 <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>
 <CAHu5PraCiz1hCsH5xBXApC-PAtXRjdwBP9fR8fDOts9=c9Cvjg@mail.gmail.com>
 <54C7F8E8.8080708@g.nevcal.com>
Message-ID: <CAHu5Prbh3cc8XsEjWfHS9BOmaJ8kPw89Zx5dTzMzm10P39-hew@mail.gmail.com>

On Tue, Jan 27, 2015 at 2:45 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
> On 1/27/2015 8:04 AM, Cyd Haselton wrote:
>
> Noted. Will proceed with the 3.x releass.
>
> I had been excited you were working on Android Python until I realized you
> were working on 2.x. I started with Python 3, and have only dabbled in 2.x
> for a couple projects that had unported dependency needs. One of them,
> reportlab, was ported in the last year, and the other is my web server CGI
> ports which is still constrained by a couple libraries, and although I think
> they may have recently been ported, I haven't had time to upgrade it.
> Probably will by April, when I have to ditch my Google OpenID login system,
> because they have abandoned it.
>
Well, you can start being cautiously excited again now that I'm
working on the 3.4.2 release.

I use cautiously for several reasons...chief among them being the
severely limited Android libc.

From greg at krypto.org  Wed Jan 28 08:14:12 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Wed, 28 Jan 2015 07:14:12 +0000
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
 <CAMSv6X253ps=khg_4wKyQEvMmzSTjatqwfHn0SBR9hJhqQRmzA@mail.gmail.com>
Message-ID: <CAGE7PNLNN49QEx30PMDRxy2Ge+U+j563gos9TJww=-cWuTio0Q@mail.gmail.com>

On Tue Jan 27 2015 at 2:13:08 PM Armin Rigo <arigo at tunes.org> wrote:

> Hi all,
>
> On 24 January 2015 at 11:50, Maciej Fijalkowski <fijall at gmail.com> wrote:
> > I would like to point out that we implemented rhettingers idea in PyPy
> > that makes all the dicts ordered by default and we don't have any
> > adverse performance effects (in fact, there is quite significant
> > memory saving coming from it). The measurments on CPython could be
> > different, but in principle OrderedDict can be implemented as
> > efficiently as normal dict.
>
> I would like to add that http://bugs.python.org/issue16991 is the same
> as today's dicts with an additional doubly-linked list for the order.
> I'm unsure why you would do that after the 2012 thread started by
> Raymond Hettinger, but anyway, don't conclude from only this that in
> the CPython case ordered dictionaries would be slower and bigger.  My
> guess is that, with a simple port of what is now in PyPy, they would
> not be (but of course no-one can be sure at this point).  Let's say,
> if you could imagine that CPython's dictionaries, tomorrow, are always
> magically fully ordered, then would it still be a bad idea?
>

It is a potentially bad idea if order is the default behavior of iteration,
items(), keys() and values(). Ideally order should only be exposed when
explicitly asked for to help prevent bugs and mitigate potential
information leaks.

But I'm not sure how big of a deal this actually is. The insertion order
nicely doesn't give away anything related to the hash seed used for hash
randomization which is a nice bonus over today's implementation (and 2.7 &
3.3's very poor hash randomization implementation).

Experience cleaning up our huge code base at work to turn on hash
randomization by default a couple years ago has shown that people depend on
iteration order in code often without intending to. This often leads to
latent bugs. Keep iteration order unstable by default and you prevent
people from doing that. Make people request an ordered or stable iteration
when their code explicitly needs it.

If such a discussion would resurface (soon or "one day"), and if other
> related issues are resolved (like what to do in Jython and
> IronPython), and if the conclusion would tentatively turn out
> positive... then, provided there would at that point still be no
> "Raymond-style" implementation of dicts, I would volunteer to port
> PyPy's one to CPython[1].  As you may have guessed I don't consider
> this particularly likely to occur, but it is a standing offer
> nevertheless :-)
>

CPython should benefit from it regardless for the memory savings alone.

-gps


>
> A bient?t,
>
> Armin.
>
>
> [1]  Someone could also do such a port for the goal of getting an
> alternate `odictobject.c`.  He would be welcome to #pypy to get some
> help from the PyPy guys, including me --- but my offer above doesn't
> apply in this case.  I want to remove a thorn in the foot of
> python-dev discussing about the language; I'm not really interested in
> contributing to the `collections.OrderedDict` type.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/499e3dd1/attachment-0001.html>

From chris at simplistix.co.uk  Wed Jan 28 10:14:21 2015
From: chris at simplistix.co.uk (Chris Withers)
Date: Wed, 28 Jan 2015 09:14:21 +0000
Subject: [Python-Dev] PEP 468 (Ordered kwargs)
In-Reply-To: <CAGE7PNLNN49QEx30PMDRxy2Ge+U+j563gos9TJww=-cWuTio0Q@mail.gmail.com>
References: <CAK5idxRh-OAPCbrJ65p_PrzifRPHvXWnRX2AY6exq+YtU7R4DQ@mail.gmail.com>
 <CAMSv6X253ps=khg_4wKyQEvMmzSTjatqwfHn0SBR9hJhqQRmzA@mail.gmail.com>
 <CAGE7PNLNN49QEx30PMDRxy2Ge+U+j563gos9TJww=-cWuTio0Q@mail.gmail.com>
Message-ID: <54C8A86D.1060401@simplistix.co.uk>

On 28/01/2015 07:14, Gregory P. Smith wrote:
>
> It is a potentially bad idea if order is the default behavior of
> iteration, items(), keys() and values(). Ideally order should only be
> exposed when explicitly asked for to help prevent bugs and mitigate
> potential information leaks.

I have to be honest, I think that's the opposite of most new users 
assumption...

> Experience cleaning up our huge code base at work to turn on hash
> randomization by default a couple years ago has shown that people depend
> on iteration order in code often without intending to. This often leads
> to latent bugs. Keep iteration order unstable by default and you prevent
> people from doing that.

Hmm, well, or you could say that always having ordering would mean the 
behaviour would match new users experimental understanding and so 
eliminate all bugs that occur when people accidentally rely on ordering.

Personally, I'd prefer to see us be explicit about data structures used 
when "security matters", an explicit RandomOrderedDict would make that 
clear.

cheers,

Chris

From agriff at tin.it  Wed Jan 28 10:22:04 2015
From: agriff at tin.it (Andrea Griffini)
Date: Wed, 28 Jan 2015 10:22:04 +0100
Subject: [Python-Dev] Why co_names? Wouldn't be simpler to just use
	co_consts?
Message-ID: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>

Sorry if the question is naive, but why is co_names needed? Wouldn't be
simpler to just use co_consts?

Andrea
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/7d0eee0c/attachment.html>

From greg.ewing at canterbury.ac.nz  Wed Jan 28 12:21:08 2015
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Thu, 29 Jan 2015 00:21:08 +1300
Subject: [Python-Dev] Why co_names? Wouldn't be simpler to just
	use	co_consts?
In-Reply-To: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>
References: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>
Message-ID: <54C8C624.8050802@canterbury.ac.nz>

Andrea Griffini wrote:
> Sorry if the question is naive, but why is co_names needed? Wouldn't be 
> simpler to just use co_consts?

One reason might be that keeping them separate means
you can have up to 256 names and 256 consts using
1-byte opcode arguments. Otherwise, you'd be limited
to a total of 256 of both.

-- 
Greg

From ncoghlan at gmail.com  Wed Jan 28 13:40:13 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 28 Jan 2015 22:40:13 +1000
Subject: [Python-Dev] Why co_names? Wouldn't be simpler to just use
	co_consts?
In-Reply-To: <54C8C624.8050802@canterbury.ac.nz>
References: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>
 <54C8C624.8050802@canterbury.ac.nz>
Message-ID: <CADiSq7c5Yh7npVPL-8fhpK3VQ_GiEy7Kk7oh7mFdt_maGxV0qg@mail.gmail.com>

On 28 January 2015 at 21:21, Greg Ewing <greg.ewing at canterbury.ac.nz> wrote:
> Andrea Griffini wrote:
>>
>> Sorry if the question is naive, but why is co_names needed? Wouldn't be
>> simpler to just use co_consts?
>
> One reason might be that keeping them separate means
> you can have up to 256 names and 256 consts using
> 1-byte opcode arguments. Otherwise, you'd be limited
> to a total of 256 of both.

They're logically distinct things accessed by different opcodes for
very different purposes. While you theoretically *could* use one array
to hold both, it would make the eval code harder to read, and various
introspection tasks (like "tell me all the names referenced from this
code object") significantly more difficult.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From aarmour at cipmail.org  Wed Jan 28 15:23:37 2015
From: aarmour at cipmail.org (Alan Armour)
Date: Wed, 28 Jan 2015 09:23:37 -0500
Subject: [Python-Dev] development
Message-ID: <CAC=V4WebbeYHmqCbryGTN3RpSXUyKG42SstzKrJcC3CG9S6s0Q@mail.gmail.com>

can you guys develop an audio kit that works around jackd or on windows
directx? and tutorials to write synthesizers.  and drum machines like a
tr-606 with triggers ( I want to trigger a drum synth like the March
UDS(Soviet).... Coolest drum synth EVER.


Also, I think you should have a way to write assembler functions to really
optimize speed and have a table and stuff for assembler learning for all
cpus and stuff. even asm graphics and audio would be super useful in some
instances.



Thanks for your time
-Ty
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/d98defec/attachment.html>

From chaselton at gmail.com  Wed Jan 28 15:43:27 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Wed, 28 Jan 2015 08:43:27 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
Message-ID: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>

All,
I recently ditched my attempts to port Python 2.7.8 to Android in
favor of Python 3.4.2.  Unfortunately, after using the same configure
options in the same environment, and modifying the setup.py as needed,
the newly built binary throws a segfault when the generate-posix-vars
portion of the build is reached...and when it is run as well (i.e.
./python --help, ./python -E -S -m sysconfig, or similar)

I took a strace of ./python, however I'm a bit lost when reviewing it.
Any ideas as to what may be going on...i.e. why Python 2.7 works but
3.x throws a segfault?

Thanks in advance,
Cyd

From brian at python.org  Wed Jan 28 15:52:09 2015
From: brian at python.org (Brian Curtin)
Date: Wed, 28 Jan 2015 08:52:09 -0600
Subject: [Python-Dev] development
In-Reply-To: <CAC=V4WebbeYHmqCbryGTN3RpSXUyKG42SstzKrJcC3CG9S6s0Q@mail.gmail.com>
References: <CAC=V4WebbeYHmqCbryGTN3RpSXUyKG42SstzKrJcC3CG9S6s0Q@mail.gmail.com>
Message-ID: <CAD+XWwp4E0RErTcy7HjKUhZ6jcdHLJkgi9AM2mzHpK-zwonC+A@mail.gmail.com>

On Wednesday, January 28, 2015, Alan Armour <aarmour at cipmail.org> wrote:

> can you guys develop an audio kit that works around jackd or on windows
> directx? and tutorials to write synthesizers.  and drum machines like a
> tr-606 with triggers ( I want to trigger a drum synth like the March
> UDS(Soviet).... Coolest drum synth EVER.
>
>
> Also, I think you should have a way to write assembler functions to really
> optimize speed and have a table and stuff for assembler learning for all
> cpus and stuff. even asm graphics and audio would be super useful in some
> instances.
>

That's not how this works.

If you would like to write all of that code and allow it to mature in the
wild while building a following around it and ensuring it is the best of
its kind and a general enough solution to be included in the Python
standard library, inclusion of that could be discussed in the future.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/0a915ee9/attachment.html>

From aarmour at cipmail.org  Wed Jan 28 15:39:25 2015
From: aarmour at cipmail.org (Alan Armour)
Date: Wed, 28 Jan 2015 09:39:25 -0500
Subject: [Python-Dev] also
Message-ID: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>

if you can do this....

a chemical physics and element physics like everything from melting points
to how much heat you need to add two chemicals together

and physics like aerodynamics, space dynamics, and hydrodynamics etcetera
for propellers and motors and stuff.

just having this in a main language seems to make a shit ton of sense.


Just like all the physics you can think of from electrical equipment to
building microchips to oscillators and resistors and stuff like that.

thanks
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/0af27f43/attachment.html>

From brett at python.org  Wed Jan 28 16:29:12 2015
From: brett at python.org (Brett Cannon)
Date: Wed, 28 Jan 2015 15:29:12 +0000
Subject: [Python-Dev] also
References: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>
Message-ID: <CAP1=2W4PjWWuTRL9Kbv2eD1k21BtekBHLTQ5HM8Chf8BO+E9Tw@mail.gmail.com>

You have the wrong mailing list for this sort of request. This is list is
about the development *of* Python, not *with* it. And since Python the
language is not in the business of providing libraries for such specific
needs this kind of request isn't appropriate here. You can try asking
somewhere like python-list to see if a library already exists for your
needs, though.

On Wed Jan 28 2015 at 10:13:03 AM Alan Armour <aarmour at cipmail.org> wrote:

> if you can do this....
>
> a chemical physics and element physics like everything from melting points
> to how much heat you need to add two chemicals together
>
> and physics like aerodynamics, space dynamics, and hydrodynamics etcetera
> for propellers and motors and stuff.
>
> just having this in a main language seems to make a shit ton of sense.
>
>
> Just like all the physics you can think of from electrical equipment to
> building microchips to oscillators and resistors and stuff like that.
>
> thanks
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/f112aad6/attachment.html>

From guido at python.org  Wed Jan 28 16:43:42 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 28 Jan 2015 07:43:42 -0800
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
Message-ID: <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>

There could be a million differences relevant (unicode, ints, ...). Perhaps
the importlib bootstrap is failing. Perhaps the dynamic loading code
changed. Did you get a stack track? (IIRC strace shows a syscall trace --
also useful, but doesn't tell you precisely how it segfaulted.)

On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com> wrote:

> All,
> I recently ditched my attempts to port Python 2.7.8 to Android in
> favor of Python 3.4.2.  Unfortunately, after using the same configure
> options in the same environment, and modifying the setup.py as needed,
> the newly built binary throws a segfault when the generate-posix-vars
> portion of the build is reached...and when it is run as well (i.e.
> ./python --help, ./python -E -S -m sysconfig, or similar)
>
> I took a strace of ./python, however I'm a bit lost when reviewing it.
> Any ideas as to what may be going on...i.e. why Python 2.7 works but
> 3.x throws a segfault?
>
> Thanks in advance,
> Cyd
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/a447ae5e/attachment.html>

From python at mrabarnett.plus.com  Wed Jan 28 16:52:27 2015
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 28 Jan 2015 15:52:27 +0000
Subject: [Python-Dev] also
In-Reply-To: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>
References: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>
Message-ID: <54C905BB.3050605@mrabarnett.plus.com>

On 2015-01-28 14:39, Alan Armour wrote:
> if you can do this....
>
> a chemical physics and element physics like everything from melting
> points to how much heat you need to add two chemicals together
>
> and physics like aerodynamics, space dynamics, and hydrodynamics
> etcetera for propellers and motors and stuff.
>
> just having this in a main language seems to make a shit ton of sense.
>
>
> Just like all the physics you can think of from electrical equipment to
> building microchips to oscillators and resistors and stuff like that.
>
> thanks
>
You should be looking at Wolfram Alpha instead...

From guido at python.org  Wed Jan 28 17:43:00 2015
From: guido at python.org (Guido van Rossum)
Date: Wed, 28 Jan 2015 08:43:00 -0800
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
Message-ID: <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>

What I see in the strace:

... load libpython3.4m.so.1.0
... load libm
... open /dev/__properties__ and do something to it (what?)
... get current time
... allocate memory
... getuid
... segfault

That's not a lot to go on, but it doesn't look as if it has started to load
modules yet.

Does /dev/__properties__ ring a bell? Not to me.

That stack trace would be really helpful.

On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com> wrote:

> Apologies...I'm not sure what a stack track is, but I do have the
> strace.  Nearest I can tell, it happens due to an open call, though I
> am probably wrong.
> Attaching the strace output to this email.  I'm going to head back to
> the documentation and to back out of some Android-related changes in
> _localemodule.c
>
> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
> wrote:
> > There could be a million differences relevant (unicode, ints, ...).
> Perhaps
> > the importlib bootstrap is failing. Perhaps the dynamic loading code
> > changed. Did you get a stack track? (IIRC strace shows a syscall trace --
> > also useful, but doesn't tell you precisely how it segfaulted.)
> >
> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >> All,
> >> I recently ditched my attempts to port Python 2.7.8 to Android in
> >> favor of Python 3.4.2.  Unfortunately, after using the same configure
> >> options in the same environment, and modifying the setup.py as needed,
> >> the newly built binary throws a segfault when the generate-posix-vars
> >> portion of the build is reached...and when it is run as well (i.e.
> >> ./python --help, ./python -E -S -m sysconfig, or similar)
> >>
> >> I took a strace of ./python, however I'm a bit lost when reviewing it.
> >> Any ideas as to what may be going on...i.e. why Python 2.7 works but
> >> 3.x throws a segfault?
> >>
> >> Thanks in advance,
> >> Cyd
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe:
> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
> >
> >
> >
> > --
> > --Guido van Rossum (python.org/~guido)
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/81c7849f/attachment.html>

From chaselton at gmail.com  Wed Jan 28 17:34:04 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Wed, 28 Jan 2015 10:34:04 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
Message-ID: <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>

Apologies...I'm not sure what a stack track is, but I do have the
strace.  Nearest I can tell, it happens due to an open call, though I
am probably wrong.
Attaching the strace output to this email.  I'm going to head back to
the documentation and to back out of some Android-related changes in
_localemodule.c

On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org> wrote:
> There could be a million differences relevant (unicode, ints, ...). Perhaps
> the importlib bootstrap is failing. Perhaps the dynamic loading code
> changed. Did you get a stack track? (IIRC strace shows a syscall trace --
> also useful, but doesn't tell you precisely how it segfaulted.)
>
> On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> All,
>> I recently ditched my attempts to port Python 2.7.8 to Android in
>> favor of Python 3.4.2.  Unfortunately, after using the same configure
>> options in the same environment, and modifying the setup.py as needed,
>> the newly built binary throws a segfault when the generate-posix-vars
>> portion of the build is reached...and when it is run as well (i.e.
>> ./python --help, ./python -E -S -m sysconfig, or similar)
>>
>> I took a strace of ./python, however I'm a bit lost when reviewing it.
>> Any ideas as to what may be going on...i.e. why Python 2.7 works but
>> 3.x throws a segfault?
>>
>> Thanks in advance,
>> Cyd
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sout
Type: application/octet-stream
Size: 9149 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/afe9400d/attachment-0001.obj>

From rymg19 at gmail.com  Wed Jan 28 18:23:53 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Wed, 28 Jan 2015 11:23:53 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
Message-ID: <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>

On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org> wrote:

> What I see in the strace:
>
> ... load libpython3.4m.so.1.0
> ... load libm
> ... open /dev/__properties__ and do something to it (what?)
> ... get current time
> ... allocate memory
> ... getuid
> ... segfault
>
> That's not a lot to go on, but it doesn't look as if it has started to
> load modules yet.
>
> Does /dev/__properties__ ring a bell? Not to me.
>
>
https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
is the code that works with that file.

This
<http://sssslide.com/www.slideshare.net/tetsu.koba/interprocess-communication-of-android>
explains
it a bit (slides 24-29). Looks like something to do with interprocess
communication. Likely has nothing to do with Python itself.

Maybe this
<http://www.andrew-kirkpatrick.com/2013/01/get-android-stack-trace-from-device-using-debug-bridge/>
would
be useful?


> That stack trace would be really helpful.
>
> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>
>> Apologies...I'm not sure what a stack track is, but I do have the
>> strace.  Nearest I can tell, it happens due to an open call, though I
>> am probably wrong.
>> Attaching the strace output to this email.  I'm going to head back to
>> the documentation and to back out of some Android-related changes in
>> _localemodule.c
>>
>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>> wrote:
>> > There could be a million differences relevant (unicode, ints, ...).
>> Perhaps
>> > the importlib bootstrap is failing. Perhaps the dynamic loading code
>> > changed. Did you get a stack track? (IIRC strace shows a syscall trace
>> --
>> > also useful, but doesn't tell you precisely how it segfaulted.)
>> >
>> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>> wrote:
>> >>
>> >> All,
>> >> I recently ditched my attempts to port Python 2.7.8 to Android in
>> >> favor of Python 3.4.2.  Unfortunately, after using the same configure
>> >> options in the same environment, and modifying the setup.py as needed,
>> >> the newly built binary throws a segfault when the generate-posix-vars
>> >> portion of the build is reached...and when it is run as well (i.e.
>> >> ./python --help, ./python -E -S -m sysconfig, or similar)
>> >>
>> >> I took a strace of ./python, however I'm a bit lost when reviewing it.
>> >> Any ideas as to what may be going on...i.e. why Python 2.7 works but
>> >> 3.x throws a segfault?
>> >>
>> >> Thanks in advance,
>> >> Cyd
>> >> _______________________________________________
>> >> Python-Dev mailing list
>> >> Python-Dev at python.org
>> >> https://mail.python.org/mailman/listinfo/python-dev
>> >> Unsubscribe:
>> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >
>> >
>> >
>> >
>> > --
>> > --Guido van Rossum (python.org/~guido)
>>
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>
>


-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/4ce66698/attachment.html>

From agriff at tin.it  Wed Jan 28 19:53:20 2015
From: agriff at tin.it (Andrea Griffini)
Date: Wed, 28 Jan 2015 19:53:20 +0100
Subject: [Python-Dev] Why co_names? Wouldn't be simpler to just use
	co_consts?
In-Reply-To: <CADiSq7c5Yh7npVPL-8fhpK3VQ_GiEy7Kk7oh7mFdt_maGxV0qg@mail.gmail.com>
References: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>
 <54C8C624.8050802@canterbury.ac.nz>
 <CADiSq7c5Yh7npVPL-8fhpK3VQ_GiEy7Kk7oh7mFdt_maGxV0qg@mail.gmail.com>
Message-ID: <CALy7ODs_Ejm6xoC2ZNGN6jUtjQC0A==-RLAMw9+KrencmAeOng@mail.gmail.com>

The names stored in op_names are totally unrelated as they can be attribute
names, module names, global names; you basically don't know much about them
unless you also inspect the actual bytecode using them (and the same name
can be used in completely different ways in different parts of the same
code object). In my opinion introspection code telling me that the name
`foo` is used but not knowing if it's about a global, a module name or an
attribute name is not going to be that useful, on the other hand if you do
inspect the bytecode then using co_consts doesn't make things more
complicate.

Anyway I was just curious to know if there was any technical reason (that I
couldn't see) or if it was more a style/historic reason.

Thank you for the clarification

On Wed, Jan 28, 2015 at 1:40 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On 28 January 2015 at 21:21, Greg Ewing <greg.ewing at canterbury.ac.nz>
> wrote:
> > Andrea Griffini wrote:
> >>
> >> Sorry if the question is naive, but why is co_names needed? Wouldn't be
> >> simpler to just use co_consts?
> >
> > One reason might be that keeping them separate means
> > you can have up to 256 names and 256 consts using
> > 1-byte opcode arguments. Otherwise, you'd be limited
> > to a total of 256 of both.
>
> They're logically distinct things accessed by different opcodes for
> very different purposes. While you theoretically *could* use one array
> to hold both, it would make the eval code harder to read, and various
> introspection tasks (like "tell me all the names referenced from this
> code object") significantly more difficult.
>
> Cheers,
> Nick.
>
> --
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/agriff%40tin.it
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150128/9f8f2294/attachment.html>

From chaselton at gmail.com  Wed Jan 28 23:42:33 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Wed, 28 Jan 2015 16:42:33 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
Message-ID: <CAHu5PraetgQHyDD1OmAm-rivbtegw7Q=wjMkAmhvC9=+ZhVUaQ@mail.gmail.com>

That is interesting.../dev/__properties__ is in memory...not the
filesystem, apparently proccesses read global properties from it. It's
read-only...not sure why the build or the python binary would access
it...or if that's the cause of the segfault.

I have root access on the tablet so I was able to check for the
traces.txt file. There are a number of them, but none contain
information about the segfault.

On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org> wrote:
>>
>> What I see in the strace:
>>
>> ... load libpython3.4m.so.1.0
>> ... load libm
>> ... open /dev/__properties__ and do something to it (what?)
>> ... get current time
>> ... allocate memory
>> ... getuid
>> ... segfault
>>
>> That's not a lot to go on, but it doesn't look as if it has started to
>> load modules yet.
>>
>> Does /dev/__properties__ ring a bell? Not to me.
>>
>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> is the code that works with that file.
>
> This explains it a bit (slides 24-29). Looks like something to do with
> interprocess communication. Likely has nothing to do with Python itself.
>
> Maybe this would be useful?
>
>>
>> That stack trace would be really helpful.
>>
>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> Apologies...I'm not sure what a stack track is, but I do have the
>>> strace.  Nearest I can tell, it happens due to an open call, though I
>>> am probably wrong.
>>> Attaching the strace output to this email.  I'm going to head back to
>>> the documentation and to back out of some Android-related changes in
>>> _localemodule.c
>>>
>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>> wrote:
>>> > There could be a million differences relevant (unicode, ints, ...).
>>> > Perhaps
>>> > the importlib bootstrap is failing. Perhaps the dynamic loading code
>>> > changed. Did you get a stack track? (IIRC strace shows a syscall trace
>>> > --
>>> > also useful, but doesn't tell you precisely how it segfaulted.)
>>> >
>>> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>> > wrote:
>>> >>
>>> >> All,
>>> >> I recently ditched my attempts to port Python 2.7.8 to Android in
>>> >> favor of Python 3.4.2.  Unfortunately, after using the same configure
>>> >> options in the same environment, and modifying the setup.py as needed,
>>> >> the newly built binary throws a segfault when the generate-posix-vars
>>> >> portion of the build is reached...and when it is run as well (i.e.
>>> >> ./python --help, ./python -E -S -m sysconfig, or similar)
>>> >>
>>> >> I took a strace of ./python, however I'm a bit lost when reviewing it.
>>> >> Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>> >> 3.x throws a segfault?
>>> >>
>>> >> Thanks in advance,
>>> >> Cyd
>>> >> _______________________________________________
>>> >> Python-Dev mailing list
>>> >> Python-Dev at python.org
>>> >> https://mail.python.org/mailman/listinfo/python-dev
>>> >> Unsubscribe:
>>> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > --Guido van Rossum (python.org/~guido)
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From steve at pearwood.info  Thu Jan 29 00:38:51 2015
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 29 Jan 2015 10:38:51 +1100
Subject: [Python-Dev] also
In-Reply-To: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>
References: <CAC=V4WeOS5O0p8AwpOw66HyU3Kr02Xt00_HWMO7J5CyVPZ1z4g@mail.gmail.com>
Message-ID: <20150128233851.GA2498@ando.pearwood.info>

On Wed, Jan 28, 2015 at 09:39:25AM -0500, Alan Armour wrote:
> if you can do this....
> 
> a chemical physics and element physics like everything from melting points
> to how much heat you need to add two chemicals together
> 
> and physics like aerodynamics, space dynamics, and hydrodynamics etcetera
> for propellers and motors and stuff.
> 
> just having this in a main language seems to make a shit ton of sense.

You should check out Frink:

http://futureboy.us/frinkdocs/


-- 
Steve

From chris.barker at noaa.gov  Thu Jan 29 06:08:09 2015
From: chris.barker at noaa.gov (Chris Barker - NOAA Federal)
Date: Wed, 28 Jan 2015 21:08:09 -0800
Subject: [Python-Dev] Pydoc Replacement for Python's help()?
In-Reply-To: <CAHu5Prbh3cc8XsEjWfHS9BOmaJ8kPw89Zx5dTzMzm10P39-hew@mail.gmail.com>
References: <CAHu5PrbHx9n1AvCdv4FRfqCXO4AJ4Tu8538GYurETLyAio=ZPQ@mail.gmail.com>
 <CANc-5UzeDHiiOLA4p9h4Dgj7DOy04J1-1_qeQj=+zdS9YPEA9g@mail.gmail.com>
 <CAHu5PrZdAYCakZSh1juQWpC12LmgOCwGgp_1X1q-AvoQoyQGww@mail.gmail.com>
 <CAHu5PrbLZjzq=utm84Wg4Ayw4awb5ixLdMqvm_8=EgrjxPY3SQ@mail.gmail.com>
 <CANc-5Uzj=CT0NOoswtUdYtCZ2RwMDVyznB3kAwTrgg6k9Q5B2w@mail.gmail.com>
 <CAHu5PraCiz1hCsH5xBXApC-PAtXRjdwBP9fR8fDOts9=c9Cvjg@mail.gmail.com>
 <54C7F8E8.8080708@g.nevcal.com>
 <CAHu5Prbh3cc8XsEjWfHS9BOmaJ8kPw89Zx5dTzMzm10P39-hew@mail.gmail.com>
Message-ID: <-5603876186594649743@unknownmsgid>

Just want to make sure you're aware of the new mobile-sig:

https://mail.python.org/mailman/listinfo/mobile-sig

No need to do this on your own.

-Chris

> On Jan 27, 2015, at 4:21 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>
>> On Tue, Jan 27, 2015 at 2:45 PM, Glenn Linderman <v+python at g.nevcal.com> wrote:
>> On 1/27/2015 8:04 AM, Cyd Haselton wrote:
>>
>> Noted. Will proceed with the 3.x releass.
>>
>> I had been excited you were working on Android Python until I realized you
>> were working on 2.x. I started with Python 3, and have only dabbled in 2.x
>> for a couple projects that had unported dependency needs. One of them,
>> reportlab, was ported in the last year, and the other is my web server CGI
>> ports which is still constrained by a couple libraries, and although I think
>> they may have recently been ported, I haven't had time to upgrade it.
>> Probably will by April, when I have to ditch my Google OpenID login system,
>> because they have abandoned it.
> Well, you can start being cautiously excited again now that I'm
> working on the 3.4.2 release.
>
> I use cautiously for several reasons...chief among them being the
> severely limited Android libc.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/chris.barker%40noaa.gov

From ncoghlan at gmail.com  Thu Jan 29 15:08:16 2015
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 30 Jan 2015 00:08:16 +1000
Subject: [Python-Dev] Why co_names? Wouldn't be simpler to just use
	co_consts?
In-Reply-To: <CALy7ODs_Ejm6xoC2ZNGN6jUtjQC0A==-RLAMw9+KrencmAeOng@mail.gmail.com>
References: <CALy7ODsDuaWji+h0KuTczEJqQGV6YbHQxd=ewcGwqZmus0KT-A@mail.gmail.com>
 <54C8C624.8050802@canterbury.ac.nz>
 <CADiSq7c5Yh7npVPL-8fhpK3VQ_GiEy7Kk7oh7mFdt_maGxV0qg@mail.gmail.com>
 <CALy7ODs_Ejm6xoC2ZNGN6jUtjQC0A==-RLAMw9+KrencmAeOng@mail.gmail.com>
Message-ID: <CADiSq7eTYzSXyffM7Sm7VeH06XabYEPsqHOLMKokSLnnCDDdww@mail.gmail.com>

On 29 January 2015 at 04:53, Andrea Griffini <agriff at tin.it> wrote:
> The names stored in op_names are totally unrelated as they can be attribute
> names, module names, global names; you basically don't know much about them
> unless you also inspect the actual bytecode using them (and the same name
> can be used in completely different ways in different parts of the same code
> object). In my opinion introspection code telling me that the name `foo` is
> used but not knowing if it's about a global, a module name or an attribute
> name is not going to be that useful, on the other hand if you do inspect the
> bytecode then using co_consts doesn't make things more complicate.

It makes more sense once you know there are a couple of related passes
over the AST in the compiler, one to build the symbol table (i.e. just
looking at what names exist, where they're defined, and what
references them), and a second pass to do the actual code generation
(already armed with the symbol details from the symbol table pass).

So co_names relates to things that come up in the symbol table pass,
while co_consts is only relevant to actual code generation and
execution. co_names also doesn't get used in isolation - it's combined
with co_varnames and various other attributes in a way that's actually
structured more for the convenience of the interpreter eval loop than
it is for human readers.

As a general rule, by the time we get to the code object level, we're
well down into the weeds of needing to think about how the underlying
Python interpreter *works*, rather than just what it allows a Python
programmer to do.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brunocauet at gmail.com  Thu Jan 29 18:49:16 2015
From: brunocauet at gmail.com (Bruno Cauet)
Date: Thu, 29 Jan 2015 18:49:16 +0100
Subject: [Python-Dev] Results of the python 2.x and 3.x use survey,
	2014 edition
Message-ID: <CAMA3c49yP8C4MS50qsH7P3waKO49RPNtFXgd5WgYJQHk0VSbJw@mail.gmail.com>

Hi!
Finally, here are the results:
http://blog.frite-camembert.net/python-survey-2014.html
Here is the auto-generated Google Forms recap:
https://docs.google.com/forms/d/1DqxkNi4GvyTCu54usSdE1DjW29zw1tc52iMeH3z4heg/viewanalytics
(more elegant than my matplotlib graphs - I'd have no future as a
designer).

Overall people started writing more python 3: +15 points in "I ever
wrote python 3 code", +10 points in "I write more python 3 than 2".
Transition is still ongoing and I hope a tipping point will be soon be
attained.
Users definitely seem to want to switch to python 3, but dependencies
keep them with 2.7 (I weep for the few ones still on 2.5).

I also posted the results on HN
(https://news.ycombinator.com/item?id=8967645) and reddit
(http://www.reddit.com/r/Python/comments/2u3oh0/results_of_the_python_2x_and_3x_use_survey_2014/).
Have a nice day, and a year full of python 3!

Bruno Cauet

From chaselton at gmail.com  Fri Jan 30 00:32:45 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Thu, 29 Jan 2015 17:32:45 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
Message-ID: <CAHu5Pra5ySr0Rxboh3sVio5Dy5JjHK1LPZgatpmVtA+30UYWvw@mail.gmail.com>

Found a patch for at bugs.python.org addressing a segfault issue for
android but even afternapplying it i'm still getting a segfault.

I ran a strace with the verbose option and am attaching it to this
update.  If not helpful, I'll see if I can hook up the debugging
bridge to the tablet but as mentioned earlier there was no helpful
info in the anr/traces files

On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org> wrote:
>>
>> What I see in the strace:
>>
>> ... load libpython3.4m.so.1.0
>> ... load libm
>> ... open /dev/__properties__ and do something to it (what?)
>> ... get current time
>> ... allocate memory
>> ... getuid
>> ... segfault
>>
>> That's not a lot to go on, but it doesn't look as if it has started to
>> load modules yet.
>>
>> Does /dev/__properties__ ring a bell? Not to me.
>>
>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> is the code that works with that file.
>
> This explains it a bit (slides 24-29). Looks like something to do with
> interprocess communication. Likely has nothing to do with Python itself.
>
> Maybe this would be useful?
>
>>
>> That stack trace would be really helpful.
>>
>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> Apologies...I'm not sure what a stack track is, but I do have the
>>> strace.  Nearest I can tell, it happens due to an open call, though I
>>> am probably wrong.
>>> Attaching the strace output to this email.  I'm going to head back to
>>> the documentation and to back out of some Android-related changes in
>>> _localemodule.c
>>>
>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>> wrote:
>>> > There could be a million differences relevant (unicode, ints, ...).
>>> > Perhaps
>>> > the importlib bootstrap is failing. Perhaps the dynamic loading code
>>> > changed. Did you get a stack track? (IIRC strace shows a syscall trace
>>> > --
>>> > also useful, but doesn't tell you precisely how it segfaulted.)
>>> >
>>> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>> > wrote:
>>> >>
>>> >> All,
>>> >> I recently ditched my attempts to port Python 2.7.8 to Android in
>>> >> favor of Python 3.4.2.  Unfortunately, after using the same configure
>>> >> options in the same environment, and modifying the setup.py as needed,
>>> >> the newly built binary throws a segfault when the generate-posix-vars
>>> >> portion of the build is reached...and when it is run as well (i.e.
>>> >> ./python --help, ./python -E -S -m sysconfig, or similar)
>>> >>
>>> >> I took a strace of ./python, however I'm a bit lost when reviewing it.
>>> >> Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>> >> 3.x throws a segfault?
>>> >>
>>> >> Thanks in advance,
>>> >> Cyd
>>> >> _______________________________________________
>>> >> Python-Dev mailing list
>>> >> Python-Dev at python.org
>>> >> https://mail.python.org/mailman/listinfo/python-dev
>>> >> Unsubscribe:
>>> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > --Guido van Rossum (python.org/~guido)
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: sout-1024
Type: application/octet-stream
Size: 11701 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/5b1eb9a0/attachment.obj>

From solipsis at pitrou.net  Fri Jan 30 02:21:38 2015
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 30 Jan 2015 02:21:38 +0100
Subject: [Python-Dev] Pronouncing on PEP 475 - last call
Message-ID: <20150130022138.6f6ba887@fsol>


Hello,

As BDFL-delegate, I'd like to pronounce soon on PEP 475 - "Retry system
calls failing with EINTR".  I've just pushed some small rephrasings,
which should appear soon (at https://www.python.org/dev/peps/pep-0475/).
There is a working implementation by Charles-Fran?ois at
http://bugs.python.org/issue23285.

If you have anything to say, please chime in quick :-)

Regards

Antoine.



From chaselton at gmail.com  Fri Jan 30 02:32:56 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Thu, 29 Jan 2015 19:32:56 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
Message-ID: <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>

Managed to get this out of logcat:
F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
11914 (python)  (libc)

[ 01-29 19:30:55.855 23373:23373 F/libc     ]
Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)

Less detail than strace but it seems to be that python is segfaulting libc...

On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org> wrote:
>>
>> What I see in the strace:
>>
>> ... load libpython3.4m.so.1.0
>> ... load libm
>> ... open /dev/__properties__ and do something to it (what?)
>> ... get current time
>> ... allocate memory
>> ... getuid
>> ... segfault
>>
>> That's not a lot to go on, but it doesn't look as if it has started to
>> load modules yet.
>>
>> Does /dev/__properties__ ring a bell? Not to me.
>>
>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> is the code that works with that file.
>
> This explains it a bit (slides 24-29). Looks like something to do with
> interprocess communication. Likely has nothing to do with Python itself.
>
> Maybe this would be useful?
>
>>
>> That stack trace would be really helpful.
>>
>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> Apologies...I'm not sure what a stack track is, but I do have the
>>> strace.  Nearest I can tell, it happens due to an open call, though I
>>> am probably wrong.
>>> Attaching the strace output to this email.  I'm going to head back to
>>> the documentation and to back out of some Android-related changes in
>>> _localemodule.c
>>>
>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>> wrote:
>>> > There could be a million differences relevant (unicode, ints, ...).
>>> > Perhaps
>>> > the importlib bootstrap is failing. Perhaps the dynamic loading code
>>> > changed. Did you get a stack track? (IIRC strace shows a syscall trace
>>> > --
>>> > also useful, but doesn't tell you precisely how it segfaulted.)
>>> >
>>> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>> > wrote:
>>> >>
>>> >> All,
>>> >> I recently ditched my attempts to port Python 2.7.8 to Android in
>>> >> favor of Python 3.4.2.  Unfortunately, after using the same configure
>>> >> options in the same environment, and modifying the setup.py as needed,
>>> >> the newly built binary throws a segfault when the generate-posix-vars
>>> >> portion of the build is reached...and when it is run as well (i.e.
>>> >> ./python --help, ./python -E -S -m sysconfig, or similar)
>>> >>
>>> >> I took a strace of ./python, however I'm a bit lost when reviewing it.
>>> >> Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>> >> 3.x throws a segfault?
>>> >>
>>> >> Thanks in advance,
>>> >> Cyd
>>> >> _______________________________________________
>>> >> Python-Dev mailing list
>>> >> Python-Dev at python.org
>>> >> https://mail.python.org/mailman/listinfo/python-dev
>>> >> Unsubscribe:
>>> >> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > --Guido van Rossum (python.org/~guido)
>>
>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Fri Jan 30 03:26:26 2015
From: rymg19 at gmail.com (Ryan)
Date: Thu, 29 Jan 2015 20:26:26 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
Message-ID: <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>

Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They allow you to get a better idea of where libc is crashing.

Cyd Haselton <chaselton at gmail.com> wrote:
>Managed to get this out of logcat:
>F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>11914 (python)  (libc)
>
>[ 01-29 19:30:55.855 23373:23373 F/libc     ]
>Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
>
>Less detail than strace but it seems to be that python is segfaulting
>libc...
>
>On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>> On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
>wrote:
>>>
>>> What I see in the strace:
>>>
>>> ... load libpython3.4m.so.1.0
>>> ... load libm
>>> ... open /dev/__properties__ and do something to it (what?)
>>> ... get current time
>>> ... allocate memory
>>> ... getuid
>>> ... segfault
>>>
>>> That's not a lot to go on, but it doesn't look as if it has started
>to
>>> load modules yet.
>>>
>>> Does /dev/__properties__ ring a bell? Not to me.
>>>
>>
>>
>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> is the code that works with that file.
>>
>> This explains it a bit (slides 24-29). Looks like something to do
>with
>> interprocess communication. Likely has nothing to do with Python
>itself.
>>
>> Maybe this would be useful?
>>
>>>
>>> That stack trace would be really helpful.
>>>
>>> On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
>wrote:
>>>>
>>>> Apologies...I'm not sure what a stack track is, but I do have the
>>>> strace.  Nearest I can tell, it happens due to an open call, though
>I
>>>> am probably wrong.
>>>> Attaching the strace output to this email.  I'm going to head back
>to
>>>> the documentation and to back out of some Android-related changes
>in
>>>> _localemodule.c
>>>>
>>>> On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
><guido at python.org>
>>>> wrote:
>>>> > There could be a million differences relevant (unicode, ints,
>...).
>>>> > Perhaps
>>>> > the importlib bootstrap is failing. Perhaps the dynamic loading
>code
>>>> > changed. Did you get a stack track? (IIRC strace shows a syscall
>trace
>>>> > --
>>>> > also useful, but doesn't tell you precisely how it segfaulted.)
>>>> >
>>>> > On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
><chaselton at gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> All,
>>>> >> I recently ditched my attempts to port Python 2.7.8 to Android
>in
>>>> >> favor of Python 3.4.2.  Unfortunately, after using the same
>configure
>>>> >> options in the same environment, and modifying the setup.py as
>needed,
>>>> >> the newly built binary throws a segfault when the
>generate-posix-vars
>>>> >> portion of the build is reached...and when it is run as well
>(i.e.
>>>> >> ./python --help, ./python -E -S -m sysconfig, or similar)
>>>> >>
>>>> >> I took a strace of ./python, however I'm a bit lost when
>reviewing it.
>>>> >> Any ideas as to what may be going on...i.e. why Python 2.7 works
>but
>>>> >> 3.x throws a segfault?
>>>> >>
>>>> >> Thanks in advance,
>>>> >> Cyd
>>>> >> _______________________________________________
>>>> >> Python-Dev mailing list
>>>> >> Python-Dev at python.org
>>>> >> https://mail.python.org/mailman/listinfo/python-dev
>>>> >> Unsubscribe:
>>>> >>
>https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > --Guido van Rossum (python.org/~guido)
>>>
>>>
>>>
>>>
>>> --
>>> --Guido van Rossum (python.org/~guido)
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>>
>https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>
>>
>>
>>
>> --
>> Ryan
>> If anybody ever asks me why I prefer C++ to C, my answer will be
>simple:
>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>was
>> nul-terminated."
>> Personal reality distortion fields are immune to contradictory
>evidence. -
>> srean
>> Check out my website: http://kirbyfan64.github.io/

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/c974cf05/attachment-0001.html>

From mistersheik at gmail.com  Fri Jan 30 13:07:51 2015
From: mistersheik at gmail.com (Neil Girdhar)
Date: Fri, 30 Jan 2015 07:07:51 -0500
Subject: [Python-Dev] Code review for PEP 448
Message-ID: <CAA68w_n00P465o0T_OzehuY4UnH6w3wEvqaOvJGXcy-B5vnmuw@mail.gmail.com>

Hello all,

PEP 448 (https://www.python.org/dev/peps/pep-0448/) is mostly implemented
now based on some early implementations by twouters (in 2008) and fhahn
(2013) and recently by Joshua and I.

The issue tracker http://bugs.python.org/issue2292  has:
* a working patch, and
* discussion and updates to the PEP (the most conservative interpretations
were taken).

I was wondering if anyone would mind reviewing the patch or even just
trying it out to let us know if there are any corner cases that we missed.
 (I need to get back to my actual work, so it would be good to have this
reviewed before I forget everything if possible.)

Thank you,

Neil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/3e3b3c26/attachment.html>

From chaselton at gmail.com  Fri Jan 30 16:05:13 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 09:05:13 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
Message-ID: <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>

Absolutely.  Good thing I have addr2line on device

/bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 0008bbc8
_PyMem_RawStrdup
/bld/python/Python-3.4.2/Objects/obmalloc.c:323
/bld/python/Python-3.4.2 $



On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
> allow you to get a better idea of where libc is crashing.
>
> Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Managed to get this out of logcat:
>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> 11914 (python)  (libc)
>>
>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
>>
>> Less detail than strace but it seems to be that python is segfaulting
>> libc...
>>
>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>
>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>>
>>>>
>>>>  What I see in the strace:
>>>>
>>>>  ... load libpython3.4m.so.1.0
>>>>  ... load libm
>>>>  ... open /dev/__properties__ and do something to it
>>>> (what?)
>>>>  ... get current time
>>>>  ... allocate memory
>>>>  ... getuid
>>>>  ... segfault
>>>>
>>>>  That's not a lot to go on, but it doesn't look as if it has started to
>>>>  load modules yet.
>>>>
>>>>  Does /dev/__properties__ ring a bell? Not to me.
>>>
>>>
>>>
>>>
>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>  is the code that works with that file.
>>>
>>>  This explains it a bit (slides 24-29). Looks like something to do with
>>>  interprocess communication. Likely has nothing to do with Python itself.
>>>
>>>  Maybe this would be useful?
>>>
>>>>
>>>>  That stack trace would be really helpful.
>>>>
>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
>>>> wrote:
>>>>>
>>>>>
>>>>>  Apologies...I'm not sure what a stack track is, but I do have the
>>>>>  strace.  Nearest I can tell, it happens due to an open call, though I
>>>>>  am probably wrong.
>>>>>  Attaching the strace output to this email.  I'm going to head back to
>>>>>  the documentation and to back out of some Android-related changes in
>>>>>  _localemodule.c
>>>>>
>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>>>>  wrote:
>>>>>>
>>>>>>  There could be a million differences relevant (unicode, ints, ...).
>>>>>>  Perhaps
>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading code
>>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>>>>>> trace
>>>>>>  --
>>>>>>  also useful, but doesn't tell you precisely how
>>>>>> it segfaulted.)
>>>>>>
>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>>>>>  wrote:
>>>>>>>
>>>>>>>
>>>>>>>  All,
>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>>>>>> configure
>>>>>>>  options in the same environment, and modifying the setup.py as
>>>>>>> needed,
>>>>>>>  the newly built binary throws a segfault when the
>>>>>>> generate-posix-vars
>>>>>>>  portion of the build is reached...and when it is run as well (i.e.
>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>>>>>>
>>>>>>>  I took a strace of ./python, however I'm a bit lost when reviewing
>>>>>>> it.
>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>>>>>>  3.x throws a segfault?
>>>>>>>
>>>>>>>  Thanks in advance,
>>>>>>>  Cyd
>>>>>>> ________________________________
>>>>>>>
>>>>>>>  Python-Dev mailing list
>>>>>>>
>>>>>>> Python-Dev at python.org
>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>>>>  Unsubscribe:
>>>>>>>
>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  --
>>>>>>  --Guido van Rossum (python.org/~guido)
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  --
>>>>  --Guido van Rossum (python.org/~guido)
>>>>
>>>> ________________________________
>>>>
>>>>  Python-Dev mailing list
>>>>  Python-Dev at python.org
>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>  Unsubscribe:
>>>>  https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>
>>>
>>>
>>>
>>>
>>>  --
>>>  Ryan
>>>  If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>>  nul-terminated."
>>>  Personal reality distortion fields are immune to contradictory evidence.
>>> -
>>>  srean
>>>  Check out my website: http://kirbyfan64.github.io/
>
>
> --
> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> Check out my website: http://kirbyfan64.github.io/

From chaselton at gmail.com  Fri Jan 30 16:29:05 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 09:29:05 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
Message-ID: <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>

There's a related strdup patch for readline.c, mentioned
here:http://bugs.python.org/issue21390 and here
https://github.com/rave-engine/python3-android/issues/2.
There's a patch, but I'm not sure how to modify it for obmalloc.c, as
(I think) the functions all belong to Python...they're all prefixed
with _PyXx

On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> Absolutely.  Good thing I have addr2line on device
>
> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 0008bbc8
> _PyMem_RawStrdup
> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> /bld/python/Python-3.4.2 $
>
>
>
> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
>> allow you to get a better idea of where libc is crashing.
>>
>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> Managed to get this out of logcat:
>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>>> 11914 (python)  (libc)
>>>
>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
>>>
>>> Less detail than strace but it seems to be that python is segfaulting
>>> libc...
>>>
>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>>
>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
>>>> wrote:
>>>>>
>>>>>
>>>>>  What I see in the strace:
>>>>>
>>>>>  ... load libpython3.4m.so.1.0
>>>>>  ... load libm
>>>>>  ... open /dev/__properties__ and do something to it
>>>>> (what?)
>>>>>  ... get current time
>>>>>  ... allocate memory
>>>>>  ... getuid
>>>>>  ... segfault
>>>>>
>>>>>  That's not a lot to go on, but it doesn't look as if it has started to
>>>>>  load modules yet.
>>>>>
>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>>>>
>>>>
>>>>
>>>>
>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>>  is the code that works with that file.
>>>>
>>>>  This explains it a bit (slides 24-29). Looks like something to do with
>>>>  interprocess communication. Likely has nothing to do with Python itself.
>>>>
>>>>  Maybe this would be useful?
>>>>
>>>>>
>>>>>  That stack trace would be really helpful.
>>>>>
>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>  Apologies...I'm not sure what a stack track is, but I do have the
>>>>>>  strace.  Nearest I can tell, it happens due to an open call, though I
>>>>>>  am probably wrong.
>>>>>>  Attaching the strace output to this email.  I'm going to head back to
>>>>>>  the documentation and to back out of some Android-related changes in
>>>>>>  _localemodule.c
>>>>>>
>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>>>>>  wrote:
>>>>>>>
>>>>>>>  There could be a million differences relevant (unicode, ints, ...).
>>>>>>>  Perhaps
>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading code
>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>>>>>>> trace
>>>>>>>  --
>>>>>>>  also useful, but doesn't tell you precisely how
>>>>>>> it segfaulted.)
>>>>>>>
>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>  All,
>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>>>>>>> configure
>>>>>>>>  options in the same environment, and modifying the setup.py as
>>>>>>>> needed,
>>>>>>>>  the newly built binary throws a segfault when the
>>>>>>>> generate-posix-vars
>>>>>>>>  portion of the build is reached...and when it is run as well (i.e.
>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>>>>>>>
>>>>>>>>  I took a strace of ./python, however I'm a bit lost when reviewing
>>>>>>>> it.
>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>>>>>>>  3.x throws a segfault?
>>>>>>>>
>>>>>>>>  Thanks in advance,
>>>>>>>>  Cyd
>>>>>>>> ________________________________
>>>>>>>>
>>>>>>>>  Python-Dev mailing list
>>>>>>>>
>>>>>>>> Python-Dev at python.org
>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>>>>>  Unsubscribe:
>>>>>>>>
>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>  --
>>>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>
>>>>> ________________________________
>>>>>
>>>>>  Python-Dev mailing list
>>>>>  Python-Dev at python.org
>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>>  Unsubscribe:
>>>>>  https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>  --
>>>>  Ryan
>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>>>  nul-terminated."
>>>>  Personal reality distortion fields are immune to contradictory evidence.
>>>> -
>>>>  srean
>>>>  Check out my website: http://kirbyfan64.github.io/
>>
>>
>> --
>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> Check out my website: http://kirbyfan64.github.io/

From status at bugs.python.org  Fri Jan 30 18:08:12 2015
From: status at bugs.python.org (Python tracker)
Date: Fri, 30 Jan 2015 18:08:12 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20150130170812.EB84C562DB@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2015-01-23 - 2015-01-30)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    4764 ( +4)
  closed 30351 (+44)
  total  35115 (+48)

Open issues with patches: 2224 


Issues opened (38)
==================

#21076: Turn signal.SIG* constants into enums
http://bugs.python.org/issue21076  reopened by serhiy.storchaka

#23306: Within zipfile, use of zlib.crc32 raises OverflowError at argu
http://bugs.python.org/issue23306  opened by Danny.Yoo

#23308: a bug in Instructions section 4.1
http://bugs.python.org/issue23308  opened by Dmot

#23309: Hang on interpreter shutdown if daemon thread prints to stdout
http://bugs.python.org/issue23309  opened by marienz

#23310: MagicMock constructor configuration fails for magic methods
http://bugs.python.org/issue23310  opened by berdario

#23311: Update PC/example_nt and extending/windows.rst
http://bugs.python.org/issue23311  opened by steve.dower

#23312: google thinks the docs are mobile unfriendly
http://bugs.python.org/issue23312  opened by benjamin.peterson

#23314: Disabling CRT asserts in debug build
http://bugs.python.org/issue23314  opened by steve.dower

#23315: tempfile.mkdtemp fails with non-ascii paths on Python 2
http://bugs.python.org/issue23315  opened by akira

#23316: Incorrect evaluation order of function arguments with *args
http://bugs.python.org/issue23316  opened by Joshua.Landau

#23317: Incorrect description of descriptor invocation in Python Langu
http://bugs.python.org/issue23317  opened by Justin.Eldridge

#23319: Missing SWAP_INT??in I_set_sw
http://bugs.python.org/issue23319  opened by mgautierfr

#23320: devguide should mention rules about "paragraph reflow" in the 
http://bugs.python.org/issue23320  opened by akira

#23321: Crash in str.decode() with special error handler
http://bugs.python.org/issue23321  opened by serhiy.storchaka

#23322: parser module docs missing second example
http://bugs.python.org/issue23322  opened by Devin Jeanpierre

#23323: Issue with imaplib and append messages passing a tuple with fl
http://bugs.python.org/issue23323  opened by Pilessio

#23324: Nonblocking serial io using Arduino and Ubuntu 14.10 (Python 3
http://bugs.python.org/issue23324  opened by MrYsLab

#23325: Turn SIG_DFL and SIG_IGN into functions
http://bugs.python.org/issue23325  opened by serhiy.storchaka

#23326: Remove redundant __ne__ implementations
http://bugs.python.org/issue23326  opened by serhiy.storchaka

#23327: zipimport to import from non-ascii pathname on Windows
http://bugs.python.org/issue23327  opened by amswap

#23328: urllib2 fails for proxy credentials that contain a '/' charact
http://bugs.python.org/issue23328  opened by Andy.Reitz

#23330: h2py.py regular expression missing
http://bugs.python.org/issue23330  opened by Thomas.Roos

#23331: Add non-interactive version of Bdb.runcall
http://bugs.python.org/issue23331  opened by etanol

#23332: datetime.isoformat() -> explicitly mark UTC string as such
http://bugs.python.org/issue23332  opened by mirkovogt

#23333: asyncio: add a new Protocol.connection_failed() method
http://bugs.python.org/issue23333  opened by haypo

#23336: Thread.LockType is misnamed
http://bugs.python.org/issue23336  opened by cindykrafft

#23337: Run python with restricted rights
http://bugs.python.org/issue23337  opened by marcd

#23338: PyErr_Format in ctypes uses invalid parameter
http://bugs.python.org/issue23338  opened by mkato

#23342: run() - unified high-level interface for subprocess
http://bugs.python.org/issue23342  opened by takluyver

#23344: Faster marshalling
http://bugs.python.org/issue23344  opened by serhiy.storchaka

#23345: test_ssl fails on OS X 10.10.2 with latest patch level of Open
http://bugs.python.org/issue23345  opened by ned.deily

#23346: shutil.rmtree doesn't work correctly on FreeBSD.
http://bugs.python.org/issue23346  opened by negval

#23348: distutils.LooseVersion fails to compare two valid versions
http://bugs.python.org/issue23348  opened by maethor

#23349: PyBuffer_ToContiguous() off-by-one error for non-contiguous bu
http://bugs.python.org/issue23349  opened by rhansen

#23350: Content-length is incorrect when request body is a list or tup
http://bugs.python.org/issue23350  opened by demian.brecht

#23351: socket.settimeout(5.0) does not have any effect
http://bugs.python.org/issue23351  opened by piotrjurkiewicz

#23352: PyBuffer_IsContiguous() sometimes returns wrong result if subo
http://bugs.python.org/issue23352  opened by rhansen

#23353: gnerator bug with exception: tstate->exc_value is not cleared 
http://bugs.python.org/issue23353  opened by haypo



Most recent 15 issues with no replies (15)
==========================================

#23349: PyBuffer_ToContiguous() off-by-one error for non-contiguous bu
http://bugs.python.org/issue23349

#23344: Faster marshalling
http://bugs.python.org/issue23344

#23336: Thread.LockType is misnamed
http://bugs.python.org/issue23336

#23331: Add non-interactive version of Bdb.runcall
http://bugs.python.org/issue23331

#23330: h2py.py regular expression missing
http://bugs.python.org/issue23330

#23326: Remove redundant __ne__ implementations
http://bugs.python.org/issue23326

#23325: Turn SIG_DFL and SIG_IGN into functions
http://bugs.python.org/issue23325

#23319: Missing SWAP_INT??in I_set_sw
http://bugs.python.org/issue23319

#23314: Disabling CRT asserts in debug build
http://bugs.python.org/issue23314

#23310: MagicMock constructor configuration fails for magic methods
http://bugs.python.org/issue23310

#23309: Hang on interpreter shutdown if daemon thread prints to stdout
http://bugs.python.org/issue23309

#23291: Documentation about Py_Finalize(): Freeing objects
http://bugs.python.org/issue23291

#23289: concurrent.futures.Executor.map is not equivalent to map.
http://bugs.python.org/issue23289

#23287: ctypes.util.find_library needlessly call crle on Solaris
http://bugs.python.org/issue23287

#23279: test_site/test_startup_imports fails when mpl_toolkit or logil
http://bugs.python.org/issue23279



Most recent 15 issues waiting for review (15)
=============================================

#23352: PyBuffer_IsContiguous() sometimes returns wrong result if subo
http://bugs.python.org/issue23352

#23350: Content-length is incorrect when request body is a list or tup
http://bugs.python.org/issue23350

#23349: PyBuffer_ToContiguous() off-by-one error for non-contiguous bu
http://bugs.python.org/issue23349

#23344: Faster marshalling
http://bugs.python.org/issue23344

#23342: run() - unified high-level interface for subprocess
http://bugs.python.org/issue23342

#23338: PyErr_Format in ctypes uses invalid parameter
http://bugs.python.org/issue23338

#23333: asyncio: add a new Protocol.connection_failed() method
http://bugs.python.org/issue23333

#23330: h2py.py regular expression missing
http://bugs.python.org/issue23330

#23327: zipimport to import from non-ascii pathname on Windows
http://bugs.python.org/issue23327

#23326: Remove redundant __ne__ implementations
http://bugs.python.org/issue23326

#23325: Turn SIG_DFL and SIG_IGN into functions
http://bugs.python.org/issue23325

#23321: Crash in str.decode() with special error handler
http://bugs.python.org/issue23321

#23319: Missing SWAP_INT??in I_set_sw
http://bugs.python.org/issue23319

#23304: Unused Superclass in calendar.py
http://bugs.python.org/issue23304

#23298: Add ArgumentParser.add_mutually_dependence_group
http://bugs.python.org/issue23298



Top 10 most discussed issues (10)
=================================

#2292: Missing *-unpacking generalizations
http://bugs.python.org/issue2292  29 msgs

#23332: datetime.isoformat() -> explicitly mark UTC string as such
http://bugs.python.org/issue23332  16 msgs

#17911: traceback: add a new thin class storing a traceback without st
http://bugs.python.org/issue17911  15 msgs

#23255: SimpleHTTPRequestHandler refactor for more extensible usage.
http://bugs.python.org/issue23255  11 msgs

#23284: Improve termcap detection in setup.py
http://bugs.python.org/issue23284  11 msgs

#23285: PEP 475 - EINTR handling
http://bugs.python.org/issue23285  11 msgs

#23316: Incorrect evaluation order of function arguments with *args
http://bugs.python.org/issue23316   9 msgs

#23333: asyncio: add a new Protocol.connection_failed() method
http://bugs.python.org/issue23333   9 msgs

#17797: Visual C++ 11.0 reports fileno(stdin) == 0 for non-console pro
http://bugs.python.org/issue17797   7 msgs

#23327: zipimport to import from non-ascii pathname on Windows
http://bugs.python.org/issue23327   7 msgs



Issues closed (41)
==================

#7665: test_urllib2 and test_ntpath fail if path contains "\"
http://bugs.python.org/issue7665  closed by serhiy.storchaka

#8094: Multiprocessing infinite loop
http://bugs.python.org/issue8094  closed by davin

#8144: muliprocessing shutdown infinite loop
http://bugs.python.org/issue8144  closed by davin

#15101: multiprocessing pool finalizer can fail if triggered in backgr
http://bugs.python.org/issue15101  closed by davin

#18813: Speed up slice object processing
http://bugs.python.org/issue18813  closed by scoder

#19361: Specialize exceptions thrown by JSON parser
http://bugs.python.org/issue19361  closed by serhiy.storchaka

#19949: Explicitly skip or mask skipped/disabled tests in test_xpickle
http://bugs.python.org/issue19949  closed by serhiy.storchaka

#19996: httplib infinite read on invalid header
http://bugs.python.org/issue19996  closed by python-dev

#20188: ALPN support for TLS
http://bugs.python.org/issue20188  closed by python-dev

#20284: patch to implement PEP 461 (%-interpolation for bytes)
http://bugs.python.org/issue20284  closed by ethan.furman

#21408: delegation of `!=` to the right-hand side argument is not alwa
http://bugs.python.org/issue21408  closed by serhiy.storchaka

#21962: No timeout for asyncio.Event.wait() or asyncio.Condition.wait(
http://bugs.python.org/issue21962  closed by haypo

#22286: Allow backslashreplace error handler to be used on input
http://bugs.python.org/issue22286  closed by serhiy.storchaka

#22668: memoryview.format is corrupted due to dangling shared pointer
http://bugs.python.org/issue22668  closed by python-dev

#23046: asyncio.BaseEventLoop is documented, but only exported via asy
http://bugs.python.org/issue23046  closed by haypo

#23055: PyUnicode_FromFormatV crasher
http://bugs.python.org/issue23055  closed by serhiy.storchaka

#23094: Unpickler failing with PicklingError at frame end on readline 
http://bugs.python.org/issue23094  closed by serhiy.storchaka

#23187: Segmentation fault, possibly asyncio related
http://bugs.python.org/issue23187  closed by haypo

#23191: fnmatch regex cache use is not threadsafe
http://bugs.python.org/issue23191  closed by serhiy.storchaka

#23202: pyvenv does not fail like documented when a venv already exist
http://bugs.python.org/issue23202  closed by python-dev

#23207: logging.basicConfig does not validate keyword arguments
http://bugs.python.org/issue23207  closed by python-dev

#23208: asyncio: add BaseEventLoop._current_handle (only used in debug
http://bugs.python.org/issue23208  closed by haypo

#23243: asyncio: emit ResourceWarning warnings if transports/event loo
http://bugs.python.org/issue23243  closed by haypo

#23253: Delay-load ShellExecute
http://bugs.python.org/issue23253  closed by python-dev

#23257: Update Windows build/setup instructions in devguide
http://bugs.python.org/issue23257  closed by steve.dower

#23268: Fix comparison of ipaddress classes
http://bugs.python.org/issue23268  closed by serhiy.storchaka

#23282: Slightly faster set lookup
http://bugs.python.org/issue23282  closed by rhettinger

#23286: A typo in the tutorial
http://bugs.python.org/issue23286  closed by berker.peksag

#23292: Enum doc suggestion
http://bugs.python.org/issue23292  closed by mark

#23305: RotatingFileHandler does not rotate if backupCount is 0
http://bugs.python.org/issue23305  closed by vinay.sajip

#23307: python hangs on call to numpy.outer
http://bugs.python.org/issue23307  closed by mark.dickinson

#23313: Wrong complex variable being altered
http://bugs.python.org/issue23313  closed by benjamin.peterson

#23318: (compiled RegEx).split gives unexpected results if () in patte
http://bugs.python.org/issue23318  closed by SilentGhost

#23329: _ssl cannot be compiled with LibreSSL anymore (on OpenBSD 5.5)
http://bugs.python.org/issue23329  closed by python-dev

#23334: http.client refactor
http://bugs.python.org/issue23334  closed by demian.brecht

#23335: _ssl.c cannot be compiled with older versions of OpenSSL
http://bugs.python.org/issue23335  closed by python-dev

#23339: dict_values should be comparable with a set
http://bugs.python.org/issue23339  closed by r.david.murray

#23340: uClibc C++ exceptions issue
http://bugs.python.org/issue23340  closed by r.david.murray

#23341: Issue parsing valid cookie
http://bugs.python.org/issue23341  closed by dlamotte

#23343: operator precedence table for `not x` has an operand, while th
http://bugs.python.org/issue23343  closed by Hobson.Lane

#23347: asyncio: fix and refactor creation of subprocess transports
http://bugs.python.org/issue23347  closed by haypo

From rymg19 at gmail.com  Fri Jan 30 18:52:41 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 11:52:41 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
Message-ID: <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>

Is it possible at all to get a stack trace of the crash using gdb? Try the
steps here <http://stackoverflow.com/a/10539883/2097780>.

That way we can see where Python's own strdup function is getting called.

On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com> wrote:

> Absolutely.  Good thing I have addr2line on device
>
> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> 0008bbc8
> _PyMem_RawStrdup
> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> /bld/python/Python-3.4.2 $
>
>
>
> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> > Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
> They
> > allow you to get a better idea of where libc is crashing.
> >
> > Cyd Haselton <chaselton at gmail.com> wrote:
> >>
> >> Managed to get this out of logcat:
> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
> >> 11914 (python)  (libc)
> >>
> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
> >>
> >> Less detail than strace but it seems to be that python is segfaulting
> >> libc...
> >>
> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
> wrote:
> >>>
> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
> >>> wrote:
> >>>>
> >>>>
> >>>>  What I see in the strace:
> >>>>
> >>>>  ... load libpython3.4m.so.1.0
> >>>>  ... load libm
> >>>>  ... open /dev/__properties__ and do something to it
> >>>> (what?)
> >>>>  ... get current time
> >>>>  ... allocate memory
> >>>>  ... getuid
> >>>>  ... segfault
> >>>>
> >>>>  That's not a lot to go on, but it doesn't look as if it has started
> to
> >>>>  load modules yet.
> >>>>
> >>>>  Does /dev/__properties__ ring a bell? Not to me.
> >>>
> >>>
> >>>
> >>>
> >>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >>>  is the code that works with that file.
> >>>
> >>>  This explains it a bit (slides 24-29). Looks like something to do with
> >>>  interprocess communication. Likely has nothing to do with Python
> itself.
> >>>
> >>>  Maybe this would be useful?
> >>>
> >>>>
> >>>>  That stack trace would be really helpful.
> >>>>
> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
> >>>> wrote:
> >>>>>
> >>>>>
> >>>>>  Apologies...I'm not sure what a stack track is, but I do have the
> >>>>>  strace.  Nearest I can tell, it happens due to an open call, though
> I
> >>>>>  am probably wrong.
> >>>>>  Attaching the strace output to this email.  I'm going to head back
> to
> >>>>>  the documentation and to back out of some Android-related changes in
> >>>>>  _localemodule.c
> >>>>>
> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org
> >
> >>>>>  wrote:
> >>>>>>
> >>>>>>  There could be a million differences relevant (unicode, ints, ...).
> >>>>>>  Perhaps
> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
> code
> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
> >>>>>> trace
> >>>>>>  --
> >>>>>>  also useful, but doesn't tell you precisely how
> >>>>>> it segfaulted.)
> >>>>>>
> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com
> >
> >>>>>>  wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>  All,
> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >>>>>>> configure
> >>>>>>>  options in the same environment, and modifying the setup.py as
> >>>>>>> needed,
> >>>>>>>  the newly built binary throws a segfault when the
> >>>>>>> generate-posix-vars
> >>>>>>>  portion of the build is reached...and when it is run as well (i.e.
> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >>>>>>>
> >>>>>>>  I took a strace of ./python, however I'm a bit lost when reviewing
> >>>>>>> it.
> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works
> but
> >>>>>>>  3.x throws a segfault?
> >>>>>>>
> >>>>>>>  Thanks in advance,
> >>>>>>>  Cyd
> >>>>>>> ________________________________
> >>>>>>>
> >>>>>>>  Python-Dev mailing list
> >>>>>>>
> >>>>>>> Python-Dev at python.org
> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >>>>>>>  Unsubscribe:
> >>>>>>>
> >>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>  --
> >>>>>>  --Guido van Rossum (python.org/~guido)
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>
> >>>>  --
> >>>>  --Guido van Rossum (python.org/~guido)
> >>>>
> >>>> ________________________________
> >>>>
> >>>>  Python-Dev mailing list
> >>>>  Python-Dev at python.org
> >>>>  https://mail.python.org/mailman/listinfo/python-dev
> >>>>  Unsubscribe:
> >>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>  --
> >>>  Ryan
> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
> was
> >>>  nul-terminated."
> >>>  Personal reality distortion fields are immune to contradictory
> evidence.
> >>> -
> >>>  srean
> >>>  Check out my website: http://kirbyfan64.github.io/
> >
> >
> > --
> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/28778086/attachment-0001.html>

From chaselton at gmail.com  Fri Jan 30 18:53:13 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 11:53:13 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
Message-ID: <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>

Alternatively, is there a hassle-free way to find out what changed in
obmalloc.c between 2.7.x and 3.4.x?


On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> There's a related strdup patch for readline.c, mentioned
> here:http://bugs.python.org/issue21390 and here
> https://github.com/rave-engine/python3-android/issues/2.
> There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> (I think) the functions all belong to Python...they're all prefixed
> with _PyXx
>
> On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>> Absolutely.  Good thing I have addr2line on device
>>
>> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0 0008bbc8
>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> /bld/python/Python-3.4.2 $
>>
>>
>>
>> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780? They
>>> allow you to get a better idea of where libc is crashing.
>>>
>>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>>
>>>> Managed to get this out of logcat:
>>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>>>> 11914 (python)  (libc)
>>>>
>>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
>>>>
>>>> Less detail than strace but it seems to be that python is segfaulting
>>>> libc...
>>>>
>>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>>>
>>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
>>>>> wrote:
>>>>>>
>>>>>>
>>>>>>  What I see in the strace:
>>>>>>
>>>>>>  ... load libpython3.4m.so.1.0
>>>>>>  ... load libm
>>>>>>  ... open /dev/__properties__ and do something to it
>>>>>> (what?)
>>>>>>  ... get current time
>>>>>>  ... allocate memory
>>>>>>  ... getuid
>>>>>>  ... segfault
>>>>>>
>>>>>>  That's not a lot to go on, but it doesn't look as if it has started to
>>>>>>  load modules yet.
>>>>>>
>>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>>>  is the code that works with that file.
>>>>>
>>>>>  This explains it a bit (slides 24-29). Looks like something to do with
>>>>>  interprocess communication. Likely has nothing to do with Python itself.
>>>>>
>>>>>  Maybe this would be useful?
>>>>>
>>>>>>
>>>>>>  That stack trace would be really helpful.
>>>>>>
>>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
>>>>>> wrote:
>>>>>>>
>>>>>>>
>>>>>>>  Apologies...I'm not sure what a stack track is, but I do have the
>>>>>>>  strace.  Nearest I can tell, it happens due to an open call, though I
>>>>>>>  am probably wrong.
>>>>>>>  Attaching the strace output to this email.  I'm going to head back to
>>>>>>>  the documentation and to back out of some Android-related changes in
>>>>>>>  _localemodule.c
>>>>>>>
>>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <guido at python.org>
>>>>>>>  wrote:
>>>>>>>>
>>>>>>>>  There could be a million differences relevant (unicode, ints, ...).
>>>>>>>>  Perhaps
>>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading code
>>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>>>>>>>> trace
>>>>>>>>  --
>>>>>>>>  also useful, but doesn't tell you precisely how
>>>>>>>> it segfaulted.)
>>>>>>>>
>>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <chaselton at gmail.com>
>>>>>>>>  wrote:
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>  All,
>>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>>>>>>>> configure
>>>>>>>>>  options in the same environment, and modifying the setup.py as
>>>>>>>>> needed,
>>>>>>>>>  the newly built binary throws a segfault when the
>>>>>>>>> generate-posix-vars
>>>>>>>>>  portion of the build is reached...and when it is run as well (i.e.
>>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>>>>>>>>
>>>>>>>>>  I took a strace of ./python, however I'm a bit lost when reviewing
>>>>>>>>> it.
>>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works but
>>>>>>>>>  3.x throws a segfault?
>>>>>>>>>
>>>>>>>>>  Thanks in advance,
>>>>>>>>>  Cyd
>>>>>>>>> ________________________________
>>>>>>>>>
>>>>>>>>>  Python-Dev mailing list
>>>>>>>>>
>>>>>>>>> Python-Dev at python.org
>>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>>>>>>  Unsubscribe:
>>>>>>>>>
>>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>  --
>>>>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>  --
>>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>>
>>>>>> ________________________________
>>>>>>
>>>>>>  Python-Dev mailing list
>>>>>>  Python-Dev at python.org
>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>>>>  Unsubscribe:
>>>>>>  https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>  --
>>>>>  Ryan
>>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>>>>  nul-terminated."
>>>>>  Personal reality distortion fields are immune to contradictory evidence.
>>>>> -
>>>>>  srean
>>>>>  Check out my website: http://kirbyfan64.github.io/
>>>
>>>
>>> --
>>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Fri Jan 30 18:56:34 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 11:56:34 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
Message-ID: <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>

I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes when
calling strlen. It's that whatever is calling it is likely asking it to
duplicate a null pointer. Basically, it's probably the caller's fault.

You could always try modifying _PyMem_RawStrdup to return NULL when given a
null pointer and see where it then segfaults.

On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com> wrote:

> Alternatively, is there a hassle-free way to find out what changed in
> obmalloc.c between 2.7.x and 3.4.x?
>
>
> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com> wrote:
> > There's a related strdup patch for readline.c, mentioned
> > here:http://bugs.python.org/issue21390 and here
> > https://github.com/rave-engine/python3-android/issues/2.
> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> > (I think) the functions all belong to Python...they're all prefixed
> > with _PyXx
> >
> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >> Absolutely.  Good thing I have addr2line on device
> >>
> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> 0008bbc8
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> /bld/python/Python-3.4.2 $
> >>
> >>
> >>
> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> >>> Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
> They
> >>> allow you to get a better idea of where libc is crashing.
> >>>
> >>> Cyd Haselton <chaselton at gmail.com> wrote:
> >>>>
> >>>> Managed to get this out of logcat:
> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
> >>>> 11914 (python)  (libc)
> >>>>
> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
> (python)
> >>>>
> >>>> Less detail than strace but it seems to be that python is segfaulting
> >>>> libc...
> >>>>
> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
> wrote:
> >>>>>
> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <
> guido at python.org>
> >>>>> wrote:
> >>>>>>
> >>>>>>
> >>>>>>  What I see in the strace:
> >>>>>>
> >>>>>>  ... load libpython3.4m.so.1.0
> >>>>>>  ... load libm
> >>>>>>  ... open /dev/__properties__ and do something to it
> >>>>>> (what?)
> >>>>>>  ... get current time
> >>>>>>  ... allocate memory
> >>>>>>  ... getuid
> >>>>>>  ... segfault
> >>>>>>
> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
> started to
> >>>>>>  load modules yet.
> >>>>>>
> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >>>>>  is the code that works with that file.
> >>>>>
> >>>>>  This explains it a bit (slides 24-29). Looks like something to do
> with
> >>>>>  interprocess communication. Likely has nothing to do with Python
> itself.
> >>>>>
> >>>>>  Maybe this would be useful?
> >>>>>
> >>>>>>
> >>>>>>  That stack trace would be really helpful.
> >>>>>>
> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com
> >
> >>>>>> wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do have the
> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
> though I
> >>>>>>>  am probably wrong.
> >>>>>>>  Attaching the strace output to this email.  I'm going to head
> back to
> >>>>>>>  the documentation and to back out of some Android-related changes
> in
> >>>>>>>  _localemodule.c
> >>>>>>>
> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum <
> guido at python.org>
> >>>>>>>  wrote:
> >>>>>>>>
> >>>>>>>>  There could be a million differences relevant (unicode, ints,
> ...).
> >>>>>>>>  Perhaps
> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
> code
> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
> >>>>>>>> trace
> >>>>>>>>  --
> >>>>>>>>  also useful, but doesn't tell you precisely how
> >>>>>>>> it segfaulted.)
> >>>>>>>>
> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton <
> chaselton at gmail.com>
> >>>>>>>>  wrote:
> >>>>>>>>>
> >>>>>>>>>
> >>>>>>>>>  All,
> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
> in
> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >>>>>>>>> configure
> >>>>>>>>>  options in the same environment, and modifying the setup.py as
> >>>>>>>>> needed,
> >>>>>>>>>  the newly built binary throws a segfault when the
> >>>>>>>>> generate-posix-vars
> >>>>>>>>>  portion of the build is reached...and when it is run as well
> (i.e.
> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >>>>>>>>>
> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
> reviewing
> >>>>>>>>> it.
> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
> works but
> >>>>>>>>>  3.x throws a segfault?
> >>>>>>>>>
> >>>>>>>>>  Thanks in advance,
> >>>>>>>>>  Cyd
> >>>>>>>>> ________________________________
> >>>>>>>>>
> >>>>>>>>>  Python-Dev mailing list
> >>>>>>>>>
> >>>>>>>>> Python-Dev at python.org
> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >>>>>>>>>  Unsubscribe:
> >>>>>>>>>
> >>>>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>
> >>>>>>>>  --
> >>>>>>>>  --Guido van Rossum (python.org/~guido)
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>  --
> >>>>>>  --Guido van Rossum (python.org/~guido)
> >>>>>>
> >>>>>> ________________________________
> >>>>>>
> >>>>>>  Python-Dev mailing list
> >>>>>>  Python-Dev at python.org
> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >>>>>>  Unsubscribe:
> >>>>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>  --
> >>>>>  Ryan
> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> that was
> >>>>>  nul-terminated."
> >>>>>  Personal reality distortion fields are immune to contradictory
> evidence.
> >>>>> -
> >>>>>  srean
> >>>>>  Check out my website: http://kirbyfan64.github.io/
> >>>
> >>>
> >>> --
> >>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> >>> Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/3f5e163e/attachment.html>

From chaselton at gmail.com  Fri Jan 30 19:00:32 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 12:00:32 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
Message-ID: <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>

I don't have gdb on device; does the following tell you where Python's
strdup is called?

>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323

On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Is it possible at all to get a stack trace of the crash using gdb? Try the
> steps here.
>
> That way we can see where Python's own strdup function is getting called.
>
> On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Absolutely.  Good thing I have addr2line on device
>>
>> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> 0008bbc8
>> _PyMem_RawStrdup
>> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> /bld/python/Python-3.4.2 $
>>
>>
>>
>> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> > Could you try the steps at http://stackoverflow.com/a/11369475/2097780?
>> > They
>> > allow you to get a better idea of where libc is crashing.
>> >
>> > Cyd Haselton <chaselton at gmail.com> wrote:
>> >>
>> >> Managed to get this out of logcat:
>> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> 11914 (python)  (libc)
>> >>
>> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373 (python)
>> >>
>> >> Less detail than strace but it seems to be that python is segfaulting
>> >> libc...
>> >>
>> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> wrote:
>> >>>
>> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <guido at python.org>
>> >>> wrote:
>> >>>>
>> >>>>
>> >>>>  What I see in the strace:
>> >>>>
>> >>>>  ... load libpython3.4m.so.1.0
>> >>>>  ... load libm
>> >>>>  ... open /dev/__properties__ and do something to it
>> >>>> (what?)
>> >>>>  ... get current time
>> >>>>  ... allocate memory
>> >>>>  ... getuid
>> >>>>  ... segfault
>> >>>>
>> >>>>  That's not a lot to go on, but it doesn't look as if it has started
>> >>>> to
>> >>>>  load modules yet.
>> >>>>
>> >>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >>>  is the code that works with that file.
>> >>>
>> >>>  This explains it a bit (slides 24-29). Looks like something to do
>> >>> with
>> >>>  interprocess communication. Likely has nothing to do with Python
>> >>> itself.
>> >>>
>> >>>  Maybe this would be useful?
>> >>>
>> >>>>
>> >>>>  That stack trace would be really helpful.
>> >>>>
>> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <chaselton at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>
>> >>>>>  Apologies...I'm not sure what a stack track is, but I do have the
>> >>>>>  strace.  Nearest I can tell, it happens due to an open call, though
>> >>>>> I
>> >>>>>  am probably wrong.
>> >>>>>  Attaching the strace output to this email.  I'm going to head back
>> >>>>> to
>> >>>>>  the documentation and to back out of some Android-related changes
>> >>>>> in
>> >>>>>  _localemodule.c
>> >>>>>
>> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >>>>> <guido at python.org>
>> >>>>>  wrote:
>> >>>>>>
>> >>>>>>  There could be a million differences relevant (unicode, ints,
>> >>>>>> ...).
>> >>>>>>  Perhaps
>> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
>> >>>>>> code
>> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>> >>>>>> trace
>> >>>>>>  --
>> >>>>>>  also useful, but doesn't tell you precisely how
>> >>>>>> it segfaulted.)
>> >>>>>>
>> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >>>>>> <chaselton at gmail.com>
>> >>>>>>  wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>  All,
>> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android in
>> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >>>>>>> configure
>> >>>>>>>  options in the same environment, and modifying the setup.py as
>> >>>>>>> needed,
>> >>>>>>>  the newly built binary throws a segfault when the
>> >>>>>>> generate-posix-vars
>> >>>>>>>  portion of the build is reached...and when it is run as well
>> >>>>>>> (i.e.
>> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >>>>>>>
>> >>>>>>>  I took a strace of ./python, however I'm a bit lost when
>> >>>>>>> reviewing
>> >>>>>>> it.
>> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7 works
>> >>>>>>> but
>> >>>>>>>  3.x throws a segfault?
>> >>>>>>>
>> >>>>>>>  Thanks in advance,
>> >>>>>>>  Cyd
>> >>>>>>> ________________________________
>> >>>>>>>
>> >>>>>>>  Python-Dev mailing list
>> >>>>>>>
>> >>>>>>> Python-Dev at python.org
>> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >>>>>>>  Unsubscribe:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>  --
>> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>>  --
>> >>>>  --Guido van Rossum (python.org/~guido)
>> >>>>
>> >>>> ________________________________
>> >>>>
>> >>>>  Python-Dev mailing list
>> >>>>  Python-Dev at python.org
>> >>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >>>>  Unsubscribe:
>> >>>>
>> >>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>  --
>> >>>  Ryan
>> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>> >>> simple:
>> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> >>> was
>> >>>  nul-terminated."
>> >>>  Personal reality distortion fields are immune to contradictory
>> >>> evidence.
>> >>> -
>> >>>  srean
>> >>>  Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> > --
>> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Fri Jan 30 19:04:41 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 12:04:41 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
 <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>
Message-ID: <CAO41-mPQ+Wn-cE96vBn+-X3tHvdVeKBxxOoWFKTsnP=onSXhKQ@mail.gmail.com>

No...

...but I think I found the issue with grep. Try applying the attached patch
to the Python/frozenmain.c. It comments out the locale handling.

It seems that Python always calls its strdup function on the locale string.
On Android, this can apparently be null (as seen in the bug report you
linked to).

On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> I don't have gdb on device; does the following tell you where Python's
> strdup is called?
>
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>
> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > Is it possible at all to get a stack trace of the crash using gdb? Try
> the
> > steps here.
> >
> > That way we can see where Python's own strdup function is getting called.
> >
> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >> Absolutely.  Good thing I have addr2line on device
> >>
> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
> >> 0008bbc8
> >> _PyMem_RawStrdup
> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> /bld/python/Python-3.4.2 $
> >>
> >>
> >>
> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> >> > Could you try the steps at
> http://stackoverflow.com/a/11369475/2097780?
> >> > They
> >> > allow you to get a better idea of where libc is crashing.
> >> >
> >> > Cyd Haselton <chaselton at gmail.com> wrote:
> >> >>
> >> >> Managed to get this out of logcat:
> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
> >> >> 11914 (python)  (libc)
> >> >>
> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
> (python)
> >> >>
> >> >> Less detail than strace but it seems to be that python is segfaulting
> >> >> libc...
> >> >>
> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
> >> >> wrote:
> >> >>>
> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum <
> guido at python.org>
> >> >>> wrote:
> >> >>>>
> >> >>>>
> >> >>>>  What I see in the strace:
> >> >>>>
> >> >>>>  ... load libpython3.4m.so.1.0
> >> >>>>  ... load libm
> >> >>>>  ... open /dev/__properties__ and do something to it
> >> >>>> (what?)
> >> >>>>  ... get current time
> >> >>>>  ... allocate memory
> >> >>>>  ... getuid
> >> >>>>  ... segfault
> >> >>>>
> >> >>>>  That's not a lot to go on, but it doesn't look as if it has
> started
> >> >>>> to
> >> >>>>  load modules yet.
> >> >>>>
> >> >>>>  Does /dev/__properties__ ring a bell? Not to me.
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >>>  is the code that works with that file.
> >> >>>
> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
> >> >>> with
> >> >>>  interprocess communication. Likely has nothing to do with Python
> >> >>> itself.
> >> >>>
> >> >>>  Maybe this would be useful?
> >> >>>
> >> >>>>
> >> >>>>  That stack trace would be really helpful.
> >> >>>>
> >> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton <
> chaselton at gmail.com>
> >> >>>> wrote:
> >> >>>>>
> >> >>>>>
> >> >>>>>  Apologies...I'm not sure what a stack track is, but I do have the
> >> >>>>>  strace.  Nearest I can tell, it happens due to an open call,
> though
> >> >>>>> I
> >> >>>>>  am probably wrong.
> >> >>>>>  Attaching the strace output to this email.  I'm going to head
> back
> >> >>>>> to
> >> >>>>>  the documentation and to back out of some Android-related changes
> >> >>>>> in
> >> >>>>>  _localemodule.c
> >> >>>>>
> >> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >>>>> <guido at python.org>
> >> >>>>>  wrote:
> >> >>>>>>
> >> >>>>>>  There could be a million differences relevant (unicode, ints,
> >> >>>>>> ...).
> >> >>>>>>  Perhaps
> >> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
> >> >>>>>> code
> >> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
> >> >>>>>> trace
> >> >>>>>>  --
> >> >>>>>>  also useful, but doesn't tell you precisely how
> >> >>>>>> it segfaulted.)
> >> >>>>>>
> >> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >>>>>> <chaselton at gmail.com>
> >> >>>>>>  wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>  All,
> >> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
> in
> >> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >> >>>>>>> configure
> >> >>>>>>>  options in the same environment, and modifying the setup.py as
> >> >>>>>>> needed,
> >> >>>>>>>  the newly built binary throws a segfault when the
> >> >>>>>>> generate-posix-vars
> >> >>>>>>>  portion of the build is reached...and when it is run as well
> >> >>>>>>> (i.e.
> >> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >> >>>>>>>
> >> >>>>>>>  I took a strace of ./python, however I'm a bit lost when
> >> >>>>>>> reviewing
> >> >>>>>>> it.
> >> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
> works
> >> >>>>>>> but
> >> >>>>>>>  3.x throws a segfault?
> >> >>>>>>>
> >> >>>>>>>  Thanks in advance,
> >> >>>>>>>  Cyd
> >> >>>>>>> ________________________________
> >> >>>>>>>
> >> >>>>>>>  Python-Dev mailing list
> >> >>>>>>>
> >> >>>>>>> Python-Dev at python.org
> >> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >>>>>>>  Unsubscribe:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>  --
> >> >>>>>>  --Guido van Rossum (python.org/~guido)
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>  --
> >> >>>>  --Guido van Rossum (python.org/~guido)
> >> >>>>
> >> >>>> ________________________________
> >> >>>>
> >> >>>>  Python-Dev mailing list
> >> >>>>  Python-Dev at python.org
> >> >>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >>>>  Unsubscribe:
> >> >>>>
> >> >>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>
> >> >>>  --
> >> >>>  Ryan
> >> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >>> simple:
> >> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> that
> >> >>> was
> >> >>>  nul-terminated."
> >> >>>  Personal reality distortion fields are immune to contradictory
> >> >>> evidence.
> >> >>> -
> >> >>>  srean
> >> >>>  Check out my website: http://kirbyfan64.github.io/
> >> >
> >> >
> >> > --
> >> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> >> > Check out my website: http://kirbyfan64.github.io/
> >
> >
> >
> >
> > --
> > Ryan
> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> > nul-terminated."
> > Personal reality distortion fields are immune to contradictory evidence.
> -
> > srean
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/ba3a8067/attachment-0001.html>
-------------- next part --------------
22c22
<     char *oldloc = NULL;
---
>     /*char *oldloc = NULL;*/
47c47
<     oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
---
>     /*oldloc = _PyMem_RawStrdup(setlocale(LC_ALL, NULL));
53c53
<     setlocale(LC_ALL, "");
---
>     setlocale(LC_ALL, "");*/
64c64
<     setlocale(LC_ALL, oldloc);
---
>     /*setlocale(LC_ALL, oldloc);
66c66
<     oldloc = NULL;
---
>     oldloc = NULL;*/

From chaselton at gmail.com  Fri Jan 30 19:09:01 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 12:09:01 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
Message-ID: <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>

  Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
currently returning NULL when given a null pointer.

>From obmalloc.c

_PyMem_RawStrdup(const char *str)
{
    size_t size;
char *copy;
    size = strlen(str) + 1;
    copy = PyMem_RawMalloc(size);
    if (copy == NULL)
        return NULL;
memcpy(copy, str, size);
return copy;
          }

On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes when
> calling strlen. It's that whatever is calling it is likely asking it to
> duplicate a null pointer. Basically, it's probably the caller's fault.
>
> You could always try modifying _PyMem_RawStrdup to return NULL when given a
> null pointer and see where it then segfaults.
>
> On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Alternatively, is there a hassle-free way to find out what changed in
>> obmalloc.c between 2.7.x and 3.4.x?
>>
>>
>> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com> wrote:
>> > There's a related strdup patch for readline.c, mentioned
>> > here:http://bugs.python.org/issue21390 and here
>> > https://github.com/rave-engine/python3-android/issues/2.
>> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> > (I think) the functions all belong to Python...they're all prefixed
>> > with _PyXx
>> >
>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >> Absolutely.  Good thing I have addr2line on device
>> >>
>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> >> 0008bbc8
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> /bld/python/Python-3.4.2 $
>> >>
>> >>
>> >>
>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> >>> Could you try the steps at
>> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >>> allow you to get a better idea of where libc is crashing.
>> >>>
>> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >>>>
>> >>>> Managed to get this out of logcat:
>> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >>>> 11914 (python)  (libc)
>> >>>>
>> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>> >>>> (python)
>> >>>>
>> >>>> Less detail than strace but it seems to be that python is segfaulting
>> >>>> libc...
>> >>>>
>> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >>>> wrote:
>> >>>>>
>> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >>>>> <guido at python.org>
>> >>>>> wrote:
>> >>>>>>
>> >>>>>>
>> >>>>>>  What I see in the strace:
>> >>>>>>
>> >>>>>>  ... load libpython3.4m.so.1.0
>> >>>>>>  ... load libm
>> >>>>>>  ... open /dev/__properties__ and do something to it
>> >>>>>> (what?)
>> >>>>>>  ... get current time
>> >>>>>>  ... allocate memory
>> >>>>>>  ... getuid
>> >>>>>>  ... segfault
>> >>>>>>
>> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
>> >>>>>> started to
>> >>>>>>  load modules yet.
>> >>>>>>
>> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >>>>>  is the code that works with that file.
>> >>>>>
>> >>>>>  This explains it a bit (slides 24-29). Looks like something to do
>> >>>>> with
>> >>>>>  interprocess communication. Likely has nothing to do with Python
>> >>>>> itself.
>> >>>>>
>> >>>>>  Maybe this would be useful?
>> >>>>>
>> >>>>>>
>> >>>>>>  That stack trace would be really helpful.
>> >>>>>>
>> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >>>>>> <chaselton at gmail.com>
>> >>>>>> wrote:
>> >>>>>>>
>> >>>>>>>
>> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do have the
>> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
>> >>>>>>> though I
>> >>>>>>>  am probably wrong.
>> >>>>>>>  Attaching the strace output to this email.  I'm going to head
>> >>>>>>> back to
>> >>>>>>>  the documentation and to back out of some Android-related changes
>> >>>>>>> in
>> >>>>>>>  _localemodule.c
>> >>>>>>>
>> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >>>>>>> <guido at python.org>
>> >>>>>>>  wrote:
>> >>>>>>>>
>> >>>>>>>>  There could be a million differences relevant (unicode, ints,
>> >>>>>>>> ...).
>> >>>>>>>>  Perhaps
>> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
>> >>>>>>>> code
>> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a syscall
>> >>>>>>>> trace
>> >>>>>>>>  --
>> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >>>>>>>> it segfaulted.)
>> >>>>>>>>
>> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >>>>>>>> <chaselton at gmail.com>
>> >>>>>>>>  wrote:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>>  All,
>> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
>> >>>>>>>>> in
>> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >>>>>>>>> configure
>> >>>>>>>>>  options in the same environment, and modifying the setup.py as
>> >>>>>>>>> needed,
>> >>>>>>>>>  the newly built binary throws a segfault when the
>> >>>>>>>>> generate-posix-vars
>> >>>>>>>>>  portion of the build is reached...and when it is run as well
>> >>>>>>>>> (i.e.
>> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >>>>>>>>>
>> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
>> >>>>>>>>> reviewing
>> >>>>>>>>> it.
>> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>> >>>>>>>>> works but
>> >>>>>>>>>  3.x throws a segfault?
>> >>>>>>>>>
>> >>>>>>>>>  Thanks in advance,
>> >>>>>>>>>  Cyd
>> >>>>>>>>> ________________________________
>> >>>>>>>>>
>> >>>>>>>>>  Python-Dev mailing list
>> >>>>>>>>>
>> >>>>>>>>> Python-Dev at python.org
>> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >>>>>>>>>  Unsubscribe:
>> >>>>>>>>>
>> >>>>>>>>>
>> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>
>> >>>>>>>>  --
>> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>>  --
>> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >>>>>>
>> >>>>>> ________________________________
>> >>>>>>
>> >>>>>>  Python-Dev mailing list
>> >>>>>>  Python-Dev at python.org
>> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >>>>>>  Unsubscribe:
>> >>>>>>
>> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>  --
>> >>>>>  Ryan
>> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>> >>>>> simple:
>> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >>>>> that was
>> >>>>>  nul-terminated."
>> >>>>>  Personal reality distortion fields are immune to contradictory
>> >>>>> evidence.
>> >>>>> -
>> >>>>>  srean
>> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >>>
>> >>>
>> >>> --
>> >>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> >>> Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Fri Jan 30 19:10:57 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 12:10:57 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
Message-ID: <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>

No, it returns NULL if malloc gives it a raw pointer. It unconditionally
checks the length of the (possibly null) string argument first.

Please try the patch I attached in the last email. It *might* fix the
issue. Android has crappy locale handling.

On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <chaselton at gmail.com> wrote:

>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> currently returning NULL when given a null pointer.
>
> From obmalloc.c
>
> _PyMem_RawStrdup(const char *str)
> {
>     size_t size;
> char *copy;
>     size = strlen(str) + 1;
>     copy = PyMem_RawMalloc(size);
>     if (copy == NULL)
>         return NULL;
> memcpy(copy, str, size);
> return copy;
>           }
>
> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
> when
> > calling strlen. It's that whatever is calling it is likely asking it to
> > duplicate a null pointer. Basically, it's probably the caller's fault.
> >
> > You could always try modifying _PyMem_RawStrdup to return NULL when
> given a
> > null pointer and see where it then segfaults.
> >
> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >> Alternatively, is there a hassle-free way to find out what changed in
> >> obmalloc.c between 2.7.x and 3.4.x?
> >>
> >>
> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >> > There's a related strdup patch for readline.c, mentioned
> >> > here:http://bugs.python.org/issue21390 and here
> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
> >> > (I think) the functions all belong to Python...they're all prefixed
> >> > with _PyXx
> >> >
> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
> >> > wrote:
> >> >> Absolutely.  Good thing I have addr2line on device
> >> >>
> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> /lib/libpython3.4m.so.1.0
> >> >> 0008bbc8
> >> >> _PyMem_RawStrdup
> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> /bld/python/Python-3.4.2 $
> >> >>
> >> >>
> >> >>
> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> >> >>> Could you try the steps at
> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >>> allow you to get a better idea of where libc is crashing.
> >> >>>
> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
> >> >>>>
> >> >>>> Managed to get this out of logcat:
> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
> >> >>>> 11914 (python)  (libc)
> >> >>>>
> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
> >> >>>> (python)
> >> >>>>
> >> >>>> Less detail than strace but it seems to be that python is
> segfaulting
> >> >>>> libc...
> >> >>>>
> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
> >> >>>> wrote:
> >> >>>>>
> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> >>>>> <guido at python.org>
> >> >>>>> wrote:
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>  What I see in the strace:
> >> >>>>>>
> >> >>>>>>  ... load libpython3.4m.so.1.0
> >> >>>>>>  ... load libm
> >> >>>>>>  ... open /dev/__properties__ and do something to it
> >> >>>>>> (what?)
> >> >>>>>>  ... get current time
> >> >>>>>>  ... allocate memory
> >> >>>>>>  ... getuid
> >> >>>>>>  ... segfault
> >> >>>>>>
> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
> >> >>>>>> started to
> >> >>>>>>  load modules yet.
> >> >>>>>>
> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >>>>>  is the code that works with that file.
> >> >>>>>
> >> >>>>>  This explains it a bit (slides 24-29). Looks like something to do
> >> >>>>> with
> >> >>>>>  interprocess communication. Likely has nothing to do with Python
> >> >>>>> itself.
> >> >>>>>
> >> >>>>>  Maybe this would be useful?
> >> >>>>>
> >> >>>>>>
> >> >>>>>>  That stack trace would be really helpful.
> >> >>>>>>
> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
> >> >>>>>> <chaselton at gmail.com>
> >> >>>>>> wrote:
> >> >>>>>>>
> >> >>>>>>>
> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do have
> the
> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
> >> >>>>>>> though I
> >> >>>>>>>  am probably wrong.
> >> >>>>>>>  Attaching the strace output to this email.  I'm going to head
> >> >>>>>>> back to
> >> >>>>>>>  the documentation and to back out of some Android-related
> changes
> >> >>>>>>> in
> >> >>>>>>>  _localemodule.c
> >> >>>>>>>
> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >>>>>>> <guido at python.org>
> >> >>>>>>>  wrote:
> >> >>>>>>>>
> >> >>>>>>>>  There could be a million differences relevant (unicode, ints,
> >> >>>>>>>> ...).
> >> >>>>>>>>  Perhaps
> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic
> loading
> >> >>>>>>>> code
> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a
> syscall
> >> >>>>>>>> trace
> >> >>>>>>>>  --
> >> >>>>>>>>  also useful, but doesn't tell you precisely how
> >> >>>>>>>> it segfaulted.)
> >> >>>>>>>>
> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >>>>>>>> <chaselton at gmail.com>
> >> >>>>>>>>  wrote:
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>  All,
> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to
> Android
> >> >>>>>>>>> in
> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
> >> >>>>>>>>> configure
> >> >>>>>>>>>  options in the same environment, and modifying the setup.py
> as
> >> >>>>>>>>> needed,
> >> >>>>>>>>>  the newly built binary throws a segfault when the
> >> >>>>>>>>> generate-posix-vars
> >> >>>>>>>>>  portion of the build is reached...and when it is run as well
> >> >>>>>>>>> (i.e.
> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >> >>>>>>>>>
> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
> >> >>>>>>>>> reviewing
> >> >>>>>>>>> it.
> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
> >> >>>>>>>>> works but
> >> >>>>>>>>>  3.x throws a segfault?
> >> >>>>>>>>>
> >> >>>>>>>>>  Thanks in advance,
> >> >>>>>>>>>  Cyd
> >> >>>>>>>>> ________________________________
> >> >>>>>>>>>
> >> >>>>>>>>>  Python-Dev mailing list
> >> >>>>>>>>>
> >> >>>>>>>>> Python-Dev at python.org
> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >>>>>>>>>  Unsubscribe:
> >> >>>>>>>>>
> >> >>>>>>>>>
> >> >>>>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>
> >> >>>>>>>>  --
> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>
> >> >>>>>>  --
> >> >>>>>>  --Guido van Rossum (python.org/~guido)
> >> >>>>>>
> >> >>>>>> ________________________________
> >> >>>>>>
> >> >>>>>>  Python-Dev mailing list
> >> >>>>>>  Python-Dev at python.org
> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >>>>>>  Unsubscribe:
> >> >>>>>>
> >> >>>>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>
> >> >>>>>  --
> >> >>>>>  Ryan
> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >>>>> simple:
> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> >> >>>>> that was
> >> >>>>>  nul-terminated."
> >> >>>>>  Personal reality distortion fields are immune to contradictory
> >> >>>>> evidence.
> >> >>>>> -
> >> >>>>>  srean
> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
> >> >>>
> >> >>>
> >> >>> --
> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
> >> >>> Check out my website: http://kirbyfan64.github.io/
> >
> >
> >
> >
> > --
> > Ryan
> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> > nul-terminated."
> > Personal reality distortion fields are immune to contradictory evidence.
> -
> > srean
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/ff76261b/attachment.html>

From victor.stinner at gmail.com  Fri Jan 30 21:04:16 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 30 Jan 2015 21:04:16 +0100
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mPQ+Wn-cE96vBn+-X3tHvdVeKBxxOoWFKTsnP=onSXhKQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
 <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>
 <CAO41-mPQ+Wn-cE96vBn+-X3tHvdVeKBxxOoWFKTsnP=onSXhKQ@mail.gmail.com>
Message-ID: <CAMpsgwYG_EHnuJ2nVCw92SYRCd9AzPqsYxHvY8-UshsWdFixJw@mail.gmail.com>

Android provides a minimal support of locales. Most functions return a
fake result, do nothing. I'm not sure that it supports any codec. To
support Android, we may force UTF-8 for the filesystem encoding, as
done on Mac OS X.

Victor

2015-01-30 19:04 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
> No...
>
> ...but I think I found the issue with grep. Try applying the attached patch
> to the Python/frozenmain.c. It comments out the locale handling.
>
> It seems that Python always calls its strdup function on the locale string.
> On Android, this can apparently be null (as seen in the bug report you
> linked to).
>
> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> I don't have gdb on device; does the following tell you where Python's
>> strdup is called?
>>
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>
>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>> > the
>> > steps here.
>> >
>> > That way we can see where Python's own strdup function is getting
>> > called.
>> >
>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Absolutely.  Good thing I have addr2line on device
>> >>
>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>> >> 0008bbc8
>> >> _PyMem_RawStrdup
>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> /bld/python/Python-3.4.2 $
>> >>
>> >>
>> >>
>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> >> > Could you try the steps at
>> >> > http://stackoverflow.com/a/11369475/2097780?
>> >> > They
>> >> > allow you to get a better idea of where libc is crashing.
>> >> >
>> >> > Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >>
>> >> >> Managed to get this out of logcat:
>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >> 11914 (python)  (libc)
>> >> >>
>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>> >> >> (python)
>> >> >>
>> >> >> Less detail than strace but it seems to be that python is
>> >> >> segfaulting
>> >> >> libc...
>> >> >>
>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >>>
>> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >>> <guido at python.org>
>> >> >>> wrote:
>> >> >>>>
>> >> >>>>
>> >> >>>>  What I see in the strace:
>> >> >>>>
>> >> >>>>  ... load libpython3.4m.so.1.0
>> >> >>>>  ... load libm
>> >> >>>>  ... open /dev/__properties__ and do something to it
>> >> >>>> (what?)
>> >> >>>>  ... get current time
>> >> >>>>  ... allocate memory
>> >> >>>>  ... getuid
>> >> >>>>  ... segfault
>> >> >>>>
>> >> >>>>  That's not a lot to go on, but it doesn't look as if it has
>> >> >>>> started
>> >> >>>> to
>> >> >>>>  load modules yet.
>> >> >>>>
>> >> >>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >>>  is the code that works with that file.
>> >> >>>
>> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
>> >> >>> with
>> >> >>>  interprocess communication. Likely has nothing to do with Python
>> >> >>> itself.
>> >> >>>
>> >> >>>  Maybe this would be useful?
>> >> >>>
>> >> >>>>
>> >> >>>>  That stack trace would be really helpful.
>> >> >>>>
>> >> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >>>> <chaselton at gmail.com>
>> >> >>>> wrote:
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>  Apologies...I'm not sure what a stack track is, but I do have
>> >> >>>>> the
>> >> >>>>>  strace.  Nearest I can tell, it happens due to an open call,
>> >> >>>>> though
>> >> >>>>> I
>> >> >>>>>  am probably wrong.
>> >> >>>>>  Attaching the strace output to this email.  I'm going to head
>> >> >>>>> back
>> >> >>>>> to
>> >> >>>>>  the documentation and to back out of some Android-related
>> >> >>>>> changes
>> >> >>>>> in
>> >> >>>>>  _localemodule.c
>> >> >>>>>
>> >> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >>>>> <guido at python.org>
>> >> >>>>>  wrote:
>> >> >>>>>>
>> >> >>>>>>  There could be a million differences relevant (unicode, ints,
>> >> >>>>>> ...).
>> >> >>>>>>  Perhaps
>> >> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
>> >> >>>>>> code
>> >> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a
>> >> >>>>>> syscall
>> >> >>>>>> trace
>> >> >>>>>>  --
>> >> >>>>>>  also useful, but doesn't tell you precisely how
>> >> >>>>>> it segfaulted.)
>> >> >>>>>>
>> >> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >>>>>> <chaselton at gmail.com>
>> >> >>>>>>  wrote:
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>  All,
>> >> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
>> >> >>>>>>> in
>> >> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >> >>>>>>> configure
>> >> >>>>>>>  options in the same environment, and modifying the setup.py as
>> >> >>>>>>> needed,
>> >> >>>>>>>  the newly built binary throws a segfault when the
>> >> >>>>>>> generate-posix-vars
>> >> >>>>>>>  portion of the build is reached...and when it is run as well
>> >> >>>>>>> (i.e.
>> >> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >> >>>>>>>
>> >> >>>>>>>  I took a strace of ./python, however I'm a bit lost when
>> >> >>>>>>> reviewing
>> >> >>>>>>> it.
>> >> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>> >> >>>>>>> works
>> >> >>>>>>> but
>> >> >>>>>>>  3.x throws a segfault?
>> >> >>>>>>>
>> >> >>>>>>>  Thanks in advance,
>> >> >>>>>>>  Cyd
>> >> >>>>>>> ________________________________
>> >> >>>>>>>
>> >> >>>>>>>  Python-Dev mailing list
>> >> >>>>>>>
>> >> >>>>>>> Python-Dev at python.org
>> >> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>>>>  Unsubscribe:
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>  --
>> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>
>> >> >>>>  --
>> >> >>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>
>> >> >>>> ________________________________
>> >> >>>>
>> >> >>>>  Python-Dev mailing list
>> >> >>>>  Python-Dev at python.org
>> >> >>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>  Unsubscribe:
>> >> >>>>
>> >> >>>>
>> >> >>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>
>> >> >>>  --
>> >> >>>  Ryan
>> >> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >>> simple:
>> >> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >>> that
>> >> >>> was
>> >> >>>  nul-terminated."
>> >> >>>  Personal reality distortion fields are immune to contradictory
>> >> >>> evidence.
>> >> >>> -
>> >> >>>  srean
>> >> >>>  Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> > --
>> >> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>

From victor.stinner at gmail.com  Fri Jan 30 21:07:07 2015
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 30 Jan 2015 21:07:07 +0100
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAMpsgwYG_EHnuJ2nVCw92SYRCd9AzPqsYxHvY8-UshsWdFixJw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
 <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>
 <CAO41-mPQ+Wn-cE96vBn+-X3tHvdVeKBxxOoWFKTsnP=onSXhKQ@mail.gmail.com>
 <CAMpsgwYG_EHnuJ2nVCw92SYRCd9AzPqsYxHvY8-UshsWdFixJw@mail.gmail.com>
Message-ID: <CAMpsgwbBBKO79GNkDKk-o5Yc6eGPnC_hHkRYByrOnaKpFZbg5Q@mail.gmail.com>

Oh, I found my old patch to force UTF-8 on Android. I didn't test it:
see attached file.

It would be nice to start a wiki page to collect all informations on
the Python port to Android.

Victor

2015-01-30 21:04 GMT+01:00 Victor Stinner <victor.stinner at gmail.com>:
> Android provides a minimal support of locales. Most functions return a
> fake result, do nothing. I'm not sure that it supports any codec. To
> support Android, we may force UTF-8 for the filesystem encoding, as
> done on Mac OS X.
>
> Victor
>
> 2015-01-30 19:04 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
>> No...
>>
>> ...but I think I found the issue with grep. Try applying the attached patch
>> to the Python/frozenmain.c. It comments out the locale handling.
>>
>> It seems that Python always calls its strdup function on the locale string.
>> On Android, this can apparently be null (as seen in the bug report you
>> linked to).
>>
>> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> I don't have gdb on device; does the following tell you where Python's
>>> strdup is called?
>>>
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>
>>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>>> > the
>>> > steps here.
>>> >
>>> > That way we can see where Python's own strdup function is getting
>>> > called.
>>> >
>>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>>> > wrote:
>>> >>
>>> >> Absolutely.  Good thing I have addr2line on device
>>> >>
>>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>>> >> 0008bbc8
>>> >> _PyMem_RawStrdup
>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> /bld/python/Python-3.4.2 $
>>> >>
>>> >>
>>> >>
>>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>>> >> > Could you try the steps at
>>> >> > http://stackoverflow.com/a/11369475/2097780?
>>> >> > They
>>> >> > allow you to get a better idea of where libc is crashing.
>>> >> >
>>> >> > Cyd Haselton <chaselton at gmail.com> wrote:
>>> >> >>
>>> >> >> Managed to get this out of logcat:
>>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>>> >> >> 11914 (python)  (libc)
>>> >> >>
>>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>> >> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>>> >> >> (python)
>>> >> >>
>>> >> >> Less detail than strace but it seems to be that python is
>>> >> >> segfaulting
>>> >> >> libc...
>>> >> >>
>>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>>> >> >> wrote:
>>> >> >>>
>>> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>>> >> >>> <guido at python.org>
>>> >> >>> wrote:
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>  What I see in the strace:
>>> >> >>>>
>>> >> >>>>  ... load libpython3.4m.so.1.0
>>> >> >>>>  ... load libm
>>> >> >>>>  ... open /dev/__properties__ and do something to it
>>> >> >>>> (what?)
>>> >> >>>>  ... get current time
>>> >> >>>>  ... allocate memory
>>> >> >>>>  ... getuid
>>> >> >>>>  ... segfault
>>> >> >>>>
>>> >> >>>>  That's not a lot to go on, but it doesn't look as if it has
>>> >> >>>> started
>>> >> >>>> to
>>> >> >>>>  load modules yet.
>>> >> >>>>
>>> >> >>>>  Does /dev/__properties__ ring a bell? Not to me.
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>> >> >>>  is the code that works with that file.
>>> >> >>>
>>> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
>>> >> >>> with
>>> >> >>>  interprocess communication. Likely has nothing to do with Python
>>> >> >>> itself.
>>> >> >>>
>>> >> >>>  Maybe this would be useful?
>>> >> >>>
>>> >> >>>>
>>> >> >>>>  That stack trace would be really helpful.
>>> >> >>>>
>>> >> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>> >> >>>> <chaselton at gmail.com>
>>> >> >>>> wrote:
>>> >> >>>>>
>>> >> >>>>>
>>> >> >>>>>  Apologies...I'm not sure what a stack track is, but I do have
>>> >> >>>>> the
>>> >> >>>>>  strace.  Nearest I can tell, it happens due to an open call,
>>> >> >>>>> though
>>> >> >>>>> I
>>> >> >>>>>  am probably wrong.
>>> >> >>>>>  Attaching the strace output to this email.  I'm going to head
>>> >> >>>>> back
>>> >> >>>>> to
>>> >> >>>>>  the documentation and to back out of some Android-related
>>> >> >>>>> changes
>>> >> >>>>> in
>>> >> >>>>>  _localemodule.c
>>> >> >>>>>
>>> >> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>>> >> >>>>> <guido at python.org>
>>> >> >>>>>  wrote:
>>> >> >>>>>>
>>> >> >>>>>>  There could be a million differences relevant (unicode, ints,
>>> >> >>>>>> ...).
>>> >> >>>>>>  Perhaps
>>> >> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
>>> >> >>>>>> code
>>> >> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a
>>> >> >>>>>> syscall
>>> >> >>>>>> trace
>>> >> >>>>>>  --
>>> >> >>>>>>  also useful, but doesn't tell you precisely how
>>> >> >>>>>> it segfaulted.)
>>> >> >>>>>>
>>> >> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>>> >> >>>>>> <chaselton at gmail.com>
>>> >> >>>>>>  wrote:
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>  All,
>>> >> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
>>> >> >>>>>>> in
>>> >> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>> >> >>>>>>> configure
>>> >> >>>>>>>  options in the same environment, and modifying the setup.py as
>>> >> >>>>>>> needed,
>>> >> >>>>>>>  the newly built binary throws a segfault when the
>>> >> >>>>>>> generate-posix-vars
>>> >> >>>>>>>  portion of the build is reached...and when it is run as well
>>> >> >>>>>>> (i.e.
>>> >> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>> >> >>>>>>>
>>> >> >>>>>>>  I took a strace of ./python, however I'm a bit lost when
>>> >> >>>>>>> reviewing
>>> >> >>>>>>> it.
>>> >> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>>> >> >>>>>>> works
>>> >> >>>>>>> but
>>> >> >>>>>>>  3.x throws a segfault?
>>> >> >>>>>>>
>>> >> >>>>>>>  Thanks in advance,
>>> >> >>>>>>>  Cyd
>>> >> >>>>>>> ________________________________
>>> >> >>>>>>>
>>> >> >>>>>>>  Python-Dev mailing list
>>> >> >>>>>>>
>>> >> >>>>>>> Python-Dev at python.org
>>> >> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>> >> >>>>>>>  Unsubscribe:
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>>
>>> >> >>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>
>>> >> >>>>>>  --
>>> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>
>>> >> >>>>  --
>>> >> >>>>  --Guido van Rossum (python.org/~guido)
>>> >> >>>>
>>> >> >>>> ________________________________
>>> >> >>>>
>>> >> >>>>  Python-Dev mailing list
>>> >> >>>>  Python-Dev at python.org
>>> >> >>>>  https://mail.python.org/mailman/listinfo/python-dev
>>> >> >>>>  Unsubscribe:
>>> >> >>>>
>>> >> >>>>
>>> >> >>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>
>>> >> >>>  --
>>> >> >>>  Ryan
>>> >> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>>> >> >>> simple:
>>> >> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>> >> >>> that
>>> >> >>> was
>>> >> >>>  nul-terminated."
>>> >> >>>  Personal reality distortion fields are immune to contradictory
>>> >> >>> evidence.
>>> >> >>> -
>>> >> >>>  srean
>>> >> >>>  Check out my website: http://kirbyfan64.github.io/
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>> >> > Check out my website: http://kirbyfan64.github.io/
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Ryan
>>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>> > nul-terminated."
>>> > Personal reality distortion fields are immune to contradictory evidence.
>>> > -
>>> > srean
>>> > Check out my website: http://kirbyfan64.github.io/
>>
>>
>>
>>
>> --
>> Ryan
>> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> nul-terminated."
>> Personal reality distortion fields are immune to contradictory evidence. -
>> srean
>> Check out my website: http://kirbyfan64.github.io/
>>
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> https://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe:
>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: android_utf8-2.patch
Type: text/x-patch
Size: 4241 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/2bce669d/attachment-0001.bin>

From chaselton at gmail.com  Fri Jan 30 19:34:18 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 12:34:18 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
Message-ID: <CAHu5PrYnvYz00jtBHtn=LyLiW+r99S+tu79N1jt8_W15pAb=9Q@mail.gmail.com>

Ok...that makes sense.. apologies I do not do a lot of debugging. My
goal was to get Python (then spidermonkey) on my device then start
learning languages where i'd need to learn debugging.

Tried patch, see my reply, agree about Android's locale...at least
where native codeis concerned

On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> checks the length of the (possibly null) string argument first.
>
> Please try the patch I attached in the last email. It *might* fix the issue.
> Android has crappy locale handling.
>
> On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> currently returning NULL when given a null pointer.
>>
>> From obmalloc.c
>>
>> _PyMem_RawStrdup(const char *str)
>> {
>>     size_t size;
>> char *copy;
>>     size = strlen(str) + 1;
>>     copy = PyMem_RawMalloc(size);
>>     if (copy == NULL)
>>         return NULL;
>> memcpy(copy, str, size);
>> return copy;
>>           }
>>
>> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
>> > when
>> > calling strlen. It's that whatever is calling it is likely asking it to
>> > duplicate a null pointer. Basically, it's probably the caller's fault.
>> >
>> > You could always try modifying _PyMem_RawStrdup to return NULL when
>> > given a
>> > null pointer and see where it then segfaults.
>> >
>> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Alternatively, is there a hassle-free way to find out what changed in
>> >> obmalloc.c between 2.7.x and 3.4.x?
>> >>
>> >>
>> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com>
>> >> wrote:
>> >> > There's a related strdup patch for readline.c, mentioned
>> >> > here:http://bugs.python.org/issue21390 and here
>> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> >> > (I think) the functions all belong to Python...they're all prefixed
>> >> > with _PyXx
>> >> >
>> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >>
>> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> /lib/libpython3.4m.so.1.0
>> >> >> 0008bbc8
>> >> >> _PyMem_RawStrdup
>> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> /bld/python/Python-3.4.2 $
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> >> >>> Could you try the steps at
>> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >>>
>> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >>>>
>> >> >>>> Managed to get this out of logcat:
>> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >>>> 11914 (python)  (libc)
>> >> >>>>
>> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>> >> >>>> (python)
>> >> >>>>
>> >> >>>> Less detail than strace but it seems to be that python is
>> >> >>>> segfaulting
>> >> >>>> libc...
>> >> >>>>
>> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >>>> wrote:
>> >> >>>>>
>> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >>>>> <guido at python.org>
>> >> >>>>> wrote:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>  What I see in the strace:
>> >> >>>>>>
>> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >>>>>>  ... load libm
>> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >>>>>> (what?)
>> >> >>>>>>  ... get current time
>> >> >>>>>>  ... allocate memory
>> >> >>>>>>  ... getuid
>> >> >>>>>>  ... segfault
>> >> >>>>>>
>> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
>> >> >>>>>> started to
>> >> >>>>>>  load modules yet.
>> >> >>>>>>
>> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >>>>>  is the code that works with that file.
>> >> >>>>>
>> >> >>>>>  This explains it a bit (slides 24-29). Looks like something to
>> >> >>>>> do
>> >> >>>>> with
>> >> >>>>>  interprocess communication. Likely has nothing to do with Python
>> >> >>>>> itself.
>> >> >>>>>
>> >> >>>>>  Maybe this would be useful?
>> >> >>>>>
>> >> >>>>>>
>> >> >>>>>>  That stack trace would be really helpful.
>> >> >>>>>>
>> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >>>>>> <chaselton at gmail.com>
>> >> >>>>>> wrote:
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do have
>> >> >>>>>>> the
>> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
>> >> >>>>>>> though I
>> >> >>>>>>>  am probably wrong.
>> >> >>>>>>>  Attaching the strace output to this email.  I'm going to head
>> >> >>>>>>> back to
>> >> >>>>>>>  the documentation and to back out of some Android-related
>> >> >>>>>>> changes
>> >> >>>>>>> in
>> >> >>>>>>>  _localemodule.c
>> >> >>>>>>>
>> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >>>>>>> <guido at python.org>
>> >> >>>>>>>  wrote:
>> >> >>>>>>>>
>> >> >>>>>>>>  There could be a million differences relevant (unicode, ints,
>> >> >>>>>>>> ...).
>> >> >>>>>>>>  Perhaps
>> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic
>> >> >>>>>>>> loading
>> >> >>>>>>>> code
>> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a
>> >> >>>>>>>> syscall
>> >> >>>>>>>> trace
>> >> >>>>>>>>  --
>> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >>>>>>>> it segfaulted.)
>> >> >>>>>>>>
>> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >>>>>>>> <chaselton at gmail.com>
>> >> >>>>>>>>  wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>  All,
>> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to
>> >> >>>>>>>>> Android
>> >> >>>>>>>>> in
>> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >> >>>>>>>>> configure
>> >> >>>>>>>>>  options in the same environment, and modifying the setup.py
>> >> >>>>>>>>> as
>> >> >>>>>>>>> needed,
>> >> >>>>>>>>>  the newly built binary throws a segfault when the
>> >> >>>>>>>>> generate-posix-vars
>> >> >>>>>>>>>  portion of the build is reached...and when it is run as well
>> >> >>>>>>>>> (i.e.
>> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >> >>>>>>>>>
>> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
>> >> >>>>>>>>> reviewing
>> >> >>>>>>>>> it.
>> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>> >> >>>>>>>>> works but
>> >> >>>>>>>>>  3.x throws a segfault?
>> >> >>>>>>>>>
>> >> >>>>>>>>>  Thanks in advance,
>> >> >>>>>>>>>  Cyd
>> >> >>>>>>>>> ________________________________
>> >> >>>>>>>>>
>> >> >>>>>>>>>  Python-Dev mailing list
>> >> >>>>>>>>>
>> >> >>>>>>>>> Python-Dev at python.org
>> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>>>>>>  Unsubscribe:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>  --
>> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>  --
>> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>>>
>> >> >>>>>> ________________________________
>> >> >>>>>>
>> >> >>>>>>  Python-Dev mailing list
>> >> >>>>>>  Python-Dev at python.org
>> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>>>  Unsubscribe:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>  --
>> >> >>>>>  Ryan
>> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >>>>> simple:
>> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >>>>> that was
>> >> >>>>>  nul-terminated."
>> >> >>>>>  Personal reality distortion fields are immune to contradictory
>> >> >>>>> evidence.
>> >> >>>>> -
>> >> >>>>>  srean
>> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From chaselton at gmail.com  Fri Jan 30 19:58:28 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 12:58:28 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
Message-ID: <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>

Update:  I did try the patch after getting it installed correctly, but
I'm still getting a segfault on the newly built binary.
Will post info this afternoon.

On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> checks the length of the (possibly null) string argument first.
>
> Please try the patch I attached in the last email. It *might* fix the issue.
> Android has crappy locale handling.
>
> On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> currently returning NULL when given a null pointer.
>>
>> From obmalloc.c
>>
>> _PyMem_RawStrdup(const char *str)
>> {
>>     size_t size;
>> char *copy;
>>     size = strlen(str) + 1;
>>     copy = PyMem_RawMalloc(size);
>>     if (copy == NULL)
>>         return NULL;
>> memcpy(copy, str, size);
>> return copy;
>>           }
>>
>> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
>> > when
>> > calling strlen. It's that whatever is calling it is likely asking it to
>> > duplicate a null pointer. Basically, it's probably the caller's fault.
>> >
>> > You could always try modifying _PyMem_RawStrdup to return NULL when
>> > given a
>> > null pointer and see where it then segfaults.
>> >
>> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Alternatively, is there a hassle-free way to find out what changed in
>> >> obmalloc.c between 2.7.x and 3.4.x?
>> >>
>> >>
>> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com>
>> >> wrote:
>> >> > There's a related strdup patch for readline.c, mentioned
>> >> > here:http://bugs.python.org/issue21390 and here
>> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c, as
>> >> > (I think) the functions all belong to Python...they're all prefixed
>> >> > with _PyXx
>> >> >
>> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >>
>> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> /lib/libpython3.4m.so.1.0
>> >> >> 0008bbc8
>> >> >> _PyMem_RawStrdup
>> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> /bld/python/Python-3.4.2 $
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>> >> >>> Could you try the steps at
>> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >>>
>> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >>>>
>> >> >>>> Managed to get this out of logcat:
>> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >>>> 11914 (python)  (libc)
>> >> >>>>
>> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>> >> >>>> (python)
>> >> >>>>
>> >> >>>> Less detail than strace but it seems to be that python is
>> >> >>>> segfaulting
>> >> >>>> libc...
>> >> >>>>
>> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >>>> wrote:
>> >> >>>>>
>> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >>>>> <guido at python.org>
>> >> >>>>> wrote:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>  What I see in the strace:
>> >> >>>>>>
>> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >>>>>>  ... load libm
>> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >>>>>> (what?)
>> >> >>>>>>  ... get current time
>> >> >>>>>>  ... allocate memory
>> >> >>>>>>  ... getuid
>> >> >>>>>>  ... segfault
>> >> >>>>>>
>> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
>> >> >>>>>> started to
>> >> >>>>>>  load modules yet.
>> >> >>>>>>
>> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >>>>>  is the code that works with that file.
>> >> >>>>>
>> >> >>>>>  This explains it a bit (slides 24-29). Looks like something to
>> >> >>>>> do
>> >> >>>>> with
>> >> >>>>>  interprocess communication. Likely has nothing to do with Python
>> >> >>>>> itself.
>> >> >>>>>
>> >> >>>>>  Maybe this would be useful?
>> >> >>>>>
>> >> >>>>>>
>> >> >>>>>>  That stack trace would be really helpful.
>> >> >>>>>>
>> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >>>>>> <chaselton at gmail.com>
>> >> >>>>>> wrote:
>> >> >>>>>>>
>> >> >>>>>>>
>> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do have
>> >> >>>>>>> the
>> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
>> >> >>>>>>> though I
>> >> >>>>>>>  am probably wrong.
>> >> >>>>>>>  Attaching the strace output to this email.  I'm going to head
>> >> >>>>>>> back to
>> >> >>>>>>>  the documentation and to back out of some Android-related
>> >> >>>>>>> changes
>> >> >>>>>>> in
>> >> >>>>>>>  _localemodule.c
>> >> >>>>>>>
>> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >>>>>>> <guido at python.org>
>> >> >>>>>>>  wrote:
>> >> >>>>>>>>
>> >> >>>>>>>>  There could be a million differences relevant (unicode, ints,
>> >> >>>>>>>> ...).
>> >> >>>>>>>>  Perhaps
>> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic
>> >> >>>>>>>> loading
>> >> >>>>>>>> code
>> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a
>> >> >>>>>>>> syscall
>> >> >>>>>>>> trace
>> >> >>>>>>>>  --
>> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >>>>>>>> it segfaulted.)
>> >> >>>>>>>>
>> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >>>>>>>> <chaselton at gmail.com>
>> >> >>>>>>>>  wrote:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>  All,
>> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to
>> >> >>>>>>>>> Android
>> >> >>>>>>>>> in
>> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>> >> >>>>>>>>> configure
>> >> >>>>>>>>>  options in the same environment, and modifying the setup.py
>> >> >>>>>>>>> as
>> >> >>>>>>>>> needed,
>> >> >>>>>>>>>  the newly built binary throws a segfault when the
>> >> >>>>>>>>> generate-posix-vars
>> >> >>>>>>>>>  portion of the build is reached...and when it is run as well
>> >> >>>>>>>>> (i.e.
>> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>> >> >>>>>>>>>
>> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
>> >> >>>>>>>>> reviewing
>> >> >>>>>>>>> it.
>> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>> >> >>>>>>>>> works but
>> >> >>>>>>>>>  3.x throws a segfault?
>> >> >>>>>>>>>
>> >> >>>>>>>>>  Thanks in advance,
>> >> >>>>>>>>>  Cyd
>> >> >>>>>>>>> ________________________________
>> >> >>>>>>>>>
>> >> >>>>>>>>>  Python-Dev mailing list
>> >> >>>>>>>>>
>> >> >>>>>>>>> Python-Dev at python.org
>> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>>>>>>  Unsubscribe:
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>>
>> >> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>
>> >> >>>>>>>>  --
>> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>>  --
>> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >>>>>>
>> >> >>>>>> ________________________________
>> >> >>>>>>
>> >> >>>>>>  Python-Dev mailing list
>> >> >>>>>>  Python-Dev at python.org
>> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >>>>>>  Unsubscribe:
>> >> >>>>>>
>> >> >>>>>>
>> >> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>
>> >> >>>>>  --
>> >> >>>>>  Ryan
>> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >>>>> simple:
>> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >>>>> that was
>> >> >>>>>  nul-terminated."
>> >> >>>>>  Personal reality distortion fields are immune to contradictory
>> >> >>>>> evidence.
>> >> >>>>> -
>> >> >>>>>  srean
>> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >>>
>> >> >>>
>> >> >>> --
>> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From chaselton at gmail.com  Fri Jan 30 21:49:30 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 14:49:30 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAMpsgwbBBKO79GNkDKk-o5Yc6eGPnC_hHkRYByrOnaKpFZbg5Q@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAO41-mP09ao5Z3qzfcv-bj5HDXGDz3sM-=sXVd10KQJ3dXnWLQ@mail.gmail.com>
 <CAHu5PrYgFqaoK=icxGo7j+pPYQYvbfi9bjV4ghsdORfPX6qzAg@mail.gmail.com>
 <CAO41-mPQ+Wn-cE96vBn+-X3tHvdVeKBxxOoWFKTsnP=onSXhKQ@mail.gmail.com>
 <CAMpsgwYG_EHnuJ2nVCw92SYRCd9AzPqsYxHvY8-UshsWdFixJw@mail.gmail.com>
 <CAMpsgwbBBKO79GNkDKk-o5Yc6eGPnC_hHkRYByrOnaKpFZbg5Q@mail.gmail.com>
Message-ID: <CAHu5Pram20+Nre5jDks5iPjVfyPo3quJBB3n1zCKQZJF6bVrNQ@mail.gmail.com>

I've got a page of all of the modifications to source files I've used
in my attempts to get 3.4.2 working...I can post it somewhere if the
port ever succeeds.

On Fri, Jan 30, 2015 at 2:07 PM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Oh, I found my old patch to force UTF-8 on Android. I didn't test it:
> see attached file.
>
> It would be nice to start a wiki page to collect all informations on
> the Python port to Android.
>
> Victor
>
> 2015-01-30 21:04 GMT+01:00 Victor Stinner <victor.stinner at gmail.com>:
>> Android provides a minimal support of locales. Most functions return a
>> fake result, do nothing. I'm not sure that it supports any codec. To
>> support Android, we may force UTF-8 for the filesystem encoding, as
>> done on Mac OS X.
>>
>> Victor
>>
>> 2015-01-30 19:04 GMT+01:00 Ryan Gonzalez <rymg19 at gmail.com>:
>>> No...
>>>
>>> ...but I think I found the issue with grep. Try applying the attached patch
>>> to the Python/frozenmain.c. It comments out the locale handling.
>>>
>>> It seems that Python always calls its strdup function on the locale string.
>>> On Android, this can apparently be null (as seen in the bug report you
>>> linked to).
>>>
>>> On Fri, Jan 30, 2015 at 12:00 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>>>
>>>> I don't have gdb on device; does the following tell you where Python's
>>>> strdup is called?
>>>>
>>>> >> _PyMem_RawStrdup
>>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>>
>>>> On Fri, Jan 30, 2015 at 11:52 AM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>>>> > Is it possible at all to get a stack trace of the crash using gdb? Try
>>>> > the
>>>> > steps here.
>>>> >
>>>> > That way we can see where Python's own strdup function is getting
>>>> > called.
>>>> >
>>>> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com>
>>>> > wrote:
>>>> >>
>>>> >> Absolutely.  Good thing I have addr2line on device
>>>> >>
>>>> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e /lib/libpython3.4m.so.1.0
>>>> >> 0008bbc8
>>>> >> _PyMem_RawStrdup
>>>> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>> >> /bld/python/Python-3.4.2 $
>>>> >>
>>>> >>
>>>> >>
>>>> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
>>>> >> > Could you try the steps at
>>>> >> > http://stackoverflow.com/a/11369475/2097780?
>>>> >> > They
>>>> >> > allow you to get a better idea of where libc is crashing.
>>>> >> >
>>>> >> > Cyd Haselton <chaselton at gmail.com> wrote:
>>>> >> >>
>>>> >> >> Managed to get this out of logcat:
>>>> >> >> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>>>> >> >> 11914 (python)  (libc)
>>>> >> >>
>>>> >> >> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>> >> >> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
>>>> >> >> (python)
>>>> >> >>
>>>> >> >> Less detail than strace but it seems to be that python is
>>>> >> >> segfaulting
>>>> >> >> libc...
>>>> >> >>
>>>> >> >> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <rymg19 at gmail.com>
>>>> >> >> wrote:
>>>> >> >>>
>>>> >> >>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>>>> >> >>> <guido at python.org>
>>>> >> >>> wrote:
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>  What I see in the strace:
>>>> >> >>>>
>>>> >> >>>>  ... load libpython3.4m.so.1.0
>>>> >> >>>>  ... load libm
>>>> >> >>>>  ... open /dev/__properties__ and do something to it
>>>> >> >>>> (what?)
>>>> >> >>>>  ... get current time
>>>> >> >>>>  ... allocate memory
>>>> >> >>>>  ... getuid
>>>> >> >>>>  ... segfault
>>>> >> >>>>
>>>> >> >>>>  That's not a lot to go on, but it doesn't look as if it has
>>>> >> >>>> started
>>>> >> >>>> to
>>>> >> >>>>  load modules yet.
>>>> >> >>>>
>>>> >> >>>>  Does /dev/__properties__ ring a bell? Not to me.
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>> >> >>>  is the code that works with that file.
>>>> >> >>>
>>>> >> >>>  This explains it a bit (slides 24-29). Looks like something to do
>>>> >> >>> with
>>>> >> >>>  interprocess communication. Likely has nothing to do with Python
>>>> >> >>> itself.
>>>> >> >>>
>>>> >> >>>  Maybe this would be useful?
>>>> >> >>>
>>>> >> >>>>
>>>> >> >>>>  That stack trace would be really helpful.
>>>> >> >>>>
>>>> >> >>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>>> >> >>>> <chaselton at gmail.com>
>>>> >> >>>> wrote:
>>>> >> >>>>>
>>>> >> >>>>>
>>>> >> >>>>>  Apologies...I'm not sure what a stack track is, but I do have
>>>> >> >>>>> the
>>>> >> >>>>>  strace.  Nearest I can tell, it happens due to an open call,
>>>> >> >>>>> though
>>>> >> >>>>> I
>>>> >> >>>>>  am probably wrong.
>>>> >> >>>>>  Attaching the strace output to this email.  I'm going to head
>>>> >> >>>>> back
>>>> >> >>>>> to
>>>> >> >>>>>  the documentation and to back out of some Android-related
>>>> >> >>>>> changes
>>>> >> >>>>> in
>>>> >> >>>>>  _localemodule.c
>>>> >> >>>>>
>>>> >> >>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>>>> >> >>>>> <guido at python.org>
>>>> >> >>>>>  wrote:
>>>> >> >>>>>>
>>>> >> >>>>>>  There could be a million differences relevant (unicode, ints,
>>>> >> >>>>>> ...).
>>>> >> >>>>>>  Perhaps
>>>> >> >>>>>>  the importlib bootstrap is failing. Perhaps the dynamic loading
>>>> >> >>>>>> code
>>>> >> >>>>>>  changed. Did you get a stack track? (IIRC strace shows a
>>>> >> >>>>>> syscall
>>>> >> >>>>>> trace
>>>> >> >>>>>>  --
>>>> >> >>>>>>  also useful, but doesn't tell you precisely how
>>>> >> >>>>>> it segfaulted.)
>>>> >> >>>>>>
>>>> >> >>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>>>> >> >>>>>> <chaselton at gmail.com>
>>>> >> >>>>>>  wrote:
>>>> >> >>>>>>>
>>>> >> >>>>>>>
>>>> >> >>>>>>>  All,
>>>> >> >>>>>>>  I recently ditched my attempts to port Python 2.7.8 to Android
>>>> >> >>>>>>> in
>>>> >> >>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the same
>>>> >> >>>>>>> configure
>>>> >> >>>>>>>  options in the same environment, and modifying the setup.py as
>>>> >> >>>>>>> needed,
>>>> >> >>>>>>>  the newly built binary throws a segfault when the
>>>> >> >>>>>>> generate-posix-vars
>>>> >> >>>>>>>  portion of the build is reached...and when it is run as well
>>>> >> >>>>>>> (i.e.
>>>> >> >>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
>>>> >> >>>>>>>
>>>> >> >>>>>>>  I took a strace of ./python, however I'm a bit lost when
>>>> >> >>>>>>> reviewing
>>>> >> >>>>>>> it.
>>>> >> >>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
>>>> >> >>>>>>> works
>>>> >> >>>>>>> but
>>>> >> >>>>>>>  3.x throws a segfault?
>>>> >> >>>>>>>
>>>> >> >>>>>>>  Thanks in advance,
>>>> >> >>>>>>>  Cyd
>>>> >> >>>>>>> ________________________________
>>>> >> >>>>>>>
>>>> >> >>>>>>>  Python-Dev mailing list
>>>> >> >>>>>>>
>>>> >> >>>>>>> Python-Dev at python.org
>>>> >> >>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>> >> >>>>>>>  Unsubscribe:
>>>> >> >>>>>>>
>>>> >> >>>>>>>
>>>> >> >>>>>>>
>>>> >> >>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>
>>>> >> >>>>>>  --
>>>> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>>  --
>>>> >> >>>>  --Guido van Rossum (python.org/~guido)
>>>> >> >>>>
>>>> >> >>>> ________________________________
>>>> >> >>>>
>>>> >> >>>>  Python-Dev mailing list
>>>> >> >>>>  Python-Dev at python.org
>>>> >> >>>>  https://mail.python.org/mailman/listinfo/python-dev
>>>> >> >>>>  Unsubscribe:
>>>> >> >>>>
>>>> >> >>>>
>>>> >> >>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>
>>>> >> >>>  --
>>>> >> >>>  Ryan
>>>> >> >>>  If anybody ever asks me why I prefer C++ to C, my answer will be
>>>> >> >>> simple:
>>>> >> >>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>> >> >>> that
>>>> >> >>> was
>>>> >> >>>  nul-terminated."
>>>> >> >>>  Personal reality distortion fields are immune to contradictory
>>>> >> >>> evidence.
>>>> >> >>> -
>>>> >> >>>  srean
>>>> >> >>>  Check out my website: http://kirbyfan64.github.io/
>>>> >> >
>>>> >> >
>>>> >> > --
>>>> >> > Sent from my Android phone with K-9 Mail. Please excuse my brevity.
>>>> >> > Check out my website: http://kirbyfan64.github.io/
>>>> >
>>>> >
>>>> >
>>>> >
>>>> > --
>>>> > Ryan
>>>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>>> > nul-terminated."
>>>> > Personal reality distortion fields are immune to contradictory evidence.
>>>> > -
>>>> > srean
>>>> > Check out my website: http://kirbyfan64.github.io/
>>>
>>>
>>>
>>>
>>> --
>>> Ryan
>>> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>>> nul-terminated."
>>> Personal reality distortion fields are immune to contradictory evidence. -
>>> srean
>>> Check out my website: http://kirbyfan64.github.io/
>>>
>>> _______________________________________________
>>> Python-Dev mailing list
>>> Python-Dev at python.org
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> Unsubscribe:
>>> https://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com
>>>

From chaselton at gmail.com  Fri Jan 30 21:50:39 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 14:50:39 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
Message-ID: <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>

Unfortunately it is still reporting the same function :-/.

On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Yes...
>
> Can you check if it's crashing in a different function now?
>
> On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Yes I did.  I did have to enter all the information in manually...I'm
>> not familiar with automated patch application tools and even if I
>> were, I'm pretty sure I wouldn't have them on my device.
>>
>> Just so that I'm absolutely sure I got everything correct...you wanted
>> all of the lines in the patch commented out, correct?  Basically
>> everything referencing oldloc?
>>
>> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Are you sure the patch was applied correctly? I was SO sure it would
>> > work!
>> >
>> > FYI, you tried the patch I attached to the email message, right?
>> >
>> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Update:  I did try the patch after getting it installed correctly, but
>> >> I'm still getting a segfault on the newly built binary.
>> >> Will post info this afternoon.
>> >>
>> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> wrote:
>> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> > unconditionally
>> >> > checks the length of the (possibly null) string argument first.
>> >> >
>> >> > Please try the patch I attached in the last email. It *might* fix the
>> >> > issue.
>> >> > Android has crappy locale handling.
>> >> >
>> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> >> >> currently returning NULL when given a null pointer.
>> >> >>
>> >> >> From obmalloc.c
>> >> >>
>> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> {
>> >> >>     size_t size;
>> >> >> char *copy;
>> >> >>     size = strlen(str) + 1;
>> >> >>     copy = PyMem_RawMalloc(size);
>> >> >>     if (copy == NULL)
>> >> >>         return NULL;
>> >> >> memcpy(copy, str, size);
>> >> >> return copy;
>> >> >>           }
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
>> >> >> > crashes
>> >> >> > when
>> >> >> > calling strlen. It's that whatever is calling it is likely asking
>> >> >> > it
>> >> >> > to
>> >> >> > duplicate a null pointer. Basically, it's probably the caller's
>> >> >> > fault.
>> >> >> >
>> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
>> >> >> > when
>> >> >> > given a
>> >> >> > null pointer and see where it then segfaults.
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> > <chaselton at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> changed
>> >> >> >> in
>> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >>
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> <chaselton at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> > obmalloc.c,
>> >> >> >> > as
>> >> >> >> > (I think) the functions all belong to Python...they're all
>> >> >> >> > prefixed
>> >> >> >> > with _PyXx
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> > <chaselton at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >> >> >>
>> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> 0008bbc8
>> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >>> Could you try the steps at
>> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >> >> >>>
>> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >> >> >>>>
>> >> >> >> >>>> Managed to get this out of logcat:
>> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
>> >> >> >> >>>> thread
>> >> >> >> >>>> 11914 (python)  (libc)
>> >> >> >> >>>>
>> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >> >> >>>> 23373
>> >> >> >> >>>> (python)
>> >> >> >> >>>>
>> >> >> >> >>>> Less detail than strace but it seems to be that python is
>> >> >> >> >>>> segfaulting
>> >> >> >> >>>> libc...
>> >> >> >> >>>>
>> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >> >>>> <rymg19 at gmail.com>
>> >> >> >> >>>> wrote:
>> >> >> >> >>>>>
>> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >> >> >>>>> <guido at python.org>
>> >> >> >> >>>>> wrote:
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  What I see in the strace:
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >> >> >>>>>>  ... load libm
>> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >> >> >>>>>> (what?)
>> >> >> >> >>>>>>  ... get current time
>> >> >> >> >>>>>>  ... allocate memory
>> >> >> >> >>>>>>  ... getuid
>> >> >> >> >>>>>>  ... segfault
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it
>> >> >> >> >>>>>> has
>> >> >> >> >>>>>> started to
>> >> >> >> >>>>>>  load modules yet.
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >> >> >>>>>  is the code that works with that file.
>> >> >> >> >>>>>
>> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
>> >> >> >> >>>>> something
>> >> >> >> >>>>> to
>> >> >> >> >>>>> do
>> >> >> >> >>>>> with
>> >> >> >> >>>>>  interprocess communication. Likely has nothing to do with
>> >> >> >> >>>>> Python
>> >> >> >> >>>>> itself.
>> >> >> >> >>>>>
>> >> >> >> >>>>>  Maybe this would be useful?
>> >> >> >> >>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  That stack trace would be really helpful.
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> >> >>>>>> <chaselton at gmail.com>
>> >> >> >> >>>>>> wrote:
>> >> >> >> >>>>>>>
>> >> >> >> >>>>>>>
>> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do
>> >> >> >> >>>>>>> have
>> >> >> >> >>>>>>> the
>> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open
>> >> >> >> >>>>>>> call,
>> >> >> >> >>>>>>> though I
>> >> >> >> >>>>>>>  am probably wrong.
>> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm going to
>> >> >> >> >>>>>>> head
>> >> >> >> >>>>>>> back to
>> >> >> >> >>>>>>>  the documentation and to back out of some
>> >> >> >> >>>>>>> Android-related
>> >> >> >> >>>>>>> changes
>> >> >> >> >>>>>>> in
>> >> >> >> >>>>>>>  _localemodule.c
>> >> >> >> >>>>>>>
>> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >> >> >>>>>>> <guido at python.org>
>> >> >> >> >>>>>>>  wrote:
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>  There could be a million differences relevant (unicode,
>> >> >> >> >>>>>>>> ints,
>> >> >> >> >>>>>>>> ...).
>> >> >> >> >>>>>>>>  Perhaps
>> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic
>> >> >> >> >>>>>>>> loading
>> >> >> >> >>>>>>>> code
>> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows
>> >> >> >> >>>>>>>> a
>> >> >> >> >>>>>>>> syscall
>> >> >> >> >>>>>>>> trace
>> >> >> >> >>>>>>>>  --
>> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >> >> >>>>>>>> it segfaulted.)
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> >> >>>>>>>> <chaselton at gmail.com>
>> >> >> >> >>>>>>>>  wrote:
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>  All,
>> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to
>> >> >> >> >>>>>>>>> Android
>> >> >> >> >>>>>>>>> in
>> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the
>> >> >> >> >>>>>>>>> same
>> >> >> >> >>>>>>>>> configure
>> >> >> >> >>>>>>>>>  options in the same environment, and modifying the
>> >> >> >> >>>>>>>>> setup.py
>> >> >> >> >>>>>>>>> as
>> >> >> >> >>>>>>>>> needed,
>> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when the
>> >> >> >> >>>>>>>>> generate-posix-vars
>> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is run
>> >> >> >> >>>>>>>>> as
>> >> >> >> >>>>>>>>> well
>> >> >> >> >>>>>>>>> (i.e.
>> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
>> >> >> >> >>>>>>>>> similar)
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost
>> >> >> >> >>>>>>>>> when
>> >> >> >> >>>>>>>>> reviewing
>> >> >> >> >>>>>>>>> it.
>> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python
>> >> >> >> >>>>>>>>> 2.7
>> >> >> >> >>>>>>>>> works but
>> >> >> >> >>>>>>>>>  3.x throws a segfault?
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>  Thanks in advance,
>> >> >> >> >>>>>>>>>  Cyd
>> >> >> >> >>>>>>>>> ________________________________
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>  Python-Dev mailing list
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>> Python-Dev at python.org
>> >> >> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >>>>>>>>>  Unsubscribe:
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>>
>> >> >> >> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>
>> >> >> >> >>>>>>>>  --
>> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  --
>> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >>>>>>
>> >> >> >> >>>>>> ________________________________
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>  Python-Dev mailing list
>> >> >> >> >>>>>>  Python-Dev at python.org
>> >> >> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >>>>>>  Unsubscribe:
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>>
>> >> >> >> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>
>> >> >> >> >>>>>  --
>> >> >> >> >>>>>  Ryan
>> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer
>> >> >> >> >>>>> will
>> >> >> >> >>>>> be
>> >> >> >> >>>>> simple:
>> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>> >> >> >> >>>>> think
>> >> >> >> >>>>> that was
>> >> >> >> >>>>>  nul-terminated."
>> >> >> >> >>>>>  Personal reality distortion fields are immune to
>> >> >> >> >>>>> contradictory
>> >> >> >> >>>>> evidence.
>> >> >> >> >>>>> -
>> >> >> >> >>>>>  srean
>> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >>>
>> >> >> >> >>>
>> >> >> >> >>> --
>> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my
>> >> >> >> >>> brevity.
>> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Ryan
>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >> > simple:
>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> > that
>> >> >> > was
>> >> >> > nul-terminated."
>> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> > evidence.
>> >> >> > -
>> >> >> > srean
>> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Ryan
>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> > simple:
>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> >> > was
>> >> > nul-terminated."
>> >> > Personal reality distortion fields are immune to contradictory
>> >> > evidence.
>> >> > -
>> >> > srean
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Fri Jan 30 20:00:05 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 13:00:05 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
Message-ID: <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>

Are you sure the patch was applied correctly? I was SO sure it would work!

FYI, you tried the patch I attached to the email message, right?

On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> Update:  I did try the patch after getting it installed correctly, but
> I'm still getting a segfault on the newly built binary.
> Will post info this afternoon.
>
> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > No, it returns NULL if malloc gives it a raw pointer. It unconditionally
> > checks the length of the (possibly null) string argument first.
> >
> > Please try the patch I attached in the last email. It *might* fix the
> issue.
> > Android has crappy locale handling.
> >
> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> >> currently returning NULL when given a null pointer.
> >>
> >> From obmalloc.c
> >>
> >> _PyMem_RawStrdup(const char *str)
> >> {
> >>     size_t size;
> >> char *copy;
> >>     size = strlen(str) + 1;
> >>     copy = PyMem_RawMalloc(size);
> >>     if (copy == NULL)
> >>         return NULL;
> >> memcpy(copy, str, size);
> >> return copy;
> >>           }
> >>
> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com>
> wrote:
> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup crashes
> >> > when
> >> > calling strlen. It's that whatever is calling it is likely asking it
> to
> >> > duplicate a null pointer. Basically, it's probably the caller's fault.
> >> >
> >> > You could always try modifying _PyMem_RawStrdup to return NULL when
> >> > given a
> >> > null pointer and see where it then segfaults.
> >> >
> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton <chaselton at gmail.com>
> >> > wrote:
> >> >>
> >> >> Alternatively, is there a hassle-free way to find out what changed in
> >> >> obmalloc.c between 2.7.x and 3.4.x?
> >> >>
> >> >>
> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton <chaselton at gmail.com>
> >> >> wrote:
> >> >> > There's a related strdup patch for readline.c, mentioned
> >> >> > here:http://bugs.python.org/issue21390 and here
> >> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> >> > There's a patch, but I'm not sure how to modify it for obmalloc.c,
> as
> >> >> > (I think) the functions all belong to Python...they're all prefixed
> >> >> > with _PyXx
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton <chaselton at gmail.com
> >
> >> >> > wrote:
> >> >> >> Absolutely.  Good thing I have addr2line on device
> >> >> >>
> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> >> >> >> /lib/libpython3.4m.so.1.0
> >> >> >> 0008bbc8
> >> >> >> _PyMem_RawStrdup
> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> >> /bld/python/Python-3.4.2 $
> >> >> >>
> >> >> >>
> >> >> >>
> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com> wrote:
> >> >> >>> Could you try the steps at
> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >> >>> allow you to get a better idea of where libc is crashing.
> >> >> >>>
> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
> >> >> >>>>
> >> >> >>>> Managed to get this out of logcat:
> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
> thread
> >> >> >>>> 11914 (python)  (libc)
> >> >> >>>>
> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread 23373
> >> >> >>>> (python)
> >> >> >>>>
> >> >> >>>> Less detail than strace but it seems to be that python is
> >> >> >>>> segfaulting
> >> >> >>>> libc...
> >> >> >>>>
> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez <
> rymg19 at gmail.com>
> >> >> >>>> wrote:
> >> >> >>>>>
> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> >> >>>>> <guido at python.org>
> >> >> >>>>> wrote:
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>  What I see in the strace:
> >> >> >>>>>>
> >> >> >>>>>>  ... load libpython3.4m.so.1.0
> >> >> >>>>>>  ... load libm
> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
> >> >> >>>>>> (what?)
> >> >> >>>>>>  ... get current time
> >> >> >>>>>>  ... allocate memory
> >> >> >>>>>>  ... getuid
> >> >> >>>>>>  ... segfault
> >> >> >>>>>>
> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it has
> >> >> >>>>>> started to
> >> >> >>>>>>  load modules yet.
> >> >> >>>>>>
> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >> >>>>>  is the code that works with that file.
> >> >> >>>>>
> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like something to
> >> >> >>>>> do
> >> >> >>>>> with
> >> >> >>>>>  interprocess communication. Likely has nothing to do with
> Python
> >> >> >>>>> itself.
> >> >> >>>>>
> >> >> >>>>>  Maybe this would be useful?
> >> >> >>>>>
> >> >> >>>>>>
> >> >> >>>>>>  That stack trace would be really helpful.
> >> >> >>>>>>
> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
> >> >> >>>>>> <chaselton at gmail.com>
> >> >> >>>>>> wrote:
> >> >> >>>>>>>
> >> >> >>>>>>>
> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I do
> have
> >> >> >>>>>>> the
> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open call,
> >> >> >>>>>>> though I
> >> >> >>>>>>>  am probably wrong.
> >> >> >>>>>>>  Attaching the strace output to this email.  I'm going to
> head
> >> >> >>>>>>> back to
> >> >> >>>>>>>  the documentation and to back out of some Android-related
> >> >> >>>>>>> changes
> >> >> >>>>>>> in
> >> >> >>>>>>>  _localemodule.c
> >> >> >>>>>>>
> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >> >>>>>>> <guido at python.org>
> >> >> >>>>>>>  wrote:
> >> >> >>>>>>>>
> >> >> >>>>>>>>  There could be a million differences relevant (unicode,
> ints,
> >> >> >>>>>>>> ...).
> >> >> >>>>>>>>  Perhaps
> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the dynamic
> >> >> >>>>>>>> loading
> >> >> >>>>>>>> code
> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace shows a
> >> >> >>>>>>>> syscall
> >> >> >>>>>>>> trace
> >> >> >>>>>>>>  --
> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
> >> >> >>>>>>>> it segfaulted.)
> >> >> >>>>>>>>
> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >> >>>>>>>> <chaselton at gmail.com>
> >> >> >>>>>>>>  wrote:
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>  All,
> >> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8 to
> >> >> >>>>>>>>> Android
> >> >> >>>>>>>>> in
> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using the
> same
> >> >> >>>>>>>>> configure
> >> >> >>>>>>>>>  options in the same environment, and modifying the
> setup.py
> >> >> >>>>>>>>> as
> >> >> >>>>>>>>> needed,
> >> >> >>>>>>>>>  the newly built binary throws a segfault when the
> >> >> >>>>>>>>> generate-posix-vars
> >> >> >>>>>>>>>  portion of the build is reached...and when it is run as
> well
> >> >> >>>>>>>>> (i.e.
> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or similar)
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost when
> >> >> >>>>>>>>> reviewing
> >> >> >>>>>>>>> it.
> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why Python 2.7
> >> >> >>>>>>>>> works but
> >> >> >>>>>>>>>  3.x throws a segfault?
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>  Thanks in advance,
> >> >> >>>>>>>>>  Cyd
> >> >> >>>>>>>>> ________________________________
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>  Python-Dev mailing list
> >> >> >>>>>>>>>
> >> >> >>>>>>>>> Python-Dev at python.org
> >> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >> >>>>>>>>>  Unsubscribe:
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> >> >> >>>>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>
> >> >> >>>>>>>>  --
> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>  --
> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >>>>>>
> >> >> >>>>>> ________________________________
> >> >> >>>>>>
> >> >> >>>>>>  Python-Dev mailing list
> >> >> >>>>>>  Python-Dev at python.org
> >> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >> >>>>>>  Unsubscribe:
> >> >> >>>>>>
> >> >> >>>>>>
> >> >> >>>>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>
> >> >> >>>>>  --
> >> >> >>>>>  Ryan
> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer will
> be
> >> >> >>>>> simple:
> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
> think
> >> >> >>>>> that was
> >> >> >>>>>  nul-terminated."
> >> >> >>>>>  Personal reality distortion fields are immune to contradictory
> >> >> >>>>> evidence.
> >> >> >>>>> -
> >> >> >>>>>  srean
> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
> >> >> >>>
> >> >> >>>
> >> >> >>> --
> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my
> brevity.
> >> >> >>> Check out my website: http://kirbyfan64.github.io/
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Ryan
> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
> was
> >> > nul-terminated."
> >> > Personal reality distortion fields are immune to contradictory
> evidence.
> >> > -
> >> > srean
> >> > Check out my website: http://kirbyfan64.github.io/
> >
> >
> >
> >
> > --
> > Ryan
> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> > nul-terminated."
> > Personal reality distortion fields are immune to contradictory evidence.
> -
> > srean
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/0b7408aa/attachment.html>

From rymg19 at gmail.com  Fri Jan 30 23:19:02 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 16:19:02 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
Message-ID: <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>

Do you have the Android SDK and NDK installed? If so...

Using Google, I've created this series of steps, which may (or may not)
work:

1. Make sure you have a copy of Python on your computer and make sure that
it's built with debug symbols.

2. Run the following commands from a shell with your phone plugged in:

  adb forward tcp:5039 tcp:5039
  adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
  <path to ndk binary folder>/arm-linux-androideabi-gdb

Now, GDB should have opened, so type the following commands:

  set solib-search-path <directory holder libpython.so>
  file <path to local python executable>
  target remote :5055
  run

Now, wait for the program to crash and type:

  backtrace

You should now see a complete backtrace in your terminal. To GDB, type:

  quit

*crosses fingers*

On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> Unfortunately it is still reporting the same function :-/.
>
> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > Yes...
> >
> > Can you check if it's crashing in a different function now?
> >
> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >> Yes I did.  I did have to enter all the information in manually...I'm
> >> not familiar with automated patch application tools and even if I
> >> were, I'm pretty sure I wouldn't have them on my device.
> >>
> >> Just so that I'm absolutely sure I got everything correct...you wanted
> >> all of the lines in the patch commented out, correct?  Basically
> >> everything referencing oldloc?
> >>
> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
> wrote:
> >> > Are you sure the patch was applied correctly? I was SO sure it would
> >> > work!
> >> >
> >> > FYI, you tried the patch I attached to the email message, right?
> >> >
> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <chaselton at gmail.com>
> >> > wrote:
> >> >>
> >> >> Update:  I did try the patch after getting it installed correctly,
> but
> >> >> I'm still getting a segfault on the newly built binary.
> >> >> Will post info this afternoon.
> >> >>
> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com>
> >> >> wrote:
> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
> >> >> > unconditionally
> >> >> > checks the length of the (possibly null) string argument first.
> >> >> >
> >> >> > Please try the patch I attached in the last email. It *might* fix
> the
> >> >> > issue.
> >> >> > Android has crappy locale handling.
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton <
> chaselton at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
> >> >> >> currently returning NULL when given a null pointer.
> >> >> >>
> >> >> >> From obmalloc.c
> >> >> >>
> >> >> >> _PyMem_RawStrdup(const char *str)
> >> >> >> {
> >> >> >>     size_t size;
> >> >> >> char *copy;
> >> >> >>     size = strlen(str) + 1;
> >> >> >>     copy = PyMem_RawMalloc(size);
> >> >> >>     if (copy == NULL)
> >> >> >>         return NULL;
> >> >> >> memcpy(copy, str, size);
> >> >> >> return copy;
> >> >> >>           }
> >> >> >>
> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez <rymg19 at gmail.com
> >
> >> >> >> wrote:
> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
> >> >> >> > crashes
> >> >> >> > when
> >> >> >> > calling strlen. It's that whatever is calling it is likely
> asking
> >> >> >> > it
> >> >> >> > to
> >> >> >> > duplicate a null pointer. Basically, it's probably the caller's
> >> >> >> > fault.
> >> >> >> >
> >> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
> >> >> >> > when
> >> >> >> > given a
> >> >> >> > null pointer and see where it then segfaults.
> >> >> >> >
> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
> >> >> >> > <chaselton at gmail.com>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >> Alternatively, is there a hassle-free way to find out what
> >> >> >> >> changed
> >> >> >> >> in
> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
> >> >> >> >>
> >> >> >> >>
> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
> >> >> >> >> <chaselton at gmail.com>
> >> >> >> >> wrote:
> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
> >> >> >> >> > obmalloc.c,
> >> >> >> >> > as
> >> >> >> >> > (I think) the functions all belong to Python...they're all
> >> >> >> >> > prefixed
> >> >> >> >> > with _PyXx
> >> >> >> >> >
> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
> >> >> >> >> > <chaselton at gmail.com>
> >> >> >> >> > wrote:
> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
> >> >> >> >> >>
> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> >> >> >> >> >> /lib/libpython3.4m.so.1.0
> >> >> >> >> >> 0008bbc8
> >> >> >> >> >> _PyMem_RawStrdup
> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> >> >> >> /bld/python/Python-3.4.2 $
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com>
> >> >> >> >> >> wrote:
> >> >> >> >> >>> Could you try the steps at
> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >> >> >> >>> allow you to get a better idea of where libc is crashing.
> >> >> >> >> >>>
> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
> >> >> >> >> >>>>
> >> >> >> >> >>>> Managed to get this out of logcat:
> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
> >> >> >> >> >>>> thread
> >> >> >> >> >>>> 11914 (python)  (libc)
> >> >> >> >> >>>>
> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
> >> >> >> >> >>>> 23373
> >> >> >> >> >>>> (python)
> >> >> >> >> >>>>
> >> >> >> >> >>>> Less detail than strace but it seems to be that python is
> >> >> >> >> >>>> segfaulting
> >> >> >> >> >>>> libc...
> >> >> >> >> >>>>
> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
> >> >> >> >> >>>> <rymg19 at gmail.com>
> >> >> >> >> >>>> wrote:
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> >> >> >> >>>>> <guido at python.org>
> >> >> >> >> >>>>> wrote:
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  What I see in the strace:
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
> >> >> >> >> >>>>>>  ... load libm
> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
> >> >> >> >> >>>>>> (what?)
> >> >> >> >> >>>>>>  ... get current time
> >> >> >> >> >>>>>>  ... allocate memory
> >> >> >> >> >>>>>>  ... getuid
> >> >> >> >> >>>>>>  ... segfault
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as if it
> >> >> >> >> >>>>>> has
> >> >> >> >> >>>>>> started to
> >> >> >> >> >>>>>>  load modules yet.
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >> >> >> >>>>>  is the code that works with that file.
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
> >> >> >> >> >>>>> something
> >> >> >> >> >>>>> to
> >> >> >> >> >>>>> do
> >> >> >> >> >>>>> with
> >> >> >> >> >>>>>  interprocess communication. Likely has nothing to do
> with
> >> >> >> >> >>>>> Python
> >> >> >> >> >>>>> itself.
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>  Maybe this would be useful?
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  That stack trace would be really helpful.
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
> >> >> >> >> >>>>>> <chaselton at gmail.com>
> >> >> >> >> >>>>>> wrote:
> >> >> >> >> >>>>>>>
> >> >> >> >> >>>>>>>
> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I
> do
> >> >> >> >> >>>>>>> have
> >> >> >> >> >>>>>>> the
> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an open
> >> >> >> >> >>>>>>> call,
> >> >> >> >> >>>>>>> though I
> >> >> >> >> >>>>>>>  am probably wrong.
> >> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm going
> to
> >> >> >> >> >>>>>>> head
> >> >> >> >> >>>>>>> back to
> >> >> >> >> >>>>>>>  the documentation and to back out of some
> >> >> >> >> >>>>>>> Android-related
> >> >> >> >> >>>>>>> changes
> >> >> >> >> >>>>>>> in
> >> >> >> >> >>>>>>>  _localemodule.c
> >> >> >> >> >>>>>>>
> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >> >> >> >>>>>>> <guido at python.org>
> >> >> >> >> >>>>>>>  wrote:
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>  There could be a million differences relevant
> (unicode,
> >> >> >> >> >>>>>>>> ints,
> >> >> >> >> >>>>>>>> ...).
> >> >> >> >> >>>>>>>>  Perhaps
> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the
> dynamic
> >> >> >> >> >>>>>>>> loading
> >> >> >> >> >>>>>>>> code
> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace
> shows
> >> >> >> >> >>>>>>>> a
> >> >> >> >> >>>>>>>> syscall
> >> >> >> >> >>>>>>>> trace
> >> >> >> >> >>>>>>>>  --
> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
> >> >> >> >> >>>>>>>> it segfaulted.)
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
> >> >> >> >> >>>>>>>>  wrote:
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>  All,
> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8
> to
> >> >> >> >> >>>>>>>>> Android
> >> >> >> >> >>>>>>>>> in
> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using
> the
> >> >> >> >> >>>>>>>>> same
> >> >> >> >> >>>>>>>>> configure
> >> >> >> >> >>>>>>>>>  options in the same environment, and modifying the
> >> >> >> >> >>>>>>>>> setup.py
> >> >> >> >> >>>>>>>>> as
> >> >> >> >> >>>>>>>>> needed,
> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when the
> >> >> >> >> >>>>>>>>> generate-posix-vars
> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is run
> >> >> >> >> >>>>>>>>> as
> >> >> >> >> >>>>>>>>> well
> >> >> >> >> >>>>>>>>> (i.e.
> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
> >> >> >> >> >>>>>>>>> similar)
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost
> >> >> >> >> >>>>>>>>> when
> >> >> >> >> >>>>>>>>> reviewing
> >> >> >> >> >>>>>>>>> it.
> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why
> Python
> >> >> >> >> >>>>>>>>> 2.7
> >> >> >> >> >>>>>>>>> works but
> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>  Thanks in advance,
> >> >> >> >> >>>>>>>>>  Cyd
> >> >> >> >> >>>>>>>>> ________________________________
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>> Python-Dev at python.org
> >> >> >> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >> >> >> >>>>>>>>>  Unsubscribe:
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> >> >> >> >> >>>>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>
> >> >> >> >> >>>>>>>>  --
> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  --
> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>> ________________________________
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>  Python-Dev mailing list
> >> >> >> >> >>>>>>  Python-Dev at python.org
> >> >> >> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >> >> >> >>>>>>  Unsubscribe:
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> >> >> >> >> >>>>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>
> >> >> >> >> >>>>>  --
> >> >> >> >> >>>>>  Ryan
> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my answer
> >> >> >> >> >>>>> will
> >> >> >> >> >>>>> be
> >> >> >> >> >>>>> simple:
> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
> don't
> >> >> >> >> >>>>> think
> >> >> >> >> >>>>> that was
> >> >> >> >> >>>>>  nul-terminated."
> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
> >> >> >> >> >>>>> contradictory
> >> >> >> >> >>>>> evidence.
> >> >> >> >> >>>>> -
> >> >> >> >> >>>>>  srean
> >> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
> >> >> >> >> >>>
> >> >> >> >> >>>
> >> >> >> >> >>> --
> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my
> >> >> >> >> >>> brevity.
> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Ryan
> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >> >> > simple:
> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> >> >> >> > that
> >> >> >> > was
> >> >> >> > nul-terminated."
> >> >> >> > Personal reality distortion fields are immune to contradictory
> >> >> >> > evidence.
> >> >> >> > -
> >> >> >> > srean
> >> >> >> > Check out my website: http://kirbyfan64.github.io/
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Ryan
> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >> > simple:
> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> that
> >> >> > was
> >> >> > nul-terminated."
> >> >> > Personal reality distortion fields are immune to contradictory
> >> >> > evidence.
> >> >> > -
> >> >> > srean
> >> >> > Check out my website: http://kirbyfan64.github.io/
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Ryan
> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
> was
> >> > nul-terminated."
> >> > Personal reality distortion fields are immune to contradictory
> evidence.
> >> > -
> >> > srean
> >> > Check out my website: http://kirbyfan64.github.io/
> >
> >
> >
> >
> > --
> > Ryan
> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> > nul-terminated."
> > Personal reality distortion fields are immune to contradictory evidence.
> -
> > srean
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/28b362a0/attachment-0001.html>

From greg at krypto.org  Sat Jan 31 00:26:08 2015
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 30 Jan 2015 23:26:08 +0000
Subject: [Python-Dev] Can Python Be Built Without Distutils
References: <CAHu5PrbDQsiEWyJDU9gepTbm2Kg0ZinmDBFxNau-ye0bndPUrw@mail.gmail.com>
 <54C29784.2080908@ubuntu.com> <54C29ED2.2050508@egenix.com>
 <CAGE7PNLw4BH=f_YL_TUbpLp18z6cFD1X4i427F5JJN=UAywJqQ@mail.gmail.com>
 <CAP1=2W7ffnPHgLqdN8AZ5FnyAUy0fdN5xp=7dSa4n9p=SxrTJw@mail.gmail.com>
 <CAKJDb-MaZ+FseT3+SU8C2Uo4d4RHgdjCXx1GjxLCGyoS6wg+_A@mail.gmail.com>
 <54C40632.4020301@egenix.com> <20150124232430.49c859ca@fsol>
 <CAGE7PNLMfL1ye-Ypb4oi0G=xNrYtTecSHe=tibkaWTeStx6c2A@mail.gmail.com>
 <20150125160753.53b98d7b@fsol>
Message-ID: <CAGE7PNL-wXN2Gn=-xkmBLTYxzrNvKLLB36fH8Fnujy9__C2V7w@mail.gmail.com>

On Sun Jan 25 2015 at 7:08:53 AM Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 25 Jan 2015 05:22:48 +0000
> "Gregory P. Smith" <greg at krypto.org> wrote:
> > Why doesn't our Makefile supply that flag with the make parallelism level
> > in the sharedmods step?
>
> If I run "make -j4" here, it works (on the default branch).
>

Thanks, glad to see that this was added! :)

This looks like a feature that was added in 3.5.  Rather than passing the
flag to setup.py from the Makefile (where I had been looking for it),
setup.py looks at the MAKEFLAGS environment variable and turns on
distutils.common.build_ext's parallel flag if "-j" is found in that. By
default it will then do a parallel build using os.cpu_count() number of
tasks.

-gps


>
> Regards
>
> Antoine.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: https://mail.python.org/mailman/options/python-dev/
> greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/9c3f1bef/attachment.html>

From chaselton at gmail.com  Sat Jan 31 01:19:56 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Fri, 30 Jan 2015 18:19:56 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
Message-ID: <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>

This is going to take some time...here's why:

Downloading and installing the NDK/SDK won't be too hard...I have to
clear some space...but my primary machine is running Windows 7 and I'd
rather swallow hot coals than install Cygwin.  I've got next to no
experience with it, other than knowing that the NDK recommends against
it.

I've got a CentOS VM...but it's currently in tarball form on an
external drive for space reasons...and it only has the NDK installed.
If I am able to get it back up and running, there's still the task of
getting  adb connected to my device. from the VM.

Not saying it's impossible...just that it'll take time...and I'll
probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
the meantime I'll also check to see if there's anything that can a)
run in an Android terminal and b) can take a stack trace; it would be
far, far, far easier than either option above.



On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> Do you have the Android SDK and NDK installed? If so...
>
> Using Google, I've created this series of steps, which may (or may not)
> work:
>
> 1. Make sure you have a copy of Python on your computer and make sure that
> it's built with debug symbols.
>
> 2. Run the following commands from a shell with your phone plugged in:
>
>   adb forward tcp:5039 tcp:5039
>   adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
>   <path to ndk binary folder>/arm-linux-androideabi-gdb
>
> Now, GDB should have opened, so type the following commands:
>
>   set solib-search-path <directory holder libpython.so>
>   file <path to local python executable>
>   target remote :5055
>   run
>
> Now, wait for the program to crash and type:
>
>   backtrace
>
> You should now see a complete backtrace in your terminal. To GDB, type:
>
>   quit
>
> *crosses fingers*
>
> On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>>
>> Unfortunately it is still reporting the same function :-/.
>>
>> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Yes...
>> >
>> > Can you check if it's crashing in a different function now?
>> >
>> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
>> > wrote:
>> >>
>> >> Yes I did.  I did have to enter all the information in manually...I'm
>> >> not familiar with automated patch application tools and even if I
>> >> were, I'm pretty sure I wouldn't have them on my device.
>> >>
>> >> Just so that I'm absolutely sure I got everything correct...you wanted
>> >> all of the lines in the patch commented out, correct?  Basically
>> >> everything referencing oldloc?
>> >>
>> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> wrote:
>> >> > Are you sure the patch was applied correctly? I was SO sure it would
>> >> > work!
>> >> >
>> >> > FYI, you tried the patch I attached to the email message, right?
>> >> >
>> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Update:  I did try the patch after getting it installed correctly,
>> >> >> but
>> >> >> I'm still getting a segfault on the newly built binary.
>> >> >> Will post info this afternoon.
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> >> > unconditionally
>> >> >> > checks the length of the (possibly null) string argument first.
>> >> >> >
>> >> >> > Please try the patch I attached in the last email. It *might* fix
>> >> >> > the
>> >> >> > issue.
>> >> >> > Android has crappy locale handling.
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>> >> >> > <chaselton at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup is
>> >> >> >> currently returning NULL when given a null pointer.
>> >> >> >>
>> >> >> >> From obmalloc.c
>> >> >> >>
>> >> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> >> {
>> >> >> >>     size_t size;
>> >> >> >> char *copy;
>> >> >> >>     size = strlen(str) + 1;
>> >> >> >>     copy = PyMem_RawMalloc(size);
>> >> >> >>     if (copy == NULL)
>> >> >> >>         return NULL;
>> >> >> >> memcpy(copy, str, size);
>> >> >> >> return copy;
>> >> >> >>           }
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>> >> >> >> <rymg19 at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
>> >> >> >> > crashes
>> >> >> >> > when
>> >> >> >> > calling strlen. It's that whatever is calling it is likely
>> >> >> >> > asking
>> >> >> >> > it
>> >> >> >> > to
>> >> >> >> > duplicate a null pointer. Basically, it's probably the caller's
>> >> >> >> > fault.
>> >> >> >> >
>> >> >> >> > You could always try modifying _PyMem_RawStrdup to return NULL
>> >> >> >> > when
>> >> >> >> > given a
>> >> >> >> > null pointer and see where it then segfaults.
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> >> > <chaselton at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> >> changed
>> >> >> >> >> in
>> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >> >>
>> >> >> >> >>
>> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> >> <chaselton at gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> >> > obmalloc.c,
>> >> >> >> >> > as
>> >> >> >> >> > (I think) the functions all belong to Python...they're all
>> >> >> >> >> > prefixed
>> >> >> >> >> > with _PyXx
>> >> >> >> >> >
>> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >> >> >> >>
>> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> >> 0008bbc8
>> >> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >>> Could you try the steps at
>> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >> >>> allow you to get a better idea of where libc is crashing.
>> >> >> >> >> >>>
>> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> Managed to get this out of logcat:
>> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>> >> >> >> >> >>>> (code=1),
>> >> >> >> >> >>>> thread
>> >> >> >> >> >>>> 11914 (python)  (libc)
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1), thread
>> >> >> >> >> >>>> 23373
>> >> >> >> >> >>>> (python)
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> Less detail than strace but it seems to be that python is
>> >> >> >> >> >>>> segfaulting
>> >> >> >> >> >>>> libc...
>> >> >> >> >> >>>>
>> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >> >> >>>> <rymg19 at gmail.com>
>> >> >> >> >> >>>> wrote:
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >> >> >> >>>>> <guido at python.org>
>> >> >> >> >> >>>>> wrote:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  What I see in the strace:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >> >> >> >>>>>>  ... load libm
>> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >> >> >> >>>>>> (what?)
>> >> >> >> >> >>>>>>  ... get current time
>> >> >> >> >> >>>>>>  ... allocate memory
>> >> >> >> >> >>>>>>  ... getuid
>> >> >> >> >> >>>>>>  ... segfault
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as if
>> >> >> >> >> >>>>>> it
>> >> >> >> >> >>>>>> has
>> >> >> >> >> >>>>>> started to
>> >> >> >> >> >>>>>>  load modules yet.
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >> >> >> >>>>>  is the code that works with that file.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
>> >> >> >> >> >>>>> something
>> >> >> >> >> >>>>> to
>> >> >> >> >> >>>>> do
>> >> >> >> >> >>>>> with
>> >> >> >> >> >>>>>  interprocess communication. Likely has nothing to do
>> >> >> >> >> >>>>> with
>> >> >> >> >> >>>>> Python
>> >> >> >> >> >>>>> itself.
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>  Maybe this would be useful?
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  That stack trace would be really helpful.
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> >> >> >>>>>> <chaselton at gmail.com>
>> >> >> >> >> >>>>>> wrote:
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is, but I
>> >> >> >> >> >>>>>>> do
>> >> >> >> >> >>>>>>> have
>> >> >> >> >> >>>>>>> the
>> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an
>> >> >> >> >> >>>>>>> open
>> >> >> >> >> >>>>>>> call,
>> >> >> >> >> >>>>>>> though I
>> >> >> >> >> >>>>>>>  am probably wrong.
>> >> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm going
>> >> >> >> >> >>>>>>> to
>> >> >> >> >> >>>>>>> head
>> >> >> >> >> >>>>>>> back to
>> >> >> >> >> >>>>>>>  the documentation and to back out of some
>> >> >> >> >> >>>>>>> Android-related
>> >> >> >> >> >>>>>>> changes
>> >> >> >> >> >>>>>>> in
>> >> >> >> >> >>>>>>>  _localemodule.c
>> >> >> >> >> >>>>>>>
>> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >> >> >> >>>>>>> <guido at python.org>
>> >> >> >> >> >>>>>>>  wrote:
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>  There could be a million differences relevant
>> >> >> >> >> >>>>>>>> (unicode,
>> >> >> >> >> >>>>>>>> ints,
>> >> >> >> >> >>>>>>>> ...).
>> >> >> >> >> >>>>>>>>  Perhaps
>> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the
>> >> >> >> >> >>>>>>>> dynamic
>> >> >> >> >> >>>>>>>> loading
>> >> >> >> >> >>>>>>>> code
>> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace
>> >> >> >> >> >>>>>>>> shows
>> >> >> >> >> >>>>>>>> a
>> >> >> >> >> >>>>>>>> syscall
>> >> >> >> >> >>>>>>>> trace
>> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >> >> >> >>>>>>>> it segfaulted.)
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>> >> >> >> >> >>>>>>>>  wrote:
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>  All,
>> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python 2.7.8
>> >> >> >> >> >>>>>>>>> to
>> >> >> >> >> >>>>>>>>> Android
>> >> >> >> >> >>>>>>>>> in
>> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after using
>> >> >> >> >> >>>>>>>>> the
>> >> >> >> >> >>>>>>>>> same
>> >> >> >> >> >>>>>>>>> configure
>> >> >> >> >> >>>>>>>>>  options in the same environment, and modifying the
>> >> >> >> >> >>>>>>>>> setup.py
>> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >>>>>>>>> needed,
>> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when the
>> >> >> >> >> >>>>>>>>> generate-posix-vars
>> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is
>> >> >> >> >> >>>>>>>>> run
>> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >>>>>>>>> well
>> >> >> >> >> >>>>>>>>> (i.e.
>> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
>> >> >> >> >> >>>>>>>>> similar)
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit lost
>> >> >> >> >> >>>>>>>>> when
>> >> >> >> >> >>>>>>>>> reviewing
>> >> >> >> >> >>>>>>>>> it.
>> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why
>> >> >> >> >> >>>>>>>>> Python
>> >> >> >> >> >>>>>>>>> 2.7
>> >> >> >> >> >>>>>>>>> works but
>> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>  Thanks in advance,
>> >> >> >> >> >>>>>>>>>  Cyd
>> >> >> >> >> >>>>>>>>> ________________________________
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>> >> >> >> >> >>>>>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >>>>>>>>>  Unsubscribe:
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >>>>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>
>> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  --
>> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> ________________________________
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>  Python-Dev mailing list
>> >> >> >> >> >>>>>>  Python-Dev at python.org
>> >> >> >> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >>>>>>  Unsubscribe:
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>>
>> >> >> >> >> >>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>
>> >> >> >> >> >>>>>  --
>> >> >> >> >> >>>>>  Ryan
>> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my
>> >> >> >> >> >>>>> answer
>> >> >> >> >> >>>>> will
>> >> >> >> >> >>>>> be
>> >> >> >> >> >>>>> simple:
>> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>> >> >> >> >> >>>>> don't
>> >> >> >> >> >>>>> think
>> >> >> >> >> >>>>> that was
>> >> >> >> >> >>>>>  nul-terminated."
>> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
>> >> >> >> >> >>>>> contradictory
>> >> >> >> >> >>>>> evidence.
>> >> >> >> >> >>>>> -
>> >> >> >> >> >>>>>  srean
>> >> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >>>
>> >> >> >> >> >>>
>> >> >> >> >> >>> --
>> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse my
>> >> >> >> >> >>> brevity.
>> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Ryan
>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>> >> >> >> > be
>> >> >> >> > simple:
>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> >> > that
>> >> >> >> > was
>> >> >> >> > nul-terminated."
>> >> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> >> > evidence.
>> >> >> >> > -
>> >> >> >> > srean
>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Ryan
>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >> > simple:
>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> > that
>> >> >> > was
>> >> >> > nul-terminated."
>> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> > evidence.
>> >> >> > -
>> >> >> > srean
>> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Ryan
>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> > simple:
>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> >> > was
>> >> > nul-terminated."
>> >> > Personal reality distortion fields are immune to contradictory
>> >> > evidence.
>> >> > -
>> >> > srean
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory evidence.
>> > -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/

From rymg19 at gmail.com  Sat Jan 31 02:51:12 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 19:51:12 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
Message-ID: <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>

Do you have just the SDK (which doesn't require Cygwin) or a rooted phone?
If so, I can upload instructions that don't use the NDK.

On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> This is going to take some time...here's why:
>
> Downloading and installing the NDK/SDK won't be too hard...I have to
> clear some space...but my primary machine is running Windows 7 and I'd
> rather swallow hot coals than install Cygwin.  I've got next to no
> experience with it, other than knowing that the NDK recommends against
> it.
>
> I've got a CentOS VM...but it's currently in tarball form on an
> external drive for space reasons...and it only has the NDK installed.
> If I am able to get it back up and running, there's still the task of
> getting  adb connected to my device. from the VM.
>
> Not saying it's impossible...just that it'll take time...and I'll
> probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
> the meantime I'll also check to see if there's anything that can a)
> run in an Android terminal and b) can take a stack trace; it would be
> far, far, far easier than either option above.
>
>
>
> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
> > Do you have the Android SDK and NDK installed? If so...
> >
> > Using Google, I've created this series of steps, which may (or may not)
> > work:
> >
> > 1. Make sure you have a copy of Python on your computer and make sure
> that
> > it's built with debug symbols.
> >
> > 2. Run the following commands from a shell with your phone plugged in:
> >
> >   adb forward tcp:5039 tcp:5039
> >   adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
> >
> > Now, GDB should have opened, so type the following commands:
> >
> >   set solib-search-path <directory holder libpython.so>
> >   file <path to local python executable>
> >   target remote :5055
> >   run
> >
> > Now, wait for the program to crash and type:
> >
> >   backtrace
> >
> > You should now see a complete backtrace in your terminal. To GDB, type:
> >
> >   quit
> >
> > *crosses fingers*
> >
> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com>
> wrote:
> >>
> >> Unfortunately it is still reporting the same function :-/.
> >>
> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
> wrote:
> >> > Yes...
> >> >
> >> > Can you check if it's crashing in a different function now?
> >> >
> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
> >> > wrote:
> >> >>
> >> >> Yes I did.  I did have to enter all the information in manually...I'm
> >> >> not familiar with automated patch application tools and even if I
> >> >> were, I'm pretty sure I wouldn't have them on my device.
> >> >>
> >> >> Just so that I'm absolutely sure I got everything correct...you
> wanted
> >> >> all of the lines in the patch commented out, correct?  Basically
> >> >> everything referencing oldloc?
> >> >>
> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
> >> >> wrote:
> >> >> > Are you sure the patch was applied correctly? I was SO sure it
> would
> >> >> > work!
> >> >> >
> >> >> > FYI, you tried the patch I attached to the email message, right?
> >> >> >
> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
> chaselton at gmail.com>
> >> >> > wrote:
> >> >> >>
> >> >> >> Update:  I did try the patch after getting it installed correctly,
> >> >> >> but
> >> >> >> I'm still getting a segfault on the newly built binary.
> >> >> >> Will post info this afternoon.
> >> >> >>
> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <rymg19 at gmail.com
> >
> >> >> >> wrote:
> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
> >> >> >> > unconditionally
> >> >> >> > checks the length of the (possibly null) string argument first.
> >> >> >> >
> >> >> >> > Please try the patch I attached in the last email. It *might*
> fix
> >> >> >> > the
> >> >> >> > issue.
> >> >> >> > Android has crappy locale handling.
> >> >> >> >
> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
> >> >> >> > <chaselton at gmail.com>
> >> >> >> > wrote:
> >> >> >> >>
> >> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup
> is
> >> >> >> >> currently returning NULL when given a null pointer.
> >> >> >> >>
> >> >> >> >> From obmalloc.c
> >> >> >> >>
> >> >> >> >> _PyMem_RawStrdup(const char *str)
> >> >> >> >> {
> >> >> >> >>     size_t size;
> >> >> >> >> char *copy;
> >> >> >> >>     size = strlen(str) + 1;
> >> >> >> >>     copy = PyMem_RawMalloc(size);
> >> >> >> >>     if (copy == NULL)
> >> >> >> >>         return NULL;
> >> >> >> >> memcpy(copy, str, size);
> >> >> >> >> return copy;
> >> >> >> >>           }
> >> >> >> >>
> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
> >> >> >> >> <rymg19 at gmail.com>
> >> >> >> >> wrote:
> >> >> >> >> > I seriously doubt the issue is in that file; _PyMem_RawStrdup
> >> >> >> >> > crashes
> >> >> >> >> > when
> >> >> >> >> > calling strlen. It's that whatever is calling it is likely
> >> >> >> >> > asking
> >> >> >> >> > it
> >> >> >> >> > to
> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
> caller's
> >> >> >> >> > fault.
> >> >> >> >> >
> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to return
> NULL
> >> >> >> >> > when
> >> >> >> >> > given a
> >> >> >> >> > null pointer and see where it then segfaults.
> >> >> >> >> >
> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
> >> >> >> >> > <chaselton at gmail.com>
> >> >> >> >> > wrote:
> >> >> >> >> >>
> >> >> >> >> >> Alternatively, is there a hassle-free way to find out what
> >> >> >> >> >> changed
> >> >> >> >> >> in
> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
> >> >> >> >> >>
> >> >> >> >> >>
> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
> >> >> >> >> >> <chaselton at gmail.com>
> >> >> >> >> >> wrote:
> >> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
> >> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
> >> >> >> >> >> > obmalloc.c,
> >> >> >> >> >> > as
> >> >> >> >> >> > (I think) the functions all belong to Python...they're all
> >> >> >> >> >> > prefixed
> >> >> >> >> >> > with _PyXx
> >> >> >> >> >> >
> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
> >> >> >> >> >> > <chaselton at gmail.com>
> >> >> >> >> >> > wrote:
> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
> >> >> >> >> >> >>
> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
> >> >> >> >> >> >> 0008bbc8
> >> >> >> >> >> >> _PyMem_RawStrdup
> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >>
> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com>
> >> >> >> >> >> >> wrote:
> >> >> >> >> >> >>> Could you try the steps at
> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
> >> >> >> >> >> >>> allow you to get a better idea of where libc is
> crashing.
> >> >> >> >> >> >>>
> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
> >> >> >> >> >> >>>>
> >> >> >> >> >> >>>> Managed to get this out of logcat:
> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
> >> >> >> >> >> >>>> (code=1),
> >> >> >> >> >> >>>> thread
> >> >> >> >> >> >>>> 11914 (python)  (libc)
> >> >> >> >> >> >>>>
> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
> thread
> >> >> >> >> >> >>>> 23373
> >> >> >> >> >> >>>> (python)
> >> >> >> >> >> >>>>
> >> >> >> >> >> >>>> Less detail than strace but it seems to be that python
> is
> >> >> >> >> >> >>>> segfaulting
> >> >> >> >> >> >>>> libc...
> >> >> >> >> >> >>>>
> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
> >> >> >> >> >> >>>> <rymg19 at gmail.com>
> >> >> >> >> >> >>>> wrote:
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
> >> >> >> >> >> >>>>> <guido at python.org>
> >> >> >> >> >> >>>>> wrote:
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  What I see in the strace:
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
> >> >> >> >> >> >>>>>>  ... load libm
> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
> >> >> >> >> >> >>>>>> (what?)
> >> >> >> >> >> >>>>>>  ... get current time
> >> >> >> >> >> >>>>>>  ... allocate memory
> >> >> >> >> >> >>>>>>  ... getuid
> >> >> >> >> >> >>>>>>  ... segfault
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as if
> >> >> >> >> >> >>>>>> it
> >> >> >> >> >> >>>>>> has
> >> >> >> >> >> >>>>>> started to
> >> >> >> >> >> >>>>>>  load modules yet.
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
> >> >> >> >> >> >>>>>  is the code that works with that file.
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
> >> >> >> >> >> >>>>> something
> >> >> >> >> >> >>>>> to
> >> >> >> >> >> >>>>> do
> >> >> >> >> >> >>>>> with
> >> >> >> >> >> >>>>>  interprocess communication. Likely has nothing to do
> >> >> >> >> >> >>>>> with
> >> >> >> >> >> >>>>> Python
> >> >> >> >> >> >>>>> itself.
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>  Maybe this would be useful?
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  That stack trace would be really helpful.
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
> >> >> >> >> >> >>>>>> <chaselton at gmail.com>
> >> >> >> >> >> >>>>>> wrote:
> >> >> >> >> >> >>>>>>>
> >> >> >> >> >> >>>>>>>
> >> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is,
> but I
> >> >> >> >> >> >>>>>>> do
> >> >> >> >> >> >>>>>>> have
> >> >> >> >> >> >>>>>>> the
> >> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an
> >> >> >> >> >> >>>>>>> open
> >> >> >> >> >> >>>>>>> call,
> >> >> >> >> >> >>>>>>> though I
> >> >> >> >> >> >>>>>>>  am probably wrong.
> >> >> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm
> going
> >> >> >> >> >> >>>>>>> to
> >> >> >> >> >> >>>>>>> head
> >> >> >> >> >> >>>>>>> back to
> >> >> >> >> >> >>>>>>>  the documentation and to back out of some
> >> >> >> >> >> >>>>>>> Android-related
> >> >> >> >> >> >>>>>>> changes
> >> >> >> >> >> >>>>>>> in
> >> >> >> >> >> >>>>>>>  _localemodule.c
> >> >> >> >> >> >>>>>>>
> >> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
> >> >> >> >> >> >>>>>>> <guido at python.org>
> >> >> >> >> >> >>>>>>>  wrote:
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>  There could be a million differences relevant
> >> >> >> >> >> >>>>>>>> (unicode,
> >> >> >> >> >> >>>>>>>> ints,
> >> >> >> >> >> >>>>>>>> ...).
> >> >> >> >> >> >>>>>>>>  Perhaps
> >> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the
> >> >> >> >> >> >>>>>>>> dynamic
> >> >> >> >> >> >>>>>>>> loading
> >> >> >> >> >> >>>>>>>> code
> >> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace
> >> >> >> >> >> >>>>>>>> shows
> >> >> >> >> >> >>>>>>>> a
> >> >> >> >> >> >>>>>>>> syscall
> >> >> >> >> >> >>>>>>>> trace
> >> >> >> >> >> >>>>>>>>  --
> >> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
> >> >> >> >> >> >>>>>>>> it segfaulted.)
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
> >> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
> >> >> >> >> >> >>>>>>>>  wrote:
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>  All,
> >> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python
> 2.7.8
> >> >> >> >> >> >>>>>>>>> to
> >> >> >> >> >> >>>>>>>>> Android
> >> >> >> >> >> >>>>>>>>> in
> >> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after
> using
> >> >> >> >> >> >>>>>>>>> the
> >> >> >> >> >> >>>>>>>>> same
> >> >> >> >> >> >>>>>>>>> configure
> >> >> >> >> >> >>>>>>>>>  options in the same environment, and modifying
> the
> >> >> >> >> >> >>>>>>>>> setup.py
> >> >> >> >> >> >>>>>>>>> as
> >> >> >> >> >> >>>>>>>>> needed,
> >> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when the
> >> >> >> >> >> >>>>>>>>> generate-posix-vars
> >> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is
> >> >> >> >> >> >>>>>>>>> run
> >> >> >> >> >> >>>>>>>>> as
> >> >> >> >> >> >>>>>>>>> well
> >> >> >> >> >> >>>>>>>>> (i.e.
> >> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
> >> >> >> >> >> >>>>>>>>> similar)
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit
> lost
> >> >> >> >> >> >>>>>>>>> when
> >> >> >> >> >> >>>>>>>>> reviewing
> >> >> >> >> >> >>>>>>>>> it.
> >> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why
> >> >> >> >> >> >>>>>>>>> Python
> >> >> >> >> >> >>>>>>>>> 2.7
> >> >> >> >> >> >>>>>>>>> works but
> >> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>  Thanks in advance,
> >> >> >> >> >> >>>>>>>>>  Cyd
> >> >> >> >> >> >>>>>>>>> ________________________________
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>> Python-Dev at python.org
> >> >> >> >> >> >>>>>>>>>
> https://mail.python.org/mailman/listinfo/python-dev
> >> >> >> >> >> >>>>>>>>>  Unsubscribe:
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> >> >> >> >> >> >>>>>>>>>
> https://mail.python.org/mailman/options/python-dev/guido%40python.org
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>
> >> >> >> >> >> >>>>>>>>  --
> >> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  --
> >> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>> ________________________________
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>  Python-Dev mailing list
> >> >> >> >> >> >>>>>>  Python-Dev at python.org
> >> >> >> >> >> >>>>>>  https://mail.python.org/mailman/listinfo/python-dev
> >> >> >> >> >> >>>>>>  Unsubscribe:
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> >> >> >> >> >> >>>>>>
> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>
> >> >> >> >> >> >>>>>  --
> >> >> >> >> >> >>>>>  Ryan
> >> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my
> >> >> >> >> >> >>>>> answer
> >> >> >> >> >> >>>>> will
> >> >> >> >> >> >>>>> be
> >> >> >> >> >> >>>>> simple:
> >> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
> >> >> >> >> >> >>>>> don't
> >> >> >> >> >> >>>>> think
> >> >> >> >> >> >>>>> that was
> >> >> >> >> >> >>>>>  nul-terminated."
> >> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
> >> >> >> >> >> >>>>> contradictory
> >> >> >> >> >> >>>>> evidence.
> >> >> >> >> >> >>>>> -
> >> >> >> >> >> >>>>>  srean
> >> >> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
> >> >> >> >> >> >>>
> >> >> >> >> >> >>>
> >> >> >> >> >> >>> --
> >> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please excuse
> my
> >> >> >> >> >> >>> brevity.
> >> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> >
> >> >> >> >> > --
> >> >> >> >> > Ryan
> >> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
> >> >> >> >> > be
> >> >> >> >> > simple:
> >> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
> think
> >> >> >> >> > that
> >> >> >> >> > was
> >> >> >> >> > nul-terminated."
> >> >> >> >> > Personal reality distortion fields are immune to
> contradictory
> >> >> >> >> > evidence.
> >> >> >> >> > -
> >> >> >> >> > srean
> >> >> >> >> > Check out my website: http://kirbyfan64.github.io/
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> >
> >> >> >> > --
> >> >> >> > Ryan
> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >> >> > simple:
> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> >> >> >> > that
> >> >> >> > was
> >> >> >> > nul-terminated."
> >> >> >> > Personal reality distortion fields are immune to contradictory
> >> >> >> > evidence.
> >> >> >> > -
> >> >> >> > srean
> >> >> >> > Check out my website: http://kirbyfan64.github.io/
> >> >> >
> >> >> >
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Ryan
> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> >> >> > simple:
> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
> that
> >> >> > was
> >> >> > nul-terminated."
> >> >> > Personal reality distortion fields are immune to contradictory
> >> >> > evidence.
> >> >> > -
> >> >> > srean
> >> >> > Check out my website: http://kirbyfan64.github.io/
> >> >
> >> >
> >> >
> >> >
> >> > --
> >> > Ryan
> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
> simple:
> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
> was
> >> > nul-terminated."
> >> > Personal reality distortion fields are immune to contradictory
> evidence.
> >> > -
> >> > srean
> >> > Check out my website: http://kirbyfan64.github.io/
> >
> >
> >
> >
> > --
> > Ryan
> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> > nul-terminated."
> > Personal reality distortion fields are immune to contradictory evidence.
> -
> > srean
> > Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/1379f8f9/attachment-0001.html>

From +18562547597 at tmomail.net  Thu Jan 29 05:42:04 2015
From: +18562547597 at tmomail.net (+18562547597 at tmomail.net)
Date: Thu, 29 Jan 2015 04:42:04 GMT
Subject: [Python-Dev] (no subject)
Message-ID: 20150029044210443174@mavenir.com

An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/18e1efdc/attachment.html>
-------------- next part --------------
Who's this a symbol was sent to my man's phone I clicked the symbol and then it was at MESSAGE options 
-------------- next part --------------
A non-text attachment was scrubbed...
Name: tmobilespace.gif
Type: image/gif
Size: 799 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/18e1efdc/attachment.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dottedline600.gif
Type: image/gif
Size: 872 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/18e1efdc/attachment-0001.gif>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: footer.gif
Type: image/gif
Size: 3589 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150129/18e1efdc/attachment-0002.gif>

From tetsuya.morimoto at gmail.com  Sat Jan 31 05:56:17 2015
From: tetsuya.morimoto at gmail.com (Tetsuya Morimoto)
Date: Sat, 31 Jan 2015 13:56:17 +0900
Subject: [Python-Dev] Results of the python 2.x and 3.x use survey,
	2014 edition
In-Reply-To: <CAMA3c49yP8C4MS50qsH7P3waKO49RPNtFXgd5WgYJQHk0VSbJw@mail.gmail.com>
References: <CAMA3c49yP8C4MS50qsH7P3waKO49RPNtFXgd5WgYJQHk0VSbJw@mail.gmail.com>
Message-ID: <CAGBzy2qMJ3PZdMJM+aYjJT6Wtb4iu3HSSzbBizDJe1wncv+p3Q@mail.gmail.com>

Thank you for your hard work. That's very helpful!

thanks,
Tetsuya

On Fri, Jan 30, 2015 at 2:49 AM, Bruno Cauet <brunocauet at gmail.com> wrote:

> Hi!
> Finally, here are the results:
> http://blog.frite-camembert.net/python-survey-2014.html
> Here is the auto-generated Google Forms recap:
>
> https://docs.google.com/forms/d/1DqxkNi4GvyTCu54usSdE1DjW29zw1tc52iMeH3z4heg/viewanalytics
> (more elegant than my matplotlib graphs - I'd have no future as a
> designer).
>
> Overall people started writing more python 3: +15 points in "I ever
> wrote python 3 code", +10 points in "I write more python 3 than 2".
> Transition is still ongoing and I hope a tipping point will be soon be
> attained.
> Users definitely seem to want to switch to python 3, but dependencies
> keep them with 2.7 (I weep for the few ones still on 2.5).
>
> I also posted the results on HN
> (https://news.ycombinator.com/item?id=8967645) and reddit
> (
> http://www.reddit.com/r/Python/comments/2u3oh0/results_of_the_python_2x_and_3x_use_survey_2014/
> ).
> Have a nice day, and a year full of python 3!
>
> Bruno Cauet
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> https://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> https://mail.python.org/mailman/options/python-dev/tetsuya.morimoto%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/d8de4785/attachment.html>

From rymg19 at gmail.com  Sat Jan 31 02:52:47 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Fri, 30 Jan 2015 19:52:47 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
Message-ID: <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>

Regardless, if you're looking to toy more with stuff like this, I'd highly
recommend dual-booting with Ubuntu, which is what I'm doing now. (Now I
rarely ever boot into Windows!)

On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:

> Do you have just the SDK (which doesn't require Cygwin) or a rooted phone?
> If so, I can upload instructions that don't use the NDK.
>
> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>
>> This is going to take some time...here's why:
>>
>> Downloading and installing the NDK/SDK won't be too hard...I have to
>> clear some space...but my primary machine is running Windows 7 and I'd
>> rather swallow hot coals than install Cygwin.  I've got next to no
>> experience with it, other than knowing that the NDK recommends against
>> it.
>>
>> I've got a CentOS VM...but it's currently in tarball form on an
>> external drive for space reasons...and it only has the NDK installed.
>> If I am able to get it back up and running, there's still the task of
>> getting  adb connected to my device. from the VM.
>>
>> Not saying it's impossible...just that it'll take time...and I'll
>> probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
>> the meantime I'll also check to see if there's anything that can a)
>> run in an Android terminal and b) can take a stack trace; it would be
>> far, far, far easier than either option above.
>>
>>
>>
>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Do you have the Android SDK and NDK installed? If so...
>> >
>> > Using Google, I've created this series of steps, which may (or may not)
>> > work:
>> >
>> > 1. Make sure you have a copy of Python on your computer and make sure
>> that
>> > it's built with debug symbols.
>> >
>> > 2. Run the following commands from a shell with your phone plugged in:
>> >
>> >   adb forward tcp:5039 tcp:5039
>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>> >
>> > Now, GDB should have opened, so type the following commands:
>> >
>> >   set solib-search-path <directory holder libpython.so>
>> >   file <path to local python executable>
>> >   target remote :5055
>> >   run
>> >
>> > Now, wait for the program to crash and type:
>> >
>> >   backtrace
>> >
>> > You should now see a complete backtrace in your terminal. To GDB, type:
>> >
>> >   quit
>> >
>> > *crosses fingers*
>> >
>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com>
>> wrote:
>> >>
>> >> Unfortunately it is still reporting the same function :-/.
>> >>
>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> wrote:
>> >> > Yes...
>> >> >
>> >> > Can you check if it's crashing in a different function now?
>> >> >
>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Yes I did.  I did have to enter all the information in
>> manually...I'm
>> >> >> not familiar with automated patch application tools and even if I
>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>> >> >>
>> >> >> Just so that I'm absolutely sure I got everything correct...you
>> wanted
>> >> >> all of the lines in the patch commented out, correct?  Basically
>> >> >> everything referencing oldloc?
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >> > Are you sure the patch was applied correctly? I was SO sure it
>> would
>> >> >> > work!
>> >> >> >
>> >> >> > FYI, you tried the patch I attached to the email message, right?
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>> chaselton at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Update:  I did try the patch after getting it installed
>> correctly,
>> >> >> >> but
>> >> >> >> I'm still getting a segfault on the newly built binary.
>> >> >> >> Will post info this afternoon.
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>> rymg19 at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> >> >> > unconditionally
>> >> >> >> > checks the length of the (possibly null) string argument first.
>> >> >> >> >
>> >> >> >> > Please try the patch I attached in the last email. It *might*
>> fix
>> >> >> >> > the
>> >> >> >> > issue.
>> >> >> >> > Android has crappy locale handling.
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>> >> >> >> > <chaselton at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup
>> is
>> >> >> >> >> currently returning NULL when given a null pointer.
>> >> >> >> >>
>> >> >> >> >> From obmalloc.c
>> >> >> >> >>
>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> >> >> {
>> >> >> >> >>     size_t size;
>> >> >> >> >> char *copy;
>> >> >> >> >>     size = strlen(str) + 1;
>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>> >> >> >> >>     if (copy == NULL)
>> >> >> >> >>         return NULL;
>> >> >> >> >> memcpy(copy, str, size);
>> >> >> >> >> return copy;
>> >> >> >> >>           }
>> >> >> >> >>
>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>> >> >> >> >> <rymg19 at gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > I seriously doubt the issue is in that file;
>> _PyMem_RawStrdup
>> >> >> >> >> > crashes
>> >> >> >> >> > when
>> >> >> >> >> > calling strlen. It's that whatever is calling it is likely
>> >> >> >> >> > asking
>> >> >> >> >> > it
>> >> >> >> >> > to
>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>> caller's
>> >> >> >> >> > fault.
>> >> >> >> >> >
>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to return
>> NULL
>> >> >> >> >> > when
>> >> >> >> >> > given a
>> >> >> >> >> > null pointer and see where it then segfaults.
>> >> >> >> >> >
>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> >> >> changed
>> >> >> >> >> >> in
>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> >> >> <chaselton at gmail.com>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> >> >> > obmalloc.c,
>> >> >> >> >> >> > as
>> >> >> >> >> >> > (I think) the functions all belong to Python...they're
>> all
>> >> >> >> >> >> > prefixed
>> >> >> >> >> >> > with _PyXx
>> >> >> >> >> >> >
>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> >> > wrote:
>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> >> >> 0008bbc8
>> >> >> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com
>> >
>> >> >> >> >> >> >> wrote:
>> >> >> >> >> >> >>> Could you try the steps at
>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>> crashing.
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>> >> >> >> >> >> >>>> (code=1),
>> >> >> >> >> >> >>>> thread
>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
>> thread
>> >> >> >> >> >> >>>> 23373
>> >> >> >> >> >> >>>> (python)
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>> python is
>> >> >> >> >> >> >>>> segfaulting
>> >> >> >> >> >> >>>> libc...
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>> >> >> >> >> >> >>>> wrote:
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >> >> >> >> >>>>> <guido at python.org>
>> >> >> >> >> >> >>>>> wrote:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  What I see in the strace:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >> >> >> >> >>>>>>  ... load libm
>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >> >> >> >> >>>>>> (what?)
>> >> >> >> >> >> >>>>>>  ... get current time
>> >> >> >> >> >> >>>>>>  ... allocate memory
>> >> >> >> >> >> >>>>>>  ... getuid
>> >> >> >> >> >> >>>>>>  ... segfault
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as
>> if
>> >> >> >> >> >> >>>>>> it
>> >> >> >> >> >> >>>>>> has
>> >> >> >> >> >> >>>>>> started to
>> >> >> >> >> >> >>>>>>  load modules yet.
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >> >> >> >> >>>>>  is the code that works with that file.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
>> >> >> >> >> >> >>>>> something
>> >> >> >> >> >> >>>>> to
>> >> >> >> >> >> >>>>> do
>> >> >> >> >> >> >>>>> with
>> >> >> >> >> >> >>>>>  interprocess communication. Likely has nothing to do
>> >> >> >> >> >> >>>>> with
>> >> >> >> >> >> >>>>> Python
>> >> >> >> >> >> >>>>> itself.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  Maybe this would be useful?
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  That stack trace would be really helpful.
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> >> >> >> >>>>>> <chaselton at gmail.com>
>> >> >> >> >> >> >>>>>> wrote:
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is,
>> but I
>> >> >> >> >> >> >>>>>>> do
>> >> >> >> >> >> >>>>>>> have
>> >> >> >> >> >> >>>>>>> the
>> >> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an
>> >> >> >> >> >> >>>>>>> open
>> >> >> >> >> >> >>>>>>> call,
>> >> >> >> >> >> >>>>>>> though I
>> >> >> >> >> >> >>>>>>>  am probably wrong.
>> >> >> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm
>> going
>> >> >> >> >> >> >>>>>>> to
>> >> >> >> >> >> >>>>>>> head
>> >> >> >> >> >> >>>>>>> back to
>> >> >> >> >> >> >>>>>>>  the documentation and to back out of some
>> >> >> >> >> >> >>>>>>> Android-related
>> >> >> >> >> >> >>>>>>> changes
>> >> >> >> >> >> >>>>>>> in
>> >> >> >> >> >> >>>>>>>  _localemodule.c
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >> >> >> >> >>>>>>> <guido at python.org>
>> >> >> >> >> >> >>>>>>>  wrote:
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  There could be a million differences relevant
>> >> >> >> >> >> >>>>>>>> (unicode,
>> >> >> >> >> >> >>>>>>>> ints,
>> >> >> >> >> >> >>>>>>>> ...).
>> >> >> >> >> >> >>>>>>>>  Perhaps
>> >> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the
>> >> >> >> >> >> >>>>>>>> dynamic
>> >> >> >> >> >> >>>>>>>> loading
>> >> >> >> >> >> >>>>>>>> code
>> >> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace
>> >> >> >> >> >> >>>>>>>> shows
>> >> >> >> >> >> >>>>>>>> a
>> >> >> >> >> >> >>>>>>>> syscall
>> >> >> >> >> >> >>>>>>>> trace
>> >> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >> >> >> >> >>>>>>>> it segfaulted.)
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>> >> >> >> >> >> >>>>>>>>  wrote:
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  All,
>> >> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python
>> 2.7.8
>> >> >> >> >> >> >>>>>>>>> to
>> >> >> >> >> >> >>>>>>>>> Android
>> >> >> >> >> >> >>>>>>>>> in
>> >> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after
>> using
>> >> >> >> >> >> >>>>>>>>> the
>> >> >> >> >> >> >>>>>>>>> same
>> >> >> >> >> >> >>>>>>>>> configure
>> >> >> >> >> >> >>>>>>>>>  options in the same environment, and modifying
>> the
>> >> >> >> >> >> >>>>>>>>> setup.py
>> >> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >> >>>>>>>>> needed,
>> >> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when
>> the
>> >> >> >> >> >> >>>>>>>>> generate-posix-vars
>> >> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is
>> >> >> >> >> >> >>>>>>>>> run
>> >> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >> >>>>>>>>> well
>> >> >> >> >> >> >>>>>>>>> (i.e.
>> >> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
>> >> >> >> >> >> >>>>>>>>> similar)
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit
>> lost
>> >> >> >> >> >> >>>>>>>>> when
>> >> >> >> >> >> >>>>>>>>> reviewing
>> >> >> >> >> >> >>>>>>>>> it.
>> >> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why
>> >> >> >> >> >> >>>>>>>>> Python
>> >> >> >> >> >> >>>>>>>>> 2.7
>> >> >> >> >> >> >>>>>>>>> works but
>> >> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  Thanks in advance,
>> >> >> >> >> >> >>>>>>>>>  Cyd
>> >> >> >> >> >> >>>>>>>>> ________________________________
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>> >> >> >> >> >> >>>>>>>>>
>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >> >>>>>>>>>  Unsubscribe:
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  --
>> >> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>> ________________________________
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  Python-Dev mailing list
>> >> >> >> >> >> >>>>>>  Python-Dev at python.org
>> >> >> >> >> >> >>>>>>
>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >> >>>>>>  Unsubscribe:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  --
>> >> >> >> >> >> >>>>>  Ryan
>> >> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my
>> >> >> >> >> >> >>>>> answer
>> >> >> >> >> >> >>>>> will
>> >> >> >> >> >> >>>>> be
>> >> >> >> >> >> >>>>> simple:
>> >> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>> >> >> >> >> >> >>>>> don't
>> >> >> >> >> >> >>>>> think
>> >> >> >> >> >> >>>>> that was
>> >> >> >> >> >> >>>>>  nul-terminated."
>> >> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
>> >> >> >> >> >> >>>>> contradictory
>> >> >> >> >> >> >>>>> evidence.
>> >> >> >> >> >> >>>>> -
>> >> >> >> >> >> >>>>>  srean
>> >> >> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> --
>> >> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please
>> excuse my
>> >> >> >> >> >> >>> brevity.
>> >> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > Ryan
>> >> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer
>> will
>> >> >> >> >> > be
>> >> >> >> >> > simple:
>> >> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>> think
>> >> >> >> >> > that
>> >> >> >> >> > was
>> >> >> >> >> > nul-terminated."
>> >> >> >> >> > Personal reality distortion fields are immune to
>> contradictory
>> >> >> >> >> > evidence.
>> >> >> >> >> > -
>> >> >> >> >> > srean
>> >> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Ryan
>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>> be
>> >> >> >> > simple:
>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> >> > that
>> >> >> >> > was
>> >> >> >> > nul-terminated."
>> >> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> >> > evidence.
>> >> >> >> > -
>> >> >> >> > srean
>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Ryan
>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >> > simple:
>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> that
>> >> >> > was
>> >> >> > nul-terminated."
>> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> > evidence.
>> >> >> > -
>> >> >> > srean
>> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Ryan
>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> simple:
>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> was
>> >> > nul-terminated."
>> >> > Personal reality distortion fields are immune to contradictory
>> evidence.
>> >> > -
>> >> > srean
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory
>> evidence. -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150130/ba7b7ade/attachment-0001.html>

From chaselton at gmail.com  Sat Jan 31 13:10:51 2015
From: chaselton at gmail.com (chaselton at gmail.com)
Date: Sat, 31 Jan 2015 06:10:51 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAP7+vJ+-dO1KFvbojSAwkeJ7JRHSCEv8qa4jcxD3kuT1PZLFfg@mail.gmail.com>
 <CAHu5PrY4XKPeGqRSvRLnLcTnn9Y0Mdy8fk2954XrzFrxWdK-jg@mail.gmail.com>
 <CAP7+vJ+5sqbpvo4kdtUoSEGmWbs-VHN7g3QMXSXrQa9QRai=LA@mail.gmail.com>
 <CAO41-mN-OknyW_uUYuUfY=xibjDAVc+6byQck1=WH1jzsfO5Nw@mail.gmail.com>
 <CAHu5PraYez-Y7d1gZCXQOpL2o7qj-aRYzvbGthRe4JtLkvMXew@mail.gmail.com>
 <543e1054-a437-4ed4-9c1d-3298e8ad0223@email.android.com>
 <CAHu5PrbAVgyZeo9-DR7hYy0VMvbjG+o7P_JgsZOZdR-eVL1amg@mail.gmail.com>
 <CAHu5PrYj1=zbCga9Qy9F+x3as50NHhkY8D6GOG29GHGkRPsiSg@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
Message-ID: <7545e812-1a79-4389-8add-c0d6acdf03b3.maildroid@localhost>

I don't have the SDK either...but my device is rooted.

Dual-booting is not an option unfortunately...space reasons.  I'll do my best to figure out a way to get the instuctions you sent implemented, but this may be a deal-breaker for porting Python 3.4.x for me...I may go back to working on 2.7.x

Sent from my android device.

-----Original Message-----
From: Ryan Gonzalez <rymg19 at gmail.com>
To: Cyd Haselton <chaselton at gmail.com>
Cc: Python-Dev <python-dev at python.org>
Sent: Fri, 30 Jan 2015 7:53 PM
Subject: Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault

Regardless, if you're looking to toy more with stuff like this, I'd highly
recommend dual-booting with Ubuntu, which is what I'm doing now. (Now I
rarely ever boot into Windows!)

On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:

> Do you have just the SDK (which doesn't require Cygwin) or a rooted phone?
> If so, I can upload instructions that don't use the NDK.
>
> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com> wrote:
>
>> This is going to take some time...here's why:
>>
>> Downloading and installing the NDK/SDK won't be too hard...I have to
>> clear some space...but my primary machine is running Windows 7 and I'd
>> rather swallow hot coals than install Cygwin.  I've got next to no
>> experience with it, other than knowing that the NDK recommends against
>> it.
>>
>> I've got a CentOS VM...but it's currently in tarball form on an
>> external drive for space reasons...and it only has the NDK installed.
>> If I am able to get it back up and running, there's still the task of
>> getting  adb connected to my device. from the VM.
>>
>> Not saying it's impossible...just that it'll take time...and I'll
>> probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
>> the meantime I'll also check to see if there's anything that can a)
>> run in an Android terminal and b) can take a stack trace; it would be
>> far, far, far easier than either option above.
>>
>>
>>
>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>> > Do you have the Android SDK and NDK installed? If so...
>> >
>> > Using Google, I've created this series of steps, which may (or may not)
>> > work:
>> >
>> > 1. Make sure you have a copy of Python on your computer and make sure
>> that
>> > it's built with debug symbols.
>> >
>> > 2. Run the following commands from a shell with your phone plugged in:
>> >
>> >   adb forward tcp:5039 tcp:5039
>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python executable>
>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>> >
>> > Now, GDB should have opened, so type the following commands:
>> >
>> >   set solib-search-path <directory holder libpython.so>
>> >   file <path to local python executable>
>> >   target remote :5055
>> >   run
>> >
>> > Now, wait for the program to crash and type:
>> >
>> >   backtrace
>> >
>> > You should now see a complete backtrace in your terminal. To GDB, type:
>> >
>> >   quit
>> >
>> > *crosses fingers*
>> >
>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com>
>> wrote:
>> >>
>> >> Unfortunately it is still reporting the same function :-/.
>> >>
>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> wrote:
>> >> > Yes...
>> >> >
>> >> > Can you check if it's crashing in a different function now?
>> >> >
>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <chaselton at gmail.com>
>> >> > wrote:
>> >> >>
>> >> >> Yes I did.  I did have to enter all the information in
>> manually...I'm
>> >> >> not familiar with automated patch application tools and even if I
>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>> >> >>
>> >> >> Just so that I'm absolutely sure I got everything correct...you
>> wanted
>> >> >> all of the lines in the patch commented out, correct?  Basically
>> >> >> everything referencing oldloc?
>> >> >>
>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <rymg19 at gmail.com>
>> >> >> wrote:
>> >> >> > Are you sure the patch was applied correctly? I was SO sure it
>> would
>> >> >> > work!
>> >> >> >
>> >> >> > FYI, you tried the patch I attached to the email message, right?
>> >> >> >
>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>> chaselton at gmail.com>
>> >> >> > wrote:
>> >> >> >>
>> >> >> >> Update:  I did try the patch after getting it installed
>> correctly,
>> >> >> >> but
>> >> >> >> I'm still getting a segfault on the newly built binary.
>> >> >> >> Will post info this afternoon.
>> >> >> >>
>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>> rymg19 at gmail.com>
>> >> >> >> wrote:
>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>> >> >> >> > unconditionally
>> >> >> >> > checks the length of the (possibly null) string argument first.
>> >> >> >> >
>> >> >> >> > Please try the patch I attached in the last email. It *might*
>> fix
>> >> >> >> > the
>> >> >> >> > issue.
>> >> >> >> > Android has crappy locale handling.
>> >> >> >> >
>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>> >> >> >> > <chaselton at gmail.com>
>> >> >> >> > wrote:
>> >> >> >> >>
>> >> >> >> >>   Unless i'm reading something incorrectly,  _PyMem_RawStrdup
>> is
>> >> >> >> >> currently returning NULL when given a null pointer.
>> >> >> >> >>
>> >> >> >> >> From obmalloc.c
>> >> >> >> >>
>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>> >> >> >> >> {
>> >> >> >> >>     size_t size;
>> >> >> >> >> char *copy;
>> >> >> >> >>     size = strlen(str) + 1;
>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>> >> >> >> >>     if (copy == NULL)
>> >> >> >> >>         return NULL;
>> >> >> >> >> memcpy(copy, str, size);
>> >> >> >> >> return copy;
>> >> >> >> >>           }
>> >> >> >> >>
>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>> >> >> >> >> <rymg19 at gmail.com>
>> >> >> >> >> wrote:
>> >> >> >> >> > I seriously doubt the issue is in that file;
>> _PyMem_RawStrdup
>> >> >> >> >> > crashes
>> >> >> >> >> > when
>> >> >> >> >> > calling strlen. It's that whatever is calling it is likely
>> >> >> >> >> > asking
>> >> >> >> >> > it
>> >> >> >> >> > to
>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>> caller's
>> >> >> >> >> > fault.
>> >> >> >> >> >
>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to return
>> NULL
>> >> >> >> >> > when
>> >> >> >> >> > given a
>> >> >> >> >> > null pointer and see where it then segfaults.
>> >> >> >> >> >
>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> > wrote:
>> >> >> >> >> >>
>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out what
>> >> >> >> >> >> changed
>> >> >> >> >> >> in
>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>> >> >> >> >> >>
>> >> >> >> >> >>
>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>> >> >> >> >> >> <chaselton at gmail.com>
>> >> >> >> >> >> wrote:
>> >> >> >> >> >> > There's a related strdup patch for readline.c, mentioned
>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>> >> >> >> >> >> > https://github.com/rave-engine/python3-android/issues/2.
>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it for
>> >> >> >> >> >> > obmalloc.c,
>> >> >> >> >> >> > as
>> >> >> >> >> >> > (I think) the functions all belong to Python...they're
>> all
>> >> >> >> >> >> > prefixed
>> >> >> >> >> >> > with _PyXx
>> >> >> >> >> >> >
>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>> >> >> >> >> >> > <chaselton at gmail.com>
>> >> >> >> >> >> > wrote:
>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>> >> >> >> >> >> >> 0008bbc8
>> >> >> >> >> >> >> _PyMem_RawStrdup
>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >>
>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <rymg19 at gmail.com
>> >
>> >> >> >> >> >> >> wrote:
>> >> >> >> >> >> >>> Could you try the steps at
>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>> crashing.
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>> >> >> >> >> >> >>>> (code=1),
>> >> >> >> >> >> >>>> thread
>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
>> thread
>> >> >> >> >> >> >>>> 23373
>> >> >> >> >> >> >>>> (python)
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>> python is
>> >> >> >> >> >> >>>> segfaulting
>> >> >> >> >> >> >>>> libc...
>> >> >> >> >> >> >>>>
>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>> >> >> >> >> >> >>>> wrote:
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van Rossum
>> >> >> >> >> >> >>>>> <guido at python.org>
>> >> >> >> >> >> >>>>> wrote:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  What I see in the strace:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>> >> >> >> >> >> >>>>>>  ... load libm
>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something to it
>> >> >> >> >> >> >>>>>> (what?)
>> >> >> >> >> >> >>>>>>  ... get current time
>> >> >> >> >> >> >>>>>>  ... allocate memory
>> >> >> >> >> >> >>>>>>  ... getuid
>> >> >> >> >> >> >>>>>>  ... segfault
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look as
>> if
>> >> >> >> >> >> >>>>>> it
>> >> >> >> >> >> >>>>>> has
>> >> >> >> >> >> >>>>>> started to
>> >> >> >> >> >> >>>>>>  load modules yet.
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to me.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>> >> >> >> >> >> >>>>>  is the code that works with that file.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks like
>> >> >> >> >> >> >>>>> something
>> >> >> >> >> >> >>>>> to
>> >> >> >> >> >> >>>>> do
>> >> >> >> >> >> >>>>> with
>> >> >> >> >> >> >>>>>  interprocess communication. Likely has nothing to do
>> >> >> >> >> >> >>>>> with
>> >> >> >> >> >> >>>>> Python
>> >> >> >> >> >> >>>>> itself.
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  Maybe this would be useful?
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  That stack trace would be really helpful.
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>> >> >> >> >> >> >>>>>> <chaselton at gmail.com>
>> >> >> >> >> >> >>>>>> wrote:
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track is,
>> but I
>> >> >> >> >> >> >>>>>>> do
>> >> >> >> >> >> >>>>>>> have
>> >> >> >> >> >> >>>>>>> the
>> >> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due to an
>> >> >> >> >> >> >>>>>>> open
>> >> >> >> >> >> >>>>>>> call,
>> >> >> >> >> >> >>>>>>> though I
>> >> >> >> >> >> >>>>>>>  am probably wrong.
>> >> >> >> >> >> >>>>>>>  Attaching the strace output to this email.  I'm
>> going
>> >> >> >> >> >> >>>>>>> to
>> >> >> >> >> >> >>>>>>> head
>> >> >> >> >> >> >>>>>>> back to
>> >> >> >> >> >> >>>>>>>  the documentation and to back out of some
>> >> >> >> >> >> >>>>>>> Android-related
>> >> >> >> >> >> >>>>>>> changes
>> >> >> >> >> >> >>>>>>> in
>> >> >> >> >> >> >>>>>>>  _localemodule.c
>> >> >> >> >> >> >>>>>>>
>> >> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van Rossum
>> >> >> >> >> >> >>>>>>> <guido at python.org>
>> >> >> >> >> >> >>>>>>>  wrote:
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  There could be a million differences relevant
>> >> >> >> >> >> >>>>>>>> (unicode,
>> >> >> >> >> >> >>>>>>>> ints,
>> >> >> >> >> >> >>>>>>>> ...).
>> >> >> >> >> >> >>>>>>>>  Perhaps
>> >> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps the
>> >> >> >> >> >> >>>>>>>> dynamic
>> >> >> >> >> >> >>>>>>>> loading
>> >> >> >> >> >> >>>>>>>> code
>> >> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC strace
>> >> >> >> >> >> >>>>>>>> shows
>> >> >> >> >> >> >>>>>>>> a
>> >> >> >> >> >> >>>>>>>> syscall
>> >> >> >> >> >> >>>>>>>> trace
>> >> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely how
>> >> >> >> >> >> >>>>>>>> it segfaulted.)
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>> >> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>> >> >> >> >> >> >>>>>>>>  wrote:
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  All,
>> >> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port Python
>> 2.7.8
>> >> >> >> >> >> >>>>>>>>> to
>> >> >> >> >> >> >>>>>>>>> Android
>> >> >> >> >> >> >>>>>>>>> in
>> >> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately, after
>> using
>> >> >> >> >> >> >>>>>>>>> the
>> >> >> >> >> >> >>>>>>>>> same
>> >> >> >> >> >> >>>>>>>>> configure
>> >> >> >> >> >> >>>>>>>>>  options in the same environment, and modifying
>> the
>> >> >> >> >> >> >>>>>>>>> setup.py
>> >> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >> >>>>>>>>> needed,
>> >> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault when
>> the
>> >> >> >> >> >> >>>>>>>>> generate-posix-vars
>> >> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when it is
>> >> >> >> >> >> >>>>>>>>> run
>> >> >> >> >> >> >>>>>>>>> as
>> >> >> >> >> >> >>>>>>>>> well
>> >> >> >> >> >> >>>>>>>>> (i.e.
>> >> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m sysconfig, or
>> >> >> >> >> >> >>>>>>>>> similar)
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a bit
>> lost
>> >> >> >> >> >> >>>>>>>>> when
>> >> >> >> >> >> >>>>>>>>> reviewing
>> >> >> >> >> >> >>>>>>>>> it.
>> >> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e. why
>> >> >> >> >> >> >>>>>>>>> Python
>> >> >> >> >> >> >>>>>>>>> 2.7
>> >> >> >> >> >> >>>>>>>>> works but
>> >> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  Thanks in advance,
>> >> >> >> >> >> >>>>>>>>>  Cyd
>> >> >> >> >> >> >>>>>>>>> ________________________________
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>> >> >> >> >> >> >>>>>>>>>
>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >> >>>>>>>>>  Unsubscribe:
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> >> >> >> >> >> >>>>>>>>>
>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>
>> >> >> >> >> >> >>>>>>>>  --
>> >> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  --
>> >> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>> ________________________________
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>  Python-Dev mailing list
>> >> >> >> >> >> >>>>>>  Python-Dev at python.org
>> >> >> >> >> >> >>>>>>
>> https://mail.python.org/mailman/listinfo/python-dev
>> >> >> >> >> >> >>>>>>  Unsubscribe:
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> >> >> >> >> >> >>>>>>
>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>
>> >> >> >> >> >> >>>>>  --
>> >> >> >> >> >> >>>>>  Ryan
>> >> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C, my
>> >> >> >> >> >> >>>>> answer
>> >> >> >> >> >> >>>>> will
>> >> >> >> >> >> >>>>> be
>> >> >> >> >> >> >>>>> simple:
>> >> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>> >> >> >> >> >> >>>>> don't
>> >> >> >> >> >> >>>>> think
>> >> >> >> >> >> >>>>> that was
>> >> >> >> >> >> >>>>>  nul-terminated."
>> >> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
>> >> >> >> >> >> >>>>> contradictory
>> >> >> >> >> >> >>>>> evidence.
>> >> >> >> >> >> >>>>> -
>> >> >> >> >> >> >>>>>  srean
>> >> >> >> >> >> >>>>>  Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>>
>> >> >> >> >> >> >>> --
>> >> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please
>> excuse my
>> >> >> >> >> >> >>> brevity.
>> >> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> >
>> >> >> >> >> > --
>> >> >> >> >> > Ryan
>> >> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer
>> will
>> >> >> >> >> > be
>> >> >> >> >> > simple:
>> >> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>> think
>> >> >> >> >> > that
>> >> >> >> >> > was
>> >> >> >> >> > nul-terminated."
>> >> >> >> >> > Personal reality distortion fields are immune to
>> contradictory
>> >> >> >> >> > evidence.
>> >> >> >> >> > -
>> >> >> >> >> > srean
>> >> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> >
>> >> >> >> > --
>> >> >> >> > Ryan
>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>> be
>> >> >> >> > simple:
>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> >> >> >> > that
>> >> >> >> > was
>> >> >> >> > nul-terminated."
>> >> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> >> > evidence.
>> >> >> >> > -
>> >> >> >> > srean
>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> > --
>> >> >> > Ryan
>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> >> >> > simple:
>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>> that
>> >> >> > was
>> >> >> > nul-terminated."
>> >> >> > Personal reality distortion fields are immune to contradictory
>> >> >> > evidence.
>> >> >> > -
>> >> >> > srean
>> >> >> > Check out my website: http://kirbyfan64.github.io/
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Ryan
>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>> simple:
>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>> was
>> >> > nul-terminated."
>> >> > Personal reality distortion fields are immune to contradictory
>> evidence.
>> >> > -
>> >> > srean
>> >> > Check out my website: http://kirbyfan64.github.io/
>> >
>> >
>> >
>> >
>> > --
>> > Ryan
>> > If anybody ever asks me why I prefer C++ to C, my answer will be simple:
>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
>> > nul-terminated."
>> > Personal reality distortion fields are immune to contradictory
>> evidence. -
>> > srean
>> > Check out my website: http://kirbyfan64.github.io/
>>
>
>
>
> --
> Ryan
> If anybody ever asks me why I prefer C++ to C, my answer will be simple:
> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
> nul-terminated."
> Personal reality distortion fields are immune to contradictory evidence. -
> srean
> Check out my website: http://kirbyfan64.github.io/
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/

From chaselton at gmail.com  Sat Jan 31 19:12:02 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 31 Jan 2015 12:12:02 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAHu5PragHoXuifGWDLk-Un9e6L6-UMdGEP-EiNNydnH=RNgUmA@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
Message-ID: <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>

Question:
When you said earlier that you found the problem by using grep...were you looking for places where strdup called locale?

On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com> wrote:
>Regardless, if you're looking to toy more with stuff like this, I'd
>highly
>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now I
>rarely ever boot into Windows!)
>
>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>
>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>phone?
>> If so, I can upload instructions that don't use the NDK.
>>
>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>wrote:
>>
>>> This is going to take some time...here's why:
>>>
>>> Downloading and installing the NDK/SDK won't be too hard...I have to
>>> clear some space...but my primary machine is running Windows 7 and
>I'd
>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>> experience with it, other than knowing that the NDK recommends
>against
>>> it.
>>>
>>> I've got a CentOS VM...but it's currently in tarball form on an
>>> external drive for space reasons...and it only has the NDK
>installed.
>>> If I am able to get it back up and running, there's still the task
>of
>>> getting  adb connected to my device. from the VM.
>>>
>>> Not saying it's impossible...just that it'll take time...and I'll
>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>In
>>> the meantime I'll also check to see if there's anything that can a)
>>> run in an Android terminal and b) can take a stack trace; it would
>be
>>> far, far, far easier than either option above.
>>>
>>>
>>>
>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>>> > Do you have the Android SDK and NDK installed? If so...
>>> >
>>> > Using Google, I've created this series of steps, which may (or may
>not)
>>> > work:
>>> >
>>> > 1. Make sure you have a copy of Python on your computer and make
>sure
>>> that
>>> > it's built with debug symbols.
>>> >
>>> > 2. Run the following commands from a shell with your phone plugged
>in:
>>> >
>>> >   adb forward tcp:5039 tcp:5039
>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>executable>
>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>> >
>>> > Now, GDB should have opened, so type the following commands:
>>> >
>>> >   set solib-search-path <directory holder libpython.so>
>>> >   file <path to local python executable>
>>> >   target remote :5055
>>> >   run
>>> >
>>> > Now, wait for the program to crash and type:
>>> >
>>> >   backtrace
>>> >
>>> > You should now see a complete backtrace in your terminal. To GDB,
>type:
>>> >
>>> >   quit
>>> >
>>> > *crosses fingers*
>>> >
>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
><chaselton at gmail.com>
>>> wrote:
>>> >>
>>> >> Unfortunately it is still reporting the same function :-/.
>>> >>
>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>> wrote:
>>> >> > Yes...
>>> >> >
>>> >> > Can you check if it's crashing in a different function now?
>>> >> >
>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
><chaselton at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Yes I did.  I did have to enter all the information in
>>> manually...I'm
>>> >> >> not familiar with automated patch application tools and even
>if I
>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>> >> >>
>>> >> >> Just so that I'm absolutely sure I got everything
>correct...you
>>> wanted
>>> >> >> all of the lines in the patch commented out, correct? 
>Basically
>>> >> >> everything referencing oldloc?
>>> >> >>
>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>> >> >> wrote:
>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>it
>>> would
>>> >> >> > work!
>>> >> >> >
>>> >> >> > FYI, you tried the patch I attached to the email message,
>right?
>>> >> >> >
>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>> chaselton at gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Update:  I did try the patch after getting it installed
>>> correctly,
>>> >> >> >> but
>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>> >> >> >> Will post info this afternoon.
>>> >> >> >>
>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>> rymg19 at gmail.com>
>>> >> >> >> wrote:
>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>> >> >> >> > unconditionally
>>> >> >> >> > checks the length of the (possibly null) string argument
>first.
>>> >> >> >> >
>>> >> >> >> > Please try the patch I attached in the last email. It
>*might*
>>> fix
>>> >> >> >> > the
>>> >> >> >> > issue.
>>> >> >> >> > Android has crappy locale handling.
>>> >> >> >> >
>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>_PyMem_RawStrdup
>>> is
>>> >> >> >> >> currently returning NULL when given a null pointer.
>>> >> >> >> >>
>>> >> >> >> >> From obmalloc.c
>>> >> >> >> >>
>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>> >> >> >> >> {
>>> >> >> >> >>     size_t size;
>>> >> >> >> >> char *copy;
>>> >> >> >> >>     size = strlen(str) + 1;
>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>> >> >> >> >>     if (copy == NULL)
>>> >> >> >> >>         return NULL;
>>> >> >> >> >> memcpy(copy, str, size);
>>> >> >> >> >> return copy;
>>> >> >> >> >>           }
>>> >> >> >> >>
>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>> >> >> >> >> <rymg19 at gmail.com>
>>> >> >> >> >> wrote:
>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>> _PyMem_RawStrdup
>>> >> >> >> >> > crashes
>>> >> >> >> >> > when
>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>likely
>>> >> >> >> >> > asking
>>> >> >> >> >> > it
>>> >> >> >> >> > to
>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>>> caller's
>>> >> >> >> >> > fault.
>>> >> >> >> >> >
>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>return
>>> NULL
>>> >> >> >> >> > when
>>> >> >> >> >> > given a
>>> >> >> >> >> > null pointer and see where it then segfaults.
>>> >> >> >> >> >
>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> > wrote:
>>> >> >> >> >> >>
>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out
>what
>>> >> >> >> >> >> changed
>>> >> >> >> >> >> in
>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>> >> >> >> >> >>
>>> >> >> >> >> >>
>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>> >> >> >> >> >> <chaselton at gmail.com>
>>> >> >> >> >> >> wrote:
>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>mentioned
>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>> >> >> >> >> >> >
>https://github.com/rave-engine/python3-android/issues/2.
>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>for
>>> >> >> >> >> >> > obmalloc.c,
>>> >> >> >> >> >> > as
>>> >> >> >> >> >> > (I think) the functions all belong to
>Python...they're
>>> all
>>> >> >> >> >> >> > prefixed
>>> >> >> >> >> >> > with _PyXx
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>> >> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> >> > wrote:
>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>> >> >> >> >> >> >> 0008bbc8
>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
><rymg19 at gmail.com
>>> >
>>> >> >> >> >> >> >> wrote:
>>> >> >> >> >> >> >>> Could you try the steps at
>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>> crashing.
>>> >> >> >> >> >> >>>
>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>>> >> >> >> >> >> >>>> (code=1),
>>> >> >> >> >> >> >>>> thread
>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>(code=1),
>>> thread
>>> >> >> >> >> >> >>>> 23373
>>> >> >> >> >> >> >>>> (python)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>> python is
>>> >> >> >> >> >> >>>> segfaulting
>>> >> >> >> >> >> >>>> libc...
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>> >> >> >> >> >> >>>> wrote:
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>Rossum
>>> >> >> >> >> >> >>>>> <guido at python.org>
>>> >> >> >> >> >> >>>>> wrote:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>> >> >> >> >> >> >>>>>>  ... load libm
>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something
>to it
>>> >> >> >> >> >> >>>>>> (what?)
>>> >> >> >> >> >> >>>>>>  ... get current time
>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>> >> >> >> >> >> >>>>>>  ... getuid
>>> >> >> >> >> >> >>>>>>  ... segfault
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't
>look as
>>> if
>>> >> >> >> >> >> >>>>>> it
>>> >> >> >> >> >> >>>>>> has
>>> >> >> >> >> >> >>>>>> started to
>>> >> >> >> >> >> >>>>>>  load modules yet.
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>me.
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>>
>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c

-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/a38b1b30/attachment-0001.html>

From rymg19 at gmail.com  Sat Jan 31 20:10:18 2015
From: rymg19 at gmail.com (Ryan)
Date: Sat, 31 Jan 2015 13:10:18 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>
Message-ID: <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>

No; I was looking for all uses of _PyRaw_Strdup. Surprisingly, it's used only a few times.

Cyd Haselton <chaselton at gmail.com> wrote:
>Question:
>When you said earlier that you found the problem by using grep...were
>you looking for places where strdup called locale?
>
>On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>>Regardless, if you're looking to toy more with stuff like this, I'd
>>highly
>>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now
>I
>>rarely ever boot into Windows!)
>>
>>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>
>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>phone?
>>> If so, I can upload instructions that don't use the NDK.
>>>
>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>wrote:
>>>
>>>> This is going to take some time...here's why:
>>>>
>>>> Downloading and installing the NDK/SDK won't be too hard...I have
>to
>>>> clear some space...but my primary machine is running Windows 7 and
>>I'd
>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>> experience with it, other than knowing that the NDK recommends
>>against
>>>> it.
>>>>
>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>> external drive for space reasons...and it only has the NDK
>>installed.
>>>> If I am able to get it back up and running, there's still the task
>>of
>>>> getting  adb connected to my device. from the VM.
>>>>
>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>>In
>>>> the meantime I'll also check to see if there's anything that can a)
>>>> run in an Android terminal and b) can take a stack trace; it would
>>be
>>>> far, far, far easier than either option above.
>>>>
>>>>
>>>>
>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>>> > Do you have the Android SDK and NDK installed? If so...
>>>> >
>>>> > Using Google, I've created this series of steps, which may (or
>may
>>not)
>>>> > work:
>>>> >
>>>> > 1. Make sure you have a copy of Python on your computer and make
>>sure
>>>> that
>>>> > it's built with debug symbols.
>>>> >
>>>> > 2. Run the following commands from a shell with your phone
>plugged
>>in:
>>>> >
>>>> >   adb forward tcp:5039 tcp:5039
>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>executable>
>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>> >
>>>> > Now, GDB should have opened, so type the following commands:
>>>> >
>>>> >   set solib-search-path <directory holder libpython.so>
>>>> >   file <path to local python executable>
>>>> >   target remote :5055
>>>> >   run
>>>> >
>>>> > Now, wait for the program to crash and type:
>>>> >
>>>> >   backtrace
>>>> >
>>>> > You should now see a complete backtrace in your terminal. To GDB,
>>type:
>>>> >
>>>> >   quit
>>>> >
>>>> > *crosses fingers*
>>>> >
>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Unfortunately it is still reporting the same function :-/.
>>>> >>
>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>>> wrote:
>>>> >> > Yes...
>>>> >> >
>>>> >> > Can you check if it's crashing in a different function now?
>>>> >> >
>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Yes I did.  I did have to enter all the information in
>>>> manually...I'm
>>>> >> >> not familiar with automated patch application tools and even
>>if I
>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>> >> >>
>>>> >> >> Just so that I'm absolutely sure I got everything
>>correct...you
>>>> wanted
>>>> >> >> all of the lines in the patch commented out, correct? 
>>Basically
>>>> >> >> everything referencing oldloc?
>>>> >> >>
>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
>><rymg19 at gmail.com>
>>>> >> >> wrote:
>>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>>it
>>>> would
>>>> >> >> > work!
>>>> >> >> >
>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>right?
>>>> >> >> >
>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>> chaselton at gmail.com>
>>>> >> >> > wrote:
>>>> >> >> >>
>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>> correctly,
>>>> >> >> >> but
>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>> >> >> >> Will post info this afternoon.
>>>> >> >> >>
>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>> rymg19 at gmail.com>
>>>> >> >> >> wrote:
>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>>> >> >> >> > unconditionally
>>>> >> >> >> > checks the length of the (possibly null) string argument
>>first.
>>>> >> >> >> >
>>>> >> >> >> > Please try the patch I attached in the last email. It
>>*might*
>>>> fix
>>>> >> >> >> > the
>>>> >> >> >> > issue.
>>>> >> >> >> > Android has crappy locale handling.
>>>> >> >> >> >
>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> > wrote:
>>>> >> >> >> >>
>>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>>_PyMem_RawStrdup
>>>> is
>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>> >> >> >> >>
>>>> >> >> >> >> From obmalloc.c
>>>> >> >> >> >>
>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>> >> >> >> >> {
>>>> >> >> >> >>     size_t size;
>>>> >> >> >> >> char *copy;
>>>> >> >> >> >>     size = strlen(str) + 1;
>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>> >> >> >> >>     if (copy == NULL)
>>>> >> >> >> >>         return NULL;
>>>> >> >> >> >> memcpy(copy, str, size);
>>>> >> >> >> >> return copy;
>>>> >> >> >> >>           }
>>>> >> >> >> >>
>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>> >> >> >> >> <rymg19 at gmail.com>
>>>> >> >> >> >> wrote:
>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>> _PyMem_RawStrdup
>>>> >> >> >> >> > crashes
>>>> >> >> >> >> > when
>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>likely
>>>> >> >> >> >> > asking
>>>> >> >> >> >> > it
>>>> >> >> >> >> > to
>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably
>the
>>>> caller's
>>>> >> >> >> >> > fault.
>>>> >> >> >> >> >
>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>return
>>>> NULL
>>>> >> >> >> >> > when
>>>> >> >> >> >> > given a
>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>> >> >> >> >> >
>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> > wrote:
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find
>out
>>what
>>>> >> >> >> >> >> changed
>>>> >> >> >> >> >> in
>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>> >> >> >> >> >>
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>mentioned
>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>> >> >> >> >> >> >
>>https://github.com/rave-engine/python3-android/issues/2.
>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>>for
>>>> >> >> >> >> >> > obmalloc.c,
>>>> >> >> >> >> >> > as
>>>> >> >> >> >> >> > (I think) the functions all belong to
>>Python...they're
>>>> all
>>>> >> >> >> >> >> > prefixed
>>>> >> >> >> >> >> > with _PyXx
>>>> >> >> >> >> >> >
>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> >> > wrote:
>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on
>device
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>> >> >> >> >> >> >> 0008bbc8
>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
>><rymg19 at gmail.com
>>>> >
>>>> >> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> >>> Could you try the steps at
>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780?
>They
>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>> crashing.
>>>> >> >> >> >> >> >>>
>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at
>0x00000000
>>>> >> >> >> >> >> >>>> (code=1),
>>>> >> >> >> >> >> >>>> thread
>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>>(code=1),
>>>> thread
>>>> >> >> >> >> >> >>>> 23373
>>>> >> >> >> >> >> >>>> (python)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>>> python is
>>>> >> >> >> >> >> >>>> segfaulting
>>>> >> >> >> >> >> >>>> libc...
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>>> >> >> >> >> >> >>>> wrote:
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>>Rossum
>>>> >> >> >> >> >> >>>>> <guido at python.org>
>>>> >> >> >> >> >> >>>>> wrote:
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>>> >> >> >> >> >> >>>>>>  ... load libm
>>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do
>something
>>to it
>>>> >> >> >> >> >> >>>>>> (what?)
>>>> >> >> >> >> >> >>>>>>  ... get current time
>>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>>> >> >> >> >> >> >>>>>>  ... getuid
>>>> >> >> >> >> >> >>>>>>  ... segfault
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't
>>look as
>>>> if
>>>> >> >> >> >> >> >>>>>> it
>>>> >> >> >> >> >> >>>>>> has
>>>> >> >> >> >> >> >>>>>> started to
>>>> >> >> >> >> >> >>>>>>  load modules yet.
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>>me.
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>>
>>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/a10b417d/attachment.html>

From chaselton at gmail.com  Sat Jan 31 20:15:08 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 31 Jan 2015 13:15:08 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>
 <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>
Message-ID: <371EAB7B-4965-49AB-8CE7-479B5515B091@gmail.com>

There were references to oldloc in Modules/python.c...a duplicate of the line in frozenmain.c and some others. On a hunch I commented them out, ran make clean && make.

If it fails, I'll revert and use the method you sent in the email after this (thanks...by the way!)

On January 31, 2015 1:10:18 PM CST, Ryan <rymg19 at gmail.com> wrote:
>No; I was looking for all uses of _PyRaw_Strdup. Surprisingly, it's
>used only a few times.
>
>Cyd Haselton <chaselton at gmail.com> wrote:
>>Question:
>>When you said earlier that you found the problem by using grep...were
>>you looking for places where strdup called locale?
>>
>>On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>>Regardless, if you're looking to toy more with stuff like this, I'd
>>>highly
>>>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now
>>I
>>>rarely ever boot into Windows!)
>>>
>>>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>wrote:
>>>
>>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>>phone?
>>>> If so, I can upload instructions that don't use the NDK.
>>>>
>>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>>wrote:
>>>>
>>>>> This is going to take some time...here's why:
>>>>>
>>>>> Downloading and installing the NDK/SDK won't be too hard...I have
>>to
>>>>> clear some space...but my primary machine is running Windows 7 and
>>>I'd
>>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>>> experience with it, other than knowing that the NDK recommends
>>>against
>>>>> it.
>>>>>
>>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>>> external drive for space reasons...and it only has the NDK
>>>installed.
>>>>> If I am able to get it back up and running, there's still the task
>>>of
>>>>> getting  adb connected to my device. from the VM.
>>>>>
>>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest).
>
>>>In
>>>>> the meantime I'll also check to see if there's anything that can
>a)
>>>>> run in an Android terminal and b) can take a stack trace; it would
>>>be
>>>>> far, far, far easier than either option above.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>wrote:
>>>>> > Do you have the Android SDK and NDK installed? If so...
>>>>> >
>>>>> > Using Google, I've created this series of steps, which may (or
>>may
>>>not)
>>>>> > work:
>>>>> >
>>>>> > 1. Make sure you have a copy of Python on your computer and make
>>>sure
>>>>> that
>>>>> > it's built with debug symbols.
>>>>> >
>>>>> > 2. Run the following commands from a shell with your phone
>>plugged
>>>in:
>>>>> >
>>>>> >   adb forward tcp:5039 tcp:5039
>>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>>executable>
>>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>>> >
>>>>> > Now, GDB should have opened, so type the following commands:
>>>>> >
>>>>> >   set solib-search-path <directory holder libpython.so>
>>>>> >   file <path to local python executable>
>>>>> >   target remote :5055
>>>>> >   run
>>>>> >
>>>>> > Now, wait for the program to crash and type:
>>>>> >
>>>>> >   backtrace
>>>>> >
>>>>> > You should now see a complete backtrace in your terminal. To
>GDB,
>>>type:
>>>>> >
>>>>> >   quit
>>>>> >
>>>>> > *crosses fingers*
>>>>> >
>>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
>>><chaselton at gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Unfortunately it is still reporting the same function :-/.
>>>>> >>
>>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez
>><rymg19 at gmail.com>
>>>>> wrote:
>>>>> >> > Yes...
>>>>> >> >
>>>>> >> > Can you check if it's crashing in a different function now?
>>>>> >> >
>>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
>>><chaselton at gmail.com>
>>>>> >> > wrote:
>>>>> >> >>
>>>>> >> >> Yes I did.  I did have to enter all the information in
>>>>> manually...I'm
>>>>> >> >> not familiar with automated patch application tools and even
>>>if I
>>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>>> >> >>
>>>>> >> >> Just so that I'm absolutely sure I got everything
>>>correct...you
>>>>> wanted
>>>>> >> >> all of the lines in the patch commented out, correct? 
>>>Basically
>>>>> >> >> everything referencing oldloc?
>>>>> >> >>
>>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
>>><rymg19 at gmail.com>
>>>>> >> >> wrote:
>>>>> >> >> > Are you sure the patch was applied correctly? I was SO
>sure
>>>it
>>>>> would
>>>>> >> >> > work!
>>>>> >> >> >
>>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>>right?
>>>>> >> >> >
>>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>>> chaselton at gmail.com>
>>>>> >> >> > wrote:
>>>>> >> >> >>
>>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>>> correctly,
>>>>> >> >> >> but
>>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>>> >> >> >> Will post info this afternoon.
>>>>> >> >> >>
>>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>>> rymg19 at gmail.com>
>>>>> >> >> >> wrote:
>>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer.
>It
>>>>> >> >> >> > unconditionally
>>>>> >> >> >> > checks the length of the (possibly null) string
>argument
>>>first.
>>>>> >> >> >> >
>>>>> >> >> >> > Please try the patch I attached in the last email. It
>>>*might*
>>>>> fix
>>>>> >> >> >> > the
>>>>> >> >> >> > issue.
>>>>> >> >> >> > Android has crappy locale handling.
>>>>> >> >> >> >
>>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>>> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> > wrote:
>>>>> >> >> >> >>
>>>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>>>_PyMem_RawStrdup
>>>>> is
>>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>>> >> >> >> >>
>>>>> >> >> >> >> From obmalloc.c
>>>>> >> >> >> >>
>>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>>> >> >> >> >> {
>>>>> >> >> >> >>     size_t size;
>>>>> >> >> >> >> char *copy;
>>>>> >> >> >> >>     size = strlen(str) + 1;
>>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>>> >> >> >> >>     if (copy == NULL)
>>>>> >> >> >> >>         return NULL;
>>>>> >> >> >> >> memcpy(copy, str, size);
>>>>> >> >> >> >> return copy;
>>>>> >> >> >> >>           }
>>>>> >> >> >> >>
>>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>>> >> >> >> >> <rymg19 at gmail.com>
>>>>> >> >> >> >> wrote:
>>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>>> _PyMem_RawStrdup
>>>>> >> >> >> >> > crashes
>>>>> >> >> >> >> > when
>>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>>likely
>>>>> >> >> >> >> > asking
>>>>> >> >> >> >> > it
>>>>> >> >> >> >> > to
>>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably
>>the
>>>>> caller's
>>>>> >> >> >> >> > fault.
>>>>> >> >> >> >> >
>>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>>return
>>>>> NULL
>>>>> >> >> >> >> > when
>>>>> >> >> >> >> > given a
>>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>>> >> >> >> >> >
>>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>>> >> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> >> > wrote:
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find
>>out
>>>what
>>>>> >> >> >> >> >> changed
>>>>> >> >> >> >> >> in
>>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>>> >> >> >> >> >> wrote:
>>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>>mentioned
>>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>>> >> >> >> >> >> >
>>>https://github.com/rave-engine/python3-android/issues/2.
>>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify
>it
>>>for
>>>>> >> >> >> >> >> > obmalloc.c,
>>>>> >> >> >> >> >> > as
>>>>> >> >> >> >> >> > (I think) the functions all belong to
>>>Python...they're
>>>>> all
>>>>> >> >> >> >> >> > prefixed
>>>>> >> >> >> >> >> > with _PyXx
>>>>> >> >> >> >> >> >
>>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> >> >> > wrote:
>>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on
>>device
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>>> >> >> >> >> >> >> 0008bbc8
>>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
>>><rymg19 at gmail.com
>>>>> >
>>>>> >> >> >> >> >> >> wrote:
>>>>> >> >> >> >> >> >>> Could you try the steps at
>>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780?
>>They
>>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>>> crashing.
>>>>> >> >> >> >> >> >>>
>>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at
>>0x00000000
>>>>> >> >> >> >> >> >>>> (code=1),
>>>>> >> >> >> >> >> >>>> thread
>>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>>>(code=1),
>>>>> thread
>>>>> >> >> >> >> >> >>>> 23373
>>>>> >> >> >> >> >> >>>> (python)
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be
>that
>>>>> python is
>>>>> >> >> >> >> >> >>>> segfaulting
>>>>> >> >> >> >> >> >>>> libc...
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan
>Gonzalez
>>>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>>>> >> >> >> >> >> >>>> wrote:
>>>>> >> >> >> >> >> >>>>>
>>>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>>>Rossum
>>>>> >> >> >> >> >> >>>>> <guido at python.org>
>>>>> >> >> >> >> >> >>>>> wrote:
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>>>> >> >> >> >> >> >>>>>>  ... load libm
>>>>> >> >> >> >> >> >>>>>>  ... open /dev/__properti
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/9f96afe5/attachment-0001.html>

From chaselton at gmail.com  Sat Jan 31 20:21:43 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 31 Jan 2015 13:21:43 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>
 <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>
Message-ID: <B42BA6C5-A292-42D2-A99D-7DEF95B9FE1C@gmail.com>

Very interesting. I got this error

Fatal Python error: Py_Initialize: Unable to get the locale encoding
NotImplementedError
Aborted
generate-posix-vars failed
make: *** [pybuilddir.txt] Error 1

...but it didn't (of course) segfault.  I'll pull gdb, get the results and send them.

On January 31, 2015 1:10:18 PM CST, Ryan <rymg19 at gmail.com> wrote:
>No; I was looking for all uses of _PyRaw_Strdup. Surprisingly, it's
>used only a few times.
>
>Cyd Haselton <chaselton at gmail.com> wrote:
>>Question:
>>When you said earlier that you found the problem by using grep...were
>>you looking for places where strdup called locale?
>>
>>On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>>Regardless, if you're looking to toy more with stuff like this, I'd
>>>highly
>>>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now
>>I
>>>rarely ever boot into Windows!)
>>>
>>>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>wrote:
>>>
>>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>>phone?
>>>> If so, I can upload instructions that don't use the NDK.
>>>>
>>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>>wrote:
>>>>
>>>>> This is going to take some time...here's why:
>>>>>
>>>>> Downloading and installing the NDK/SDK won't be too hard...I have
>>to
>>>>> clear some space...but my primary machine is running Windows 7 and
>>>I'd
>>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>>> experience with it, other than knowing that the NDK recommends
>>>against
>>>>> it.
>>>>>
>>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>>> external drive for space reasons...and it only has the NDK
>>>installed.
>>>>> If I am able to get it back up and running, there's still the task
>>>of
>>>>> getting  adb connected to my device. from the VM.
>>>>>
>>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest).
>
>>>In
>>>>> the meantime I'll also check to see if there's anything that can
>a)
>>>>> run in an Android terminal and b) can take a stack trace; it would
>>>be
>>>>> far, far, far easier than either option above.
>>>>>
>>>>>
>>>>>
>>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>wrote:
>>>>> > Do you have the Android SDK and NDK installed? If so...
>>>>> >
>>>>> > Using Google, I've created this series of steps, which may (or
>>may
>>>not)
>>>>> > work:
>>>>> >
>>>>> > 1. Make sure you have a copy of Python on your computer and make
>>>sure
>>>>> that
>>>>> > it's built with debug symbols.
>>>>> >
>>>>> > 2. Run the following commands from a shell with your phone
>>plugged
>>>in:
>>>>> >
>>>>> >   adb forward tcp:5039 tcp:5039
>>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>>executable>
>>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>>> >
>>>>> > Now, GDB should have opened, so type the following commands:
>>>>> >
>>>>> >   set solib-search-path <directory holder libpython.so>
>>>>> >   file <path to local python executable>
>>>>> >   target remote :5055
>>>>> >   run
>>>>> >
>>>>> > Now, wait for the program to crash and type:
>>>>> >
>>>>> >   backtrace
>>>>> >
>>>>> > You should now see a complete backtrace in your terminal. To
>GDB,
>>>type:
>>>>> >
>>>>> >   quit
>>>>> >
>>>>> > *crosses fingers*
>>>>> >
>>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
>>><chaselton at gmail.com>
>>>>> wrote:
>>>>> >>
>>>>> >> Unfortunately it is still reporting the same function :-/.
>>>>> >>
>>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez
>><rymg19 at gmail.com>
>>>>> wrote:
>>>>> >> > Yes...
>>>>> >> >
>>>>> >> > Can you check if it's crashing in a different function now?
>>>>> >> >
>>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
>>><chaselton at gmail.com>
>>>>> >> > wrote:
>>>>> >> >>
>>>>> >> >> Yes I did.  I did have to enter all the information in
>>>>> manually...I'm
>>>>> >> >> not familiar with automated patch application tools and even
>>>if I
>>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>>> >> >>
>>>>> >> >> Just so that I'm absolutely sure I got everything
>>>correct...you
>>>>> wanted
>>>>> >> >> all of the lines in the patch commented out, correct? 
>>>Basically
>>>>> >> >> everything referencing oldloc?
>>>>> >> >>
>>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
>>><rymg19 at gmail.com>
>>>>> >> >> wrote:
>>>>> >> >> > Are you sure the patch was applied correctly? I was SO
>sure
>>>it
>>>>> would
>>>>> >> >> > work!
>>>>> >> >> >
>>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>>right?
>>>>> >> >> >
>>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>>> chaselton at gmail.com>
>>>>> >> >> > wrote:
>>>>> >> >> >>
>>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>>> correctly,
>>>>> >> >> >> but
>>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>>> >> >> >> Will post info this afternoon.
>>>>> >> >> >>
>>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>>> rymg19 at gmail.com>
>>>>> >> >> >> wrote:
>>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer.
>It
>>>>> >> >> >> > unconditionally
>>>>> >> >> >> > checks the length of the (possibly null) string
>argument
>>>first.
>>>>> >> >> >> >
>>>>> >> >> >> > Please try the patch I attached in the last email. It
>>>*might*
>>>>> fix
>>>>> >> >> >> > the
>>>>> >> >> >> > issue.
>>>>> >> >> >> > Android has crappy locale handling.
>>>>> >> >> >> >
>>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>>> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> > wrote:
>>>>> >> >> >> >>
>>>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>>>_PyMem_RawStrdup
>>>>> is
>>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>>> >> >> >> >>
>>>>> >> >> >> >> From obmalloc.c
>>>>> >> >> >> >>
>>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>>> >> >> >> >> {
>>>>> >> >> >> >>     size_t size;
>>>>> >> >> >> >> char *copy;
>>>>> >> >> >> >>     size = strlen(str) + 1;
>>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>>> >> >> >> >>     if (copy == NULL)
>>>>> >> >> >> >>         return NULL;
>>>>> >> >> >> >> memcpy(copy, str, size);
>>>>> >> >> >> >> return copy;
>>>>> >> >> >> >>           }
>>>>> >> >> >> >>
>>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>>> >> >> >> >> <rymg19 at gmail.com>
>>>>> >> >> >> >> wrote:
>>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>>> _PyMem_RawStrdup
>>>>> >> >> >> >> > crashes
>>>>> >> >> >> >> > when
>>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>>likely
>>>>> >> >> >> >> > asking
>>>>> >> >> >> >> > it
>>>>> >> >> >> >> > to
>>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably
>>the
>>>>> caller's
>>>>> >> >> >> >> > fault.
>>>>> >> >> >> >> >
>>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>>return
>>>>> NULL
>>>>> >> >> >> >> > when
>>>>> >> >> >> >> > given a
>>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>>> >> >> >> >> >
>>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>>> >> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> >> > wrote:
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find
>>out
>>>what
>>>>> >> >> >> >> >> changed
>>>>> >> >> >> >> >> in
>>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >>
>>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>>> >> >> >> >> >> wrote:
>>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>>mentioned
>>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>>> >> >> >> >> >> >
>>>https://github.com/rave-engine/python3-android/issues/2.
>>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify
>it
>>>for
>>>>> >> >> >> >> >> > obmalloc.c,
>>>>> >> >> >> >> >> > as
>>>>> >> >> >> >> >> > (I think) the functions all belong to
>>>Python...they're
>>>>> all
>>>>> >> >> >> >> >> > prefixed
>>>>> >> >> >> >> >> > with _PyXx
>>>>> >> >> >> >> >> >
>>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>>> >> >> >> >> >> > wrote:
>>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on
>>device
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>>> >> >> >> >> >> >> 0008bbc8
>>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >>
>>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
>>><rymg19 at gmail.com
>>>>> >
>>>>> >> >> >> >> >> >> wrote:
>>>>> >> >> >> >> >> >>> Could you try the steps at
>>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780?
>>They
>>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>>> crashing.
>>>>> >> >> >> >> >> >>>
>>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at
>>0x00000000
>>>>> >> >> >> >> >> >>>> (code=1),
>>>>> >> >> >> >> >> >>>> thread
>>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>>>(code=1),
>>>>> thread
>>>>> >> >> >> >> >> >>>> 23373
>>>>> >> >> >> >> >> >>>> (python)
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be
>that
>>>>> python is
>>>>> >> >> >> >> >> >>>> segfaulting
>>>>> >> >> >> >> >> >>>> libc...
>>>>> >> >> >> >> >> >>>>
>>>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan
>Gonzalez
>>>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>>>> >> >> >> >> >> >>>> wrote:
>>>>> >> >> >> >> >> >>>>>
>>>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>>>Rossum
>>>>> >> >> >> >> >> >>>>> <guido at python.org>
>>>>> >> >> >> >> >> >>>>> wrote:
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>>>> >> >> >> >> >> >>>>>>
>>>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>>>> >> >> >> >> >> >>>>>>  ... load libm
>>>>> >> >> >> >> >> >>>>>>  ... open /dev/__properti
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/d49be8ae/attachment-0001.html>

From rymg19 at gmail.com  Sat Jan 31 20:09:22 2015
From: rymg19 at gmail.com (Ryan)
Date: Sat, 31 Jan 2015 13:09:22 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <7545e812-1a79-4389-8add-c0d6acdf03b3.maildroid@localhost>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAO41-mOxsAq=KDn67jVK5Y=zfQmyRM9GUSncG3xch6hztkmUtw@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <7545e812-1a79-4389-8add-c0d6acdf03b3.maildroid@localhost>
Message-ID: <b8fb2129-c394-4739-a06f-8d43cfdd9f65@email.android.com>

Ok...try this (based on http://dan.drown.org/android/howto/gdb.html):

- Install BusyBox and a Terminal Emulator
- Inside the Terminal Emulator, run:

  su
  cd /data/local/tmp
  wget http://dan.drown.org/android/gdb-
static.tar.gz
  tar zxf gdb-static.tar.gz
  ./gdb <path to python interpreter>

Now, inside gdb, type:

  set logging file /mnt/sdcard/bt.txt
  set logging on
  run

Wait for Python to crash, then type:

  backtrace

A backtrace should be printed to the screen and saved to a file named bt.txt on the SD card. After that, type:

  quit

to quit GDB. Send the list the bt.txt file on your SD card.

chaselton at gmail.com wrote:
>I don't have the SDK either...but my device is rooted.
>
>Dual-booting is not an option unfortunately...space reasons.  I'll do
>my best to figure out a way to get the instuctions you sent
>implemented, but this may be a deal-breaker for porting Python 3.4.x
>for me...I may go back to working on 2.7.x
>
>Sent from my android device.
>
>-----Original Message-----
>From: Ryan Gonzalez <rymg19 at gmail.com>
>To: Cyd Haselton <chaselton at gmail.com>
>Cc: Python-Dev <python-dev at python.org>
>Sent: Fri, 30 Jan 2015 7:53 PM
>Subject: Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault
>
>Regardless, if you're looking to toy more with stuff like this, I'd
>highly
>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now I
>rarely ever boot into Windows!)
>
>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>
>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>phone?
>> If so, I can upload instructions that don't use the NDK.
>>
>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>wrote:
>>
>>> This is going to take some time...here's why:
>>>
>>> Downloading and installing the NDK/SDK won't be too hard...I have to
>>> clear some space...but my primary machine is running Windows 7 and
>I'd
>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>> experience with it, other than knowing that the NDK recommends
>against
>>> it.
>>>
>>> I've got a CentOS VM...but it's currently in tarball form on an
>>> external drive for space reasons...and it only has the NDK
>installed.
>>> If I am able to get it back up and running, there's still the task
>of
>>> getting  adb connected to my device. from the VM.
>>>
>>> Not saying it's impossible...just that it'll take time...and I'll
>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>In
>>> the meantime I'll also check to see if there's anything that can a)
>>> run in an Android terminal and b) can take a stack trace; it would
>be
>>> far, far, far easier than either option above.
>>>
>>>
>>>
>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>>> > Do you have the Android SDK and NDK installed? If so...
>>> >
>>> > Using Google, I've created this series of steps, which may (or may
>not)
>>> > work:
>>> >
>>> > 1. Make sure you have a copy of Python on your computer and make
>sure
>>> that
>>> > it's built with debug symbols.
>>> >
>>> > 2. Run the following commands from a shell with your phone plugged
>in:
>>> >
>>> >   adb forward tcp:5039 tcp:5039
>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>executable>
>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>> >
>>> > Now, GDB should have opened, so type the following commands:
>>> >
>>> >   set solib-search-path <directory holder libpython.so>
>>> >   file <path to local python executable>
>>> >   target remote :5055
>>> >   run
>>> >
>>> > Now, wait for the program to crash and type:
>>> >
>>> >   backtrace
>>> >
>>> > You should now see a complete backtrace in your terminal. To GDB,
>type:
>>> >
>>> >   quit
>>> >
>>> > *crosses fingers*
>>> >
>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
><chaselton at gmail.com>
>>> wrote:
>>> >>
>>> >> Unfortunately it is still reporting the same function :-/.
>>> >>
>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>> wrote:
>>> >> > Yes...
>>> >> >
>>> >> > Can you check if it's crashing in a different function now?
>>> >> >
>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
><chaselton at gmail.com>
>>> >> > wrote:
>>> >> >>
>>> >> >> Yes I did.  I did have to enter all the information in
>>> manually...I'm
>>> >> >> not familiar with automated patch application tools and even
>if I
>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>> >> >>
>>> >> >> Just so that I'm absolutely sure I got everything
>correct...you
>>> wanted
>>> >> >> all of the lines in the patch commented out, correct? 
>Basically
>>> >> >> everything referencing oldloc?
>>> >> >>
>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>> >> >> wrote:
>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>it
>>> would
>>> >> >> > work!
>>> >> >> >
>>> >> >> > FYI, you tried the patch I attached to the email message,
>right?
>>> >> >> >
>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>> chaselton at gmail.com>
>>> >> >> > wrote:
>>> >> >> >>
>>> >> >> >> Update:  I did try the patch after getting it installed
>>> correctly,
>>> >> >> >> but
>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>> >> >> >> Will post info this afternoon.
>>> >> >> >>
>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>> rymg19 at gmail.com>
>>> >> >> >> wrote:
>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>> >> >> >> > unconditionally
>>> >> >> >> > checks the length of the (possibly null) string argument
>first.
>>> >> >> >> >
>>> >> >> >> > Please try the patch I attached in the last email. It
>*might*
>>> fix
>>> >> >> >> > the
>>> >> >> >> > issue.
>>> >> >> >> > Android has crappy locale handling.
>>> >> >> >> >
>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> > wrote:
>>> >> >> >> >>
>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>_PyMem_RawStrdup
>>> is
>>> >> >> >> >> currently returning NULL when given a null pointer.
>>> >> >> >> >>
>>> >> >> >> >> From obmalloc.c
>>> >> >> >> >>
>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>> >> >> >> >> {
>>> >> >> >> >>     size_t size;
>>> >> >> >> >> char *copy;
>>> >> >> >> >>     size = strlen(str) + 1;
>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>> >> >> >> >>     if (copy == NULL)
>>> >> >> >> >>         return NULL;
>>> >> >> >> >> memcpy(copy, str, size);
>>> >> >> >> >> return copy;
>>> >> >> >> >>           }
>>> >> >> >> >>
>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>> >> >> >> >> <rymg19 at gmail.com>
>>> >> >> >> >> wrote:
>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>> _PyMem_RawStrdup
>>> >> >> >> >> > crashes
>>> >> >> >> >> > when
>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>likely
>>> >> >> >> >> > asking
>>> >> >> >> >> > it
>>> >> >> >> >> > to
>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>>> caller's
>>> >> >> >> >> > fault.
>>> >> >> >> >> >
>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>return
>>> NULL
>>> >> >> >> >> > when
>>> >> >> >> >> > given a
>>> >> >> >> >> > null pointer and see where it then segfaults.
>>> >> >> >> >> >
>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> > wrote:
>>> >> >> >> >> >>
>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out
>what
>>> >> >> >> >> >> changed
>>> >> >> >> >> >> in
>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>> >> >> >> >> >>
>>> >> >> >> >> >>
>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>> >> >> >> >> >> <chaselton at gmail.com>
>>> >> >> >> >> >> wrote:
>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>mentioned
>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>> >> >> >> >> >> >
>https://github.com/rave-engine/python3-android/issues/2.
>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>for
>>> >> >> >> >> >> > obmalloc.c,
>>> >> >> >> >> >> > as
>>> >> >> >> >> >> > (I think) the functions all belong to
>Python...they're
>>> all
>>> >> >> >> >> >> > prefixed
>>> >> >> >> >> >> > with _PyXx
>>> >> >> >> >> >> >
>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>> >> >> >> >> >> > <chaselton at gmail.com>
>>> >> >> >> >> >> > wrote:
>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>> >> >> >> >> >> >> 0008bbc8
>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >>
>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
><rymg19 at gmail.com
>>> >
>>> >> >> >> >> >> >> wrote:
>>> >> >> >> >> >> >>> Could you try the steps at
>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>> crashing.
>>> >> >> >> >> >> >>>
>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>>> >> >> >> >> >> >>>> (code=1),
>>> >> >> >> >> >> >>>> thread
>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>(code=1),
>>> thread
>>> >> >> >> >> >> >>>> 23373
>>> >> >> >> >> >> >>>> (python)
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>> python is
>>> >> >> >> >> >> >>>> segfaulting
>>> >> >> >> >> >> >>>> libc...
>>> >> >> >> >> >> >>>>
>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>> >> >> >> >> >> >>>> wrote:
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>Rossum
>>> >> >> >> >> >> >>>>> <guido at python.org>
>>> >> >> >> >> >> >>>>> wrote:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>> >> >> >> >> >> >>>>>>  ... load libm
>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something
>to it
>>> >> >> >> >> >> >>>>>> (what?)
>>> >> >> >> >> >> >>>>>>  ... get current time
>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>> >> >> >> >> >> >>>>>>  ... getuid
>>> >> >> >> >> >> >>>>>>  ... segfault
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't
>look as
>>> if
>>> >> >> >> >> >> >>>>>> it
>>> >> >> >> >> >> >>>>>> has
>>> >> >> >> >> >> >>>>>> started to
>>> >> >> >> >> >> >>>>>>  load modules yet.
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>me.
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>>
>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>> >> >> >> >> >> >>>>>  is the code that works with that file.
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks
>like
>>> >> >> >> >> >> >>>>> something
>>> >> >> >> >> >> >>>>> to
>>> >> >> >> >> >> >>>>> do
>>> >> >> >> >> >> >>>>> with
>>> >> >> >> >> >> >>>>>  interprocess communication. Likely has nothing
>to do
>>> >> >> >> >> >> >>>>> with
>>> >> >> >> >> >> >>>>> Python
>>> >> >> >> >> >> >>>>> itself.
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  Maybe this would be useful?
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  That stack trace would be really helpful.
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>> >> >> >> >> >> >>>>>> <chaselton at gmail.com>
>>> >> >> >> >> >> >>>>>> wrote:
>>> >> >> >> >> >> >>>>>>>
>>> >> >> >> >> >> >>>>>>>
>>> >> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track
>is,
>>> but I
>>> >> >> >> >> >> >>>>>>> do
>>> >> >> >> >> >> >>>>>>> have
>>> >> >> >> >> >> >>>>>>> the
>>> >> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due
>to an
>>> >> >> >> >> >> >>>>>>> open
>>> >> >> >> >> >> >>>>>>> call,
>>> >> >> >> >> >> >>>>>>> though I
>>> >> >> >> >> >> >>>>>>>  am probably wrong.
>>> >> >> >> >> >> >>>>>>>  Attaching the strace output to this email. 
>I'm
>>> going
>>> >> >> >> >> >> >>>>>>> to
>>> >> >> >> >> >> >>>>>>> head
>>> >> >> >> >> >> >>>>>>> back to
>>> >> >> >> >> >> >>>>>>>  the documentation and to back out of some
>>> >> >> >> >> >> >>>>>>> Android-related
>>> >> >> >> >> >> >>>>>>> changes
>>> >> >> >> >> >> >>>>>>> in
>>> >> >> >> >> >> >>>>>>>  _localemodule.c
>>> >> >> >> >> >> >>>>>>>
>>> >> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van
>Rossum
>>> >> >> >> >> >> >>>>>>> <guido at python.org>
>>> >> >> >> >> >> >>>>>>>  wrote:
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>  There could be a million differences
>relevant
>>> >> >> >> >> >> >>>>>>>> (unicode,
>>> >> >> >> >> >> >>>>>>>> ints,
>>> >> >> >> >> >> >>>>>>>> ...).
>>> >> >> >> >> >> >>>>>>>>  Perhaps
>>> >> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps
>the
>>> >> >> >> >> >> >>>>>>>> dynamic
>>> >> >> >> >> >> >>>>>>>> loading
>>> >> >> >> >> >> >>>>>>>> code
>>> >> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC
>strace
>>> >> >> >> >> >> >>>>>>>> shows
>>> >> >> >> >> >> >>>>>>>> a
>>> >> >> >> >> >> >>>>>>>> syscall
>>> >> >> >> >> >> >>>>>>>> trace
>>> >> >> >> >> >> >>>>>>>>  --
>>> >> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely
>how
>>> >> >> >> >> >> >>>>>>>> it segfaulted.)
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd
>Haselton
>>> >> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>>> >> >> >> >> >> >>>>>>>>  wrote:
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>  All,
>>> >> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port
>Python
>>> 2.7.8
>>> >> >> >> >> >> >>>>>>>>> to
>>> >> >> >> >> >> >>>>>>>>> Android
>>> >> >> >> >> >> >>>>>>>>> in
>>> >> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately,
>after
>>> using
>>> >> >> >> >> >> >>>>>>>>> the
>>> >> >> >> >> >> >>>>>>>>> same
>>> >> >> >> >> >> >>>>>>>>> configure
>>> >> >> >> >> >> >>>>>>>>>  options in the same environment, and
>modifying
>>> the
>>> >> >> >> >> >> >>>>>>>>> setup.py
>>> >> >> >> >> >> >>>>>>>>> as
>>> >> >> >> >> >> >>>>>>>>> needed,
>>> >> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault
>when
>>> the
>>> >> >> >> >> >> >>>>>>>>> generate-posix-vars
>>> >> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when
>it is
>>> >> >> >> >> >> >>>>>>>>> run
>>> >> >> >> >> >> >>>>>>>>> as
>>> >> >> >> >> >> >>>>>>>>> well
>>> >> >> >> >> >> >>>>>>>>> (i.e.
>>> >> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m
>sysconfig, or
>>> >> >> >> >> >> >>>>>>>>> similar)
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a
>bit
>>> lost
>>> >> >> >> >> >> >>>>>>>>> when
>>> >> >> >> >> >> >>>>>>>>> reviewing
>>> >> >> >> >> >> >>>>>>>>> it.
>>> >> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going
>on...i.e. why
>>> >> >> >> >> >> >>>>>>>>> Python
>>> >> >> >> >> >> >>>>>>>>> 2.7
>>> >> >> >> >> >> >>>>>>>>> works but
>>> >> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>  Thanks in advance,
>>> >> >> >> >> >> >>>>>>>>>  Cyd
>>> >> >> >> >> >> >>>>>>>>> ________________________________
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>>> >> >> >> >> >> >>>>>>>>>
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> >> >> >> >> >> >>>>>>>>>  Unsubscribe:
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>> >> >> >> >> >> >>>>>>>>>
>>>
>https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>
>>> >> >> >> >> >> >>>>>>>>  --
>>> >> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  --
>>> >> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>> ________________________________
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>  Python-Dev mailing list
>>> >> >> >> >> >> >>>>>>  Python-Dev at python.org
>>> >> >> >> >> >> >>>>>>
>>> https://mail.python.org/mailman/listinfo/python-dev
>>> >> >> >> >> >> >>>>>>  Unsubscribe:
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>> >> >> >> >> >> >>>>>>
>>>
>https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>
>>> >> >> >> >> >> >>>>>  --
>>> >> >> >> >> >> >>>>>  Ryan
>>> >> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C,
>my
>>> >> >> >> >> >> >>>>> answer
>>> >> >> >> >> >> >>>>> will
>>> >> >> >> >> >> >>>>> be
>>> >> >> >> >> >> >>>>> simple:
>>> >> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT.
>Wait, I
>>> >> >> >> >> >> >>>>> don't
>>> >> >> >> >> >> >>>>> think
>>> >> >> >> >> >> >>>>> that was
>>> >> >> >> >> >> >>>>>  nul-terminated."
>>> >> >> >> >> >> >>>>>  Personal reality distortion fields are immune
>to
>>> >> >> >> >> >> >>>>> contradictory
>>> >> >> >> >> >> >>>>> evidence.
>>> >> >> >> >> >> >>>>> -
>>> >> >> >> >> >> >>>>>  srean
>>> >> >> >> >> >> >>>>>  Check out my website:
>http://kirbyfan64.github.io/
>>> >> >> >> >> >> >>>
>>> >> >> >> >> >> >>>
>>> >> >> >> >> >> >>> --
>>> >> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please
>>> excuse my
>>> >> >> >> >> >> >>> brevity.
>>> >> >> >> >> >> >>> Check out my website:
>http://kirbyfan64.github.io/
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> >
>>> >> >> >> >> > --
>>> >> >> >> >> > Ryan
>>> >> >> >> >> > If anybody ever asks me why I prefer C++ to C, my
>answer
>>> will
>>> >> >> >> >> > be
>>> >> >> >> >> > simple:
>>> >> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>don't
>>> think
>>> >> >> >> >> > that
>>> >> >> >> >> > was
>>> >> >> >> >> > nul-terminated."
>>> >> >> >> >> > Personal reality distortion fields are immune to
>>> contradictory
>>> >> >> >> >> > evidence.
>>> >> >> >> >> > -
>>> >> >> >> >> > srean
>>> >> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> >
>>> >> >> >> > --
>>> >> >> >> > Ryan
>>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer
>will
>>> be
>>> >> >> >> > simple:
>>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>think
>>> >> >> >> > that
>>> >> >> >> > was
>>> >> >> >> > nul-terminated."
>>> >> >> >> > Personal reality distortion fields are immune to
>contradictory
>>> >> >> >> > evidence.
>>> >> >> >> > -
>>> >> >> >> > srean
>>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> >
>>> >> >> > --
>>> >> >> > Ryan
>>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer
>will be
>>> >> >> > simple:
>>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>think
>>> that
>>> >> >> > was
>>> >> >> > nul-terminated."
>>> >> >> > Personal reality distortion fields are immune to
>contradictory
>>> >> >> > evidence.
>>> >> >> > -
>>> >> >> > srean
>>> >> >> > Check out my website: http://kirbyfan64.github.io/
>>> >> >
>>> >> >
>>> >> >
>>> >> >
>>> >> > --
>>> >> > Ryan
>>> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>be
>>> simple:
>>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>that
>>> was
>>> >> > nul-terminated."
>>> >> > Personal reality distortion fields are immune to contradictory
>>> evidence.
>>> >> > -
>>> >> > srean
>>> >> > Check out my website: http://kirbyfan64.github.io/
>>> >
>>> >
>>> >
>>> >
>>> > --
>>> > Ryan
>>> > If anybody ever asks me why I prefer C++ to C, my answer will be
>simple:
>>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>that was
>>> > nul-terminated."
>>> > Personal reality distortion fields are immune to contradictory
>>> evidence. -
>>> > srean
>>> > Check out my website: http://kirbyfan64.github.io/
>>>
>>
>>
>>
>> --
>> Ryan
>> If anybody ever asks me why I prefer C++ to C, my answer will be
>simple:
>> "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>was
>> nul-terminated."
>> Personal reality distortion fields are immune to contradictory
>evidence. -
>> srean
>> Check out my website: http://kirbyfan64.github.io/
>>
>
>
>
>-- 
>Ryan
>If anybody ever asks me why I prefer C++ to C, my answer will be
>simple:
>"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that
>was
>nul-terminated."
>Personal reality distortion fields are immune to contradictory
>evidence. -
>srean
>Check out my website: http://kirbyfan64.github.io/

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/3ce2dcb3/attachment-0001.html>

From chaselton at gmail.com  Sat Jan 31 20:45:55 2015
From: chaselton at gmail.com (Cyd Haselton)
Date: Sat, 31 Jan 2015 13:45:55 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <b8fb2129-c394-4739-a06f-8d43cfdd9f65@email.android.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <7545e812-1a79-4389-8add-c0d6acdf03b3.maildroid@localhost>
 <b8fb2129-c394-4739-a06f-8d43cfdd9f65@email.android.com>
Message-ID: <D9DA3196-86FB-45DE-BE03-E7D4787F0734@gmail.com>

Ergh...this may not work.
I've already got BusyBox and a terminal installed, but the python build...and all other utilities that I use...are running in a fakechrooted environment inside the terminal. That environment only filters dynamic calls to libc, so the statically compiled gdb won't work in the environment and running it outside the environment against the python built IN the environment won't work, I think...

Let me do some looking around and tinkering...I may just need to build gdb for my environment.

On January 31, 2015 1:09:22 PM CST, Ryan <rymg19 at gmail.com> wrote:
>Ok...try this (based on http://dan.drown.org/android/howto/gdb.html):
>
>- Install BusyBox and a Terminal Emulator
>- Inside the Terminal Emulator, run:
>
>  su
>  cd /data/local/tmp
>  wget http://dan.drown.org/android/gdb-
>static.tar.gz
>  tar zxf gdb-static.tar.gz
>  ./gdb <path to python interpreter>
>
>Now, inside gdb, type:
>
>  set logging file /mnt/sdcard/bt.txt
>  set logging on
>  run
>
>Wait for Python to crash, then type:
>
>  backtrace
>
>A backtrace should be printed to the screen and saved to a file named
>bt.txt on the SD card. After that, type:
>
>  quit
>
>to quit GDB. Send the list the bt.txt file on your SD card.
>
>chaselton at gmail.com wrote:
>>I don't have the SDK either...but my device is rooted.
>>
>>Dual-booting is not an option unfortunately...space reasons.  I'll do
>>my best to figure out a way to get the instuctions you sent
>>implemented, but this may be a deal-breaker for porting Python 3.4.x
>>for me...I may go back to working on 2.7.x
>>
>>Sent from my android device.
>>
>>-----Original Message-----
>>From: Ryan Gonzalez <rymg19 at gmail.com>
>>To: Cyd Haselton <chaselton at gmail.com>
>>Cc: Python-Dev <python-dev at python.org>
>>Sent: Fri, 30 Jan 2015 7:53 PM
>>Subject: Re: [Python-Dev] Newly Built Python3 Binary Throws Segfault
>>
>>Regardless, if you're looking to toy more with stuff like this, I'd
>>highly
>>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now
>I
>>rarely ever boot into Windows!)
>>
>>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>
>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>phone?
>>> If so, I can upload instructions that don't use the NDK.
>>>
>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>wrote:
>>>
>>>> This is going to take some time...here's why:
>>>>
>>>> Downloading and installing the NDK/SDK won't be too hard...I have
>to
>>>> clear some space...but my primary machine is running Windows 7 and
>>I'd
>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>> experience with it, other than knowing that the NDK recommends
>>against
>>>> it.
>>>>
>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>> external drive for space reasons...and it only has the NDK
>>installed.
>>>> If I am able to get it back up and running, there's still the task
>>of
>>>> getting  adb connected to my device. from the VM.
>>>>
>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>>In
>>>> the meantime I'll also check to see if there's anything that can a)
>>>> run in an Android terminal and b) can take a stack trace; it would
>>be
>>>> far, far, far easier than either option above.
>>>>
>>>>
>>>>
>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>>> > Do you have the Android SDK and NDK installed? If so...
>>>> >
>>>> > Using Google, I've created this series of steps, which may (or
>may
>>not)
>>>> > work:
>>>> >
>>>> > 1. Make sure you have a copy of Python on your computer and make
>>sure
>>>> that
>>>> > it's built with debug symbols.
>>>> >
>>>> > 2. Run the following commands from a shell with your phone
>plugged
>>in:
>>>> >
>>>> >   adb forward tcp:5039 tcp:5039
>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>executable>
>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>> >
>>>> > Now, GDB should have opened, so type the following commands:
>>>> >
>>>> >   set solib-search-path <directory holder libpython.so>
>>>> >   file <path to local python executable>
>>>> >   target remote :5055
>>>> >   run
>>>> >
>>>> > Now, wait for the program to crash and type:
>>>> >
>>>> >   backtrace
>>>> >
>>>> > You should now see a complete backtrace in your terminal. To GDB,
>>type:
>>>> >
>>>> >   quit
>>>> >
>>>> > *crosses fingers*
>>>> >
>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Unfortunately it is still reporting the same function :-/.
>>>> >>
>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>>> wrote:
>>>> >> > Yes...
>>>> >> >
>>>> >> > Can you check if it's crashing in a different function now?
>>>> >> >
>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Yes I did.  I did have to enter all the information in
>>>> manually...I'm
>>>> >> >> not familiar with automated patch application tools and even
>>if I
>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>> >> >>
>>>> >> >> Just so that I'm absolutely sure I got everything
>>correct...you
>>>> wanted
>>>> >> >> all of the lines in the patch commented out, correct? 
>>Basically
>>>> >> >> everything referencing oldloc?
>>>> >> >>
>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
>><rymg19 at gmail.com>
>>>> >> >> wrote:
>>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>>it
>>>> would
>>>> >> >> > work!
>>>> >> >> >
>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>right?
>>>> >> >> >
>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>> chaselton at gmail.com>
>>>> >> >> > wrote:
>>>> >> >> >>
>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>> correctly,
>>>> >> >> >> but
>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>> >> >> >> Will post info this afternoon.
>>>> >> >> >>
>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>> rymg19 at gmail.com>
>>>> >> >> >> wrote:
>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>>> >> >> >> > unconditionally
>>>> >> >> >> > checks the length of the (possibly null) string argument
>>first.
>>>> >> >> >> >
>>>> >> >> >> > Please try the patch I attached in the last email. It
>>*might*
>>>> fix
>>>> >> >> >> > the
>>>> >> >> >> > issue.
>>>> >> >> >> > Android has crappy locale handling.
>>>> >> >> >> >
>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> > wrote:
>>>> >> >> >> >>
>>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>>_PyMem_RawStrdup
>>>> is
>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>> >> >> >> >>
>>>> >> >> >> >> From obmalloc.c
>>>> >> >> >> >>
>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>> >> >> >> >> {
>>>> >> >> >> >>     size_t size;
>>>> >> >> >> >> char *copy;
>>>> >> >> >> >>     size = strlen(str) + 1;
>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>> >> >> >> >>     if (copy == NULL)
>>>> >> >> >> >>         return NULL;
>>>> >> >> >> >> memcpy(copy, str, size);
>>>> >> >> >> >> return copy;
>>>> >> >> >> >>           }
>>>> >> >> >> >>
>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>> >> >> >> >> <rymg19 at gmail.com>
>>>> >> >> >> >> wrote:
>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>> _PyMem_RawStrdup
>>>> >> >> >> >> > crashes
>>>> >> >> >> >> > when
>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>likely
>>>> >> >> >> >> > asking
>>>> >> >> >> >> > it
>>>> >> >> >> >> > to
>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably
>the
>>>> caller's
>>>> >> >> >> >> > fault.
>>>> >> >> >> >> >
>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>return
>>>> NULL
>>>> >> >> >> >> > when
>>>> >> >> >> >> > given a
>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>> >> >> >> >> >
>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> > wrote:
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find
>out
>>what
>>>> >> >> >> >> >> changed
>>>> >> >> >> >> >> in
>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>> >> >> >> >> >>
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>mentioned
>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>> >> >> >> >> >> >
>>https://github.com/rave-engine/python3-android/issues/2.
>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>>for
>>>> >> >> >> >> >> > obmalloc.c,
>>>> >> >> >> >> >> > as
>>>> >> >> >> >> >> > (I think) the functions all belong to
>>Python...they're
>>>> all
>>>> >> >> >> >> >> > prefixed
>>>> >> >> >> >> >> > with _PyXx
>>>> >> >> >> >> >> >
>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> >> > wrote:
>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on
>device
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>> >> >> >> >> >> >> 0008bbc8
>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
>><rymg19 at gmail.com
>>>> >
>>>> >> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> >>> Could you try the steps at
>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780?
>They
>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>> crashing.
>>>> >> >> >> >> >> >>>
>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at
>0x00000000
>>>> >> >> >> >> >> >>>> (code=1),
>>>> >> >> >> >> >> >>>> thread
>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>>(code=1),
>>>> thread
>>>> >> >> >> >> >> >>>> 23373
>>>> >> >> >> >> >> >>>> (python)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>>> python is
>>>> >> >> >> >> >> >>>> segfaulting
>>>> >> >> >> >> >> >>>> libc...
>>>> >> >> >> >> >> >>
-- 
Sent from my Android device with K-9 Mail. Please excuse my brevity.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/cd030bb9/attachment-0001.html>

From rymg19 at gmail.com  Sat Jan 31 23:08:58 2015
From: rymg19 at gmail.com (Ryan Gonzalez)
Date: Sat, 31 Jan 2015 16:08:58 -0600
Subject: [Python-Dev] Newly Built Python3 Binary Throws Segfault
In-Reply-To: <B42BA6C5-A292-42D2-A99D-7DEF95B9FE1C@gmail.com>
References: <CAHu5PrY5GM2yvEMc7QNGaHZfrUEeX0T_G6ZVOzRXLPv7C6NL3Q@mail.gmail.com>
 <CAHu5Prbv3UJeUohVMUPeXuVvNa2EcfmL9RJwgQxkJePRqqHRRg@mail.gmail.com>
 <CAO41-mMuq=TXLg3XA6K2-Pd91+MaOW95i7qRfRjur08QzVSUDQ@mail.gmail.com>
 <CAHu5PrZckvQNXgSZoqkbqsPD3qeTyq98X8PFbS+SUS+pgak1kg@mail.gmail.com>
 <CAO41-mO254AxoRVUfswVJ2b2XoJXDWhNxxWTxf88+TX6W6dQDg@mail.gmail.com>
 <CAHu5Prae5HZ8e=cLva-N7B+xz4tHTtSnpLFTSYM00+42aMfAgA@mail.gmail.com>
 <CAO41-mM0uLcLGJKvbxAuV=Hv=vArSCnn9rQn72VA8fE_P3LCqA@mail.gmail.com>
 <CAHu5Prb-98oQ86Zf3R=dMd_EctwKowW+z1Z0gh5V_JOrS-kH+A@mail.gmail.com>
 <CAO41-mNPv3pYuCL+6d9sX9TyR+UAjcG0Mqn2eH5uisRDN6mrQQ@mail.gmail.com>
 <CAHu5PrbZsgG2jxUM6sHQ36c4yJf7+TxE98D-eqCnj8M+o63fSg@mail.gmail.com>
 <CAO41-mMrzwy9GmgL+jbx9KwBRndj23g2TRhG52tWHXQFcN+nZQ@mail.gmail.com>
 <CAO41-mPdxN6YMP6X_xUvKW0SMB41kpW2ZC3soVVu6MNdJ-CaTw@mail.gmail.com>
 <FB8BBDBF-6430-43D0-A52E-666EABC9DE6A@gmail.com>
 <3f23f67c-514e-4e91-89b2-5800ede08317@email.android.com>
 <B42BA6C5-A292-42D2-A99D-7DEF95B9FE1C@gmail.com>
Message-ID: <CAO41-mPydhLL5M1+zWO_tJ8MnBdETqpgSp+7pG252HP3261Atg@mail.gmail.com>

That looks better! Looks like now the real encoding issues are coming up.
Try going to line 269 of pythonrun.c and changing:

PyErr_SetNone(PyExc_NotImplementedError);
return NULL;

to:

char* m = malloc(6);
strcpy(m, "ascii");
return m;

This just sets a default encoding.

On Sat, Jan 31, 2015 at 1:21 PM, Cyd Haselton <chaselton at gmail.com> wrote:

> Very interesting. I got this error
>
> Fatal Python error: Py_Initialize: Unable to get the locale encoding
> NotImplementedError
> Aborted
> generate-posix-vars failed
> make: *** [pybuilddir.txt] Error 1
>
> ...but it didn't (of course) segfault. I'll pull gdb, get the results and
> send them.
>
> On January 31, 2015 1:10:18 PM CST, Ryan <rymg19 at gmail.com> wrote:
>
>> No; I was looking for all uses of _PyRaw_Strdup. Surprisingly, it's used
>> only a few times.
>>
>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>
>>> Question:
>>> When you said earlier that you found the problem by using grep...were
>>> you looking for places where strdup called locale?
>>>
>>> On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com>
>>> wrote:
>>>>
>>>> Regardless, if you're looking to toy more with stuff like this, I'd
>>>> highly recommend dual-booting with Ubuntu, which is what I'm doing now.
>>>> (Now I rarely ever boot into Windows!)
>>>>
>>>> On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>> wrote:
>>>>
>>>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>>>> phone? If so, I can upload instructions that don't use the NDK.
>>>>>
>>>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>>>> wrote:
>>>>>
>>>>>> This is going to take some time...here's why:
>>>>>>
>>>>>> Downloading and installing the NDK/SDK won't be too hard...I have to
>>>>>> clear some space...but my primary machine is running Windows 7 and I'd
>>>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>>>> experience with it, other than knowing that the NDK recommends against
>>>>>> it.
>>>>>>
>>>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>>>> external drive for space reasons...and it only has the NDK installed.
>>>>>> If I am able to get it back up and running, there's still the task of
>>>>>> getting  adb connected to my device. from the VM.
>>>>>>
>>>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest).  In
>>>>>> the meantime I'll also check to see if there's anything that can a)
>>>>>> run in an Android terminal and b) can take a stack trace; it would be
>>>>>> far, far, far easier than either option above.
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>>>> wrote:
>>>>>> > Do you have the Android SDK and NDK installed? If so...
>>>>>> >
>>>>>> > Using Google, I've created this series of steps, which may (or may
>>>>>> not)
>>>>>> > work:
>>>>>> >
>>>>>> > 1. Make sure you have a copy of Python on your computer and make
>>>>>> sure that
>>>>>> > it's built with debug symbols.
>>>>>> >
>>>>>> > 2. Run the following commands from a shell with your phone plugged
>>>>>> in:
>>>>>> >
>>>>>> >   adb forward tcp:5039 tcp:5039
>>>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>>>>> executable>
>>>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>>>> >
>>>>>> > Now, GDB should have opened, so type the following commands:
>>>>>> >
>>>>>> >   set solib-search-path <directory holder libpython.so>
>>>>>> >   file <path to local python executable>
>>>>>> >   target remote :5055
>>>>>> >   run
>>>>>> >
>>>>>> > Now, wait for the program to crash and type:
>>>>>> >
>>>>>> >   backtrace
>>>>>> >
>>>>>> > You should now see a complete backtrace in your terminal. To GDB,
>>>>>> type:
>>>>>> >
>>>>>> >   quit
>>>>>> >
>>>>>> > *crosses fingers*
>>>>>> >
>>>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton <chaselton at gmail.com>
>>>>>> wrote:
>>>>>> >>
>>>>>> >> Unfortunately it is still reporting the same function :-/.
>>>>>> >>
>>>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>>>>> wrote:
>>>>>> >> > Yes...
>>>>>> >> >
>>>>>> >> > Can you check if it's crashing in a different function now?
>>>>>> >> >
>>>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton <
>>>>>> chaselton at gmail.com>
>>>>>> >> > wrote:
>>>>>> >> >>
>>>>>> >> >> Yes I did.  I did have to enter all the information in
>>>>>> manually...I'm
>>>>>> >> >> not familiar with automated patch application tools and even if
>>>>>> I
>>>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>>>> >> >>
>>>>>> >> >> Just so that I'm absolutely sure I got everything correct...you
>>>>>> wanted
>>>>>> >> >> all of the lines in the patch commented out, correct?  Basically
>>>>>> >> >> everything referencing oldloc?
>>>>>> >> >>
>>>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez <
>>>>>> rymg19 at gmail.com>
>>>>>> >> >> wrote:
>>>>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>>>>>> it would
>>>>>> >> >> > work!
>>>>>> >> >> >
>>>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>>>>> right?
>>>>>> >> >> >
>>>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>>>> chaselton at gmail.com>
>>>>>> >> >> > wrote:
>>>>>> >> >> >>
>>>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>>>> correctly,
>>>>>> >> >> >> but
>>>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>>>> >> >> >> Will post info this afternoon.
>>>>>> >> >> >>
>>>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>>>> rymg19 at gmail.com>
>>>>>> >> >> >> wrote:
>>>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>>>>> >> >> >> > unconditionally
>>>>>> >> >> >> > checks the length of the (possibly null) string argument
>>>>>> first.
>>>>>> >> >> >> >
>>>>>> >> >> >> > Please try the patch I attached in the last email. It
>>>>>> *might* fix
>>>>>> >> >> >> > the
>>>>>> >> >> >> > issue.
>>>>>> >> >> >> > Android has crappy locale handling.
>>>>>> >> >> >> >
>>>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>>>> >> >> >> > <chaselton at gmail.com>
>>>>>> >> >> >> > wrote:
>>>>>> >> >> >> >>
>>>>>> >> >> >> >>   Unless i'm reading something incorrectly,
>>>>>> _PyMem_RawStrdup is
>>>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>>>> >> >> >> >>
>>>>>> >> >> >> >> From obmalloc.c
>>>>>> >> >> >> >>
>>>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>>>> >> >> >> >> {
>>>>>> >> >> >> >>     size_t size;
>>>>>> >> >> >> >> char *copy;
>>>>>> >> >> >> >>     size = strlen(str) + 1;
>>>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>>>> >> >> >> >>     if (copy == NULL)
>>>>>> >> >> >> >>         return NULL;
>>>>>> >> >> >> >> memcpy(copy, str, size);
>>>>>> >> >> >> >> return copy;
>>>>>> >> >> >> >>           }
>>>>>> >> >> >> >>
>>>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>>>> >> >> >> >> <rymg19 at gmail.com>
>>>>>> >> >> >> >> wrote:
>>>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>>>> _PyMem_RawStrdup
>>>>>> >> >> >> >> > crashes
>>>>>> >> >> >> >> > when
>>>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>>>>> likely
>>>>>> >> >> >> >> > asking
>>>>>> >> >> >> >> > it
>>>>>> >> >> >> >> > to
>>>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably the
>>>>>> caller's
>>>>>> >> >> >> >> > fault.
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>>>>> return NULL
>>>>>> >> >> >> >> > when
>>>>>> >> >> >> >> > given a
>>>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>>>> >> >> >> >> > <chaselton at gmail.com>
>>>>>> >> >> >> >> > wrote:
>>>>>> >> >> >> >> >>
>>>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find out
>>>>>> what
>>>>>> >> >> >> >> >> changed
>>>>>> >> >> >> >> >> in
>>>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>>>> >> >> >> >> >>
>>>>>> >> >> >> >> >>
>>>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>>>> >> >> >> >> >> wrote:
>>>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>>>>> mentioned
>>>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>>>> >> >> >> >> >> >
>>>>>> https://github.com/rave-engine/python3-android/issues/2.
>>>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>>>>>> for
>>>>>> >> >> >> >> >> > obmalloc.c,
>>>>>> >> >> >> >> >> > as
>>>>>> >> >> >> >> >> > (I think) the functions all belong to
>>>>>> Python...they're all
>>>>>> >> >> >> >> >> > prefixed
>>>>>> >> >> >> >> >> > with _PyXx
>>>>>> >> >> >> >> >> >
>>>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>>>> >> >> >> >> >> > wrote:
>>>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on device
>>>>>> >> >> >> >> >> >>
>>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>>>> >> >> >> >> >> >> 0008bbc8
>>>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>>>> >> >> >> >> >> >>
>>>>>> >> >> >> >> >> >>
>>>>>> >> >> >> >> >> >>
>>>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan <
>>>>>> rymg19 at gmail.com>
>>>>>> >> >> >> >> >> >> wrote:
>>>>>> >> >> >> >> >> >>> Could you try the steps at
>>>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780? They
>>>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>>>> crashing.
>>>>>> >> >> >> >> >> >>>
>>>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>>>> >> >> >> >> >> >>>>
>>>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at 0x00000000
>>>>>> >> >> >> >> >> >>>> (code=1),
>>>>>> >> >> >> >> >> >>>> thread
>>>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>>>> >> >> >> >> >> >>>>
>>>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000 (code=1),
>>>>>> thread
>>>>>> >> >> >> >> >> >>>> 23373
>>>>>> >> >> >> >> >> >>>> (python)
>>>>>> >> >> >> >> >> >>>>
>>>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>>>>> python is
>>>>>> >> >> >> >> >> >>>> segfaulting
>>>>>> >> >> >> >> >> >>>> libc...
>>>>>> >> >> >> >> >> >>>>
>>>>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>>>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>>>>> >> >> >> >> >> >>>> wrote:
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>>>>>> Rossum
>>>>>> >> >> >> >> >> >>>>> <guido at python.org>
>>>>>> >> >> >> >> >> >>>>> wrote:
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>>>>> >> >> >> >> >> >>>>>>  ... load libm
>>>>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do something
>>>>>> to it
>>>>>> >> >> >> >> >> >>>>>> (what?)
>>>>>> >> >> >> >> >> >>>>>>  ... get current time
>>>>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>>>>> >> >> >> >> >> >>>>>>  ... getuid
>>>>>> >> >> >> >> >> >>>>>>  ... segfault
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't look
>>>>>> as if
>>>>>> >> >> >> >> >> >>>>>> it
>>>>>> >> >> >> >> >> >>>>>> has
>>>>>> >> >> >> >> >> >>>>>> started to
>>>>>> >> >> >> >> >> >>>>>>  load modules yet.
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>>>>>> me.
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>>>>>> >> >> >> >> >> >>>>>  is the code that works with that file.
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>  This explains it a bit (slides 24-29). Looks
>>>>>> like
>>>>>> >> >> >> >> >> >>>>> something
>>>>>> >> >> >> >> >> >>>>> to
>>>>>> >> >> >> >> >> >>>>> do
>>>>>> >> >> >> >> >> >>>>> with
>>>>>> >> >> >> >> >> >>>>>  interprocess communication. Likely has nothing
>>>>>> to do
>>>>>> >> >> >> >> >> >>>>> with
>>>>>> >> >> >> >> >> >>>>> Python
>>>>>> >> >> >> >> >> >>>>> itself.
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>  Maybe this would be useful?
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  That stack trace would be really helpful.
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  On Wed, Jan 28, 2015 at 8:34 AM, Cyd Haselton
>>>>>> >> >> >> >> >> >>>>>> <chaselton at gmail.com>
>>>>>> >> >> >> >> >> >>>>>> wrote:
>>>>>> >> >> >> >> >> >>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>  Apologies...I'm not sure what a stack track
>>>>>> is, but I
>>>>>> >> >> >> >> >> >>>>>>> do
>>>>>> >> >> >> >> >> >>>>>>> have
>>>>>> >> >> >> >> >> >>>>>>> the
>>>>>> >> >> >> >> >> >>>>>>>  strace.  Nearest I can tell, it happens due
>>>>>> to an
>>>>>> >> >> >> >> >> >>>>>>> open
>>>>>> >> >> >> >> >> >>>>>>> call,
>>>>>> >> >> >> >> >> >>>>>>> though I
>>>>>> >> >> >> >> >> >>>>>>>  am probably wrong.
>>>>>> >> >> >> >> >> >>>>>>>  Attaching the strace output to this email.
>>>>>> I'm going
>>>>>> >> >> >> >> >> >>>>>>> to
>>>>>> >> >> >> >> >> >>>>>>> head
>>>>>> >> >> >> >> >> >>>>>>> back to
>>>>>> >> >> >> >> >> >>>>>>>  the documentation and to back out of some
>>>>>> >> >> >> >> >> >>>>>>> Android-related
>>>>>> >> >> >> >> >> >>>>>>> changes
>>>>>> >> >> >> >> >> >>>>>>> in
>>>>>> >> >> >> >> >> >>>>>>>  _localemodule.c
>>>>>> >> >> >> >> >> >>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>  On Wed, Jan 28, 2015 at 9:43 AM, Guido van
>>>>>> Rossum
>>>>>> >> >> >> >> >> >>>>>>> <guido at python.org>
>>>>>> >> >> >> >> >> >>>>>>>  wrote:
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>  There could be a million differences relevant
>>>>>> >> >> >> >> >> >>>>>>>> (unicode,
>>>>>> >> >> >> >> >> >>>>>>>> ints,
>>>>>> >> >> >> >> >> >>>>>>>> ...).
>>>>>> >> >> >> >> >> >>>>>>>>  Perhaps
>>>>>> >> >> >> >> >> >>>>>>>>  the importlib bootstrap is failing. Perhaps
>>>>>> the
>>>>>> >> >> >> >> >> >>>>>>>> dynamic
>>>>>> >> >> >> >> >> >>>>>>>> loading
>>>>>> >> >> >> >> >> >>>>>>>> code
>>>>>> >> >> >> >> >> >>>>>>>>  changed. Did you get a stack track? (IIRC
>>>>>> strace
>>>>>> >> >> >> >> >> >>>>>>>> shows
>>>>>> >> >> >> >> >> >>>>>>>> a
>>>>>> >> >> >> >> >> >>>>>>>> syscall
>>>>>> >> >> >> >> >> >>>>>>>> trace
>>>>>> >> >> >> >> >> >>>>>>>>  --
>>>>>> >> >> >> >> >> >>>>>>>>  also useful, but doesn't tell you precisely
>>>>>> how
>>>>>> >> >> >> >> >> >>>>>>>> it segfaulted.)
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>  On Wed, Jan 28, 2015 at 6:43 AM, Cyd Haselton
>>>>>> >> >> >> >> >> >>>>>>>> <chaselton at gmail.com>
>>>>>> >> >> >> >> >> >>>>>>>>  wrote:
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>  All,
>>>>>> >> >> >> >> >> >>>>>>>>>  I recently ditched my attempts to port
>>>>>> Python 2.7.8
>>>>>> >> >> >> >> >> >>>>>>>>> to
>>>>>> >> >> >> >> >> >>>>>>>>> Android
>>>>>> >> >> >> >> >> >>>>>>>>> in
>>>>>> >> >> >> >> >> >>>>>>>>>  favor of Python 3.4.2.  Unfortunately,
>>>>>> after using
>>>>>> >> >> >> >> >> >>>>>>>>> the
>>>>>> >> >> >> >> >> >>>>>>>>> same
>>>>>> >> >> >> >> >> >>>>>>>>> configure
>>>>>> >> >> >> >> >> >>>>>>>>>  options in the same environment, and
>>>>>> modifying the
>>>>>> >> >> >> >> >> >>>>>>>>> setup.py
>>>>>> >> >> >> >> >> >>>>>>>>> as
>>>>>> >> >> >> >> >> >>>>>>>>> needed,
>>>>>> >> >> >> >> >> >>>>>>>>>  the newly built binary throws a segfault
>>>>>> when the
>>>>>> >> >> >> >> >> >>>>>>>>> generate-posix-vars
>>>>>> >> >> >> >> >> >>>>>>>>>  portion of the build is reached...and when
>>>>>> it is
>>>>>> >> >> >> >> >> >>>>>>>>> run
>>>>>> >> >> >> >> >> >>>>>>>>> as
>>>>>> >> >> >> >> >> >>>>>>>>> well
>>>>>> >> >> >> >> >> >>>>>>>>> (i.e.
>>>>>> >> >> >> >> >> >>>>>>>>>  ./python --help, ./python -E -S -m
>>>>>> sysconfig, or
>>>>>> >> >> >> >> >> >>>>>>>>> similar)
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>  I took a strace of ./python, however I'm a
>>>>>> bit lost
>>>>>> >> >> >> >> >> >>>>>>>>> when
>>>>>> >> >> >> >> >> >>>>>>>>> reviewing
>>>>>> >> >> >> >> >> >>>>>>>>> it.
>>>>>> >> >> >> >> >> >>>>>>>>>  Any ideas as to what may be going on...i.e.
>>>>>> why
>>>>>> >> >> >> >> >> >>>>>>>>> Python
>>>>>> >> >> >> >> >> >>>>>>>>> 2.7
>>>>>> >> >> >> >> >> >>>>>>>>> works but
>>>>>> >> >> >> >> >> >>>>>>>>>  3.x throws a segfault?
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>  Thanks in advance,
>>>>>> >> >> >> >> >> >>>>>>>>>  Cyd
>>>>>> >> >> >> >> >> >>>>>>>>> ________________________________
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>  Python-Dev mailing list
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>> Python-Dev at python.org
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> https://mail.python.org/mailman/listinfo/python-dev
>>>>>> >> >> >> >> >> >>>>>>>>>  Unsubscribe:
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>>
>>>>>> https://mail.python.org/mailman/options/python-dev/guido%40python.org
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>
>>>>>> >> >> >> >> >> >>>>>>>>  --
>>>>>> >> >> >> >> >> >>>>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  --
>>>>>> >> >> >> >> >> >>>>>>  --Guido van Rossum (python.org/~guido)
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>> ________________________________
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>  Python-Dev mailing list
>>>>>> >> >> >> >> >> >>>>>>  Python-Dev at python.org
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> https://mail.python.org/mailman/listinfo/python-dev
>>>>>> >> >> >> >> >> >>>>>>  Unsubscribe:
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> >> >> >> >> >> >>>>>>
>>>>>> https://mail.python.org/mailman/options/python-dev/rymg19%40gmail.com
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>
>>>>>> >> >> >> >> >> >>>>>  --
>>>>>> >> >> >> >> >> >>>>>  Ryan
>>>>>> >> >> >> >> >> >>>>>  If anybody ever asks me why I prefer C++ to C,
>>>>>> my
>>>>>> >> >> >> >> >> >>>>> answer
>>>>>> >> >> >> >> >> >>>>> will
>>>>>> >> >> >> >> >> >>>>> be
>>>>>> >> >> >> >> >> >>>>> simple:
>>>>>> >> >> >> >> >> >>>>>  "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT.
>>>>>> Wait, I
>>>>>> >> >> >> >> >> >>>>> don't
>>>>>> >> >> >> >> >> >>>>> think
>>>>>> >> >> >> >> >> >>>>> that was
>>>>>> >> >> >> >> >> >>>>>  nul-terminated."
>>>>>> >> >> >> >> >> >>>>>  Personal reality distortion fields are immune to
>>>>>> >> >> >> >> >> >>>>> contradictory
>>>>>> >> >> >> >> >> >>>>> evidence.
>>>>>> >> >> >> >> >> >>>>> -
>>>>>> >> >> >> >> >> >>>>>  srean
>>>>>> >> >> >> >> >> >>>>>  Check out my website:
>>>>>> http://kirbyfan64.github.io/
>>>>>> >> >> >> >> >> >>>
>>>>>> >> >> >> >> >> >>>
>>>>>> >> >> >> >> >> >>> --
>>>>>> >> >> >> >> >> >>> Sent from my Android phone with K-9 Mail. Please
>>>>>> excuse my
>>>>>> >> >> >> >> >> >>> brevity.
>>>>>> >> >> >> >> >> >>> Check out my website: http://kirbyfan64.github.io/
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> >
>>>>>> >> >> >> >> > --
>>>>>> >> >> >> >> > Ryan
>>>>>> >> >> >> >> > If anybody ever asks me why I prefer C++ to C, my
>>>>>> answer will
>>>>>> >> >> >> >> > be
>>>>>> >> >> >> >> > simple:
>>>>>> >> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I
>>>>>> don't think
>>>>>> >> >> >> >> > that
>>>>>> >> >> >> >> > was
>>>>>> >> >> >> >> > nul-terminated."
>>>>>> >> >> >> >> > Personal reality distortion fields are immune to
>>>>>> contradictory
>>>>>> >> >> >> >> > evidence.
>>>>>> >> >> >> >> > -
>>>>>> >> >> >> >> > srean
>>>>>> >> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>>>>>> >> >> >> >
>>>>>> >> >> >> >
>>>>>> >> >> >> >
>>>>>> >> >> >> >
>>>>>> >> >> >> > --
>>>>>> >> >> >> > Ryan
>>>>>> >> >> >> > If anybody ever asks me why I prefer C++ to C, my answer
>>>>>> will be
>>>>>> >> >> >> > simple:
>>>>>> >> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>>>>>> think
>>>>>> >> >> >> > that
>>>>>> >> >> >> > was
>>>>>> >> >> >> > nul-terminated."
>>>>>> >> >> >> > Personal reality distortion fields are immune to
>>>>>> contradictory
>>>>>> >> >> >> > evidence.
>>>>>> >> >> >> > -
>>>>>> >> >> >> > srean
>>>>>> >> >> >> > Check out my website: http://kirbyfan64.github.io/
>>>>>> >> >> >
>>>>>> >> >> >
>>>>>> >> >> >
>>>>>> >> >> >
>>>>>> >> >> > --
>>>>>> >> >> > Ryan
>>>>>> >> >> > If anybody ever asks me why I prefer C++ to C, my answer will
>>>>>> be
>>>>>> >> >> > simple:
>>>>>> >> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't
>>>>>> think that
>>>>>> >> >> > was
>>>>>> >> >> > nul-terminated."
>>>>>> >> >> > Personal reality distortion fields are immune to contradictory
>>>>>> >> >> > evidence.
>>>>>> >> >> > -
>>>>>> >> >> > srean
>>>>>> >> >> > Check out my website: http://kirbyfan64.github.io/
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> >> >
>>>>>> >> > --
>>>>>> >> > Ryan
>>>>>> >> > If anybody ever asks me why I prefer C++ to C, my answer will be
>>>>>> simple:
>>>>>> >> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>>>> that was
>>>>>> >> > nul-terminated."
>>>>>> >> > Personal reality distortion fields are immune to contradictory
>>>>>> evidence.
>>>>>> >> > -
>>>>>> >> > srean
>>>>>> >> > Check out my website: http://kirbyfan64.github.io/
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> >
>>>>>> > --
>>>>>> > Ryan
>>>>>> > If anybody ever asks me why I prefer C++ to C, my answer will be
>>>>>> simple:
>>>>>> > "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>>>> that was
>>>>>> > nul-terminated."
>>>>>> > Personal reality distortion fields are immune to contradictory
>>>>>> evidence. -
>>>>>> > srean
>>>>>> > Check out my website: http://kirbyfan64.github.io/
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Ryan
>>>>> If anybody ever asks me why I prefer C++ to C, my answer will be
>>>>> simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>>> that was nul-terminated."
>>>>> Personal reality distortion fields are immune to contradictory
>>>>> evidence. - srean
>>>>> Check out my website: http://kirbyfan64.github.io/
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Ryan
>>>> If anybody ever asks me why I prefer C++ to C, my answer will be
>>>> simple: "It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think
>>>> that was nul-terminated."
>>>> Personal reality distortion fields are immune to contradictory
>>>> evidence. - srean
>>>> Check out my website: http://kirbyfan64.github.io/
>>>>
>>>
>>>
>>
> --
> Sent from my Android device with K-9 Mail. Please excuse my brevity.
>



-- 
Ryan
If anybody ever asks me why I prefer C++ to C, my answer will be simple:
"It's becauseslejfp23(@#Q*(E*EIdc-SEGFAULT. Wait, I don't think that was
nul-terminated."
Personal reality distortion fields are immune to contradictory evidence. -
srean
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/9ce386ef/attachment-0001.html>