[Python-Dev] Newly Built Python3 Binary Throws Segfault

Ryan rymg19 at gmail.com
Sat Jan 31 20:10:18 CET 2015 

No; I was looking for all uses of _PyRaw_Strdup. Surprisingly, it's used only a few times.

Cyd Haselton <chaselton at gmail.com> wrote:
>Question:
>When you said earlier that you found the problem by using grep...were
>you looking for places where strdup called locale?
>
>On January 30, 2015 7:52:47 PM CST, Ryan Gonzalez <rymg19 at gmail.com>
>wrote:
>>Regardless, if you're looking to toy more with stuff like this, I'd
>>highly
>>recommend dual-booting with Ubuntu, which is what I'm doing now. (Now
>I
>>rarely ever boot into Windows!)
>>
>>On Fri, Jan 30, 2015 at 7:51 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>
>>> Do you have just the SDK (which doesn't require Cygwin) or a rooted
>>phone?
>>> If so, I can upload instructions that don't use the NDK.
>>>
>>> On Fri, Jan 30, 2015 at 6:19 PM, Cyd Haselton <chaselton at gmail.com>
>>wrote:
>>>
>>>> This is going to take some time...here's why:
>>>>
>>>> Downloading and installing the NDK/SDK won't be too hard...I have
>to
>>>> clear some space...but my primary machine is running Windows 7 and
>>I'd
>>>> rather swallow hot coals than install Cygwin.  I've got next to no
>>>> experience with it, other than knowing that the NDK recommends
>>against
>>>> it.
>>>>
>>>> I've got a CentOS VM...but it's currently in tarball form on an
>>>> external drive for space reasons...and it only has the NDK
>>installed.
>>>> If I am able to get it back up and running, there's still the task
>>of
>>>> getting  adb connected to my device. from the VM.
>>>>
>>>> Not saying it's impossible...just that it'll take time...and I'll
>>>> probably have to tackle it tomorrow (earliest) or Sunday (latest). 
>>In
>>>> the meantime I'll also check to see if there's anything that can a)
>>>> run in an Android terminal and b) can take a stack trace; it would
>>be
>>>> far, far, far easier than either option above.
>>>>
>>>>
>>>>
>>>> On Fri, Jan 30, 2015 at 4:19 PM, Ryan Gonzalez <rymg19 at gmail.com>
>>wrote:
>>>> > Do you have the Android SDK and NDK installed? If so...
>>>> >
>>>> > Using Google, I've created this series of steps, which may (or
>may
>>not)
>>>> > work:
>>>> >
>>>> > 1. Make sure you have a copy of Python on your computer and make
>>sure
>>>> that
>>>> > it's built with debug symbols.
>>>> >
>>>> > 2. Run the following commands from a shell with your phone
>plugged
>>in:
>>>> >
>>>> >   adb forward tcp:5039 tcp:5039
>>>> >   adb shell /system/bin/gdbserver tcp:5039 <path to python
>>executable>
>>>> >   <path to ndk binary folder>/arm-linux-androideabi-gdb
>>>> >
>>>> > Now, GDB should have opened, so type the following commands:
>>>> >
>>>> >   set solib-search-path <directory holder libpython.so>
>>>> >   file <path to local python executable>
>>>> >   target remote :5055
>>>> >   run
>>>> >
>>>> > Now, wait for the program to crash and type:
>>>> >
>>>> >   backtrace
>>>> >
>>>> > You should now see a complete backtrace in your terminal. To GDB,
>>type:
>>>> >
>>>> >   quit
>>>> >
>>>> > *crosses fingers*
>>>> >
>>>> > On Fri, Jan 30, 2015 at 2:50 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> wrote:
>>>> >>
>>>> >> Unfortunately it is still reporting the same function :-/.
>>>> >>
>>>> >> On Fri, Jan 30, 2015 at 1:44 PM, Ryan Gonzalez
><rymg19 at gmail.com>
>>>> wrote:
>>>> >> > Yes...
>>>> >> >
>>>> >> > Can you check if it's crashing in a different function now?
>>>> >> >
>>>> >> > On Fri, Jan 30, 2015 at 1:39 PM, Cyd Haselton
>><chaselton at gmail.com>
>>>> >> > wrote:
>>>> >> >>
>>>> >> >> Yes I did.  I did have to enter all the information in
>>>> manually...I'm
>>>> >> >> not familiar with automated patch application tools and even
>>if I
>>>> >> >> were, I'm pretty sure I wouldn't have them on my device.
>>>> >> >>
>>>> >> >> Just so that I'm absolutely sure I got everything
>>correct...you
>>>> wanted
>>>> >> >> all of the lines in the patch commented out, correct? 
>>Basically
>>>> >> >> everything referencing oldloc?
>>>> >> >>
>>>> >> >> On Fri, Jan 30, 2015 at 1:00 PM, Ryan Gonzalez
>><rymg19 at gmail.com>
>>>> >> >> wrote:
>>>> >> >> > Are you sure the patch was applied correctly? I was SO sure
>>it
>>>> would
>>>> >> >> > work!
>>>> >> >> >
>>>> >> >> > FYI, you tried the patch I attached to the email message,
>>right?
>>>> >> >> >
>>>> >> >> > On Fri, Jan 30, 2015 at 12:58 PM, Cyd Haselton <
>>>> chaselton at gmail.com>
>>>> >> >> > wrote:
>>>> >> >> >>
>>>> >> >> >> Update:  I did try the patch after getting it installed
>>>> correctly,
>>>> >> >> >> but
>>>> >> >> >> I'm still getting a segfault on the newly built binary.
>>>> >> >> >> Will post info this afternoon.
>>>> >> >> >>
>>>> >> >> >> On Fri, Jan 30, 2015 at 12:10 PM, Ryan Gonzalez <
>>>> rymg19 at gmail.com>
>>>> >> >> >> wrote:
>>>> >> >> >> > No, it returns NULL if malloc gives it a raw pointer. It
>>>> >> >> >> > unconditionally
>>>> >> >> >> > checks the length of the (possibly null) string argument
>>first.
>>>> >> >> >> >
>>>> >> >> >> > Please try the patch I attached in the last email. It
>>*might*
>>>> fix
>>>> >> >> >> > the
>>>> >> >> >> > issue.
>>>> >> >> >> > Android has crappy locale handling.
>>>> >> >> >> >
>>>> >> >> >> > On Fri, Jan 30, 2015 at 12:09 PM, Cyd Haselton
>>>> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> > wrote:
>>>> >> >> >> >>
>>>> >> >> >> >>   Unless i'm reading something incorrectly, 
>>_PyMem_RawStrdup
>>>> is
>>>> >> >> >> >> currently returning NULL when given a null pointer.
>>>> >> >> >> >>
>>>> >> >> >> >> From obmalloc.c
>>>> >> >> >> >>
>>>> >> >> >> >> _PyMem_RawStrdup(const char *str)
>>>> >> >> >> >> {
>>>> >> >> >> >>     size_t size;
>>>> >> >> >> >> char *copy;
>>>> >> >> >> >>     size = strlen(str) + 1;
>>>> >> >> >> >>     copy = PyMem_RawMalloc(size);
>>>> >> >> >> >>     if (copy == NULL)
>>>> >> >> >> >>         return NULL;
>>>> >> >> >> >> memcpy(copy, str, size);
>>>> >> >> >> >> return copy;
>>>> >> >> >> >>           }
>>>> >> >> >> >>
>>>> >> >> >> >> On Fri, Jan 30, 2015 at 11:56 AM, Ryan Gonzalez
>>>> >> >> >> >> <rymg19 at gmail.com>
>>>> >> >> >> >> wrote:
>>>> >> >> >> >> > I seriously doubt the issue is in that file;
>>>> _PyMem_RawStrdup
>>>> >> >> >> >> > crashes
>>>> >> >> >> >> > when
>>>> >> >> >> >> > calling strlen. It's that whatever is calling it is
>>likely
>>>> >> >> >> >> > asking
>>>> >> >> >> >> > it
>>>> >> >> >> >> > to
>>>> >> >> >> >> > duplicate a null pointer. Basically, it's probably
>the
>>>> caller's
>>>> >> >> >> >> > fault.
>>>> >> >> >> >> >
>>>> >> >> >> >> > You could always try modifying _PyMem_RawStrdup to
>>return
>>>> NULL
>>>> >> >> >> >> > when
>>>> >> >> >> >> > given a
>>>> >> >> >> >> > null pointer and see where it then segfaults.
>>>> >> >> >> >> >
>>>> >> >> >> >> > On Fri, Jan 30, 2015 at 11:53 AM, Cyd Haselton
>>>> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> > wrote:
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> Alternatively, is there a hassle-free way to find
>out
>>what
>>>> >> >> >> >> >> changed
>>>> >> >> >> >> >> in
>>>> >> >> >> >> >> obmalloc.c between 2.7.x and 3.4.x?
>>>> >> >> >> >> >>
>>>> >> >> >> >> >>
>>>> >> >> >> >> >> On Fri, Jan 30, 2015 at 9:29 AM, Cyd Haselton
>>>> >> >> >> >> >> <chaselton at gmail.com>
>>>> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> > There's a related strdup patch for readline.c,
>>mentioned
>>>> >> >> >> >> >> > here:http://bugs.python.org/issue21390 and here
>>>> >> >> >> >> >> >
>>https://github.com/rave-engine/python3-android/issues/2.
>>>> >> >> >> >> >> > There's a patch, but I'm not sure how to modify it
>>for
>>>> >> >> >> >> >> > obmalloc.c,
>>>> >> >> >> >> >> > as
>>>> >> >> >> >> >> > (I think) the functions all belong to
>>Python...they're
>>>> all
>>>> >> >> >> >> >> > prefixed
>>>> >> >> >> >> >> > with _PyXx
>>>> >> >> >> >> >> >
>>>> >> >> >> >> >> > On Fri, Jan 30, 2015 at 9:05 AM, Cyd Haselton
>>>> >> >> >> >> >> > <chaselton at gmail.com>
>>>> >> >> >> >> >> > wrote:
>>>> >> >> >> >> >> >> Absolutely.  Good thing I have addr2line on
>device
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $ addr2line -C -f -e
>>>> >> >> >> >> >> >> /lib/libpython3.4m.so.1.0
>>>> >> >> >> >> >> >> 0008bbc8
>>>> >> >> >> >> >> >> _PyMem_RawStrdup
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2/Objects/obmalloc.c:323
>>>> >> >> >> >> >> >> /bld/python/Python-3.4.2 $
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >>
>>>> >> >> >> >> >> >> On Thu, Jan 29, 2015 at 8:26 PM, Ryan
>><rymg19 at gmail.com
>>>> >
>>>> >> >> >> >> >> >> wrote:
>>>> >> >> >> >> >> >>> Could you try the steps at
>>>> >> >> >> >> >> >>> http://stackoverflow.com/a/11369475/2097780?
>They
>>>> >> >> >> >> >> >>> allow you to get a better idea of where libc is
>>>> crashing.
>>>> >> >> >> >> >> >>>
>>>> >> >> >> >> >> >>> Cyd Haselton <chaselton at gmail.com> wrote:
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Managed to get this out of logcat:
>>>> >> >> >> >> >> >>>> F(11914) Fatal signal 11 (SIGSEGV) at
>0x00000000
>>>> >> >> >> >> >> >>>> (code=1),
>>>> >> >> >> >> >> >>>> thread
>>>> >> >> >> >> >> >>>> 11914 (python)  (libc)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> [ 01-29 19:30:55.855 23373:23373 F/libc     ]
>>>> >> >> >> >> >> >>>> Fatal signal 11 (SIGSEGV) at 0x00000000
>>(code=1),
>>>> thread
>>>> >> >> >> >> >> >>>> 23373
>>>> >> >> >> >> >> >>>> (python)
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> Less detail than strace but it seems to be that
>>>> python is
>>>> >> >> >> >> >> >>>> segfaulting
>>>> >> >> >> >> >> >>>> libc...
>>>> >> >> >> >> >> >>>>
>>>> >> >> >> >> >> >>>> On Wed, Jan 28, 2015 at 11:23 AM, Ryan Gonzalez
>>>> >> >> >> >> >> >>>> <rymg19 at gmail.com>
>>>> >> >> >> >> >> >>>> wrote:
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>  On Wed, Jan 28, 2015 at 10:43 AM, Guido van
>>Rossum
>>>> >> >> >> >> >> >>>>> <guido at python.org>
>>>> >> >> >> >> >> >>>>> wrote:
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  What I see in the strace:
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  ... load libpython3.4m.so.1.0
>>>> >> >> >> >> >> >>>>>>  ... load libm
>>>> >> >> >> >> >> >>>>>>  ... open /dev/__properties__ and do
>something
>>to it
>>>> >> >> >> >> >> >>>>>> (what?)
>>>> >> >> >> >> >> >>>>>>  ... get current time
>>>> >> >> >> >> >> >>>>>>  ... allocate memory
>>>> >> >> >> >> >> >>>>>>  ... getuid
>>>> >> >> >> >> >> >>>>>>  ... segfault
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  That's not a lot to go on, but it doesn't
>>look as
>>>> if
>>>> >> >> >> >> >> >>>>>> it
>>>> >> >> >> >> >> >>>>>> has
>>>> >> >> >> >> >> >>>>>> started to
>>>> >> >> >> >> >> >>>>>>  load modules yet.
>>>> >> >> >> >> >> >>>>>>
>>>> >> >> >> >> >> >>>>>>  Does /dev/__properties__ ring a bell? Not to
>>me.
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>> >> >> >> >> >> >>>>>
>>>>
>>https://android.googlesource.com/platform/system/core/+/tools_r22/init/property_service.c
>
>-- 
>Sent from my Android device with K-9 Mail. Please excuse my brevity.

-- 
Sent from my Android phone with K-9 Mail. Please excuse my brevity.
Check out my website: http://kirbyfan64.github.io/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20150131/a10b417d/attachment.html>

More information about the Python-Dev mailing list