[Mobile-sig] python on android

[Guido van Rossum]

My only suggestion: try to understand *exactly* what the Makefile is doing
at that point.

If it's really just invoking the wrong Python binary to do the
byte-compilation, you could also skip that until you're further along. The
byte-compilation is optional (it slows startup down a bit, but you
currently don't even start up, so who cares. :-)

On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee <
russell at keith-magee.com> wrote:

>
> Ok - so, yes, libPython is just copying and byte compiling. The catch is
> which version of Python and Python library you use to do the compiling.
>
> To compile cross platform, you need to do 2 builds - a "host" build (x86
> for most desktop purposes), and then a "target" build (the architecture of
> your phone).
>
> (As an aside - in the case of iOS, you actually need to do 2 target builds
> - one for the simulator (which is x86, but with a different libc to OSX),
> and one for the device (which is an arm7/arm7s/arm64 triple target build) -
> but that's beside the point. The problems I'm having exist without this
> added complication).
>
> So you do the host build, then you do the target build. The target build
> needs to call Python to invoke the compilation of modules etc; so you need
> to invoke the host python, but use the setup.py that was configured for the
> target Python.
>
> This much I have working (at least, I think I do - I can't test yet
> because I don't have a working build, but all signs are positive).
>
> But then, you get to libinstall, and you need to invoke Python to byte
> compile the pyc files. To do this, you need to invoke the host python,
> using the host Python's library, but over the *target* Python's sources.
> It's this last step that is tripping me up at the moment - I haven't worked
> out how to drive Autoconf to drive configure to pass in the set of
> arguments to invoke Python so that it will use the right binary and library
> with the right library tree. I keep end up running the iOS binary (which
> doesn't start), or the x86 binary with the iOS library tree (which is
> missing some parts that Python needs).
>
> The problem manifests as an "ImportError: No module named _struct",
> because the compiled parts of the struct module aren't in the iOS tree (or,
> at least, aren't compiled for x86 in the iOS tree)
>
> I appreciate that this isn't the easiest problem to debug over a mailing
> list. It's not even strictly "debugging" - it's a matter of working out
> what combination of arguments I need to pass in, and working backwards to
> the automake script. I've already made a bunch of changes to get this far,
> and I'm guessing any suggestions would have to be informed by what I've
> already done, which isn't a trivial thing to communicate without dumping a
> huge patch on your lap and saying "hey, fix this for me".
>
> This is why, to date, I haven't sought out help - I've just been beavering
> through the problems one at a time :-) If you've got any suggestions on
> more productive ways to tackle this, I'm all ears. And, again, I'm happy to
> share what I've got to date if anyone is interested in helping out.
>
> Yours,
> Russ Magee %-)
>
> On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum <guido at python.org>
> wrote:
>
>> Well, libinstall is nearly 100 lines, not counting dependencies. OTOH
>> it's just copying a lot of files and then byte-compiling them into .pyc
>> files -- and .pyc files are portable. So perhaps you can go into a little
>> more detail?
>>
>> On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee <
>> russell at keith-magee.com> wrote:
>>
>>> Hi Guido,
>>>
>>> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it
>>> up at all is because I keep hearing arguments that seem to dispute that
>>> getting a libPython build working *is* the first step. You've now put that
>>> argument to bed, so I agree - lets move on.
>>>
>>> For what it's worth, I've got a reasonable handle on how to compile
>>> libPython for mobile at this point - what I don't have is a good handle on
>>> is the intricacies of Python's build system, and in particular, how to
>>> drive Autoconf to support cross-platform builds.
>>>
>>> I've almost worked out the patches to the Python 2.7.1 source tree to
>>> generate an iOS-compatible libPython. Once I've got that working, I'm
>>> planning to merge those changes up to the tip of 2.7 and 3.X, and submit
>>> those patches for inclusion in the source tree. However, at the moment, I'm
>>> hitting problems with cross-platform execution in the libinstall target;
>>> I'm happy to share what I have so far with anyone interested in
>>> collaborating.
>>>
>>> Yours,
>>> Russ Magee %-)
>>>
>>> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum <guido at python.org>
>>> wrote:
>>>
>>>> Can we stop arguing about Kivy vs Toga and focus on the one thing that
>>>> they have in common, the need for a working Python 3 port on Android and
>>>> iOS (for a start)? This is apparently mostly a matter of solving a lot of
>>>> small things with the build system, dependencies, improved config files,
>>>> and getting stuff integrated upstream so it can be built out of the box,
>>>> right? After that the layers 2-4 stuff can compete, but everybody wins when
>>>> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody
>>>> apparently knows how to reproduce the build steps, and everybody just
>>>> copies the one Python 2.7.{1,2} binary that someone built out of sheer
>>>> willpower.
>>>>
>>>> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee <
>>>> russell at keith-magee.com> wrote:
>>>>
>>>>>
>>>>> Hi Bill,
>>>>>
>>>>> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen <janssen at parc.com>
>>>>> wrote:
>>>>>
>>>>>> Russell Keith-Magee <russell at keith-magee.com> wrote:
>>>>>>
>>>>>> >  b) I don't like the Kivy build tools. They're a lot more complex
>>>>>> than they
>>>>>> > need to be.
>>>>>>
>>>>>> I didn't find it troublesome, but of course this wasn't my first
>>>>>> rodeo.
>>>>>> I'd certainly agree it's not a push-button solution.  So, what would a
>>>>>> less complex system be like?
>>>>>>
>>>>>
>>>>> A less complex system is what Toga does.
>>>>>
>>>>>  1. You use cookiecutter to generate a stub project. This gives you
>>>>> the full source tree for a project you can load into XCode (iOS), or build
>>>>> with ant (Android), including a "hello world" __main__.py
>>>>>
>>>>>  2. You download the pre-compiled Python.framework for iOS, or
>>>>> libPython for Android, and copy it into a libs directory
>>>>>
>>>>>  3. You start writing Python code, replacing the __main__.py with your
>>>>> own logic.
>>>>>
>>>>>  4. You compile and deploy your project using XCode/ant.
>>>>>
>>>>> Compare this with Kivy - My experience was spending a couple of days
>>>>> getting the Kivy build process to actually work - trying to find versions
>>>>> of the Android NDK that aren't being distributed any more (but are the only
>>>>> hard coded options in the build system), working out that the provided code
>>>>> doesn't work with the most recent versions of Cython, and sourcing
>>>>> libraries for all sorts of dependencies, so that I could compile SDL and a
>>>>> bunch of OpenGL stuff - none of which I needed. It took me a couple of days
>>>>> to get to "hello world" - and all because of something that could have been
>>>>> shipped as a pre-compiled binary.
>>>>>
>>>>>
>>>>>> > I'm going to guess the Kivy people are all Linux users, because
>>>>>> > they don't appear to have worked out that binary compatibility is a
>>>>>> thing.
>>>>>>
>>>>>> Sorry -- why is that a Linux thing?
>>>>>>
>>>>>
>>>>> If I want to distribute an app for OS/X or Windows, I give you an
>>>>> executable, and It Just Works (tm). Source *might* be provided as an option
>>>>> in the interest of being open source, but it's not how you distribute
>>>>> anything in practice. The "Linux way" for distribution is to distribute
>>>>> source, and tell you to compile it yourself. Distributing binaries is an
>>>>> afterthought, because ABI compatibility makes building and distributing
>>>>> binaries painful. Most projects don't have the infrastructure to distribute
>>>>> binaries for multiple platforms, so unless you can get the OS to provide a
>>>>> recent binary for you, you compile from source.
>>>>>
>>>>> I see reflections of that bias here. Even though ABI compatibility
>>>>> exists for both iOS and Android as platforms, Kivy chooses to distribute as
>>>>> source.
>>>>>
>>>>>
>>>>>> > You don't need to have every user compile Python and the rest of
>>>>>> the Kivy
>>>>>> > stack - you can just ship a binary library, and it will work on
>>>>>> every phone
>>>>>> > with the same hardware (I know, because Toga does this. The Toga
>>>>>> > bootstrapping process is "clone this repo, and copy this file". You
>>>>>> could
>>>>>> > reduce this to "clone this repo" if you were happy putting binary
>>>>>> artefacts
>>>>>> > into version control.)
>>>>>>
>>>>>> Sure.
>>>>>>
>>>>>> >  c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on
>>>>>> Android;
>>>>>>
>>>>>> I believe the mobile platform packaging tools are still stuck at 2.7.
>>>>>> Kivy 1.8+ will run on Python 3 on desktop, though.
>>>>>
>>>>>
>>>>> Yes - but in practice, the absence of Python 3 on Mobile means that
>>>>> Kivy on Mobile is Python 2 only, and an old version at that.
>>>>>
>>>>>
>>>>>> > and if you build them on OSX, when you're on the device, they report
>>>>>> > sys.platform as "darwin".
>>>>>>
>>>>>> Seems like a bug; I imagine you're suggesting that the Kivy build
>>>>>> process should patch that file to return "android"?  Although I never
>>>>>> know what to look at to get that platform info correctly -- this is a
>>>>>> larger Python issue.
>>>>>>
>>>>>
>>>>> Yes, it's a bug (or at least a missing feature). The build system
>>>>> patches that Kivy use doesn't introduce anything for targeted builds (i.e.,
>>>>> using an x86 platform to compile ARM64 binaries - which is what you're
>>>>> doing when you compile for mobile), and doesn't provide a platform
>>>>> definition for mobile.
>>>>>
>>>>>
>>>>>> And kivy.utils.platform seems to return the proper thing.
>>>>>>
>>>>>
>>>>> So... instead of they've introduced their own way to get access to
>>>>> information that Python already has a standard way of providing.
>>>>>
>>>>>
>>>>>> > Going back to my post
>>>>>>
>>>>>> For those of you following along at home, here's Jeff's list (I had to
>>>>>> go and look it up).
>>>>>
>>>>>
>>>>> Jeff?
>>>>>
>>>>>
>>>>>> >  1. A library build of Python
>>>>>> >  2. Templates to stub out a working Python project
>>>>>> >  3. Libraries to do bridge between native language environments and
>>>>>> Python
>>>>>> >  (for me, that's Rubicon)
>>>>>> >  4. Libraries for utilising native system services (for me, that's
>>>>>> Toga)
>>>>>>
>>>>>> > - I agree with Kivy on layer 1, and I was able to use
>>>>>> > their build tools to bootstrap my own. However, I have very
>>>>>> different
>>>>>> > opinions on layers 2-4.
>>>>>>
>>>>>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 --
>>>>>> it
>>>>>> provides examples, and you're supposed to extrapolate from them.  I
>>>>>> guess that's a form of template.  For #3, there's Kivy's "pyjnius" (to
>>>>>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus"
>>>>>> (to
>>>>>> access Objective-C via runtime reflection)
>>>>>> (https://github.com/kivy/pyjnius).  For #4, as I said, there's
>>>>>> "plyer"
>>>>>> (https://github.com/kivy/plyer).
>>>>>
>>>>>
>>>>> I'd also include most of Kivy in #4 as well, because that's how
>>>>> they're tackling the widget issue.
>>>>>
>>>>> We can have a separate discussion sometime about Django vs. Tornado
>>>>>> :-).
>>>>>>
>>>>>
>>>>> So - a monkey knife-fight at dawn, then :-)
>>>>>
>>>>> Yours,
>>>>> Russ Magee %-)
>>>>>
>>>>> _______________________________________________
>>>>> Mobile-sig mailing list
>>>>> Mobile-sig at python.org
>>>>> https://mail.python.org/mailman/listinfo/mobile-sig
>>>>>
>>>>>
>>>>
>>>>
>>>> --
>>>> --Guido van Rossum (python.org/~guido)
>>>>
>>>
>>>
>>
>>
>> --
>> --Guido van Rossum (python.org/~guido)
>>
>
>


-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20150129/aac7d353/attachment-0001.html>