From fetchinson at googlemail.com Tue Jan 27 17:26:08 2015 From: fetchinson at googlemail.com (Fetchinson .) Date: Tue, 27 Jan 2015 17:26:08 +0100 Subject: [Mobile-sig] python on android Message-ID: Hi all, First of all thanks a lot for creating this mailing list. I sent the original mail to python-ideas [1] referenced by Jeff on meta-sig [2]. My perspective is that of a developer of python programs (mostly hobby projects) and not developer of python itself. What I'd like to see is a python distribution for android (I'm aware of the fact that this is mobile-sig and not android-sig, but for me personally android matters a lot more than other mobile platforms) that is as reliable as the linux/macos/windows distributions released by the python core dev team. Ideally, each time a new python version is released not only linux/macos/windows installers should be available on python.org but also android probably in the form of an installable apk. And just as there are platform specific features for e.g. windows that are not available on linux, there of course would be android-only features in the android distribution like hardware related libraries like camera, gps, accelerator, etc. sensors and probably more. Currently there is no single officially blessed (i.e. psf blessed) python distribution for android. There are various approaches, kivy [3], sl4a [4] and more [5]. But these are either dead projects or accomplish much more than a vanilla python distribution. What I'd like to see is an environment whose goal is to be "just" the python core plus about as much platform specific feature as there are for the others, linux/macos/windows. Of course the whole enterprise depends on the willingness of core python dev folks to dedicate time and energy to maintaining a code base that one can compile for android also which is not a small feat. I'm not sure they have this time and energy and I of course wouldn't blame them, this is open source, people do it in their free time. I merely would like to gauge the community's interest in something like this. Maybe if many users show interest then the core python dev folks can be convinced that this would be a worthy effort. Cheers, Daniel [1] https://mail.python.org/pipermail/python-ideas/2014-December/030490.html [2] https://mail.python.org/pipermail/meta-sig/2015-January/001412.html [3] http://kivy.org/ [4] https://github.com/damonkohler/sl4a [5] https://github.com/pybee/Python-Android-support -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From jacob at blindza.co.za Wed Jan 28 10:37:42 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Wed, 28 Jan 2015 11:37:42 +0200 Subject: [Mobile-sig] python on android Message-ID: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> ---original message--- > What I'd like to see is a python distribution for android (I'm aware > of the fact that this is mobile-sig and not android-sig, but for me > personally android matters a lot more than other mobile platforms) I agree, and android is my smartphone platform of choice, and while have played around with SL4A - TTS output calls it slang for android - in past, among a few others, would like something relatively stable/reliable, and best would be to be able to generate .apk files, which would include something like python interpreter wrappers around the code, or something. A while ago, I did play around with python on symbian/nokia phones, but, that was also just experimentation at the time. Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell at keith-magee.com Wed Jan 28 12:38:05 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Wed, 28 Jan 2015 19:38:05 +0800 Subject: [Mobile-sig] python on android In-Reply-To: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> Message-ID: Hi all, Introducing myself - I'm Russell Keith-Magee, the person behind Python-Android-support [1], Python-iOS-support [2], rubicon [3] and toga [4]. In terms of past history, I've been a member of the Django core team for 9 years. For the record, iOS is my platform of personal choice, and that's the platform that has received the bulk of my attention to date. However, the reason I'm looking at Python on mobile at all is for cross-platform purposes, so Android is still of interest to me. For that matter, so is Windows Mobile, Tizen, Sailfish and so on, but at least for the moment, the big two are where most of the market is. My personal end goal is to make Toga a viable platform for mobile application development. In attempting to reach that goal, I've been very careful to separate the goals of getting Python working on mobile, and getting Toga working on mobile. To my mind, there are 4 layers to the story: 1. A library build of Python 2. Templates to stub out a working Python project 3. Libraries to do bridge between native language environments and Python (for me, that's Rubicon) 4. Libraries for utilising native system services (for me, that's Toga) Of these, in my mind only layer 1 is something that should be targeted at Python's core. I've gone to great lengths to keep the concerns separate. If I had to criticise some of the other approaches mentioned, it would be that they *haven't* necessarily kept that separation (although, in their defence, that's possibly been done in the name of simplifying their own projects). To date, I've managed layer 1 with an external build process that applies patches and packages up a usable framework/library. My packages leverage heavily off what Kivy provided - but without all the layer 2-4 stuff. However, I've been working on patches for Python's own build system, so that the same product could be generated "out of the box". There are some complications associated with this, but that's the subject for another email (when I've got a little more to show for my work). In various discussions to date, some people have stated a goal of wrapping mobile facilities like Geolocation into a "standard" API in the stdlib. I'd like to go on the record that I think this is a bad idea, and possibly even an unachievable goal. The reason I think this is that you can't just "link" native system tools (at least, not trivially). You need to a cross language barrier - either Objc or Java, depending on your platform. I don't see ObjC or Java integration layers as being part of the mandate of Python core, but without a language integration layer, you can't use system native services [5]. To be clear, I think a common cross-platform API for these services is definitely worthwhile - I just don't think it's something that Python's *core* needs to be concerned with. For me, it's "layer 4" in my description above. I'm also of the opinion that while batteries included is a good philosophy, committing to a set of batteries too early is a bad idea - witness what has happened in Python's stdlib with HTTP handling or argument parsing. I'd rather see third party libraries, easily installed using pip et al, as the community's first response to this problem. Another proposal is to have an official install of Python for mobile platforms. While this is slightly more achievable, I doubt whether it's worthwhile, compared to the effort involved. The simplest interpretation of an "installable Python" would be a "Python shell on mobile". While this is an interesting novelty, I don't see it as being particularly useful in the long run; and, in order to have this novelty, you'd need to essentially write native terminal emulators for iOS and Android. Python on Windows doesn't ship with a *windows native* terminal emulator - it uses a DOS console, or Tk to launch IDLE. iOS doesn't have a native terminal capability, Android's isn't surfaced by default, and getting Tk to launch on either platform would be.... an interesting exercise :-) An alternate approach would be to have a "Python-running activity" which an APK might provide - SL4A does something like this, giving other apps the ability to execute Python scripts. While I'm sure this can be made to work, my experience as an end user was pretty poor. Perhaps this is my iOS bias showing, but when I use an app, I want to go to an app store, pick it, and install it. Android's ability to install an app from a URL is also appealing, albeit unavailable to iOS. What isn't appealing is trying to make sure that you've got a series of pre-requisite apps installed, and that they're correctly configured, and so on. On top of all that, no analogous service or capability exists on iOS, so even if you did this for Android, there wouldn't be cross-platform parity in what a "Mobile python" is able to do. So - after all that: what do I see as the way forward? To my mind, it's pretty minimal: - a set of patches to the existing build system to provide better support for cross-platform, targeted builds for libPython - a set of patches to sys.platform to provide an "android" and "iOS" platform definition - a set of patches to os to provide an "android" and "iOS" operating system definition And that's it. In order to get these patches into core, there are a bunch of questions to be answered about build systems and release testing, but the first step is to get the patches ready. Beyond that, there's still plenty of work to do to get a *useful* Python on a mobile platform - but what constitutes "useful" will vary wildly between users. I want to write standard, AppStore apps that use system native widgets. Kivy's approach is to write apps with a custom, but common widget appearance. Others might actually be interested in the SL4A "Python-running service" approach. SL4A differs from Kivy and Toga at layer 2; Kivy and Toga differ at layer 3. However, all these end-user approaches have the same layer 1 - a libPython that can be used on a mobile device. However, the different approaches to layers 2-4 can all co-exist, and do so *outside* Python's core. Regardless of what comes of this SIG, I'm going to keep beavering away; but if anyone in the community is similarly aligned, and we can work together (especially on the Android bits, which fall into the "secondary interest" pile for me personally), I'd love to be able to work together. Yours, Russ Magee %-) [1] https://github.com/pybee/Python-Android-support [2] https://github.com/pybee/Python-iOS-support [3] http://pybee.org/rubicon/ [4] http://pybee.org/toga [5] At least, I haven't been able to find a way. On iOS, you definitely need to cross the ObjC barrier - there isn't a "raw C" API for any basic system services that a native module could link against. On Android, I haven't found any way to link to native capabilities - the APIs are all in Java, and expect to be called through Java. However, as I said in my intro, Android isn't my platform of strength, so I'll defer to expertise on this. -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Wed Jan 28 19:01:57 2015 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 Jan 2015 10:01:57 -0800 Subject: [Mobile-sig] python on android In-Reply-To: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> Message-ID: <24279.1422468117@parc.com> Jacob Kruger wrote: > ---original message--- > > What I'd like to see is a python distribution for android (I'm aware > > of the fact that this is mobile-sig and not android-sig, but for me > > personally android matters a lot more than other mobile platforms) > > I agree, and android is my smartphone platform of choice, and while > have played around with SL4A - TTS output calls it slang for android - > in past, among a few others, would like something relatively > stable/reliable, and best would be to be able to generate .apk files, > which would include something like python interpreter wrappers around > the code, or something. To me, this is a solved problem. I'm running several apps completely written in Python on my Android phone. I used Kivy (http://kivy.org/, but https://github.com/kivy is more useful if you're a coder), for the UI, the API to Android services, and for the packaging into an APK. The UI is the Kivy set of widgets: http://kivy.org/docs/api-kivy.html The generic OS API is "plyer": http://plyer.readthedocs.org/en/latest/ The packager is buildozer: http://buildozer.readthedocs.org/en/latest/ I'm amazed at how well it all works. Bill From jacob at blindza.co.za Wed Jan 28 19:17:45 2015 From: jacob at blindza.co.za (Jacob Kruger) Date: Wed, 28 Jan 2015 20:17:45 +0200 Subject: [Mobile-sig] python on android In-Reply-To: <24279.1422468117@parc.com> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> Message-ID: <253C51F819C04DECB4BFFAC300741003@JakesPC> ----- Original Message ----- > To me, this is a solved problem. I'm running several apps completely > written in Python on my Android phone. I used Kivy (http://kivy.org/, > but https://github.com/kivy is more useful if you're a coder), for the > UI, the API to Android services, and for the packaging into an APK. > > The UI is the Kivy set of widgets: http://kivy.org/docs/api-kivy.html > > The generic OS API is "plyer": http://plyer.readthedocs.org/en/latest/ > > The packager is buildozer: http://buildozer.readthedocs.org/en/latest/ > > I'm amazed at how well it all works. > > Bill Will have another look at it, but, think issue last time glanced at it was download sources/packages, and, if it uses it's own form of UI elements, not sure about if that will work perfectly well along with talkback screen reader, etc. - but, won't know until try... Jacob Kruger Blind Biker Skype: BlindZA "Roger Wilco wants to welcome you...to the space janitor's closet..." From russell at keith-magee.com Thu Jan 29 02:07:57 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Thu, 29 Jan 2015 09:07:57 +0800 Subject: [Mobile-sig] python on android In-Reply-To: <24279.1422468117@parc.com> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> Message-ID: On Thu, Jan 29, 2015 at 2:01 AM, Bill Janssen wrote: > Jacob Kruger wrote: > > > ---original message--- > > > What I'd like to see is a python distribution for android (I'm aware > > > of the fact that this is mobile-sig and not android-sig, but for me > > > personally android matters a lot more than other mobile platforms) > > > > I agree, and android is my smartphone platform of choice, and while > > have played around with SL4A - TTS output calls it slang for android - > > in past, among a few others, would like something relatively > > stable/reliable, and best would be to be able to generate .apk files, > > which would include something like python interpreter wrappers around > > the code, or something. > > To me, this is a solved problem. I'm running several apps completely > written in Python on my Android phone. I used Kivy (http://kivy.org/, > but https://github.com/kivy is more useful if you're a coder), for the > UI, the API to Android services, and for the packaging into an APK. > > The UI is the Kivy set of widgets: http://kivy.org/docs/api-kivy.html > > The generic OS API is "plyer": http://plyer.readthedocs.org/en/latest/ > > The packager is buildozer: http://buildozer.readthedocs.org/en/latest/ > > I'm amazed at how well it all works. > I'm glad that you find Kivy good enough for your purposes, but I don't. a) I don't like the Kivy widget set. It might be fine for games, but if I'm using a platform, I want native widgets, and if I'm using native widgets, I don't want to carry the overhead of a customised widget toolkit I'm not using. b) I don't like the Kivy build tools. They're a lot more complex than they need to be. I'm going to guess the Kivy people are all Linux users, because they don't appear to have worked out that binary compatibility is a thing. You don't need to have every user compile Python and the rest of the Kivy stack - you can just ship a binary library, and it will work on every phone with the same hardware (I know, because Toga does this. The Toga bootstrapping process is "clone this repo, and copy this file". You could reduce this to "clone this repo" if you were happy putting binary artefacts into version control.) c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on Android; and if you build them on OSX, when you're on the device, they report sys.platform as "darwin". Going back to my post - I agree with Kivy on layer 1, and I was able to use their build tools to bootstrap my own. However, I have very different opinions on layers 2-4. Kivy is *a* solution for Python on mobile; it isn't the *only* possible solution. On top of that, Kivy still has a lot of work to do on point (c). If effort is going to be spent fixing libPython for mobile, I'd like to see it done in Python's core - this isn't something that should need to be duplicated by every project that has an interest in this space. I also like to see this in core as a formal recognition that mobile platforms matter in the landscape of modern computing. And once a mobile libPython exists, Kivy can use it however they want; I can use it however I want; and SL4A can use it however they want. Hopefully, we'll be able to share effort along the way on other parts of the project - but as I detailed in my original response, I don't think that necessarily needs to be part of Python core. Yours, Russ Magee %-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Thu Jan 29 03:41:31 2015 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 Jan 2015 18:41:31 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> Message-ID: <7129.1422499291@parc.com> Russell Keith-Magee wrote: > I'm glad that you find Kivy good enough for your purposes, but I don't. Don't get me wrong! I think your core idea for a set of patches is a great one, and I'm sure the Kivy folks would appreciate it, too. They bit off an awful lot to chew there. > a) I don't like the Kivy widget set. It might be fine for games, but if > I'm using a platform, I want native widgets, and if I'm using native > widgets, I don't want to carry the overhead of a customised widget toolkit > I'm not using. Yes, I don't care about native widgets. Of course, the UI widget set is there for a different purpose, portability across platforms, and if you don't care about that, you can always use the native widget set with Kivy, instead. But I'm happy enough with the Kivy widgets. > b) I don't like the Kivy build tools. They're a lot more complex than they > need to be. I didn't find it troublesome, but of course this wasn't my first rodeo. I'd certainly agree it's not a push-button solution. So, what would a less complex system be like? > I'm going to guess the Kivy people are all Linux users, because > they don't appear to have worked out that binary compatibility is a thing. Sorry -- why is that a Linux thing? > You don't need to have every user compile Python and the rest of the Kivy > stack - you can just ship a binary library, and it will work on every phone > with the same hardware (I know, because Toga does this. The Toga > bootstrapping process is "clone this repo, and copy this file". You could > reduce this to "clone this repo" if you were happy putting binary artefacts > into version control.) Sure. > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on Android; I believe the mobile platform packaging tools are still stuck at 2.7. Kivy 1.8+ will run on Python 3 on desktop, though. > and if you build them on OSX, when you're on the device, they report > sys.platform as "darwin". Seems like a bug; I imagine you're suggesting that the Kivy build process should patch that file to return "android"? Although I never know what to look at to get that platform info correctly -- this is a larger Python issue. And kivy.utils.platform seems to return the proper thing. > Going back to my post For those of you following along at home, here's Jeff's list (I had to go and look it up). > 1. A library build of Python > 2. Templates to stub out a working Python project > 3. Libraries to do bridge between native language environments and Python > (for me, that's Rubicon) > 4. Libraries for utilising native system services (for me, that's Toga) > - I agree with Kivy on layer 1, and I was able to use > their build tools to bootstrap my own. However, I have very different > opinions on layers 2-4. Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it provides examples, and you're supposed to extrapolate from them. I guess that's a form of template. For #3, there's Kivy's "pyjnius" (to access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to access Objective-C via runtime reflection) (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" (https://github.com/kivy/plyer). > Kivy is *a* solution for Python on mobile; it isn't the *only* > possible solution. Of course. > On top of that, Kivy still has a lot of work to do on point (c). If effort > is going to be spent fixing libPython for mobile, I'd like to see it done > in Python's core - this isn't something that should need to be duplicated > by every project that has an interest in this space. I wouldn't imagine they'd argue with that. > I also like to see this in core as a formal recognition that mobile > platforms matter in the landscape of modern computing. Sure. > And once a mobile libPython exists, Kivy can use it however they want; I > can use it however I want; and SL4A can use it however they want. Sure. We can have a separate discussion sometime about Django vs. Tornado :-). Bill From russell at keith-magee.com Thu Jan 29 05:04:42 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Thu, 29 Jan 2015 12:04:42 +0800 Subject: [Mobile-sig] python on android In-Reply-To: <7129.1422499291@parc.com> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: Hi Bill, On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: > Russell Keith-Magee wrote: > > > b) I don't like the Kivy build tools. They're a lot more complex than > they > > need to be. > > I didn't find it troublesome, but of course this wasn't my first rodeo. > I'd certainly agree it's not a push-button solution. So, what would a > less complex system be like? > A less complex system is what Toga does. 1. You use cookiecutter to generate a stub project. This gives you the full source tree for a project you can load into XCode (iOS), or build with ant (Android), including a "hello world" __main__.py 2. You download the pre-compiled Python.framework for iOS, or libPython for Android, and copy it into a libs directory 3. You start writing Python code, replacing the __main__.py with your own logic. 4. You compile and deploy your project using XCode/ant. Compare this with Kivy - My experience was spending a couple of days getting the Kivy build process to actually work - trying to find versions of the Android NDK that aren't being distributed any more (but are the only hard coded options in the build system), working out that the provided code doesn't work with the most recent versions of Cython, and sourcing libraries for all sorts of dependencies, so that I could compile SDL and a bunch of OpenGL stuff - none of which I needed. It took me a couple of days to get to "hello world" - and all because of something that could have been shipped as a pre-compiled binary. > > I'm going to guess the Kivy people are all Linux users, because > > they don't appear to have worked out that binary compatibility is a > thing. > > Sorry -- why is that a Linux thing? > If I want to distribute an app for OS/X or Windows, I give you an executable, and It Just Works (tm). Source *might* be provided as an option in the interest of being open source, but it's not how you distribute anything in practice. The "Linux way" for distribution is to distribute source, and tell you to compile it yourself. Distributing binaries is an afterthought, because ABI compatibility makes building and distributing binaries painful. Most projects don't have the infrastructure to distribute binaries for multiple platforms, so unless you can get the OS to provide a recent binary for you, you compile from source. I see reflections of that bias here. Even though ABI compatibility exists for both iOS and Android as platforms, Kivy chooses to distribute as source. > > You don't need to have every user compile Python and the rest of the Kivy > > stack - you can just ship a binary library, and it will work on every > phone > > with the same hardware (I know, because Toga does this. The Toga > > bootstrapping process is "clone this repo, and copy this file". You could > > reduce this to "clone this repo" if you were happy putting binary > artefacts > > into version control.) > > Sure. > > > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on > Android; > > I believe the mobile platform packaging tools are still stuck at 2.7. > Kivy 1.8+ will run on Python 3 on desktop, though. Yes - but in practice, the absence of Python 3 on Mobile means that Kivy on Mobile is Python 2 only, and an old version at that. > > and if you build them on OSX, when you're on the device, they report > > sys.platform as "darwin". > > Seems like a bug; I imagine you're suggesting that the Kivy build > process should patch that file to return "android"? Although I never > know what to look at to get that platform info correctly -- this is a > larger Python issue. > Yes, it's a bug (or at least a missing feature). The build system patches that Kivy use doesn't introduce anything for targeted builds (i.e., using an x86 platform to compile ARM64 binaries - which is what you're doing when you compile for mobile), and doesn't provide a platform definition for mobile. > And kivy.utils.platform seems to return the proper thing. > So... instead of they've introduced their own way to get access to information that Python already has a standard way of providing. > > Going back to my post > > For those of you following along at home, here's Jeff's list (I had to > go and look it up). Jeff? > > 1. A library build of Python > > 2. Templates to stub out a working Python project > > 3. Libraries to do bridge between native language environments and > Python > > (for me, that's Rubicon) > > 4. Libraries for utilising native system services (for me, that's Toga) > > > - I agree with Kivy on layer 1, and I was able to use > > their build tools to bootstrap my own. However, I have very different > > opinions on layers 2-4. > > Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it > provides examples, and you're supposed to extrapolate from them. I > guess that's a form of template. For #3, there's Kivy's "pyjnius" (to > access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to > access Objective-C via runtime reflection) > (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" > (https://github.com/kivy/plyer). I'd also include most of Kivy in #4 as well, because that's how they're tackling the widget issue. We can have a separate discussion sometime about Django vs. Tornado :-). > So - a monkey knife-fight at dawn, then :-) Yours, Russ Magee %-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jan 29 05:31:49 2015 From: guido at python.org (Guido van Rossum) Date: Wed, 28 Jan 2015 20:31:49 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: Can we stop arguing about Kivy vs Toga and focus on the one thing that they have in common, the need for a working Python 3 port on Android and iOS (for a start)? This is apparently mostly a matter of solving a lot of small things with the build system, dependencies, improved config files, and getting stuff integrated upstream so it can be built out of the box, right? After that the layers 2-4 stuff can compete, but everybody wins when layer 1 is dealt with (even imperfectly). It's pretty sad that nobody apparently knows how to reproduce the build steps, and everybody just copies the one Python 2.7.{1,2} binary that someone built out of sheer willpower. On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < russell at keith-magee.com> wrote: > > Hi Bill, > > On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: > >> Russell Keith-Magee wrote: >> >> > b) I don't like the Kivy build tools. They're a lot more complex than >> they >> > need to be. >> >> I didn't find it troublesome, but of course this wasn't my first rodeo. >> I'd certainly agree it's not a push-button solution. So, what would a >> less complex system be like? >> > > A less complex system is what Toga does. > > 1. You use cookiecutter to generate a stub project. This gives you the > full source tree for a project you can load into XCode (iOS), or build with > ant (Android), including a "hello world" __main__.py > > 2. You download the pre-compiled Python.framework for iOS, or libPython > for Android, and copy it into a libs directory > > 3. You start writing Python code, replacing the __main__.py with your own > logic. > > 4. You compile and deploy your project using XCode/ant. > > Compare this with Kivy - My experience was spending a couple of days > getting the Kivy build process to actually work - trying to find versions > of the Android NDK that aren't being distributed any more (but are the only > hard coded options in the build system), working out that the provided code > doesn't work with the most recent versions of Cython, and sourcing > libraries for all sorts of dependencies, so that I could compile SDL and a > bunch of OpenGL stuff - none of which I needed. It took me a couple of days > to get to "hello world" - and all because of something that could have been > shipped as a pre-compiled binary. > > >> > I'm going to guess the Kivy people are all Linux users, because >> > they don't appear to have worked out that binary compatibility is a >> thing. >> >> Sorry -- why is that a Linux thing? >> > > If I want to distribute an app for OS/X or Windows, I give you an > executable, and It Just Works (tm). Source *might* be provided as an option > in the interest of being open source, but it's not how you distribute > anything in practice. The "Linux way" for distribution is to distribute > source, and tell you to compile it yourself. Distributing binaries is an > afterthought, because ABI compatibility makes building and distributing > binaries painful. Most projects don't have the infrastructure to distribute > binaries for multiple platforms, so unless you can get the OS to provide a > recent binary for you, you compile from source. > > I see reflections of that bias here. Even though ABI compatibility exists > for both iOS and Android as platforms, Kivy chooses to distribute as source. > > >> > You don't need to have every user compile Python and the rest of the >> Kivy >> > stack - you can just ship a binary library, and it will work on every >> phone >> > with the same hardware (I know, because Toga does this. The Toga >> > bootstrapping process is "clone this repo, and copy this file". You >> could >> > reduce this to "clone this repo" if you were happy putting binary >> artefacts >> > into version control.) >> >> Sure. >> >> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >> Android; >> >> I believe the mobile platform packaging tools are still stuck at 2.7. >> Kivy 1.8+ will run on Python 3 on desktop, though. > > > Yes - but in practice, the absence of Python 3 on Mobile means that Kivy > on Mobile is Python 2 only, and an old version at that. > > >> > and if you build them on OSX, when you're on the device, they report >> > sys.platform as "darwin". >> >> Seems like a bug; I imagine you're suggesting that the Kivy build >> process should patch that file to return "android"? Although I never >> know what to look at to get that platform info correctly -- this is a >> larger Python issue. >> > > Yes, it's a bug (or at least a missing feature). The build system patches > that Kivy use doesn't introduce anything for targeted builds (i.e., using > an x86 platform to compile ARM64 binaries - which is what you're doing when > you compile for mobile), and doesn't provide a platform definition for > mobile. > > >> And kivy.utils.platform seems to return the proper thing. >> > > So... instead of they've introduced their own way to get access to > information that Python already has a standard way of providing. > > >> > Going back to my post >> >> For those of you following along at home, here's Jeff's list (I had to >> go and look it up). > > > Jeff? > > >> > 1. A library build of Python >> > 2. Templates to stub out a working Python project >> > 3. Libraries to do bridge between native language environments and >> Python >> > (for me, that's Rubicon) >> > 4. Libraries for utilising native system services (for me, that's Toga) >> >> > - I agree with Kivy on layer 1, and I was able to use >> > their build tools to bootstrap my own. However, I have very different >> > opinions on layers 2-4. >> >> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it >> provides examples, and you're supposed to extrapolate from them. I >> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to >> access Objective-C via runtime reflection) >> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >> (https://github.com/kivy/plyer). > > > I'd also include most of Kivy in #4 as well, because that's how they're > tackling the widget issue. > > We can have a separate discussion sometime about Django vs. Tornado :-). >> > > So - a monkey knife-fight at dawn, then :-) > > Yours, > Russ Magee %-) > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell at keith-magee.com Thu Jan 29 06:10:35 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Thu, 29 Jan 2015 13:10:35 +0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: Hi Guido, I'll glady stop arguing about Kivy vs Toga. The only reason I brought it up at all is because I keep hearing arguments that seem to dispute that getting a libPython build working *is* the first step. You've now put that argument to bed, so I agree - lets move on. For what it's worth, I've got a reasonable handle on how to compile libPython for mobile at this point - what I don't have is a good handle on is the intricacies of Python's build system, and in particular, how to drive Autoconf to support cross-platform builds. I've almost worked out the patches to the Python 2.7.1 source tree to generate an iOS-compatible libPython. Once I've got that working, I'm planning to merge those changes up to the tip of 2.7 and 3.X, and submit those patches for inclusion in the source tree. However, at the moment, I'm hitting problems with cross-platform execution in the libinstall target; I'm happy to share what I have so far with anyone interested in collaborating. Yours, Russ Magee %-) On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum wrote: > Can we stop arguing about Kivy vs Toga and focus on the one thing that > they have in common, the need for a working Python 3 port on Android and > iOS (for a start)? This is apparently mostly a matter of solving a lot of > small things with the build system, dependencies, improved config files, > and getting stuff integrated upstream so it can be built out of the box, > right? After that the layers 2-4 stuff can compete, but everybody wins when > layer 1 is dealt with (even imperfectly). It's pretty sad that nobody > apparently knows how to reproduce the build steps, and everybody just > copies the one Python 2.7.{1,2} binary that someone built out of sheer > willpower. > > On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > >> >> Hi Bill, >> >> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: >> >>> Russell Keith-Magee wrote: >>> >>> > b) I don't like the Kivy build tools. They're a lot more complex than >>> they >>> > need to be. >>> >>> I didn't find it troublesome, but of course this wasn't my first rodeo. >>> I'd certainly agree it's not a push-button solution. So, what would a >>> less complex system be like? >>> >> >> A less complex system is what Toga does. >> >> 1. You use cookiecutter to generate a stub project. This gives you the >> full source tree for a project you can load into XCode (iOS), or build with >> ant (Android), including a "hello world" __main__.py >> >> 2. You download the pre-compiled Python.framework for iOS, or libPython >> for Android, and copy it into a libs directory >> >> 3. You start writing Python code, replacing the __main__.py with your >> own logic. >> >> 4. You compile and deploy your project using XCode/ant. >> >> Compare this with Kivy - My experience was spending a couple of days >> getting the Kivy build process to actually work - trying to find versions >> of the Android NDK that aren't being distributed any more (but are the only >> hard coded options in the build system), working out that the provided code >> doesn't work with the most recent versions of Cython, and sourcing >> libraries for all sorts of dependencies, so that I could compile SDL and a >> bunch of OpenGL stuff - none of which I needed. It took me a couple of days >> to get to "hello world" - and all because of something that could have been >> shipped as a pre-compiled binary. >> >> >>> > I'm going to guess the Kivy people are all Linux users, because >>> > they don't appear to have worked out that binary compatibility is a >>> thing. >>> >>> Sorry -- why is that a Linux thing? >>> >> >> If I want to distribute an app for OS/X or Windows, I give you an >> executable, and It Just Works (tm). Source *might* be provided as an option >> in the interest of being open source, but it's not how you distribute >> anything in practice. The "Linux way" for distribution is to distribute >> source, and tell you to compile it yourself. Distributing binaries is an >> afterthought, because ABI compatibility makes building and distributing >> binaries painful. Most projects don't have the infrastructure to distribute >> binaries for multiple platforms, so unless you can get the OS to provide a >> recent binary for you, you compile from source. >> >> I see reflections of that bias here. Even though ABI compatibility exists >> for both iOS and Android as platforms, Kivy chooses to distribute as source. >> >> >>> > You don't need to have every user compile Python and the rest of the >>> Kivy >>> > stack - you can just ship a binary library, and it will work on every >>> phone >>> > with the same hardware (I know, because Toga does this. The Toga >>> > bootstrapping process is "clone this repo, and copy this file". You >>> could >>> > reduce this to "clone this repo" if you were happy putting binary >>> artefacts >>> > into version control.) >>> >>> Sure. >>> >>> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >>> Android; >>> >>> I believe the mobile platform packaging tools are still stuck at 2.7. >>> Kivy 1.8+ will run on Python 3 on desktop, though. >> >> >> Yes - but in practice, the absence of Python 3 on Mobile means that Kivy >> on Mobile is Python 2 only, and an old version at that. >> >> >>> > and if you build them on OSX, when you're on the device, they report >>> > sys.platform as "darwin". >>> >>> Seems like a bug; I imagine you're suggesting that the Kivy build >>> process should patch that file to return "android"? Although I never >>> know what to look at to get that platform info correctly -- this is a >>> larger Python issue. >>> >> >> Yes, it's a bug (or at least a missing feature). The build system patches >> that Kivy use doesn't introduce anything for targeted builds (i.e., using >> an x86 platform to compile ARM64 binaries - which is what you're doing when >> you compile for mobile), and doesn't provide a platform definition for >> mobile. >> >> >>> And kivy.utils.platform seems to return the proper thing. >>> >> >> So... instead of they've introduced their own way to get access to >> information that Python already has a standard way of providing. >> >> >>> > Going back to my post >>> >>> For those of you following along at home, here's Jeff's list (I had to >>> go and look it up). >> >> >> Jeff? >> >> >>> > 1. A library build of Python >>> > 2. Templates to stub out a working Python project >>> > 3. Libraries to do bridge between native language environments and >>> Python >>> > (for me, that's Rubicon) >>> > 4. Libraries for utilising native system services (for me, that's >>> Toga) >>> >>> > - I agree with Kivy on layer 1, and I was able to use >>> > their build tools to bootstrap my own. However, I have very different >>> > opinions on layers 2-4. >>> >>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it >>> provides examples, and you're supposed to extrapolate from them. I >>> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to >>> access Objective-C via runtime reflection) >>> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >>> (https://github.com/kivy/plyer). >> >> >> I'd also include most of Kivy in #4 as well, because that's how they're >> tackling the widget issue. >> >> We can have a separate discussion sometime about Django vs. Tornado :-). >>> >> >> So - a monkey knife-fight at dawn, then :-) >> >> Yours, >> Russ Magee %-) >> >> _______________________________________________ >> Mobile-sig mailing list >> Mobile-sig at python.org >> https://mail.python.org/mailman/listinfo/mobile-sig >> >> > > > -- > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From guido at python.org Thu Jan 29 07:17:30 2015 From: guido at python.org (Guido van Rossum) Date: Wed, 28 Jan 2015 22:17:30 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: Well, libinstall is nearly 100 lines, not counting dependencies. OTOH it's just copying a lot of files and then byte-compiling them into .pyc files -- and .pyc files are portable. So perhaps you can go into a little more detail? On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < russell at keith-magee.com> wrote: > Hi Guido, > > I'll glady stop arguing about Kivy vs Toga. The only reason I brought it > up at all is because I keep hearing arguments that seem to dispute that > getting a libPython build working *is* the first step. You've now put that > argument to bed, so I agree - lets move on. > > For what it's worth, I've got a reasonable handle on how to compile > libPython for mobile at this point - what I don't have is a good handle on > is the intricacies of Python's build system, and in particular, how to > drive Autoconf to support cross-platform builds. > > I've almost worked out the patches to the Python 2.7.1 source tree to > generate an iOS-compatible libPython. Once I've got that working, I'm > planning to merge those changes up to the tip of 2.7 and 3.X, and submit > those patches for inclusion in the source tree. However, at the moment, I'm > hitting problems with cross-platform execution in the libinstall target; > I'm happy to share what I have so far with anyone interested in > collaborating. > > Yours, > Russ Magee %-) > > On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum > wrote: > >> Can we stop arguing about Kivy vs Toga and focus on the one thing that >> they have in common, the need for a working Python 3 port on Android and >> iOS (for a start)? This is apparently mostly a matter of solving a lot of >> small things with the build system, dependencies, improved config files, >> and getting stuff integrated upstream so it can be built out of the box, >> right? After that the layers 2-4 stuff can compete, but everybody wins when >> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody >> apparently knows how to reproduce the build steps, and everybody just >> copies the one Python 2.7.{1,2} binary that someone built out of sheer >> willpower. >> >> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < >> russell at keith-magee.com> wrote: >> >>> >>> Hi Bill, >>> >>> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: >>> >>>> Russell Keith-Magee wrote: >>>> >>>> > b) I don't like the Kivy build tools. They're a lot more complex >>>> than they >>>> > need to be. >>>> >>>> I didn't find it troublesome, but of course this wasn't my first rodeo. >>>> I'd certainly agree it's not a push-button solution. So, what would a >>>> less complex system be like? >>>> >>> >>> A less complex system is what Toga does. >>> >>> 1. You use cookiecutter to generate a stub project. This gives you the >>> full source tree for a project you can load into XCode (iOS), or build with >>> ant (Android), including a "hello world" __main__.py >>> >>> 2. You download the pre-compiled Python.framework for iOS, or libPython >>> for Android, and copy it into a libs directory >>> >>> 3. You start writing Python code, replacing the __main__.py with your >>> own logic. >>> >>> 4. You compile and deploy your project using XCode/ant. >>> >>> Compare this with Kivy - My experience was spending a couple of days >>> getting the Kivy build process to actually work - trying to find versions >>> of the Android NDK that aren't being distributed any more (but are the only >>> hard coded options in the build system), working out that the provided code >>> doesn't work with the most recent versions of Cython, and sourcing >>> libraries for all sorts of dependencies, so that I could compile SDL and a >>> bunch of OpenGL stuff - none of which I needed. It took me a couple of days >>> to get to "hello world" - and all because of something that could have been >>> shipped as a pre-compiled binary. >>> >>> >>>> > I'm going to guess the Kivy people are all Linux users, because >>>> > they don't appear to have worked out that binary compatibility is a >>>> thing. >>>> >>>> Sorry -- why is that a Linux thing? >>>> >>> >>> If I want to distribute an app for OS/X or Windows, I give you an >>> executable, and It Just Works (tm). Source *might* be provided as an option >>> in the interest of being open source, but it's not how you distribute >>> anything in practice. The "Linux way" for distribution is to distribute >>> source, and tell you to compile it yourself. Distributing binaries is an >>> afterthought, because ABI compatibility makes building and distributing >>> binaries painful. Most projects don't have the infrastructure to distribute >>> binaries for multiple platforms, so unless you can get the OS to provide a >>> recent binary for you, you compile from source. >>> >>> I see reflections of that bias here. Even though ABI compatibility >>> exists for both iOS and Android as platforms, Kivy chooses to distribute as >>> source. >>> >>> >>>> > You don't need to have every user compile Python and the rest of the >>>> Kivy >>>> > stack - you can just ship a binary library, and it will work on every >>>> phone >>>> > with the same hardware (I know, because Toga does this. The Toga >>>> > bootstrapping process is "clone this repo, and copy this file". You >>>> could >>>> > reduce this to "clone this repo" if you were happy putting binary >>>> artefacts >>>> > into version control.) >>>> >>>> Sure. >>>> >>>> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >>>> Android; >>>> >>>> I believe the mobile platform packaging tools are still stuck at 2.7. >>>> Kivy 1.8+ will run on Python 3 on desktop, though. >>> >>> >>> Yes - but in practice, the absence of Python 3 on Mobile means that Kivy >>> on Mobile is Python 2 only, and an old version at that. >>> >>> >>>> > and if you build them on OSX, when you're on the device, they report >>>> > sys.platform as "darwin". >>>> >>>> Seems like a bug; I imagine you're suggesting that the Kivy build >>>> process should patch that file to return "android"? Although I never >>>> know what to look at to get that platform info correctly -- this is a >>>> larger Python issue. >>>> >>> >>> Yes, it's a bug (or at least a missing feature). The build system >>> patches that Kivy use doesn't introduce anything for targeted builds (i.e., >>> using an x86 platform to compile ARM64 binaries - which is what you're >>> doing when you compile for mobile), and doesn't provide a platform >>> definition for mobile. >>> >>> >>>> And kivy.utils.platform seems to return the proper thing. >>>> >>> >>> So... instead of they've introduced their own way to get access to >>> information that Python already has a standard way of providing. >>> >>> >>>> > Going back to my post >>>> >>>> For those of you following along at home, here's Jeff's list (I had to >>>> go and look it up). >>> >>> >>> Jeff? >>> >>> >>>> > 1. A library build of Python >>>> > 2. Templates to stub out a working Python project >>>> > 3. Libraries to do bridge between native language environments and >>>> Python >>>> > (for me, that's Rubicon) >>>> > 4. Libraries for utilising native system services (for me, that's >>>> Toga) >>>> >>>> > - I agree with Kivy on layer 1, and I was able to use >>>> > their build tools to bootstrap my own. However, I have very different >>>> > opinions on layers 2-4. >>>> >>>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it >>>> provides examples, and you're supposed to extrapolate from them. I >>>> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >>>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" >>>> (to >>>> access Objective-C via runtime reflection) >>>> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >>>> (https://github.com/kivy/plyer). >>> >>> >>> I'd also include most of Kivy in #4 as well, because that's how they're >>> tackling the widget issue. >>> >>> We can have a separate discussion sometime about Django vs. Tornado :-). >>>> >>> >>> So - a monkey knife-fight at dawn, then :-) >>> >>> Yours, >>> Russ Magee %-) >>> >>> _______________________________________________ >>> Mobile-sig mailing list >>> Mobile-sig at python.org >>> https://mail.python.org/mailman/listinfo/mobile-sig >>> >>> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell at keith-magee.com Thu Jan 29 07:58:14 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Thu, 29 Jan 2015 14:58:14 +0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: Ok - so, yes, libPython is just copying and byte compiling. The catch is which version of Python and Python library you use to do the compiling. To compile cross platform, you need to do 2 builds - a "host" build (x86 for most desktop purposes), and then a "target" build (the architecture of your phone). (As an aside - in the case of iOS, you actually need to do 2 target builds - one for the simulator (which is x86, but with a different libc to OSX), and one for the device (which is an arm7/arm7s/arm64 triple target build) - but that's beside the point. The problems I'm having exist without this added complication). So you do the host build, then you do the target build. The target build needs to call Python to invoke the compilation of modules etc; so you need to invoke the host python, but use the setup.py that was configured for the target Python. This much I have working (at least, I think I do - I can't test yet because I don't have a working build, but all signs are positive). But then, you get to libinstall, and you need to invoke Python to byte compile the pyc files. To do this, you need to invoke the host python, using the host Python's library, but over the *target* Python's sources. It's this last step that is tripping me up at the moment - I haven't worked out how to drive Autoconf to drive configure to pass in the set of arguments to invoke Python so that it will use the right binary and library with the right library tree. I keep end up running the iOS binary (which doesn't start), or the x86 binary with the iOS library tree (which is missing some parts that Python needs). The problem manifests as an "ImportError: No module named _struct", because the compiled parts of the struct module aren't in the iOS tree (or, at least, aren't compiled for x86 in the iOS tree) I appreciate that this isn't the easiest problem to debug over a mailing list. It's not even strictly "debugging" - it's a matter of working out what combination of arguments I need to pass in, and working backwards to the automake script. I've already made a bunch of changes to get this far, and I'm guessing any suggestions would have to be informed by what I've already done, which isn't a trivial thing to communicate without dumping a huge patch on your lap and saying "hey, fix this for me". This is why, to date, I haven't sought out help - I've just been beavering through the problems one at a time :-) If you've got any suggestions on more productive ways to tackle this, I'm all ears. And, again, I'm happy to share what I've got to date if anyone is interested in helping out. Yours, Russ Magee %-) On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum wrote: > Well, libinstall is nearly 100 lines, not counting dependencies. OTOH it's > just copying a lot of files and then byte-compiling them into .pyc files -- > and .pyc files are portable. So perhaps you can go into a little more > detail? > > On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > >> Hi Guido, >> >> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it >> up at all is because I keep hearing arguments that seem to dispute that >> getting a libPython build working *is* the first step. You've now put that >> argument to bed, so I agree - lets move on. >> >> For what it's worth, I've got a reasonable handle on how to compile >> libPython for mobile at this point - what I don't have is a good handle on >> is the intricacies of Python's build system, and in particular, how to >> drive Autoconf to support cross-platform builds. >> >> I've almost worked out the patches to the Python 2.7.1 source tree to >> generate an iOS-compatible libPython. Once I've got that working, I'm >> planning to merge those changes up to the tip of 2.7 and 3.X, and submit >> those patches for inclusion in the source tree. However, at the moment, I'm >> hitting problems with cross-platform execution in the libinstall target; >> I'm happy to share what I have so far with anyone interested in >> collaborating. >> >> Yours, >> Russ Magee %-) >> >> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum >> wrote: >> >>> Can we stop arguing about Kivy vs Toga and focus on the one thing that >>> they have in common, the need for a working Python 3 port on Android and >>> iOS (for a start)? This is apparently mostly a matter of solving a lot of >>> small things with the build system, dependencies, improved config files, >>> and getting stuff integrated upstream so it can be built out of the box, >>> right? After that the layers 2-4 stuff can compete, but everybody wins when >>> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody >>> apparently knows how to reproduce the build steps, and everybody just >>> copies the one Python 2.7.{1,2} binary that someone built out of sheer >>> willpower. >>> >>> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < >>> russell at keith-magee.com> wrote: >>> >>>> >>>> Hi Bill, >>>> >>>> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen >>>> wrote: >>>> >>>>> Russell Keith-Magee wrote: >>>>> >>>>> > b) I don't like the Kivy build tools. They're a lot more complex >>>>> than they >>>>> > need to be. >>>>> >>>>> I didn't find it troublesome, but of course this wasn't my first rodeo. >>>>> I'd certainly agree it's not a push-button solution. So, what would a >>>>> less complex system be like? >>>>> >>>> >>>> A less complex system is what Toga does. >>>> >>>> 1. You use cookiecutter to generate a stub project. This gives you the >>>> full source tree for a project you can load into XCode (iOS), or build with >>>> ant (Android), including a "hello world" __main__.py >>>> >>>> 2. You download the pre-compiled Python.framework for iOS, or >>>> libPython for Android, and copy it into a libs directory >>>> >>>> 3. You start writing Python code, replacing the __main__.py with your >>>> own logic. >>>> >>>> 4. You compile and deploy your project using XCode/ant. >>>> >>>> Compare this with Kivy - My experience was spending a couple of days >>>> getting the Kivy build process to actually work - trying to find versions >>>> of the Android NDK that aren't being distributed any more (but are the only >>>> hard coded options in the build system), working out that the provided code >>>> doesn't work with the most recent versions of Cython, and sourcing >>>> libraries for all sorts of dependencies, so that I could compile SDL and a >>>> bunch of OpenGL stuff - none of which I needed. It took me a couple of days >>>> to get to "hello world" - and all because of something that could have been >>>> shipped as a pre-compiled binary. >>>> >>>> >>>>> > I'm going to guess the Kivy people are all Linux users, because >>>>> > they don't appear to have worked out that binary compatibility is a >>>>> thing. >>>>> >>>>> Sorry -- why is that a Linux thing? >>>>> >>>> >>>> If I want to distribute an app for OS/X or Windows, I give you an >>>> executable, and It Just Works (tm). Source *might* be provided as an option >>>> in the interest of being open source, but it's not how you distribute >>>> anything in practice. The "Linux way" for distribution is to distribute >>>> source, and tell you to compile it yourself. Distributing binaries is an >>>> afterthought, because ABI compatibility makes building and distributing >>>> binaries painful. Most projects don't have the infrastructure to distribute >>>> binaries for multiple platforms, so unless you can get the OS to provide a >>>> recent binary for you, you compile from source. >>>> >>>> I see reflections of that bias here. Even though ABI compatibility >>>> exists for both iOS and Android as platforms, Kivy chooses to distribute as >>>> source. >>>> >>>> >>>>> > You don't need to have every user compile Python and the rest of the >>>>> Kivy >>>>> > stack - you can just ship a binary library, and it will work on >>>>> every phone >>>>> > with the same hardware (I know, because Toga does this. The Toga >>>>> > bootstrapping process is "clone this repo, and copy this file". You >>>>> could >>>>> > reduce this to "clone this repo" if you were happy putting binary >>>>> artefacts >>>>> > into version control.) >>>>> >>>>> Sure. >>>>> >>>>> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >>>>> Android; >>>>> >>>>> I believe the mobile platform packaging tools are still stuck at 2.7. >>>>> Kivy 1.8+ will run on Python 3 on desktop, though. >>>> >>>> >>>> Yes - but in practice, the absence of Python 3 on Mobile means that >>>> Kivy on Mobile is Python 2 only, and an old version at that. >>>> >>>> >>>>> > and if you build them on OSX, when you're on the device, they report >>>>> > sys.platform as "darwin". >>>>> >>>>> Seems like a bug; I imagine you're suggesting that the Kivy build >>>>> process should patch that file to return "android"? Although I never >>>>> know what to look at to get that platform info correctly -- this is a >>>>> larger Python issue. >>>>> >>>> >>>> Yes, it's a bug (or at least a missing feature). The build system >>>> patches that Kivy use doesn't introduce anything for targeted builds (i.e., >>>> using an x86 platform to compile ARM64 binaries - which is what you're >>>> doing when you compile for mobile), and doesn't provide a platform >>>> definition for mobile. >>>> >>>> >>>>> And kivy.utils.platform seems to return the proper thing. >>>>> >>>> >>>> So... instead of they've introduced their own way to get access to >>>> information that Python already has a standard way of providing. >>>> >>>> >>>>> > Going back to my post >>>>> >>>>> For those of you following along at home, here's Jeff's list (I had to >>>>> go and look it up). >>>> >>>> >>>> Jeff? >>>> >>>> >>>>> > 1. A library build of Python >>>>> > 2. Templates to stub out a working Python project >>>>> > 3. Libraries to do bridge between native language environments and >>>>> Python >>>>> > (for me, that's Rubicon) >>>>> > 4. Libraries for utilising native system services (for me, that's >>>>> Toga) >>>>> >>>>> > - I agree with Kivy on layer 1, and I was able to use >>>>> > their build tools to bootstrap my own. However, I have very different >>>>> > opinions on layers 2-4. >>>>> >>>>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- >>>>> it >>>>> provides examples, and you're supposed to extrapolate from them. I >>>>> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >>>>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" >>>>> (to >>>>> access Objective-C via runtime reflection) >>>>> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >>>>> (https://github.com/kivy/plyer). >>>> >>>> >>>> I'd also include most of Kivy in #4 as well, because that's how they're >>>> tackling the widget issue. >>>> >>>> We can have a separate discussion sometime about Django vs. Tornado :-). >>>>> >>>> >>>> So - a monkey knife-fight at dawn, then :-) >>>> >>>> Yours, >>>> Russ Magee %-) >>>> >>>> _______________________________________________ >>>> Mobile-sig mailing list >>>> Mobile-sig at python.org >>>> https://mail.python.org/mailman/listinfo/mobile-sig >>>> >>>> >>> >>> >>> -- >>> --Guido van Rossum (python.org/~guido) >>> >> >> > > > -- > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From janssen at parc.com Thu Jan 29 08:32:15 2015 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 Jan 2015 23:32:15 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: <12898.1422516735@parc.com> Russell Keith-Magee wrote: > > For those of you following along at home, here's Jeff's list (I had to > > go and look it up). > > > Jeff? Sorry, don't know how that snuck in there. Bill From janssen at parc.com Thu Jan 29 08:36:24 2015 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 Jan 2015 23:36:24 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: <13019.1422516984@parc.com> Russell Keith-Magee wrote: > If I want to distribute an app for OS/X or Windows, I give you an > executable, and It Just Works (tm). Source *might* be provided as an option > in the interest of being open source, but it's not how you distribute > anything in practice. The "Linux way" for distribution is to distribute > source, and tell you to compile it yourself. Ah, I see what you mean. As an old open-source guy, I compile everything from source everywhere, so I guess I fall into this Linux camp, too. And, in practice, this Just Works thing isn't all it's cracked up to be. The Kivy folks do provide a binary VirtualBox image, which has everything installed correctly. Though the buildozer process *still* compiles everything from source, sigh. Bill From janssen at parc.com Thu Jan 29 08:56:40 2015 From: janssen at parc.com (Bill Janssen) Date: Wed, 28 Jan 2015 23:56:40 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: <13421.1422518200@parc.com> Guido van Rossum wrote: > Can we stop arguing about Kivy vs Toga and focus on the one thing that they > have in common, the need for a working Python 3 port on Android and iOS > (for a start)? Hi, Guido. Does that not already exist? I guess you'd know. > This is apparently mostly a matter of solving a lot of small > things with the build system, dependencies, improved config files, and > getting stuff integrated upstream so it can be built out of the box, right? Must be. The Kivy buildozer script just compiles Python from source, so the first experiment would be to drop in a different version and see what breaks. There's no ancient Python binary in the Kivy build process, but there are a dozen or so source patches against 2.7.2: https://github.com/kivy/python-for-android/tree/master/recipes/python/patches They'd have to be updated. Most are one or two line changes. Bill From janssen at parc.com Thu Jan 29 09:09:50 2015 From: janssen at parc.com (Bill Janssen) Date: Thu, 29 Jan 2015 00:09:50 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: <13744.1422518990@parc.com> Russell Keith-Magee wrote: > I appreciate that this isn't the easiest problem to debug over a mailing > list. It's not even strictly "debugging" - it's a matter of working out > what combination of arguments I need to pass in, and working backwards to > the automake script. Don't know if this helps, but the Kivy build script for iOS is at https://github.com/kivy/kivy-ios/blob/master/tools/build-python.sh, and it references http://randomsplat.com/id5-cross-compiling-python-for-embedded-linux.html, which details a process for cross-compiling said to work for Python 3, as well. Bill From guido at python.org Thu Jan 29 16:54:28 2015 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2015 07:54:28 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: My only suggestion: try to understand *exactly* what the Makefile is doing at that point. If it's really just invoking the wrong Python binary to do the byte-compilation, you could also skip that until you're further along. The byte-compilation is optional (it slows startup down a bit, but you currently don't even start up, so who cares. :-) On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee < russell at keith-magee.com> wrote: > > Ok - so, yes, libPython is just copying and byte compiling. The catch is > which version of Python and Python library you use to do the compiling. > > To compile cross platform, you need to do 2 builds - a "host" build (x86 > for most desktop purposes), and then a "target" build (the architecture of > your phone). > > (As an aside - in the case of iOS, you actually need to do 2 target builds > - one for the simulator (which is x86, but with a different libc to OSX), > and one for the device (which is an arm7/arm7s/arm64 triple target build) - > but that's beside the point. The problems I'm having exist without this > added complication). > > So you do the host build, then you do the target build. The target build > needs to call Python to invoke the compilation of modules etc; so you need > to invoke the host python, but use the setup.py that was configured for the > target Python. > > This much I have working (at least, I think I do - I can't test yet > because I don't have a working build, but all signs are positive). > > But then, you get to libinstall, and you need to invoke Python to byte > compile the pyc files. To do this, you need to invoke the host python, > using the host Python's library, but over the *target* Python's sources. > It's this last step that is tripping me up at the moment - I haven't worked > out how to drive Autoconf to drive configure to pass in the set of > arguments to invoke Python so that it will use the right binary and library > with the right library tree. I keep end up running the iOS binary (which > doesn't start), or the x86 binary with the iOS library tree (which is > missing some parts that Python needs). > > The problem manifests as an "ImportError: No module named _struct", > because the compiled parts of the struct module aren't in the iOS tree (or, > at least, aren't compiled for x86 in the iOS tree) > > I appreciate that this isn't the easiest problem to debug over a mailing > list. It's not even strictly "debugging" - it's a matter of working out > what combination of arguments I need to pass in, and working backwards to > the automake script. I've already made a bunch of changes to get this far, > and I'm guessing any suggestions would have to be informed by what I've > already done, which isn't a trivial thing to communicate without dumping a > huge patch on your lap and saying "hey, fix this for me". > > This is why, to date, I haven't sought out help - I've just been beavering > through the problems one at a time :-) If you've got any suggestions on > more productive ways to tackle this, I'm all ears. And, again, I'm happy to > share what I've got to date if anyone is interested in helping out. > > Yours, > Russ Magee %-) > > On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum > wrote: > >> Well, libinstall is nearly 100 lines, not counting dependencies. OTOH >> it's just copying a lot of files and then byte-compiling them into .pyc >> files -- and .pyc files are portable. So perhaps you can go into a little >> more detail? >> >> On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < >> russell at keith-magee.com> wrote: >> >>> Hi Guido, >>> >>> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it >>> up at all is because I keep hearing arguments that seem to dispute that >>> getting a libPython build working *is* the first step. You've now put that >>> argument to bed, so I agree - lets move on. >>> >>> For what it's worth, I've got a reasonable handle on how to compile >>> libPython for mobile at this point - what I don't have is a good handle on >>> is the intricacies of Python's build system, and in particular, how to >>> drive Autoconf to support cross-platform builds. >>> >>> I've almost worked out the patches to the Python 2.7.1 source tree to >>> generate an iOS-compatible libPython. Once I've got that working, I'm >>> planning to merge those changes up to the tip of 2.7 and 3.X, and submit >>> those patches for inclusion in the source tree. However, at the moment, I'm >>> hitting problems with cross-platform execution in the libinstall target; >>> I'm happy to share what I have so far with anyone interested in >>> collaborating. >>> >>> Yours, >>> Russ Magee %-) >>> >>> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum >>> wrote: >>> >>>> Can we stop arguing about Kivy vs Toga and focus on the one thing that >>>> they have in common, the need for a working Python 3 port on Android and >>>> iOS (for a start)? This is apparently mostly a matter of solving a lot of >>>> small things with the build system, dependencies, improved config files, >>>> and getting stuff integrated upstream so it can be built out of the box, >>>> right? After that the layers 2-4 stuff can compete, but everybody wins when >>>> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody >>>> apparently knows how to reproduce the build steps, and everybody just >>>> copies the one Python 2.7.{1,2} binary that someone built out of sheer >>>> willpower. >>>> >>>> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < >>>> russell at keith-magee.com> wrote: >>>> >>>>> >>>>> Hi Bill, >>>>> >>>>> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen >>>>> wrote: >>>>> >>>>>> Russell Keith-Magee wrote: >>>>>> >>>>>> > b) I don't like the Kivy build tools. They're a lot more complex >>>>>> than they >>>>>> > need to be. >>>>>> >>>>>> I didn't find it troublesome, but of course this wasn't my first >>>>>> rodeo. >>>>>> I'd certainly agree it's not a push-button solution. So, what would a >>>>>> less complex system be like? >>>>>> >>>>> >>>>> A less complex system is what Toga does. >>>>> >>>>> 1. You use cookiecutter to generate a stub project. This gives you >>>>> the full source tree for a project you can load into XCode (iOS), or build >>>>> with ant (Android), including a "hello world" __main__.py >>>>> >>>>> 2. You download the pre-compiled Python.framework for iOS, or >>>>> libPython for Android, and copy it into a libs directory >>>>> >>>>> 3. You start writing Python code, replacing the __main__.py with your >>>>> own logic. >>>>> >>>>> 4. You compile and deploy your project using XCode/ant. >>>>> >>>>> Compare this with Kivy - My experience was spending a couple of days >>>>> getting the Kivy build process to actually work - trying to find versions >>>>> of the Android NDK that aren't being distributed any more (but are the only >>>>> hard coded options in the build system), working out that the provided code >>>>> doesn't work with the most recent versions of Cython, and sourcing >>>>> libraries for all sorts of dependencies, so that I could compile SDL and a >>>>> bunch of OpenGL stuff - none of which I needed. It took me a couple of days >>>>> to get to "hello world" - and all because of something that could have been >>>>> shipped as a pre-compiled binary. >>>>> >>>>> >>>>>> > I'm going to guess the Kivy people are all Linux users, because >>>>>> > they don't appear to have worked out that binary compatibility is a >>>>>> thing. >>>>>> >>>>>> Sorry -- why is that a Linux thing? >>>>>> >>>>> >>>>> If I want to distribute an app for OS/X or Windows, I give you an >>>>> executable, and It Just Works (tm). Source *might* be provided as an option >>>>> in the interest of being open source, but it's not how you distribute >>>>> anything in practice. The "Linux way" for distribution is to distribute >>>>> source, and tell you to compile it yourself. Distributing binaries is an >>>>> afterthought, because ABI compatibility makes building and distributing >>>>> binaries painful. Most projects don't have the infrastructure to distribute >>>>> binaries for multiple platforms, so unless you can get the OS to provide a >>>>> recent binary for you, you compile from source. >>>>> >>>>> I see reflections of that bias here. Even though ABI compatibility >>>>> exists for both iOS and Android as platforms, Kivy chooses to distribute as >>>>> source. >>>>> >>>>> >>>>>> > You don't need to have every user compile Python and the rest of >>>>>> the Kivy >>>>>> > stack - you can just ship a binary library, and it will work on >>>>>> every phone >>>>>> > with the same hardware (I know, because Toga does this. The Toga >>>>>> > bootstrapping process is "clone this repo, and copy this file". You >>>>>> could >>>>>> > reduce this to "clone this repo" if you were happy putting binary >>>>>> artefacts >>>>>> > into version control.) >>>>>> >>>>>> Sure. >>>>>> >>>>>> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >>>>>> Android; >>>>>> >>>>>> I believe the mobile platform packaging tools are still stuck at 2.7. >>>>>> Kivy 1.8+ will run on Python 3 on desktop, though. >>>>> >>>>> >>>>> Yes - but in practice, the absence of Python 3 on Mobile means that >>>>> Kivy on Mobile is Python 2 only, and an old version at that. >>>>> >>>>> >>>>>> > and if you build them on OSX, when you're on the device, they report >>>>>> > sys.platform as "darwin". >>>>>> >>>>>> Seems like a bug; I imagine you're suggesting that the Kivy build >>>>>> process should patch that file to return "android"? Although I never >>>>>> know what to look at to get that platform info correctly -- this is a >>>>>> larger Python issue. >>>>>> >>>>> >>>>> Yes, it's a bug (or at least a missing feature). The build system >>>>> patches that Kivy use doesn't introduce anything for targeted builds (i.e., >>>>> using an x86 platform to compile ARM64 binaries - which is what you're >>>>> doing when you compile for mobile), and doesn't provide a platform >>>>> definition for mobile. >>>>> >>>>> >>>>>> And kivy.utils.platform seems to return the proper thing. >>>>>> >>>>> >>>>> So... instead of they've introduced their own way to get access to >>>>> information that Python already has a standard way of providing. >>>>> >>>>> >>>>>> > Going back to my post >>>>>> >>>>>> For those of you following along at home, here's Jeff's list (I had to >>>>>> go and look it up). >>>>> >>>>> >>>>> Jeff? >>>>> >>>>> >>>>>> > 1. A library build of Python >>>>>> > 2. Templates to stub out a working Python project >>>>>> > 3. Libraries to do bridge between native language environments and >>>>>> Python >>>>>> > (for me, that's Rubicon) >>>>>> > 4. Libraries for utilising native system services (for me, that's >>>>>> Toga) >>>>>> >>>>>> > - I agree with Kivy on layer 1, and I was able to use >>>>>> > their build tools to bootstrap my own. However, I have very >>>>>> different >>>>>> > opinions on layers 2-4. >>>>>> >>>>>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- >>>>>> it >>>>>> provides examples, and you're supposed to extrapolate from them. I >>>>>> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >>>>>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" >>>>>> (to >>>>>> access Objective-C via runtime reflection) >>>>>> (https://github.com/kivy/pyjnius). For #4, as I said, there's >>>>>> "plyer" >>>>>> (https://github.com/kivy/plyer). >>>>> >>>>> >>>>> I'd also include most of Kivy in #4 as well, because that's how >>>>> they're tackling the widget issue. >>>>> >>>>> We can have a separate discussion sometime about Django vs. Tornado >>>>>> :-). >>>>>> >>>>> >>>>> So - a monkey knife-fight at dawn, then :-) >>>>> >>>>> Yours, >>>>> Russ Magee %-) >>>>> >>>>> _______________________________________________ >>>>> Mobile-sig mailing list >>>>> Mobile-sig at python.org >>>>> https://mail.python.org/mailman/listinfo/mobile-sig >>>>> >>>>> >>>> >>>> >>>> -- >>>> --Guido van Rossum (python.org/~guido) >>>> >>> >>> >> >> >> -- >> --Guido van Rossum (python.org/~guido) >> > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From fetchinson at googlemail.com Thu Jan 29 17:45:05 2015 From: fetchinson at googlemail.com (Fetchinson .) Date: Thu, 29 Jan 2015 17:45:05 +0100 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: On 1/29/15, Guido van Rossum wrote: > Can we stop arguing about Kivy vs Toga and focus on the one thing that they > have in common, the need for a working Python 3 port on Android and iOS > (for a start)? This is apparently mostly a matter of solving a lot of small > things with the build system, dependencies, improved config files, and > getting stuff integrated upstream so it can be built out of the box, right? I think so. And even better would be if binary android or iOS packages are distributed on python.org just like for windows and people don't have to compile themselves since even if the source tree is perfectly okay and can be compiled for android or iOS the actual compilation will be complicated for an average user of (and not developer of) a python program on a phone. > After that the layers 2-4 stuff can compete, but everybody wins when layer > 1 is dealt with (even imperfectly). Exactly. All the layers 2-4 stuff from various approaches would use the same layer 1 stuff just as various GUI toolkits on desktop use the same python core distribution. I understand from Russell that hardware integration on the phone is tricky and is definitely separate from layer 1, so all the fancy phone related hardware stuff can live in competing packages. But a core python (layer 1) should be there for all developers to count on. Cheers, Daniel > It's pretty sad that nobody apparently > knows how to reproduce the build steps, and everybody just copies the one > Python 2.7.{1,2} binary that someone built out of sheer willpower. > > On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > >> >> Hi Bill, >> >> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: >> >>> Russell Keith-Magee wrote: >>> >>> > b) I don't like the Kivy build tools. They're a lot more complex than >>> they >>> > need to be. >>> >>> I didn't find it troublesome, but of course this wasn't my first rodeo. >>> I'd certainly agree it's not a push-button solution. So, what would a >>> less complex system be like? >>> >> >> A less complex system is what Toga does. >> >> 1. You use cookiecutter to generate a stub project. This gives you the >> full source tree for a project you can load into XCode (iOS), or build >> with >> ant (Android), including a "hello world" __main__.py >> >> 2. You download the pre-compiled Python.framework for iOS, or libPython >> for Android, and copy it into a libs directory >> >> 3. You start writing Python code, replacing the __main__.py with your >> own >> logic. >> >> 4. You compile and deploy your project using XCode/ant. >> >> Compare this with Kivy - My experience was spending a couple of days >> getting the Kivy build process to actually work - trying to find versions >> of the Android NDK that aren't being distributed any more (but are the >> only >> hard coded options in the build system), working out that the provided >> code >> doesn't work with the most recent versions of Cython, and sourcing >> libraries for all sorts of dependencies, so that I could compile SDL and >> a >> bunch of OpenGL stuff - none of which I needed. It took me a couple of >> days >> to get to "hello world" - and all because of something that could have >> been >> shipped as a pre-compiled binary. >> >> >>> > I'm going to guess the Kivy people are all Linux users, because >>> > they don't appear to have worked out that binary compatibility is a >>> thing. >>> >>> Sorry -- why is that a Linux thing? >>> >> >> If I want to distribute an app for OS/X or Windows, I give you an >> executable, and It Just Works (tm). Source *might* be provided as an >> option >> in the interest of being open source, but it's not how you distribute >> anything in practice. The "Linux way" for distribution is to distribute >> source, and tell you to compile it yourself. Distributing binaries is an >> afterthought, because ABI compatibility makes building and distributing >> binaries painful. Most projects don't have the infrastructure to >> distribute >> binaries for multiple platforms, so unless you can get the OS to provide >> a >> recent binary for you, you compile from source. >> >> I see reflections of that bias here. Even though ABI compatibility exists >> for both iOS and Android as platforms, Kivy chooses to distribute as >> source. >> >> >>> > You don't need to have every user compile Python and the rest of the >>> Kivy >>> > stack - you can just ship a binary library, and it will work on every >>> phone >>> > with the same hardware (I know, because Toga does this. The Toga >>> > bootstrapping process is "clone this repo, and copy this file". You >>> could >>> > reduce this to "clone this repo" if you were happy putting binary >>> artefacts >>> > into version control.) >>> >>> Sure. >>> >>> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >>> Android; >>> >>> I believe the mobile platform packaging tools are still stuck at 2.7. >>> Kivy 1.8+ will run on Python 3 on desktop, though. >> >> >> Yes - but in practice, the absence of Python 3 on Mobile means that Kivy >> on Mobile is Python 2 only, and an old version at that. >> >> >>> > and if you build them on OSX, when you're on the device, they report >>> > sys.platform as "darwin". >>> >>> Seems like a bug; I imagine you're suggesting that the Kivy build >>> process should patch that file to return "android"? Although I never >>> know what to look at to get that platform info correctly -- this is a >>> larger Python issue. >>> >> >> Yes, it's a bug (or at least a missing feature). The build system patches >> that Kivy use doesn't introduce anything for targeted builds (i.e., using >> an x86 platform to compile ARM64 binaries - which is what you're doing >> when >> you compile for mobile), and doesn't provide a platform definition for >> mobile. >> >> >>> And kivy.utils.platform seems to return the proper thing. >>> >> >> So... instead of they've introduced their own way to get access to >> information that Python already has a standard way of providing. >> >> >>> > Going back to my post >>> >>> For those of you following along at home, here's Jeff's list (I had to >>> go and look it up). >> >> >> Jeff? >> >> >>> > 1. A library build of Python >>> > 2. Templates to stub out a working Python project >>> > 3. Libraries to do bridge between native language environments and >>> Python >>> > (for me, that's Rubicon) >>> > 4. Libraries for utilising native system services (for me, that's >>> > Toga) >>> >>> > - I agree with Kivy on layer 1, and I was able to use >>> > their build tools to bootstrap my own. However, I have very different >>> > opinions on layers 2-4. >>> >>> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it >>> provides examples, and you're supposed to extrapolate from them. I >>> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >>> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to >>> access Objective-C via runtime reflection) >>> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >>> (https://github.com/kivy/plyer). >> >> >> I'd also include most of Kivy in #4 as well, because that's how they're >> tackling the widget issue. >> >> We can have a separate discussion sometime about Django vs. Tornado :-). >>> >> >> So - a monkey knife-fight at dawn, then :-) >> >> Yours, >> Russ Magee %-) >> >> _______________________________________________ >> Mobile-sig mailing list >> Mobile-sig at python.org >> https://mail.python.org/mailman/listinfo/mobile-sig >> >> > > > -- > --Guido van Rossum (python.org/~guido) > -- Psss, psss, put it down! - http://www.cafepress.com/putitdown From guido at python.org Thu Jan 29 18:33:13 2015 From: guido at python.org (Guido van Rossum) Date: Thu, 29 Jan 2015 09:33:13 -0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: On Thu, Jan 29, 2015 at 8:45 AM, Fetchinson . wrote: > On 1/29/15, Guido van Rossum wrote: > > Can we stop arguing about Kivy vs Toga and focus on the one thing that > they > > have in common, the need for a working Python 3 port on Android and iOS > > (for a start)? This is apparently mostly a matter of solving a lot of > small > > things with the build system, dependencies, improved config files, and > > getting stuff integrated upstream so it can be built out of the box, > right? > > I think so. And even better would be if binary android or iOS packages > are distributed on python.org just like for windows and people don't > have to compile themselves since even if the source tree is perfectly > okay and can be compiled for android or iOS the actual compilation > will be complicated for an average user of (and not developer of) a > python program on a phone. > Well, *someone* has to commit to making those packages. And it's unlikely that it'll be one of the core committers, who don't have the right expertise (give us a break :-). For Mac and Windows, we coordinate the release of the binary packages carefully with new source releases -- for other platforms, I don't think such a tightly coupled schedule will work. But instead I do hope that you or someone else who cares enough and has the expertise and resources to build such binaries will build them and post them to PyPI. It then shouldn't be a big hassle to get the python.org website updated with links to the latest packages on PyPI. -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.i.frank at intel.com Thu Jan 29 18:53:54 2015 From: matthew.i.frank at intel.com (Frank, Matthew I) Date: Thu, 29 Jan 2015 17:53:54 +0000 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> Message-ID: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> I believe that I?ve seen this particular problem when building Python 3 for Android, but that it comes and goes. The current open patch for Python 3.5 that you need to apply is: http://bugs.python.org/issue22625. For Python 3 the problem is in Makefile.pre.in around line 750: $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN) Should be $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS) For cross-building Python 3.4.2 from a Linux host I used the following configure command for the target build: CPPFLAGS=-I/abs/path/to/source/Python-3.4.2/FIXLOCALE ../Python-3.4.2/configure --enable-shared --prefix=/absolute/path/to/android/install/of/python --build=x86_64-linux-gnu --host=i686-linux-android --disable-ipv6 ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no ac_cv_little_endian_double=yes --without-ensurepip The ?without-ensurepip, in particular was necessary because ensurepip needs to call the just-built Python interpreter (which won?t work when cross compiling). There are some other things that need patching for Android (around the mbs to wcs functions and nl_langinfo() and such), but that?s the only problem with the Makefile that I?m aware of. -Matt From: Guido van Rossum [mailto:guido at python.org] Sent: Thursday, January 29, 2015 9:54 AM To: Russell Keith-Magee Cc: mobile-sig Subject: Re: [Mobile-sig] python on android My only suggestion: try to understand *exactly* what the Makefile is doing at that point. If it's really just invoking the wrong Python binary to do the byte-compilation, you could also skip that until you're further along. The byte-compilation is optional (it slows startup down a bit, but you currently don't even start up, so who cares. :-) On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee > wrote: Ok - so, yes, libPython is just copying and byte compiling. The catch is which version of Python and Python library you use to do the compiling. To compile cross platform, you need to do 2 builds - a "host" build (x86 for most desktop purposes), and then a "target" build (the architecture of your phone). (As an aside - in the case of iOS, you actually need to do 2 target builds - one for the simulator (which is x86, but with a different libc to OSX), and one for the device (which is an arm7/arm7s/arm64 triple target build) - but that's beside the point. The problems I'm having exist without this added complication). So you do the host build, then you do the target build. The target build needs to call Python to invoke the compilation of modules etc; so you need to invoke the host python, but use the setup.py that was configured for the target Python. This much I have working (at least, I think I do - I can't test yet because I don't have a working build, but all signs are positive). But then, you get to libinstall, and you need to invoke Python to byte compile the pyc files. To do this, you need to invoke the host python, using the host Python's library, but over the *target* Python's sources. It's this last step that is tripping me up at the moment - I haven't worked out how to drive Autoconf to drive configure to pass in the set of arguments to invoke Python so that it will use the right binary and library with the right library tree. I keep end up running the iOS binary (which doesn't start), or the x86 binary with the iOS library tree (which is missing some parts that Python needs). The problem manifests as an "ImportError: No module named _struct", because the compiled parts of the struct module aren't in the iOS tree (or, at least, aren't compiled for x86 in the iOS tree) I appreciate that this isn't the easiest problem to debug over a mailing list. It's not even strictly "debugging" - it's a matter of working out what combination of arguments I need to pass in, and working backwards to the automake script. I've already made a bunch of changes to get this far, and I'm guessing any suggestions would have to be informed by what I've already done, which isn't a trivial thing to communicate without dumping a huge patch on your lap and saying "hey, fix this for me". This is why, to date, I haven't sought out help - I've just been beavering through the problems one at a time :-) If you've got any suggestions on more productive ways to tackle this, I'm all ears. And, again, I'm happy to share what I've got to date if anyone is interested in helping out. Yours, Russ Magee %-) On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum > wrote: Well, libinstall is nearly 100 lines, not counting dependencies. OTOH it's just copying a lot of files and then byte-compiling them into .pyc files -- and .pyc files are portable. So perhaps you can go into a little more detail? On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee > wrote: Hi Guido, I'll glady stop arguing about Kivy vs Toga. The only reason I brought it up at all is because I keep hearing arguments that seem to dispute that getting a libPython build working *is* the first step. You've now put that argument to bed, so I agree - lets move on. For what it's worth, I've got a reasonable handle on how to compile libPython for mobile at this point - what I don't have is a good handle on is the intricacies of Python's build system, and in particular, how to drive Autoconf to support cross-platform builds. I've almost worked out the patches to the Python 2.7.1 source tree to generate an iOS-compatible libPython. Once I've got that working, I'm planning to merge those changes up to the tip of 2.7 and 3.X, and submit those patches for inclusion in the source tree. However, at the moment, I'm hitting problems with cross-platform execution in the libinstall target; I'm happy to share what I have so far with anyone interested in collaborating. Yours, Russ Magee %-) On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum > wrote: Can we stop arguing about Kivy vs Toga and focus on the one thing that they have in common, the need for a working Python 3 port on Android and iOS (for a start)? This is apparently mostly a matter of solving a lot of small things with the build system, dependencies, improved config files, and getting stuff integrated upstream so it can be built out of the box, right? After that the layers 2-4 stuff can compete, but everybody wins when layer 1 is dealt with (even imperfectly). It's pretty sad that nobody apparently knows how to reproduce the build steps, and everybody just copies the one Python 2.7.{1,2} binary that someone built out of sheer willpower. On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee > wrote: Hi Bill, On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen > wrote: Russell Keith-Magee > wrote: > b) I don't like the Kivy build tools. They're a lot more complex than they > need to be. I didn't find it troublesome, but of course this wasn't my first rodeo. I'd certainly agree it's not a push-button solution. So, what would a less complex system be like? A less complex system is what Toga does. 1. You use cookiecutter to generate a stub project. This gives you the full source tree for a project you can load into XCode (iOS), or build with ant (Android), including a "hello world" __main__.py 2. You download the pre-compiled Python.framework for iOS, or libPython for Android, and copy it into a libs directory 3. You start writing Python code, replacing the __main__.py with your own logic. 4. You compile and deploy your project using XCode/ant. Compare this with Kivy - My experience was spending a couple of days getting the Kivy build process to actually work - trying to find versions of the Android NDK that aren't being distributed any more (but are the only hard coded options in the build system), working out that the provided code doesn't work with the most recent versions of Cython, and sourcing libraries for all sorts of dependencies, so that I could compile SDL and a bunch of OpenGL stuff - none of which I needed. It took me a couple of days to get to "hello world" - and all because of something that could have been shipped as a pre-compiled binary. > I'm going to guess the Kivy people are all Linux users, because > they don't appear to have worked out that binary compatibility is a thing. Sorry -- why is that a Linux thing? If I want to distribute an app for OS/X or Windows, I give you an executable, and It Just Works (tm). Source *might* be provided as an option in the interest of being open source, but it's not how you distribute anything in practice. The "Linux way" for distribution is to distribute source, and tell you to compile it yourself. Distributing binaries is an afterthought, because ABI compatibility makes building and distributing binaries painful. Most projects don't have the infrastructure to distribute binaries for multiple platforms, so unless you can get the OS to provide a recent binary for you, you compile from source. I see reflections of that bias here. Even though ABI compatibility exists for both iOS and Android as platforms, Kivy chooses to distribute as source. > You don't need to have every user compile Python and the rest of the Kivy > stack - you can just ship a binary library, and it will work on every phone > with the same hardware (I know, because Toga does this. The Toga > bootstrapping process is "clone this repo, and copy this file". You could > reduce this to "clone this repo" if you were happy putting binary artefacts > into version control.) Sure. > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on Android; I believe the mobile platform packaging tools are still stuck at 2.7. Kivy 1.8+ will run on Python 3 on desktop, though. Yes - but in practice, the absence of Python 3 on Mobile means that Kivy on Mobile is Python 2 only, and an old version at that. > and if you build them on OSX, when you're on the device, they report > sys.platform as "darwin". Seems like a bug; I imagine you're suggesting that the Kivy build process should patch that file to return "android"? Although I never know what to look at to get that platform info correctly -- this is a larger Python issue. Yes, it's a bug (or at least a missing feature). The build system patches that Kivy use doesn't introduce anything for targeted builds (i.e., using an x86 platform to compile ARM64 binaries - which is what you're doing when you compile for mobile), and doesn't provide a platform definition for mobile. And kivy.utils.platform seems to return the proper thing. So... instead of they've introduced their own way to get access to information that Python already has a standard way of providing. > Going back to my post For those of you following along at home, here's Jeff's list (I had to go and look it up). Jeff? > 1. A library build of Python > 2. Templates to stub out a working Python project > 3. Libraries to do bridge between native language environments and Python > (for me, that's Rubicon) > 4. Libraries for utilising native system services (for me, that's Toga) > - I agree with Kivy on layer 1, and I was able to use > their build tools to bootstrap my own. However, I have very different > opinions on layers 2-4. Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it provides examples, and you're supposed to extrapolate from them. I guess that's a form of template. For #3, there's Kivy's "pyjnius" (to access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to access Objective-C via runtime reflection) (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" (https://github.com/kivy/plyer). I'd also include most of Kivy in #4 as well, because that's how they're tackling the widget issue. We can have a separate discussion sometime about Django vs. Tornado :-). So - a monkey knife-fight at dawn, then :-) Yours, Russ Magee %-) _______________________________________________ Mobile-sig mailing list Mobile-sig at python.org https://mail.python.org/mailman/listinfo/mobile-sig -- --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at boddie.org.uk Thu Jan 29 18:54:26 2015 From: david at boddie.org.uk (david_boddie) Date: Thu, 29 Jan 2015 18:54:26 +0100 Subject: [Mobile-sig] Patches for cross-compiling Message-ID: <2f08b42a2a236a266917812604842a23@webmail.webfaction.com> It might be worth looking in the Python tracker for patches to help with cross-compiling. See the results of this search, for example: http://bugs.python.org/issue?%40columns=id%2Cactivity%2Ctitle%2Ccreator%2Cassignee%2Cstatus%2Ctype&%40sort=-activity&%40filter=status&%40action=searchid&ignore=file%3Acontent&%40search_text=cross-compile&submit=search&status=-1%2C1%2C2%2C3 Different search terms also produce a few more potentially useful hits. David From wes.turner at gmail.com Thu Jan 29 19:17:03 2015 From: wes.turner at gmail.com (Wes Turner) Date: Thu, 29 Jan 2015 12:17:03 -0600 Subject: [Mobile-sig] python on android In-Reply-To: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> Message-ID: Could there be buildbots for (each version of) each mobile platform? https://www.python.org/dev/buildbot/ https://wiki.python.org/moin/BuildBot On Jan 29, 2015 11:54 AM, "Frank, Matthew I" wrote: > I believe that I?ve seen this particular problem when building Python 3 > for Android, but that it comes and goes. The current open patch for Python > 3.5 that you need to apply is: http://bugs.python.org/issue22625. > > > > For Python 3 the problem is in Makefile.pre.in around line 750: > > > > $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN) > > Should be > > $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS) > > > > > > For cross-building Python 3.4.2 from a Linux host I used the following > configure command for the target build: > > > > CPPFLAGS=-I/abs/path/to/source/Python-3.4.2/FIXLOCALE > ../Python-3.4.2/configure --enable-shared > --prefix=/absolute/path/to/android/install/of/python > --build=x86_64-linux-gnu --host=i686-linux-android --disable-ipv6 > ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no > ac_cv_little_endian_double=yes --without-ensurepip > > > > The ?without-ensurepip, in particular was necessary because ensurepip > needs to call the just-built Python interpreter (which won?t work when > cross compiling). > > > > There are some other things that need patching for Android (around the mbs > to wcs functions and nl_langinfo() and such), but that?s the only problem > with the Makefile that I?m aware of. > > > > -Matt > > > > *From:* Guido van Rossum [mailto:guido at python.org] > *Sent:* Thursday, January 29, 2015 9:54 AM > *To:* Russell Keith-Magee > *Cc:* mobile-sig > *Subject:* Re: [Mobile-sig] python on android > > > > My only suggestion: try to understand *exactly* what the Makefile is doing > at that point. > > If it's really just invoking the wrong Python binary to do the > byte-compilation, you could also skip that until you're further along. The > byte-compilation is optional (it slows startup down a bit, but you > currently don't even start up, so who cares. :-) > > > > On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > > > Ok - so, yes, libPython is just copying and byte compiling. The catch is > which version of Python and Python library you use to do the compiling. > > > > To compile cross platform, you need to do 2 builds - a "host" build (x86 > for most desktop purposes), and then a "target" build (the architecture of > your phone). > > > > (As an aside - in the case of iOS, you actually need to do 2 target builds > - one for the simulator (which is x86, but with a different libc to OSX), > and one for the device (which is an arm7/arm7s/arm64 triple target build) - > but that's beside the point. The problems I'm having exist without this > added complication). > > > > So you do the host build, then you do the target build. The target build > needs to call Python to invoke the compilation of modules etc; so you need > to invoke the host python, but use the setup.py that was configured for the > target Python. > > > > This much I have working (at least, I think I do - I can't test yet > because I don't have a working build, but all signs are positive). > > > > But then, you get to libinstall, and you need to invoke Python to byte > compile the pyc files. To do this, you need to invoke the host python, > using the host Python's library, but over the *target* Python's sources. > It's this last step that is tripping me up at the moment - I haven't worked > out how to drive Autoconf to drive configure to pass in the set of > arguments to invoke Python so that it will use the right binary and library > with the right library tree. I keep end up running the iOS binary (which > doesn't start), or the x86 binary with the iOS library tree (which is > missing some parts that Python needs). > > > > The problem manifests as an "ImportError: No module named _struct", > because the compiled parts of the struct module aren't in the iOS tree (or, > at least, aren't compiled for x86 in the iOS tree) > > > > I appreciate that this isn't the easiest problem to debug over a mailing > list. It's not even strictly "debugging" - it's a matter of working out > what combination of arguments I need to pass in, and working backwards to > the automake script. I've already made a bunch of changes to get this far, > and I'm guessing any suggestions would have to be informed by what I've > already done, which isn't a trivial thing to communicate without dumping a > huge patch on your lap and saying "hey, fix this for me". > > > > This is why, to date, I haven't sought out help - I've just been beavering > through the problems one at a time :-) If you've got any suggestions on > more productive ways to tackle this, I'm all ears. And, again, I'm happy to > share what I've got to date if anyone is interested in helping out. > > > > Yours, > > Russ Magee %-) > > > > On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum > wrote: > > Well, libinstall is nearly 100 lines, not counting dependencies. OTOH > it's just copying a lot of files and then byte-compiling them into .pyc > files -- and .pyc files are portable. So perhaps you can go into a little > more detail? > > > > On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > Hi Guido, > > > > I'll glady stop arguing about Kivy vs Toga. The only reason I brought it > up at all is because I keep hearing arguments that seem to dispute that > getting a libPython build working *is* the first step. You've now put that > argument to bed, so I agree - lets move on. > > > > For what it's worth, I've got a reasonable handle on how to compile > libPython for mobile at this point - what I don't have is a good handle on > is the intricacies of Python's build system, and in particular, how to > drive Autoconf to support cross-platform builds. > > > > I've almost worked out the patches to the Python 2.7.1 source tree to > generate an iOS-compatible libPython. Once I've got that working, I'm > planning to merge those changes up to the tip of 2.7 and 3.X, and submit > those patches for inclusion in the source tree. However, at the moment, I'm > hitting problems with cross-platform execution in the libinstall target; > I'm happy to share what I have so far with anyone interested in > collaborating. > > > > Yours, > > Russ Magee %-) > > > > On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum > wrote: > > Can we stop arguing about Kivy vs Toga and focus on the one thing that > they have in common, the need for a working Python 3 port on Android and > iOS (for a start)? This is apparently mostly a matter of solving a lot of > small things with the build system, dependencies, improved config files, > and getting stuff integrated upstream so it can be built out of the box, > right? After that the layers 2-4 stuff can compete, but everybody wins when > layer 1 is dealt with (even imperfectly). It's pretty sad that nobody > apparently knows how to reproduce the build steps, and everybody just > copies the one Python 2.7.{1,2} binary that someone built out of sheer > willpower. > > > > On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > > > Hi Bill, > > > > On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: > > Russell Keith-Magee wrote: > > > b) I don't like the Kivy build tools. They're a lot more complex than > they > > need to be. > > I didn't find it troublesome, but of course this wasn't my first rodeo. > I'd certainly agree it's not a push-button solution. So, what would a > less complex system be like? > > > > A less complex system is what Toga does. > > > > 1. You use cookiecutter to generate a stub project. This gives you the > full source tree for a project you can load into XCode (iOS), or build with > ant (Android), including a "hello world" __main__.py > > > > 2. You download the pre-compiled Python.framework for iOS, or libPython > for Android, and copy it into a libs directory > > > > 3. You start writing Python code, replacing the __main__.py with your own > logic. > > > > 4. You compile and deploy your project using XCode/ant. > > > > Compare this with Kivy - My experience was spending a couple of days > getting the Kivy build process to actually work - trying to find versions > of the Android NDK that aren't being distributed any more (but are the only > hard coded options in the build system), working out that the provided code > doesn't work with the most recent versions of Cython, and sourcing > libraries for all sorts of dependencies, so that I could compile SDL and a > bunch of OpenGL stuff - none of which I needed. It took me a couple of days > to get to "hello world" - and all because of something that could have been > shipped as a pre-compiled binary. > > > > > I'm going to guess the Kivy people are all Linux users, because > > they don't appear to have worked out that binary compatibility is a > thing. > > Sorry -- why is that a Linux thing? > > > > If I want to distribute an app for OS/X or Windows, I give you an > executable, and It Just Works (tm). Source *might* be provided as an option > in the interest of being open source, but it's not how you distribute > anything in practice. The "Linux way" for distribution is to distribute > source, and tell you to compile it yourself. Distributing binaries is an > afterthought, because ABI compatibility makes building and distributing > binaries painful. Most projects don't have the infrastructure to distribute > binaries for multiple platforms, so unless you can get the OS to provide a > recent binary for you, you compile from source. > > > > I see reflections of that bias here. Even though ABI compatibility exists > for both iOS and Android as platforms, Kivy chooses to distribute as source. > > > > > You don't need to have every user compile Python and the rest of the Kivy > > stack - you can just ship a binary library, and it will work on every > phone > > with the same hardware (I know, because Toga does this. The Toga > > bootstrapping process is "clone this repo, and copy this file". You could > > reduce this to "clone this repo" if you were happy putting binary > artefacts > > into version control.) > > Sure. > > > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on > Android; > > I believe the mobile platform packaging tools are still stuck at 2.7. > Kivy 1.8+ will run on Python 3 on desktop, though. > > > > Yes - but in practice, the absence of Python 3 on Mobile means that Kivy > on Mobile is Python 2 only, and an old version at that. > > > > > and if you build them on OSX, when you're on the device, they report > > sys.platform as "darwin". > > Seems like a bug; I imagine you're suggesting that the Kivy build > process should patch that file to return "android"? Although I never > know what to look at to get that platform info correctly -- this is a > larger Python issue. > > > > Yes, it's a bug (or at least a missing feature). The build system patches > that Kivy use doesn't introduce anything for targeted builds (i.e., using > an x86 platform to compile ARM64 binaries - which is what you're doing when > you compile for mobile), and doesn't provide a platform definition for > mobile. > > > > And kivy.utils.platform seems to return the proper thing. > > > > So... instead of they've introduced their own way to get access to > information that Python already has a standard way of providing. > > > > > Going back to my post > > For those of you following along at home, here's Jeff's list (I had to > go and look it up). > > > > Jeff? > > > > > 1. A library build of Python > > 2. Templates to stub out a working Python project > > 3. Libraries to do bridge between native language environments and > Python > > (for me, that's Rubicon) > > 4. Libraries for utilising native system services (for me, that's Toga) > > > - I agree with Kivy on layer 1, and I was able to use > > their build tools to bootstrap my own. However, I have very different > > opinions on layers 2-4. > > Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it > provides examples, and you're supposed to extrapolate from them. I > guess that's a form of template. For #3, there's Kivy's "pyjnius" (to > access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to > access Objective-C via runtime reflection) > (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" > (https://github.com/kivy/plyer). > > > > I'd also include most of Kivy in #4 as well, because that's how they're > tackling the widget issue. > > > > We can have a separate discussion sometime about Django vs. Tornado :-). > > > > So - a monkey knife-fight at dawn, then :-) > > > > Yours, > > Russ Magee %-) > > > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell at keith-magee.com Fri Jan 30 01:02:31 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Fri, 30 Jan 2015 08:02:31 +0800 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> Message-ID: Hi Wes, A buildbot is possible, but complicated. An Android buildbot could be hosted on any "unix like" system - you can get the Android development kits for Linux and OS/X. However, that wouldn't allow you to run the test suite for the mobile platform - because the binary you build would need to run on the mobile platform itself. Android does provide an emulator that would allow "on server" testing, but I don't know how well it can be driven by remote control. My experience using it as a developer has been "don't", because it's so painfully slow to start and control. On iOS, there are other limitations. The host compiler *must* be iOS, because XCode isn't available for any other platform (AFAIK). Assuming that can be accommodated, iOS provides a *simulator*, not an emulator - and the simulator uses x86, not ARM. So, we'd need to run the test suite on the x86 simulator *and* a native device to be truly sure it works. So - I don't know how this would fit into the existing Python build farm infrastructure, but we'd need to add a minimum of 1 device (an iOS device) to the build farm - ideally, 2 (an Android and an iOS device) - and possibly a couple more if we want to cover testing on different OS versions. This is, of course, is somewhat premature - we need to get the build working first :-) Russ %-) On Fri, Jan 30, 2015 at 2:17 AM, Wes Turner wrote: > Could there be buildbots for (each version of) each mobile platform? > > https://www.python.org/dev/buildbot/ > > https://wiki.python.org/moin/BuildBot > On Jan 29, 2015 11:54 AM, "Frank, Matthew I" > wrote: > >> I believe that I?ve seen this particular problem when building Python 3 >> for Android, but that it comes and goes. The current open patch for Python >> 3.5 that you need to apply is: http://bugs.python.org/issue22625. >> >> >> >> For Python 3 the problem is in Makefile.pre.in around line 750: >> >> >> >> $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN) >> >> Should be >> >> $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS) >> >> >> >> >> >> For cross-building Python 3.4.2 from a Linux host I used the following >> configure command for the target build: >> >> >> >> CPPFLAGS=-I/abs/path/to/source/Python-3.4.2/FIXLOCALE >> ../Python-3.4.2/configure --enable-shared >> --prefix=/absolute/path/to/android/install/of/python >> --build=x86_64-linux-gnu --host=i686-linux-android --disable-ipv6 >> ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no >> ac_cv_little_endian_double=yes --without-ensurepip >> >> >> >> The ?without-ensurepip, in particular was necessary because ensurepip >> needs to call the just-built Python interpreter (which won?t work when >> cross compiling). >> >> >> >> There are some other things that need patching for Android (around the >> mbs to wcs functions and nl_langinfo() and such), but that?s the only >> problem with the Makefile that I?m aware of. >> >> >> >> -Matt >> >> >> >> *From:* Guido van Rossum [mailto:guido at python.org] >> *Sent:* Thursday, January 29, 2015 9:54 AM >> *To:* Russell Keith-Magee >> *Cc:* mobile-sig >> *Subject:* Re: [Mobile-sig] python on android >> >> >> >> My only suggestion: try to understand *exactly* what the Makefile is >> doing at that point. >> >> If it's really just invoking the wrong Python binary to do the >> byte-compilation, you could also skip that until you're further along. The >> byte-compilation is optional (it slows startup down a bit, but you >> currently don't even start up, so who cares. :-) >> >> >> >> On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee < >> russell at keith-magee.com> wrote: >> >> >> >> Ok - so, yes, libPython is just copying and byte compiling. The catch is >> which version of Python and Python library you use to do the compiling. >> >> >> >> To compile cross platform, you need to do 2 builds - a "host" build (x86 >> for most desktop purposes), and then a "target" build (the architecture of >> your phone). >> >> >> >> (As an aside - in the case of iOS, you actually need to do 2 target >> builds - one for the simulator (which is x86, but with a different libc to >> OSX), and one for the device (which is an arm7/arm7s/arm64 triple target >> build) - but that's beside the point. The problems I'm having exist without >> this added complication). >> >> >> >> So you do the host build, then you do the target build. The target build >> needs to call Python to invoke the compilation of modules etc; so you need >> to invoke the host python, but use the setup.py that was configured for the >> target Python. >> >> >> >> This much I have working (at least, I think I do - I can't test yet >> because I don't have a working build, but all signs are positive). >> >> >> >> But then, you get to libinstall, and you need to invoke Python to byte >> compile the pyc files. To do this, you need to invoke the host python, >> using the host Python's library, but over the *target* Python's sources. >> It's this last step that is tripping me up at the moment - I haven't worked >> out how to drive Autoconf to drive configure to pass in the set of >> arguments to invoke Python so that it will use the right binary and library >> with the right library tree. I keep end up running the iOS binary (which >> doesn't start), or the x86 binary with the iOS library tree (which is >> missing some parts that Python needs). >> >> >> >> The problem manifests as an "ImportError: No module named _struct", >> because the compiled parts of the struct module aren't in the iOS tree (or, >> at least, aren't compiled for x86 in the iOS tree) >> >> >> >> I appreciate that this isn't the easiest problem to debug over a mailing >> list. It's not even strictly "debugging" - it's a matter of working out >> what combination of arguments I need to pass in, and working backwards to >> the automake script. I've already made a bunch of changes to get this far, >> and I'm guessing any suggestions would have to be informed by what I've >> already done, which isn't a trivial thing to communicate without dumping a >> huge patch on your lap and saying "hey, fix this for me". >> >> >> >> This is why, to date, I haven't sought out help - I've just been >> beavering through the problems one at a time :-) If you've got any >> suggestions on more productive ways to tackle this, I'm all ears. And, >> again, I'm happy to share what I've got to date if anyone is interested in >> helping out. >> >> >> >> Yours, >> >> Russ Magee %-) >> >> >> >> On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum >> wrote: >> >> Well, libinstall is nearly 100 lines, not counting dependencies. OTOH >> it's just copying a lot of files and then byte-compiling them into .pyc >> files -- and .pyc files are portable. So perhaps you can go into a little >> more detail? >> >> >> >> On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < >> russell at keith-magee.com> wrote: >> >> Hi Guido, >> >> >> >> I'll glady stop arguing about Kivy vs Toga. The only reason I brought it >> up at all is because I keep hearing arguments that seem to dispute that >> getting a libPython build working *is* the first step. You've now put that >> argument to bed, so I agree - lets move on. >> >> >> >> For what it's worth, I've got a reasonable handle on how to compile >> libPython for mobile at this point - what I don't have is a good handle on >> is the intricacies of Python's build system, and in particular, how to >> drive Autoconf to support cross-platform builds. >> >> >> >> I've almost worked out the patches to the Python 2.7.1 source tree to >> generate an iOS-compatible libPython. Once I've got that working, I'm >> planning to merge those changes up to the tip of 2.7 and 3.X, and submit >> those patches for inclusion in the source tree. However, at the moment, I'm >> hitting problems with cross-platform execution in the libinstall target; >> I'm happy to share what I have so far with anyone interested in >> collaborating. >> >> >> >> Yours, >> >> Russ Magee %-) >> >> >> >> On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum >> wrote: >> >> Can we stop arguing about Kivy vs Toga and focus on the one thing that >> they have in common, the need for a working Python 3 port on Android and >> iOS (for a start)? This is apparently mostly a matter of solving a lot of >> small things with the build system, dependencies, improved config files, >> and getting stuff integrated upstream so it can be built out of the box, >> right? After that the layers 2-4 stuff can compete, but everybody wins when >> layer 1 is dealt with (even imperfectly). It's pretty sad that nobody >> apparently knows how to reproduce the build steps, and everybody just >> copies the one Python 2.7.{1,2} binary that someone built out of sheer >> willpower. >> >> >> >> On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < >> russell at keith-magee.com> wrote: >> >> >> >> Hi Bill, >> >> >> >> On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: >> >> Russell Keith-Magee wrote: >> >> > b) I don't like the Kivy build tools. They're a lot more complex than >> they >> > need to be. >> >> I didn't find it troublesome, but of course this wasn't my first rodeo. >> I'd certainly agree it's not a push-button solution. So, what would a >> less complex system be like? >> >> >> >> A less complex system is what Toga does. >> >> >> >> 1. You use cookiecutter to generate a stub project. This gives you the >> full source tree for a project you can load into XCode (iOS), or build with >> ant (Android), including a "hello world" __main__.py >> >> >> >> 2. You download the pre-compiled Python.framework for iOS, or libPython >> for Android, and copy it into a libs directory >> >> >> >> 3. You start writing Python code, replacing the __main__.py with your >> own logic. >> >> >> >> 4. You compile and deploy your project using XCode/ant. >> >> >> >> Compare this with Kivy - My experience was spending a couple of days >> getting the Kivy build process to actually work - trying to find versions >> of the Android NDK that aren't being distributed any more (but are the only >> hard coded options in the build system), working out that the provided code >> doesn't work with the most recent versions of Cython, and sourcing >> libraries for all sorts of dependencies, so that I could compile SDL and a >> bunch of OpenGL stuff - none of which I needed. It took me a couple of days >> to get to "hello world" - and all because of something that could have been >> shipped as a pre-compiled binary. >> >> >> >> > I'm going to guess the Kivy people are all Linux users, because >> > they don't appear to have worked out that binary compatibility is a >> thing. >> >> Sorry -- why is that a Linux thing? >> >> >> >> If I want to distribute an app for OS/X or Windows, I give you an >> executable, and It Just Works (tm). Source *might* be provided as an option >> in the interest of being open source, but it's not how you distribute >> anything in practice. The "Linux way" for distribution is to distribute >> source, and tell you to compile it yourself. Distributing binaries is an >> afterthought, because ABI compatibility makes building and distributing >> binaries painful. Most projects don't have the infrastructure to distribute >> binaries for multiple platforms, so unless you can get the OS to provide a >> recent binary for you, you compile from source. >> >> >> >> I see reflections of that bias here. Even though ABI compatibility exists >> for both iOS and Android as platforms, Kivy chooses to distribute as source. >> >> >> >> > You don't need to have every user compile Python and the rest of the >> Kivy >> > stack - you can just ship a binary library, and it will work on every >> phone >> > with the same hardware (I know, because Toga does this. The Toga >> > bootstrapping process is "clone this repo, and copy this file". You >> could >> > reduce this to "clone this repo" if you were happy putting binary >> artefacts >> > into version control.) >> >> Sure. >> >> > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on >> Android; >> >> I believe the mobile platform packaging tools are still stuck at 2.7. >> Kivy 1.8+ will run on Python 3 on desktop, though. >> >> >> >> Yes - but in practice, the absence of Python 3 on Mobile means that Kivy >> on Mobile is Python 2 only, and an old version at that. >> >> >> >> > and if you build them on OSX, when you're on the device, they report >> > sys.platform as "darwin". >> >> Seems like a bug; I imagine you're suggesting that the Kivy build >> process should patch that file to return "android"? Although I never >> know what to look at to get that platform info correctly -- this is a >> larger Python issue. >> >> >> >> Yes, it's a bug (or at least a missing feature). The build system patches >> that Kivy use doesn't introduce anything for targeted builds (i.e., using >> an x86 platform to compile ARM64 binaries - which is what you're doing when >> you compile for mobile), and doesn't provide a platform definition for >> mobile. >> >> >> >> And kivy.utils.platform seems to return the proper thing. >> >> >> >> So... instead of they've introduced their own way to get access to >> information that Python already has a standard way of providing. >> >> >> >> > Going back to my post >> >> For those of you following along at home, here's Jeff's list (I had to >> go and look it up). >> >> >> >> Jeff? >> >> >> >> > 1. A library build of Python >> > 2. Templates to stub out a working Python project >> > 3. Libraries to do bridge between native language environments and >> Python >> > (for me, that's Rubicon) >> > 4. Libraries for utilising native system services (for me, that's Toga) >> >> > - I agree with Kivy on layer 1, and I was able to use >> > their build tools to bootstrap my own. However, I have very different >> > opinions on layers 2-4. >> >> Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it >> provides examples, and you're supposed to extrapolate from them. I >> guess that's a form of template. For #3, there's Kivy's "pyjnius" (to >> access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to >> access Objective-C via runtime reflection) >> (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" >> (https://github.com/kivy/plyer). >> >> >> >> I'd also include most of Kivy in #4 as well, because that's how they're >> tackling the widget issue. >> >> >> >> We can have a separate discussion sometime about Django vs. Tornado :-). >> >> >> >> So - a monkey knife-fight at dawn, then :-) >> >> >> >> Yours, >> >> Russ Magee %-) >> >> >> >> _______________________________________________ >> Mobile-sig mailing list >> Mobile-sig at python.org >> https://mail.python.org/mailman/listinfo/mobile-sig >> >> >> >> >> -- >> >> --Guido van Rossum (python.org/~guido) >> >> >> >> >> >> >> -- >> >> --Guido van Rossum (python.org/~guido) >> >> >> >> >> >> >> -- >> >> --Guido van Rossum (python.org/~guido) >> >> _______________________________________________ >> Mobile-sig mailing list >> Mobile-sig at python.org >> https://mail.python.org/mailman/listinfo/mobile-sig >> >> -------------- next part -------------- An HTML attachment was scrubbed... URL: From russell at keith-magee.com Fri Jan 30 01:04:10 2015 From: russell at keith-magee.com (Russell Keith-Magee) Date: Fri, 30 Jan 2015 08:04:10 +0800 Subject: [Mobile-sig] python on android In-Reply-To: <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> Message-ID: Hi Frank, Thanks for the tips. I'll try these out and see how I go. Yours, Russ Magee %-) On Fri, Jan 30, 2015 at 1:53 AM, Frank, Matthew I wrote: > I believe that I?ve seen this particular problem when building Python 3 > for Android, but that it comes and goes. The current open patch for Python > 3.5 that you need to apply is: http://bugs.python.org/issue22625. > > > > For Python 3 the problem is in Makefile.pre.in around line 750: > > > > $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGEN) > > Should be > > $(GRAMMAR_H): $(GRAMMAR_INPUT) $(PGENSRCS) > > > > > > For cross-building Python 3.4.2 from a Linux host I used the following > configure command for the target build: > > > > CPPFLAGS=-I/abs/path/to/source/Python-3.4.2/FIXLOCALE > ../Python-3.4.2/configure --enable-shared > --prefix=/absolute/path/to/android/install/of/python > --build=x86_64-linux-gnu --host=i686-linux-android --disable-ipv6 > ac_cv_file__dev_ptmx=no ac_cv_file__dev_ptc=no > ac_cv_little_endian_double=yes --without-ensurepip > > > > The ?without-ensurepip, in particular was necessary because ensurepip > needs to call the just-built Python interpreter (which won?t work when > cross compiling). > > > > There are some other things that need patching for Android (around the mbs > to wcs functions and nl_langinfo() and such), but that?s the only problem > with the Makefile that I?m aware of. > > > > -Matt > > > > *From:* Guido van Rossum [mailto:guido at python.org] > *Sent:* Thursday, January 29, 2015 9:54 AM > *To:* Russell Keith-Magee > *Cc:* mobile-sig > *Subject:* Re: [Mobile-sig] python on android > > > > My only suggestion: try to understand *exactly* what the Makefile is doing > at that point. > > If it's really just invoking the wrong Python binary to do the > byte-compilation, you could also skip that until you're further along. The > byte-compilation is optional (it slows startup down a bit, but you > currently don't even start up, so who cares. :-) > > > > On Wed, Jan 28, 2015 at 10:58 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > > > Ok - so, yes, libPython is just copying and byte compiling. The catch is > which version of Python and Python library you use to do the compiling. > > > > To compile cross platform, you need to do 2 builds - a "host" build (x86 > for most desktop purposes), and then a "target" build (the architecture of > your phone). > > > > (As an aside - in the case of iOS, you actually need to do 2 target builds > - one for the simulator (which is x86, but with a different libc to OSX), > and one for the device (which is an arm7/arm7s/arm64 triple target build) - > but that's beside the point. The problems I'm having exist without this > added complication). > > > > So you do the host build, then you do the target build. The target build > needs to call Python to invoke the compilation of modules etc; so you need > to invoke the host python, but use the setup.py that was configured for the > target Python. > > > > This much I have working (at least, I think I do - I can't test yet > because I don't have a working build, but all signs are positive). > > > > But then, you get to libinstall, and you need to invoke Python to byte > compile the pyc files. To do this, you need to invoke the host python, > using the host Python's library, but over the *target* Python's sources. > It's this last step that is tripping me up at the moment - I haven't worked > out how to drive Autoconf to drive configure to pass in the set of > arguments to invoke Python so that it will use the right binary and library > with the right library tree. I keep end up running the iOS binary (which > doesn't start), or the x86 binary with the iOS library tree (which is > missing some parts that Python needs). > > > > The problem manifests as an "ImportError: No module named _struct", > because the compiled parts of the struct module aren't in the iOS tree (or, > at least, aren't compiled for x86 in the iOS tree) > > > > I appreciate that this isn't the easiest problem to debug over a mailing > list. It's not even strictly "debugging" - it's a matter of working out > what combination of arguments I need to pass in, and working backwards to > the automake script. I've already made a bunch of changes to get this far, > and I'm guessing any suggestions would have to be informed by what I've > already done, which isn't a trivial thing to communicate without dumping a > huge patch on your lap and saying "hey, fix this for me". > > > > This is why, to date, I haven't sought out help - I've just been beavering > through the problems one at a time :-) If you've got any suggestions on > more productive ways to tackle this, I'm all ears. And, again, I'm happy to > share what I've got to date if anyone is interested in helping out. > > > > Yours, > > Russ Magee %-) > > > > On Thu, Jan 29, 2015 at 2:17 PM, Guido van Rossum > wrote: > > Well, libinstall is nearly 100 lines, not counting dependencies. OTOH > it's just copying a lot of files and then byte-compiling them into .pyc > files -- and .pyc files are portable. So perhaps you can go into a little > more detail? > > > > On Wed, Jan 28, 2015 at 9:10 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > Hi Guido, > > > > I'll glady stop arguing about Kivy vs Toga. The only reason I brought it > up at all is because I keep hearing arguments that seem to dispute that > getting a libPython build working *is* the first step. You've now put that > argument to bed, so I agree - lets move on. > > > > For what it's worth, I've got a reasonable handle on how to compile > libPython for mobile at this point - what I don't have is a good handle on > is the intricacies of Python's build system, and in particular, how to > drive Autoconf to support cross-platform builds. > > > > I've almost worked out the patches to the Python 2.7.1 source tree to > generate an iOS-compatible libPython. Once I've got that working, I'm > planning to merge those changes up to the tip of 2.7 and 3.X, and submit > those patches for inclusion in the source tree. However, at the moment, I'm > hitting problems with cross-platform execution in the libinstall target; > I'm happy to share what I have so far with anyone interested in > collaborating. > > > > Yours, > > Russ Magee %-) > > > > On Thu, Jan 29, 2015 at 12:31 PM, Guido van Rossum > wrote: > > Can we stop arguing about Kivy vs Toga and focus on the one thing that > they have in common, the need for a working Python 3 port on Android and > iOS (for a start)? This is apparently mostly a matter of solving a lot of > small things with the build system, dependencies, improved config files, > and getting stuff integrated upstream so it can be built out of the box, > right? After that the layers 2-4 stuff can compete, but everybody wins when > layer 1 is dealt with (even imperfectly). It's pretty sad that nobody > apparently knows how to reproduce the build steps, and everybody just > copies the one Python 2.7.{1,2} binary that someone built out of sheer > willpower. > > > > On Wed, Jan 28, 2015 at 8:04 PM, Russell Keith-Magee < > russell at keith-magee.com> wrote: > > > > Hi Bill, > > > > On Thu, Jan 29, 2015 at 10:41 AM, Bill Janssen wrote: > > Russell Keith-Magee wrote: > > > b) I don't like the Kivy build tools. They're a lot more complex than > they > > need to be. > > I didn't find it troublesome, but of course this wasn't my first rodeo. > I'd certainly agree it's not a push-button solution. So, what would a > less complex system be like? > > > > A less complex system is what Toga does. > > > > 1. You use cookiecutter to generate a stub project. This gives you the > full source tree for a project you can load into XCode (iOS), or build with > ant (Android), including a "hello world" __main__.py > > > > 2. You download the pre-compiled Python.framework for iOS, or libPython > for Android, and copy it into a libs directory > > > > 3. You start writing Python code, replacing the __main__.py with your own > logic. > > > > 4. You compile and deploy your project using XCode/ant. > > > > Compare this with Kivy - My experience was spending a couple of days > getting the Kivy build process to actually work - trying to find versions > of the Android NDK that aren't being distributed any more (but are the only > hard coded options in the build system), working out that the provided code > doesn't work with the most recent versions of Cython, and sourcing > libraries for all sorts of dependencies, so that I could compile SDL and a > bunch of OpenGL stuff - none of which I needed. It took me a couple of days > to get to "hello world" - and all because of something that could have been > shipped as a pre-compiled binary. > > > > > I'm going to guess the Kivy people are all Linux users, because > > they don't appear to have worked out that binary compatibility is a > thing. > > Sorry -- why is that a Linux thing? > > > > If I want to distribute an app for OS/X or Windows, I give you an > executable, and It Just Works (tm). Source *might* be provided as an option > in the interest of being open source, but it's not how you distribute > anything in practice. The "Linux way" for distribution is to distribute > source, and tell you to compile it yourself. Distributing binaries is an > afterthought, because ABI compatibility makes building and distributing > binaries painful. Most projects don't have the infrastructure to distribute > binaries for multiple platforms, so unless you can get the OS to provide a > recent binary for you, you compile from source. > > > > I see reflections of that bias here. Even though ABI compatibility exists > for both iOS and Android as platforms, Kivy chooses to distribute as source. > > > > > You don't need to have every user compile Python and the rest of the Kivy > > stack - you can just ship a binary library, and it will work on every > phone > > with the same hardware (I know, because Toga does this. The Toga > > bootstrapping process is "clone this repo, and copy this file". You could > > reduce this to "clone this repo" if you were happy putting binary > artefacts > > into version control.) > > Sure. > > > c) Kivy's build tools are Python 2.7.1 only on iOS, and 2.7.2 on > Android; > > I believe the mobile platform packaging tools are still stuck at 2.7. > Kivy 1.8+ will run on Python 3 on desktop, though. > > > > Yes - but in practice, the absence of Python 3 on Mobile means that Kivy > on Mobile is Python 2 only, and an old version at that. > > > > > and if you build them on OSX, when you're on the device, they report > > sys.platform as "darwin". > > Seems like a bug; I imagine you're suggesting that the Kivy build > process should patch that file to return "android"? Although I never > know what to look at to get that platform info correctly -- this is a > larger Python issue. > > > > Yes, it's a bug (or at least a missing feature). The build system patches > that Kivy use doesn't introduce anything for targeted builds (i.e., using > an x86 platform to compile ARM64 binaries - which is what you're doing when > you compile for mobile), and doesn't provide a platform definition for > mobile. > > > > And kivy.utils.platform seems to return the proper thing. > > > > So... instead of they've introduced their own way to get access to > information that Python already has a standard way of providing. > > > > > Going back to my post > > For those of you following along at home, here's Jeff's list (I had to > go and look it up). > > > > Jeff? > > > > > 1. A library build of Python > > 2. Templates to stub out a working Python project > > 3. Libraries to do bridge between native language environments and > Python > > (for me, that's Rubicon) > > 4. Libraries for utilising native system services (for me, that's Toga) > > > - I agree with Kivy on layer 1, and I was able to use > > their build tools to bootstrap my own. However, I have very different > > opinions on layers 2-4. > > Just to outline the Kivy approach to 2-4: Kivy doesn't really do 2 -- it > provides examples, and you're supposed to extrapolate from them. I > guess that's a form of template. For #3, there's Kivy's "pyjnius" (to > access Java via JNI) (https://github.com/kivy/pyobjus) and "pyobjus" (to > access Objective-C via runtime reflection) > (https://github.com/kivy/pyjnius). For #4, as I said, there's "plyer" > (https://github.com/kivy/plyer). > > > > I'd also include most of Kivy in #4 as well, because that's how they're > tackling the widget issue. > > > > We can have a separate discussion sometime about Django vs. Tornado :-). > > > > So - a monkey knife-fight at dawn, then :-) > > > > Yours, > > Russ Magee %-) > > > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > > > > > > > -- > > --Guido van Rossum (python.org/~guido) > -------------- next part -------------- An HTML attachment was scrubbed... URL: From wes.turner at gmail.com Fri Jan 30 02:18:47 2015 From: wes.turner at gmail.com (Wes Turner) Date: Thu, 29 Jan 2015 19:18:47 -0600 Subject: [Mobile-sig] python on android In-Reply-To: References: <1EC1C30484FE464CAE23D84D5BEA0CCB@JakesPC> <24279.1422468117@parc.com> <7129.1422499291@parc.com> <8FEF00DDD6C58843BF4ED3C1DCF80E9F6E1384F4@FMSMSX106.amr.corp.intel.com> Message-ID: On Jan 29, 2015 6:02 PM, "Russell Keith-Magee" wrote: > > Hi Wes, > > A buildbot is possible, but complicated. > > An Android buildbot could be hosted on any "unix like" system - you can get the Android development kits for Linux and OS/X. However, that wouldn't allow you to run the test suite for the mobile platform - because the binary you build would need to run on the mobile platform itself. Android does provide an emulator that would allow "on server" testing, but I don't know how well it can be driven by remote control. My experience using it as a developer has been "don't", because it's so painfully slow to start and control. > > On iOS, there are other limitations. The host compiler *must* be iOS, because XCode isn't available for any other platform (AFAIK). Assuming that can be accommodated, iOS provides a *simulator*, not an emulator - and the simulator uses x86, not ARM. So, we'd need to run the test suite on the x86 simulator *and* a native device to be truly sure it works. > > So - I don't know how this would fit into the existing Python build farm infrastructure, but we'd need to add a minimum of 1 device (an iOS device) to the build farm - ideally, 2 (an Android and an iOS device) - and possibly a couple more if we want to cover testing on different OS versions. > > This is, of course, is somewhat premature - we need to get the build working first :-) Understood, thanks! So, there appears to be an opportunity to donate some resources to this initiative. -------------- next part -------------- An HTML attachment was scrubbed... URL: