From russell at keith-magee.com Sat Jan 2 10:12:39 2016 From: russell at keith-magee.com (Russell Keith-Magee) Date: Sat, 2 Jan 2016 23:12:39 +0800 Subject: [Mobile-sig] Progress update on Mobile Python work Message-ID: Hi all, This SIG has been a bit quiet of late, but over the last few months (and especially over the Christmas/New Year break) I?ve made some significant progress on support for Python on mobile platforms (especially Android), so I thought I?d give an update of where things (at least, from where I stand). iOS Support =========== iOS support is getting quite strong. There is a patch against CPython 3.4.2 sources: http://bugs.python.org/issue23670 This patch provides an XCode project that runs the Python test suite. There are still a handful of bugs; these are mostly related to edge cases in ctypes. The issues that previously existed with libffi have now been resolved. The biggest question with libffi support now is whether there is an opportunity to merge libffi support on OS X with the new libffi support on iOS. The generated code for libffi_osx is several years old at this point, and it would be easy to replace the current libffi_osx generated code with an updated version sharing sources with iOS. The only catch is that libffi no longer supports PowerPC architectures in their OS X builds (because Apple no longer supports those architectures). There are two impediments currently preventing this patch from being merged into trunk. The first is a bug, reported during the 3.5 alpha, that is still lingering: http://bugs.python.org/issue22625 This bug was introduced in the process of fixing support for parallel builds in the root Makefile. This problem prevents *any* cross platform build from working. I don?t have enough Makefile mojo to sort this one out. The second is the absence of a feedback from someone on the core team. The patch really needs a review from someone to indicate which of the design decisions I?ve made are acceptable to the CPython core team. In the meantime, I?ve been maintaining an external project to make it easy to build a Python framework for use in an iPhone (or iPad, or AppleTV, or Apple Watch) app: https://github.com/pybee/Python-iOS-support You can download a pre-compiled version of the libraries from the project page. Add an Objective C bridging library (like Rubicon), and you can write your iOS (or tvOS) app in Python. Android Support =============== Cyd Haselton has been working on an analogous patch for CPython to add support for Android. From what I can see, he?s having some difficulties; the code is segfaulting, and debugging the source of the segfault is proving difficult. I experimented with CPython on Android briefly at the end of 2014; I came to the conclusion that it wasn?t really viable, because once you have CPython, you need to access Java libraries to actually write an application. That means you have to use JNI, and while JNI *exists* on Android, it is constrained to the point that it makes writing apps nigh on impractical. Instead, I?ve been working on an alternate approach: transpiling CPython 3.4 bytecode directly to Java class files. The result is a project called VOC: https://github.com/pybee/voc This is a different approach to that used by Jython. There?s no Python interpreter at runtime - what you push to the phone is a pure Java class file implementing Python language semantics. The price you pay is that there?s no exec(), eval() or REPL - but if you?re writing an app for distribution on a phone, that probably doesn?t matter. Performance is passable, but not great - PyStone indicates that it?s roughly an order of magnitude slower than CPython on the same machine. However I haven?t been focussing on performance to date, and there?s a lot of low hanging opportunities for optimisation. It?s also worth noting that mobile apps (other than games) generally won?t be CPU bound - they?re network bound - so this performance issue isn?t as critical as it may seem. Here?s a sample project: https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97 When you compile that code with VOC, you get 3 class files (one for each class, and one for the module itself). If you add those class files to an Android project and upload to a phone, you get this: https://twitter.com/PyBeeWare/status/683258762816192513/video/1 There?s still a lot of work to do on VOC; there are some language constructs that it doesn?t handle yet (most notably generators and context managers), and I need to port the parts of the standard library that are written in C. However, as the video shows, even without those parts, you can write simple Android applications in Python. Packaging ========= One last thing - I?ve also created project called Briefcase to tie all these pieces together. https://github.com/pybee/briefcase Briefcase is an extension to distutils that adds setup.py targets to package a project for distribution as an app. iOS and OS/X support is quite stable. A stub command exists for Android, but it isn?t fully operational yet. Apple TV support would be trivial to add - most of the pieces are already there, they just need to be tied together. Summary ======= So - that?s where, IMHO, Python on Mobile is at right now. If you?ve got any questions, comments, or feedback - let me know. Yours, Russ Magee %-) -------------- next part -------------- An HTML attachment was scrubbed... URL: From alexanderjohntaylor at gmail.com Sun Jan 3 08:39:08 2016 From: alexanderjohntaylor at gmail.com (Alexander Taylor) Date: Sun, 3 Jan 2016 13:39:08 +0000 Subject: [Mobile-sig] Progress update on Mobile Python work In-Reply-To: <56880A3B.5050005@gmail.com> References: <56880A3B.5050005@gmail.com> Message-ID: <5689247C.7060904@gmail.com> (I sent this reply originally to just Russell, but meant it for the whole list - the information about Python with CrystaX in particular may be interesting. Sorry for the double reply Russell.) Hi Russell (just Russ?), I'm one of the Kivy developers and have been working mainly on our own python-for-android build tools recently, so I'm very interested in your updates there - thanks for the post. I haven't posted here before (I feel too inexpert!) but have read some of the previous discussion, and certainly agree that it would be great to have python build for Android without difficulty. Regarding Cyd Haselton's patch, is the issue at http://bugs.python.org/issue23496 the right place to follow it, or is there discussion elsewhere? I've recently been trying to add python3 support to Kivy's python-for-android, but had no luck with any patches, though I didn't know about Cyd's effort and mainly used SL4A's old experimental python3 support which (although allowing compilation to complete) I couldn't get running on the device. That problem could easily be my own mistake elsewhere, but I'll give this method a try instead, as it's much preferable anyway. More recently, I was actually able to sidestep the build entirely with the new support for python by the CrystaX NDK, which as of their recent 10.3.0 release includes precombiled python distributions for 2.7 and 3.5 on Android (all supported architectures). For anyone not aware, this is a drop-in replacement for google's own NDK with many features added, like newer versions of gcc/clang and better C++ standard support. This means that the CrystaX NDK can compile upstream Python with no patches at all. I gave it a try and it was extremely easy to get working, so I'll soon be able to (finally...) add python3 support to python-for-android. Using this method might be very practical for other projects as well, if anyone is interested, though of course it's still very valuable to get Python building for Android with the offical NDK. I'm sad to hear that manipulating a native-widget Android app from Python via JNI seems impractical, but good luck with VOC, I would love to see this become doable. Thanks, Alexander On 02/01/16 15:12, Russell Keith-Magee wrote: > Hi all, > > This SIG has been a bit quiet of late, but over the last few months (and > especially over the Christmas/New Year break) I?ve made some significant > progress on support for Python on mobile platforms (especially Android), > so I thought I?d give an update of where things (at least, from where I > stand). > > iOS Support > =========== > > iOS support is getting quite strong. There is a patch against CPython > 3.4.2 sources: > > http://bugs.python.org/issue23670 > > This patch provides an XCode project that runs the Python test suite. > There are still a handful of bugs; these are mostly related to edge > cases in ctypes. > > The issues that previously existed with libffi have now been resolved. > The biggest question with libffi support now is whether there is an > opportunity to merge libffi support on OS X with the new libffi support > on iOS. The generated code for libffi_osx is several years old at this > point, and it would be easy to replace the current libffi_osx generated > code with an updated version sharing sources with iOS. The only catch is > that libffi no longer supports PowerPC architectures in their OS X > builds (because Apple no longer supports those architectures). > > There are two impediments currently preventing this patch from being > merged into trunk. > > The first is a bug, reported during the 3.5 alpha, that is still lingering: > > http://bugs.python.org/issue22625 > > This bug was introduced in the process of fixing support for parallel > builds in the root Makefile. This problem prevents *any* cross platform > build from working. I don?t have enough Makefile mojo to sort this one out. > > The second is the absence of a feedback from someone on the core team. > The patch really needs a review from someone to indicate which of the > design decisions I?ve made are acceptable to the CPython core team. > > In the meantime, I?ve been maintaining an external project to make it > easy to build a Python framework for use in an iPhone (or iPad, or > AppleTV, or Apple Watch) app: > > https://github.com/pybee/Python-iOS-support > > You can download a pre-compiled version of the libraries from the > project page. Add an Objective C bridging library (like Rubicon), and > you can write your iOS (or tvOS) app in Python. > > Android Support > =============== > > Cyd Haselton has been working on an analogous patch for CPython to add > support for Android. From what I can see, he?s having some difficulties; > the code is segfaulting, and debugging the source of the segfault is > proving difficult. > > I experimented with CPython on Android briefly at the end of 2014; I > came to the conclusion that it wasn?t really viable, because once you > have CPython, you need to access Java libraries to actually write an > application. That means you have to use JNI, and while JNI *exists* on > Android, it is constrained to the point that it makes writing apps nigh > on impractical. > > Instead, I?ve been working on an alternate approach: transpiling CPython > 3.4 bytecode directly to Java class files. The result is a project > called VOC: > > https://github.com/pybee/voc > > This is a different approach to that used by Jython. There?s no Python > interpreter at runtime - what > you push to the phone is a pure Java class file implementing Python > language semantics. The price you pay is that there?s no exec(), eval() > or REPL - but if you?re writing an app for distribution on a phone, that > probably doesn?t matter. > > Performance is passable, but not great - PyStone indicates that it?s > roughly an order of magnitude slower than CPython on the same machine. > However I haven?t been focussing on performance to date, and there?s a > lot of low hanging opportunities for optimisation. It?s also worth > noting that mobile apps (other than games) generally won?t be CPU bound > - they?re network bound - so this performance issue isn?t as critical as > it may seem. > > Here?s a sample project: > > https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97 > > When you compile that code with VOC, you get 3 class files (one for each > class, and one for the module itself). If you add those class files to > an Android project and upload to a phone, you get this: > > https://twitter.com/PyBeeWare/status/683258762816192513/video/1 > > There?s still a lot of work to do on VOC; there are some language > constructs that it doesn?t handle yet (most notably generators and > context managers), and I need to port the parts of the standard library > that are written in C. However, as the video shows, even without those > parts, you can write simple Android applications in Python. > > Packaging > ========= > > One last thing - I?ve also created project called Briefcase to tie all > these pieces together. > > https://github.com/pybee/briefcase > > Briefcase is an extension to distutils that adds setup.py targets to > package a project for distribution as an app. iOS and OS/X support is > quite stable. A stub command exists for Android, but it isn?t fully > operational yet. Apple TV support would be trivial to add - most of the > pieces are already there, they just need to be tied together. > > Summary > ======= > > So - that?s where, IMHO, Python on Mobile is at right now. > > If you?ve got any questions, comments, or feedback - let me know. > > Yours, > Russ Magee %-) > > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 819 bytes Desc: OpenPGP digital signature URL: From guido at python.org Sun Jan 3 16:04:37 2016 From: guido at python.org (Guido van Rossum) Date: Sun, 3 Jan 2016 13:04:37 -0800 Subject: [Mobile-sig] Progress update on Mobile Python work In-Reply-To: References: Message-ID: Hi Russ, Thanks for the detailed progress report! I expect that in the next few years mobile support will become critical for Python, and your work (and that of others in this area) is much appreciated. I've got one recommendation: if you feel you're stuck waiting for a core developer to provide feedback, please don't hesitate to bring the issue up on python-dev. That's where we're all lurking. (I expect only a very small number of core devs read mobile-sig.) Happy 2016, --Guido On Sat, Jan 2, 2016 at 7:12 AM, Russell Keith-Magee wrote: > Hi all, > > This SIG has been a bit quiet of late, but over the last few months (and > especially over the Christmas/New Year break) I?ve made some significant > progress on support for Python on mobile platforms (especially Android), so > I thought I?d give an update of where things (at least, from where I stand). > > iOS Support > =========== > > iOS support is getting quite strong. There is a patch against CPython > 3.4.2 sources: > > http://bugs.python.org/issue23670 > > This patch provides an XCode project that runs the Python test suite. > There are still a handful of bugs; these are mostly related to edge cases > in ctypes. > > The issues that previously existed with libffi have now been resolved. The > biggest question with libffi support now is whether there is an opportunity > to merge libffi support on OS X with the new libffi support on iOS. The > generated code for libffi_osx is several years old at this point, and it > would be easy to replace the current libffi_osx generated code with an > updated version sharing sources with iOS. The only catch is that libffi no > longer supports PowerPC architectures in their OS X builds (because Apple > no longer supports those architectures). > > There are two impediments currently preventing this patch from being > merged into trunk. > > The first is a bug, reported during the 3.5 alpha, that is still lingering: > > http://bugs.python.org/issue22625 > > This bug was introduced in the process of fixing support for parallel > builds in the root Makefile. This problem prevents *any* cross platform > build from working. I don?t have enough Makefile mojo to sort this one out. > > The second is the absence of a feedback from someone on the core team. The > patch really needs a review from someone to indicate which of the design > decisions I?ve made are acceptable to the CPython core team. > > In the meantime, I?ve been maintaining an external project to make it easy > to build a Python framework for use in an iPhone (or iPad, or AppleTV, or > Apple Watch) app: > > https://github.com/pybee/Python-iOS-support > > You can download a pre-compiled version of the libraries from the project > page. Add an Objective C bridging library (like Rubicon), and you can write > your iOS (or tvOS) app in Python. > > Android Support > =============== > > Cyd Haselton has been working on an analogous patch for CPython to add > support for Android. From what I can see, he?s having some difficulties; > the code is segfaulting, and debugging the source of the segfault is > proving difficult. > > I experimented with CPython on Android briefly at the end of 2014; I came > to the conclusion that it wasn?t really viable, because once you have > CPython, you need to access Java libraries to actually write an > application. That means you have to use JNI, and while JNI *exists* on > Android, it is constrained to the point that it makes writing apps nigh on > impractical. > > Instead, I?ve been working on an alternate approach: transpiling CPython > 3.4 bytecode directly to Java class files. The result is a project called > VOC: > > https://github.com/pybee/voc > > This is a different approach to that used by Jython. There?s no Python > interpreter at runtime - what > you push to the phone is a pure Java class file implementing Python > language semantics. The price you pay is that there?s no exec(), eval() or > REPL - but if you?re writing an app for distribution on a phone, that > probably doesn?t matter. > > Performance is passable, but not great - PyStone indicates that it?s > roughly an order of magnitude slower than CPython on the same machine. > However I haven?t been focussing on performance to date, and there?s a lot > of low hanging opportunities for optimisation. It?s also worth noting that > mobile apps (other than games) generally won?t be CPU bound - they?re > network bound - so this performance issue isn?t as critical as it may seem. > > Here?s a sample project: > > https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97 > > When you compile that code with VOC, you get 3 class files (one for each > class, and one for the module itself). If you add those class files to an > Android project and upload to a phone, you get this: > > https://twitter.com/PyBeeWare/status/683258762816192513/video/1 > > There?s still a lot of work to do on VOC; there are some language > constructs that it doesn?t handle yet (most notably generators and context > managers), and I need to port the parts of the standard library that are > written in C. However, as the video shows, even without those parts, you > can write simple Android applications in Python. > > Packaging > ========= > > One last thing - I?ve also created project called Briefcase to tie all > these pieces together. > > https://github.com/pybee/briefcase > > Briefcase is an extension to distutils that adds setup.py targets to > package a project for distribution as an app. iOS and OS/X support is quite > stable. A stub command exists for Android, but it isn?t fully operational > yet. Apple TV support would be trivial to add - most of the pieces are > already there, they just need to be tied together. > > Summary > ======= > > So - that?s where, IMHO, Python on Mobile is at right now. > > If you?ve got any questions, comments, or feedback - let me know. > > Yours, > Russ Magee %-) > > _______________________________________________ > Mobile-sig mailing list > Mobile-sig at python.org > https://mail.python.org/mailman/listinfo/mobile-sig > > -- --Guido van Rossum (python.org/~guido) -------------- next part -------------- An HTML attachment was scrubbed... URL: