From xdegaye at gmail.com  Tue Apr 26 03:02:29 2016
From: xdegaye at gmail.com (Xavier de Gaye)
Date: Tue, 26 Apr 2016 09:02:29 +0200
Subject: [Mobile-sig] support of the android platform
Message-ID: <571F1285.6080409@gmail.com>

Starting with API level 21 (Android 5.0), the build of python3 with the
official android toolchains (that is, without resorting to external libraries
for wide character support) runs correctly.  With the set of patches described
in the patches/Makefile file at [1], the cpython test suite runs[2] on the
android x86 and armv7 emulators with only few errors[3].  Those errors are
listed with their corresponding error messages, this may give a raw idea of
the effort needed to support this platform.

This post has been first submitted to python-dev as:
https://mail.python.org/pipermail/python-dev/2016-April/144320.html

Xavier

[1] https://bitbucket.org/xdegaye/pyona/src
[2] To reproduce these results, follow the instructions found in INSTALL
     at https://bitbucket.org/xdegaye/pyona/wiki/install
[3] https://bitbucket.org/xdegaye/pyona/wiki/testsuite

From xdegaye at gmail.com  Tue Apr 26 11:41:51 2016
From: xdegaye at gmail.com (Xavier de Gaye)
Date: Tue, 26 Apr 2016 17:41:51 +0200
Subject: [Mobile-sig] [Python-Dev] support of the android platform
In-Reply-To: <CADiSq7doiu1qVahak8NGmJoKeoABSP_C_rkoi36Xx6zZG14sjw@mail.gmail.com>
References: <571C73A8.1030908@gmail.com>
 <CALFfu7C5+3KkCVrtwyUezLRY6J3VaE0qo4rp1GHRCDre48GRjw@mail.gmail.com>
 <loom.20160425T164909-296@post.gmane.org> <571E7D53.2000809@gmail.com>
 <CADiSq7doiu1qVahak8NGmJoKeoABSP_C_rkoi36Xx6zZG14sjw@mail.gmail.com>
Message-ID: <571F8C3F.7060803@gmail.com>

On 04/26/2016 04:02 AM, Nick Coghlan wrote:
 > On 26 April 2016 at 06:25, Xavier de Gaye <xdegaye at gmail.com <mailto:xdegaye at gmail.com>> wrote:
 >
 >     On 04/25/2016 04:53 PM, Stefan Krah wrote:
 >     > Eric Snow <ericsnowcurrently <at>gmail.com <http://gmail.com>> writes:
 >     >> On Sun, Apr 24, 2016 at 1:20 AM, Xavier de Gaye <xdegaye <at>gmail.com <http://gmail.com>>
 >     > wrote:
 >     >>> Starting with API level 21 (Android 5.0), the build of python3 with the
 >     >>> official android toolchains (that is, without resorting to external
 >     >
 >     >> How does this relate tohttp://bugs.python.org/issue23496?
 >     >
 >     > As I understand, that issue seems abandoned and the patches are
 >     > (despite core devs asking otherwise) against 3.4.
 >     >
 >     >
 >     > If Xavier is willing to do so, I think it would be best to start over
 >     > with a new issue that integrates his work into 3.6.
 >
 >     I will enter a new issue that lists all the new issues and the other already
 >     existing issues that, would have they been fixed, would have allowed a
 >     successfull cross-build and the same test suite results as described in my
 >     previous post.
 >
 >
 > Thanks for this, Xavier!
 >
 > Once you have that, in addition to posting the link back here, you may also want to ping the Mobile SIG list: https://www.python.org/community/sigs/current/mobile-sig/


Issue 26865 [1] lists issues that may have to be fixed in the perspective of a
future support of the android platform.

Xavier

[1] http://bugs.python.org/issue26865


From russell at keith-magee.com  Wed Apr 27 21:01:45 2016
From: russell at keith-magee.com (Russell Keith-Magee)
Date: Thu, 28 Apr 2016 09:01:45 +0800
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <571F1285.6080409@gmail.com>
References: <571F1285.6080409@gmail.com>
Message-ID: <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>

(Apologies for the personal repost - I forgot to reply-all on the first
attempt)

Hi Xavier,

Great stuff!

I?ve got a question about your experience integrating with the native
Android platform APIs.

Getting CPython compiled as a native binary library is a huge step, but my
experience has been that bridging between binary libraries and the
Java/Dalvik APIs is a painful process - JNI *exists*, but it?s *really*
slow, and has some pretty harsh limitations (like the kernel-imposed JNI
reference count limit).

For some applications, this won?t matter - for example, if you?re treating
the android device as a low power server, something that is terminal only
isn?t a problem. However, if you want to write a native app, then you need
to be able to create an Activity, with a View, put a Button on it, and a
Layout, and so on. Have you done any exploration of the binding to these
native APIs?

Also - are you going to be at PyCon US by any chance? If you are, I?d love
to catch up and share notes.

Yours,
Russ Magee %-)



On Tue, Apr 26, 2016 at 3:02 PM, Xavier de Gaye <xdegaye at gmail.com> wrote:

> Starting with API level 21 (Android 5.0), the build of python3 with the
> official android toolchains (that is, without resorting to external
> libraries
> for wide character support) runs correctly.  With the set of patches
> described
> in the patches/Makefile file at [1], the cpython test suite runs[2] on the
> android x86 and armv7 emulators with only few errors[3].  Those errors are
> listed with their corresponding error messages, this may give a raw idea of
> the effort needed to support this platform.
>
> This post has been first submitted to python-dev as:
> https://mail.python.org/pipermail/python-dev/2016-April/144320.html
>
> Xavier
>
> [1] https://bitbucket.org/xdegaye/pyona/src
> [2] To reproduce these results, follow the instructions found in INSTALL
>     at https://bitbucket.org/xdegaye/pyona/wiki/install
> [3] https://bitbucket.org/xdegaye/pyona/wiki/testsuite
> _______________________________________________
> Mobile-sig mailing list
> Mobile-sig at python.org
> https://mail.python.org/mailman/listinfo/mobile-sig
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20160428/cf6db8ac/attachment.html>

From xdegaye at gmail.com  Thu Apr 28 05:09:52 2016
From: xdegaye at gmail.com (Xavier de Gaye)
Date: Thu, 28 Apr 2016 11:09:52 +0200
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
Message-ID: <5721D360.7010209@gmail.com>

On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
 > (Apologies for the personal repost - I forgot to reply-all on the first attempt)
 >
 > Hi Xavier,
 >
 > Great stuff!
 >
 > I?ve got a question about your experience integrating with the native Android platform APIs.
 >
 > Getting CPython compiled as a native binary library is a huge step, but my experience has been that bridging between binary libraries and the Java/Dalvik APIs is a painful process - JNI *exists*, but
 > it?s *really* slow, and has some pretty harsh limitations (like the kernel-imposed JNI reference count limit).
 >
 > For some applications, this won?t matter - for example, if you?re treating the android device as a low power server, something that is terminal only isn?t a problem. However, if you want to write a
 > native app, then you need to be able to create an Activity, with a View, put a Button on it, and a Layout, and so on. Have you done any exploration of the binding to these native APIs?


A first-class citizen java application embeds python using JNI. The embedded
python accesses the android API by importing and invoking methods of an
android.py module.  The android.py module maps those API requests to json RPC
calls that are sent to another java application so that they may be processed
and their results returned.  This is the design of Scripting Layer for Android
(SL4A) [1], a languishing project.

IMHO you should define what 'slow' means, slow for a gaming application, for
tracking the smartphone accelerometer or for a pyephem based application that
brings the ephemeresis of stars and planets at your finger tip :)

Xavier

[1] https://github.com/damonkohler/sl4a


From dimaqq at gmail.com  Thu Apr 28 05:38:04 2016
From: dimaqq at gmail.com (Dima Tisnek)
Date: Thu, 28 Apr 2016 18:38:04 +0900
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
Message-ID: <CAGGBzX+AnpQt5GFdVwSUFKnD44m6RX2G+tDg9NOba0yJATxW8w@mail.gmail.com>

http://pyjnius.readthedocs.io/en/latest/android.html#accessing-the-activity

seems pretty solid!

On 28 April 2016 at 10:01, Russell Keith-Magee <russell at keith-magee.com> wrote:
> (Apologies for the personal repost - I forgot to reply-all on the first
> attempt)
>
> Hi Xavier,
>
> Great stuff!
>
> I?ve got a question about your experience integrating with the native
> Android platform APIs.
>
> Getting CPython compiled as a native binary library is a huge step, but my
> experience has been that bridging between binary libraries and the
> Java/Dalvik APIs is a painful process - JNI *exists*, but it?s *really*
> slow, and has some pretty harsh limitations (like the kernel-imposed JNI
> reference count limit).
>
> For some applications, this won?t matter - for example, if you?re treating
> the android device as a low power server, something that is terminal only
> isn?t a problem. However, if you want to write a native app, then you need
> to be able to create an Activity, with a View, put a Button on it, and a
> Layout, and so on. Have you done any exploration of the binding to these
> native APIs?
>
> Also - are you going to be at PyCon US by any chance? If you are, I?d love
> to catch up and share notes.
>
> Yours,
> Russ Magee %-)
>
>
>
> On Tue, Apr 26, 2016 at 3:02 PM, Xavier de Gaye <xdegaye at gmail.com> wrote:
>>
>> Starting with API level 21 (Android 5.0), the build of python3 with the
>> official android toolchains (that is, without resorting to external
>> libraries
>> for wide character support) runs correctly.  With the set of patches
>> described
>> in the patches/Makefile file at [1], the cpython test suite runs[2] on the
>> android x86 and armv7 emulators with only few errors[3].  Those errors are
>> listed with their corresponding error messages, this may give a raw idea
>> of
>> the effort needed to support this platform.
>>
>> This post has been first submitted to python-dev as:
>> https://mail.python.org/pipermail/python-dev/2016-April/144320.html
>>
>> Xavier
>>
>> [1] https://bitbucket.org/xdegaye/pyona/src
>> [2] To reproduce these results, follow the instructions found in INSTALL
>>     at https://bitbucket.org/xdegaye/pyona/wiki/install
>> [3] https://bitbucket.org/xdegaye/pyona/wiki/testsuite
>> _______________________________________________
>> Mobile-sig mailing list
>> Mobile-sig at python.org
>> https://mail.python.org/mailman/listinfo/mobile-sig
>
>
>
> _______________________________________________
> Mobile-sig mailing list
> Mobile-sig at python.org
> https://mail.python.org/mailman/listinfo/mobile-sig
>

From gabriel.pettier at gmail.com  Thu Apr 28 06:09:31 2016
From: gabriel.pettier at gmail.com (Gabriel Pettier)
Date: Thu, 28 Apr 2016 12:09:31 +0200
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <5721D360.7010209@gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
 <5721D360.7010209@gmail.com>
Message-ID: <20160428100931.GY17808@desk-dev-1>

On Thu, Apr 28, 2016 at 11:09:52AM +0200, Xavier de Gaye wrote:
>On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
>> (Apologies for the personal repost - I forgot to reply-all on the first attempt)
>>
>> Hi Xavier,
>>
>> Great stuff!
>>
>> I?ve got a question about your experience integrating with the native Android platform APIs.
>>
>> Getting CPython compiled as a native binary library is a huge step, but my experience has been that bridging between binary libraries and the Java/Dalvik APIs is a painful process - JNI *exists*, but
>> it?s *really* slow, and has some pretty harsh limitations (like the kernel-imposed JNI reference count limit).
>>
>> For some applications, this won?t matter - for example, if you?re treating the android device as a low power server, something that is terminal only isn?t a problem. However, if you want to write a
>> native app, then you need to be able to create an Activity, with a View, put a Button on it, and a Layout, and so on. Have you done any exploration of the binding to these native APIs?
>
>
>A first-class citizen java application embeds python using JNI. The embedded
>python accesses the android API by importing and invoking methods of an
>android.py module.  The android.py module maps those API requests to json RPC
>calls that are sent to another java application so that they may be processed
>and their results returned.  This is the design of Scripting Layer for Android
>(SL4A) [1], a languishing project.
>
>IMHO you should define what 'slow' means, slow for a gaming application, for
>tracking the smartphone accelerometer or for a pyephem based application that
>brings the ephemeresis of stars and planets at your finger tip :)
>
>Xavier
>
>[1] https://github.com/damonkohler/sl4a

You may be interested in pyjnius[1], which, if not working around the 
JNI limitations (speed being one, but other JNI limitations for using 
the Android API is the inhability to extend java classes), makes using 
it with python a lot easier, by using Java introspection capabilities, 
it may not be the best answer if speed is the most important criteria, 
but i would advocate for reconsidering using Python on Android if that's 
the case.

[1] https://github.com/kivy/pyjnius

From russell at keith-magee.com  Thu Apr 28 17:50:36 2016
From: russell at keith-magee.com (Russell Keith-Magee)
Date: Fri, 29 Apr 2016 05:50:36 +0800
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <20160428100931.GY17808@desk-dev-1>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
 <5721D360.7010209@gmail.com> <20160428100931.GY17808@desk-dev-1>
Message-ID: <CAJxq84_Qpim4jRaTFsuXEA2ZjSpPtbvbkAO9OiiiLm3g0DApZQ@mail.gmail.com>

On Thu, Apr 28, 2016 at 6:09 PM, Gabriel Pettier <gabriel.pettier at gmail.com>
wrote:

> On Thu, Apr 28, 2016 at 11:09:52AM +0200, Xavier de Gaye wrote:
>
>> On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
>>
>>> (Apologies for the personal repost - I forgot to reply-all on the first
>>> attempt)
>>>
>>> Hi Xavier,
>>>
>>> Great stuff!
>>>
>>> I?ve got a question about your experience integrating with the native
>>> Android platform APIs.
>>>
>>> Getting CPython compiled as a native binary library is a huge step, but
>>> my experience has been that bridging between binary libraries and the
>>> Java/Dalvik APIs is a painful process - JNI *exists*, but
>>> it?s *really* slow, and has some pretty harsh limitations (like the
>>> kernel-imposed JNI reference count limit).
>>>
>>> For some applications, this won?t matter - for example, if you?re
>>> treating the android device as a low power server, something that is
>>> terminal only isn?t a problem. However, if you want to write a
>>> native app, then you need to be able to create an Activity, with a View,
>>> put a Button on it, and a Layout, and so on. Have you done any exploration
>>> of the binding to these native APIs?
>>>
>>
>>
>> A first-class citizen java application embeds python using JNI. The
>> embedded
>> python accesses the android API by importing and invoking methods of an
>> android.py module.  The android.py module maps those API requests to json
>> RPC
>> calls that are sent to another java application so that they may be
>> processed
>> and their results returned.  This is the design of Scripting Layer for
>> Android
>> (SL4A) [1], a languishing project.
>>
>> IMHO you should define what 'slow' means, slow for a gaming application,
>> for
>> tracking the smartphone accelerometer or for a pyephem based application
>> that
>> brings the ephemeresis of stars and planets at your finger tip :)
>>
>> Xavier
>>
>> [1] https://github.com/damonkohler/sl4a
>>
>
> You may be interested in pyjnius[1], which, if not working around the JNI
> limitations (speed being one, but other JNI limitations for using the
> Android API is the inhability to extend java classes), makes using it with
> python a lot easier, by using Java introspection capabilities, it may not
> be the best answer if speed is the most important criteria, but i would
> advocate for reconsidering using Python on Android if that's the case.
>
> [1] https://github.com/kivy/pyjnius
>

Hi Gabriel,

I?m well aware of PyJNIus. I didn?t have a whole lot of luck with it,
mostly because of bugs/complications with Cython.

I also tried my hand at my own version that didn?t have the dependency on
Cython, and used ctypes instead:

https://github.com/pybee/rubicon-java

This has the benefit that it can be installed using a simple ?pip install?.

That works, but my experience was that Android?s JNI layer really wasn?t up
to the task - hence why I asked Xavier what (if any) work he had done in
this area.

Yours,
Russ Magee %-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20160429/94f263de/attachment.html>

From russell at keith-magee.com  Thu Apr 28 18:16:47 2016
From: russell at keith-magee.com (Russell Keith-Magee)
Date: Fri, 29 Apr 2016 06:16:47 +0800
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <5721D360.7010209@gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
 <5721D360.7010209@gmail.com>
Message-ID: <CAJxq8480smyw==23a2FwX7T5V1perGNGZ8tv7nzDjAiBodwuzw@mail.gmail.com>

On Thu, Apr 28, 2016 at 5:09 PM, Xavier de Gaye <xdegaye at gmail.com> wrote:

> On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
> > (Apologies for the personal repost - I forgot to reply-all on the first
> attempt)
> >
> > Hi Xavier,
> >
> > Great stuff!
> >
> > I?ve got a question about your experience integrating with the native
> Android platform APIs.
> >
> > Getting CPython compiled as a native binary library is a huge step, but
> my experience has been that bridging between binary libraries and the
> Java/Dalvik APIs is a painful process - JNI *exists*, but
> > it?s *really* slow, and has some pretty harsh limitations (like the
> kernel-imposed JNI reference count limit).
> >
> > For some applications, this won?t matter - for example, if you?re
> treating the android device as a low power server, something that is
> terminal only isn?t a problem. However, if you want to write a
> > native app, then you need to be able to create an Activity, with a View,
> put a Button on it, and a Layout, and so on. Have you done any exploration
> of the binding to these native APIs?
>
>
> A first-class citizen java application embeds python using JNI. The
> embedded
> python accesses the android API by importing and invoking methods of an
> android.py module.  The android.py module maps those API requests to json
> RPC
> calls that are sent to another java application so that they may be
> processed
> and their results returned.  This is the design of Scripting Layer for
> Android
> (SL4A) [1], a languishing project.
>

Sure. I?m aware of SL4A. It struck me as a neat hack, but not an especially
viable approach in practice - certainly not something suitable for an app
that you?re expecting to use for wide consumer adoption.


> IMHO you should define what 'slow' means, slow for a gaming application,
> for
> tracking the smartphone accelerometer or for a pyephem based application
> that
> brings the ephemeresis of stars and planets at your finger tip :)
>

My use case was a fairly simple native application, a couple of inputs and
a button. I took a Python implementation of the Cassowary widget layout
algorithm - the same one that iPhones use natively - and tried to use that
layout algorithm for widgets on the Android view - and it was almost
unusably slow.

?Slow? meant ~5 second startup time for a ?hello world? app, and observable
lags in widget redraw on device rotation. If I was getting that with only a
couple of widgets on the screen, I didn?t want to think what it would be
like with a complex UI.

My diagnosis of the situation found two sources of slowdown:

1) The Android kernel imposes a hard limit on the number of JNI references
that a single application can hold, and that limit is impractically low. I
forget the exact number but the limit is something like 1000 - but a full
rollout of the classes involved in starting Hello World involves 4500 JNI
references. So - you end up having to do on-demand JNI lookups, and
aggressive LRU caching, all of which slows down actual use - you can?t just
write off the problem as a startup cost.

2) The JNI marshalling layer itself was about an order of magnitude slower
than it is on a desktop JNI bridge. Getting a JNI reference, making a call
across the JNI bridge - all these operations took *much* longer than the
comparable operations on a desktop machine, even when you allow for the
differences in CPU speed.

FWIW - I was working on embedded CPython about 18 months ago, and gave up
because these problems didn?t seem solvable. I was hoping you might have
found a different way into the native capabilities of the Android platform.
My conclusion is that Android *REALLY* wants you to be writing Java, not
using JNI.

As a result, I gave up on the JNI approach, and took a different tack.

VOC [1] is a transpiler - but not a source code transpiler. It takes
CPython bytecode as input (instructions for a stack-based virtual machine),
and converts that bytecode to Java classfiles (bytecode for different
stack-based virtual machine). That way, you can take Python code [2],
compile it *directly* to equivalent Java class files, where it runs
completely natively [3], with fairly acceptable performance. When the JVM
throws a stack trace, it references the *Python* source code line number.

To be clear - this isn?t source code transpilation - it?s at the bytecode
level.. It?s also different to Jython, in that it doesn?t carry the
overhead of an interpreter to the runtime - you?re shipping precompiled,
ready-to-run Java bytecode. The downside is that there isn?t a REPL or an
exec()/eval() call - but for my use case (writing apps for end users),
that?s a limitation I can live with.

[1] https://github.com/pybee/voc
[2] https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97
[3] https://twitter.com/PyBeeWare/status/683258762816192513

Yours
Russ Magee %-)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20160429/3c8b0a34/attachment-0001.html>

From dimaqq at gmail.com  Thu Apr 28 20:33:26 2016
From: dimaqq at gmail.com (Dima Tisnek)
Date: Fri, 29 Apr 2016 09:33:26 +0900
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <CAJxq8480smyw==23a2FwX7T5V1perGNGZ8tv7nzDjAiBodwuzw@mail.gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
 <5721D360.7010209@gmail.com>
 <CAJxq8480smyw==23a2FwX7T5V1perGNGZ8tv7nzDjAiBodwuzw@mail.gmail.com>
Message-ID: <CAGGBzXKvhMR8hauDpyiLjeNNemtshWXfvDnZriF1fN2YLf56cQ@mail.gmail.com>

Sorry if I'm getting this off-topic, how do native apps (as in NDK) access
ui then?

IMO in-process py-java bridge is the way to go. If it's not good enough yet
(numbers please), then we have to make it better.
On 29 Apr 2016 07:16, "Russell Keith-Magee" <russell at keith-magee.com> wrote:

>
> On Thu, Apr 28, 2016 at 5:09 PM, Xavier de Gaye <xdegaye at gmail.com> wrote:
>
>> On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
>> > (Apologies for the personal repost - I forgot to reply-all on the first
>> attempt)
>> >
>> > Hi Xavier,
>> >
>> > Great stuff!
>> >
>> > I?ve got a question about your experience integrating with the native
>> Android platform APIs.
>> >
>> > Getting CPython compiled as a native binary library is a huge step, but
>> my experience has been that bridging between binary libraries and the
>> Java/Dalvik APIs is a painful process - JNI *exists*, but
>> > it?s *really* slow, and has some pretty harsh limitations (like the
>> kernel-imposed JNI reference count limit).
>> >
>> > For some applications, this won?t matter - for example, if you?re
>> treating the android device as a low power server, something that is
>> terminal only isn?t a problem. However, if you want to write a
>> > native app, then you need to be able to create an Activity, with a
>> View, put a Button on it, and a Layout, and so on. Have you done any
>> exploration of the binding to these native APIs?
>>
>>
>> A first-class citizen java application embeds python using JNI. The
>> embedded
>> python accesses the android API by importing and invoking methods of an
>> android.py module.  The android.py module maps those API requests to json
>> RPC
>> calls that are sent to another java application so that they may be
>> processed
>> and their results returned.  This is the design of Scripting Layer for
>> Android
>> (SL4A) [1], a languishing project.
>>
>
> Sure. I?m aware of SL4A. It struck me as a neat hack, but not an
> especially viable approach in practice - certainly not something suitable
> for an app that you?re expecting to use for wide consumer adoption.
>
>
>> IMHO you should define what 'slow' means, slow for a gaming application,
>> for
>> tracking the smartphone accelerometer or for a pyephem based application
>> that
>> brings the ephemeresis of stars and planets at your finger tip :)
>>
>
> My use case was a fairly simple native application, a couple of inputs and
> a button. I took a Python implementation of the Cassowary widget layout
> algorithm - the same one that iPhones use natively - and tried to use that
> layout algorithm for widgets on the Android view - and it was almost
> unusably slow.
>
> ?Slow? meant ~5 second startup time for a ?hello world? app, and
> observable lags in widget redraw on device rotation. If I was getting that
> with only a couple of widgets on the screen, I didn?t want to think what it
> would be like with a complex UI.
>
> My diagnosis of the situation found two sources of slowdown:
>
> 1) The Android kernel imposes a hard limit on the number of JNI references
> that a single application can hold, and that limit is impractically low. I
> forget the exact number but the limit is something like 1000 - but a full
> rollout of the classes involved in starting Hello World involves 4500 JNI
> references. So - you end up having to do on-demand JNI lookups, and
> aggressive LRU caching, all of which slows down actual use - you can?t just
> write off the problem as a startup cost.
>
> 2) The JNI marshalling layer itself was about an order of magnitude slower
> than it is on a desktop JNI bridge. Getting a JNI reference, making a call
> across the JNI bridge - all these operations took *much* longer than the
> comparable operations on a desktop machine, even when you allow for the
> differences in CPU speed.
>
> FWIW - I was working on embedded CPython about 18 months ago, and gave up
> because these problems didn?t seem solvable. I was hoping you might have
> found a different way into the native capabilities of the Android platform.
> My conclusion is that Android *REALLY* wants you to be writing Java, not
> using JNI.
>
> As a result, I gave up on the JNI approach, and took a different tack.
>
> VOC [1] is a transpiler - but not a source code transpiler. It takes
> CPython bytecode as input (instructions for a stack-based virtual machine),
> and converts that bytecode to Java classfiles (bytecode for different
> stack-based virtual machine). That way, you can take Python code [2],
> compile it *directly* to equivalent Java class files, where it runs
> completely natively [3], with fairly acceptable performance. When the JVM
> throws a stack trace, it references the *Python* source code line number.
>
> To be clear - this isn?t source code transpilation - it?s at the bytecode
> level.. It?s also different to Jython, in that it doesn?t carry the
> overhead of an interpreter to the runtime - you?re shipping precompiled,
> ready-to-run Java bytecode. The downside is that there isn?t a REPL or an
> exec()/eval() call - but for my use case (writing apps for end users),
> that?s a limitation I can live with.
>
> [1] https://github.com/pybee/voc
> [2] https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97
> [3] https://twitter.com/PyBeeWare/status/683258762816192513
>
> Yours
> Russ Magee %-)
>
> _______________________________________________
> Mobile-sig mailing list
> Mobile-sig at python.org
> https://mail.python.org/mailman/listinfo/mobile-sig
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/mobile-sig/attachments/20160429/3f1625ac/attachment.html>

From gabriel.pettier at gmail.com  Thu Apr 28 21:01:06 2016
From: gabriel.pettier at gmail.com (Gabriel Pettier)
Date: Fri, 29 Apr 2016 03:01:06 +0200
Subject: [Mobile-sig] support of the android platform
In-Reply-To: <CAJxq8480smyw==23a2FwX7T5V1perGNGZ8tv7nzDjAiBodwuzw@mail.gmail.com>
References: <571F1285.6080409@gmail.com>
 <CAJxq84-+Npvce51vy=zYbVr3gTwDH126jEKgBmxe1oWDsEsdJw@mail.gmail.com>
 <5721D360.7010209@gmail.com>
 <CAJxq8480smyw==23a2FwX7T5V1perGNGZ8tv7nzDjAiBodwuzw@mail.gmail.com>
Message-ID: <20160429010106.GC18801@desk-dev-1>

Sorry, i'm aware of your work, just didn't make the connection 
immediatly. Sorry.

About VOC, That's a very interesting approach, i'd be curious to see 
what code using it looks like (how do you use java classes?  
introspection api?), sad to see no examples or more than the first page 
of the tutorials covering hello world, i guess you have a lot more 
working than that :).

Congrats for all the hard work!

On Fri, Apr 29, 2016 at 06:16:47AM +0800, Russell Keith-Magee wrote:
>On Thu, Apr 28, 2016 at 5:09 PM, Xavier de Gaye <xdegaye at gmail.com> wrote:
>
>> On 04/28/2016 03:01 AM, Russell Keith-Magee wrote:
>> > (Apologies for the personal repost - I forgot to reply-all on the first
>> attempt)
>> >
>> > Hi Xavier,
>> >
>> > Great stuff!
>> >
>> > I?ve got a question about your experience integrating with the native
>> Android platform APIs.
>> >
>> > Getting CPython compiled as a native binary library is a huge step, but
>> my experience has been that bridging between binary libraries and the
>> Java/Dalvik APIs is a painful process - JNI *exists*, but
>> > it?s *really* slow, and has some pretty harsh limitations (like the
>> kernel-imposed JNI reference count limit).
>> >
>> > For some applications, this won?t matter - for example, if you?re
>> treating the android device as a low power server, something that is
>> terminal only isn?t a problem. However, if you want to write a
>> > native app, then you need to be able to create an Activity, with a View,
>> put a Button on it, and a Layout, and so on. Have you done any exploration
>> of the binding to these native APIs?
>>
>>
>> A first-class citizen java application embeds python using JNI. The
>> embedded
>> python accesses the android API by importing and invoking methods of an
>> android.py module.  The android.py module maps those API requests to json
>> RPC
>> calls that are sent to another java application so that they may be
>> processed
>> and their results returned.  This is the design of Scripting Layer for
>> Android
>> (SL4A) [1], a languishing project.
>>
>
>Sure. I?m aware of SL4A. It struck me as a neat hack, but not an especially
>viable approach in practice - certainly not something suitable for an app
>that you?re expecting to use for wide consumer adoption.
>
>
>> IMHO you should define what 'slow' means, slow for a gaming application,
>> for
>> tracking the smartphone accelerometer or for a pyephem based application
>> that
>> brings the ephemeresis of stars and planets at your finger tip :)
>>
>
>My use case was a fairly simple native application, a couple of inputs and
>a button. I took a Python implementation of the Cassowary widget layout
>algorithm - the same one that iPhones use natively - and tried to use that
>layout algorithm for widgets on the Android view - and it was almost
>unusably slow.
>
>?Slow? meant ~5 second startup time for a ?hello world? app, and observable
>lags in widget redraw on device rotation. If I was getting that with only a
>couple of widgets on the screen, I didn?t want to think what it would be
>like with a complex UI.
>
>My diagnosis of the situation found two sources of slowdown:
>
>1) The Android kernel imposes a hard limit on the number of JNI references
>that a single application can hold, and that limit is impractically low. I
>forget the exact number but the limit is something like 1000 - but a full
>rollout of the classes involved in starting Hello World involves 4500 JNI
>references. So - you end up having to do on-demand JNI lookups, and
>aggressive LRU caching, all of which slows down actual use - you can?t just
>write off the problem as a startup cost.
>
>2) The JNI marshalling layer itself was about an order of magnitude slower
>than it is on a desktop JNI bridge. Getting a JNI reference, making a call
>across the JNI bridge - all these operations took *much* longer than the
>comparable operations on a desktop machine, even when you allow for the
>differences in CPU speed.
>
>FWIW - I was working on embedded CPython about 18 months ago, and gave up
>because these problems didn?t seem solvable. I was hoping you might have
>found a different way into the native capabilities of the Android platform.
>My conclusion is that Android *REALLY* wants you to be writing Java, not
>using JNI.
>
>As a result, I gave up on the JNI approach, and took a different tack.
>
>VOC [1] is a transpiler - but not a source code transpiler. It takes
>CPython bytecode as input (instructions for a stack-based virtual machine),
>and converts that bytecode to Java classfiles (bytecode for different
>stack-based virtual machine). That way, you can take Python code [2],
>compile it *directly* to equivalent Java class files, where it runs
>completely natively [3], with fairly acceptable performance. When the JVM
>throws a stack trace, it references the *Python* source code line number.
>
>To be clear - this isn?t source code transpilation - it?s at the bytecode
>level.. It?s also different to Jython, in that it doesn?t carry the
>overhead of an interpreter to the runtime - you?re shipping precompiled,
>ready-to-run Java bytecode. The downside is that there isn?t a REPL or an
>exec()/eval() call - but for my use case (writing apps for end users),
>that?s a limitation I can live with.
>
>[1] https://github.com/pybee/voc
>[2] https://gist.github.com/freakboy3742/3c6b74e8506d47d9bd97
>[3] https://twitter.com/PyBeeWare/status/683258762816192513
>
>Yours
>Russ Magee %-)

>_______________________________________________
>Mobile-sig mailing list
>Mobile-sig at python.org
>https://mail.python.org/mailman/listinfo/mobile-sig