Loading modules from files through C++

[Roland Plüss]

On 05/17/2014 04:01 PM, Stefan Behnel wrote:
> Hi,
>
> please avoid top-posting.
>
>
> Roland Plüss, 17.05.2014 15:49:
>> On 05/17/2014 03:26 PM, Stefan Behnel wrote:
>>> Roland Plüss, 17.05.2014 15:00:
>>>> On 05/17/2014 01:58 PM, Stefan Behnel wrote:
>>>>> Roland Plüss, 17.05.2014 02:27:
>>>>>> I'm using Python in an embedded situation. In particular I have to load
>>>>>> python scripts through a memory interface so regular python module
>>>>>> loading can not be used. I got working so far a module loader object
>>>>>> I've added using C++ to sys.meta_path . Now I'm totally stuck at the
>>>>>> finally loading step.
>>>>>>
>>>>>> I've got this a C++ loader method "load_module(fullname)" which does
>>>>>> load the requested module script files into a null-terminated string. I
>>>>>> know that "load_module" has to return the module PyObject*. But I can't
>>>>>> get the python source in the c-string into a module PyObject*.
>>>>>> [...]
>>>>>> Can anybody help how in gods name one is supposed to create a module
>>>>>> from an in-memory c-string when called from within load_module (or
>>>>>> anywhere)?
>>>>> Looks like you want to implement a SourceLoader:
>>>>>
>>>>> https://docs.python.org/3.4/library/importlib.html#importlib.abc.SourceLoader
>>>>>
>>>>> I recommend implementing this in Python code instead of C code, though.
>>>>> Much easier. Cython can help with the integration between both.
>>>> That doesn't work in 2.x, doesn't it?
>>> Is there a reason you have to use Py2?
>>>
>>> Anyway, PEP 302 predates Py3 by a couple of years:
>>>
>>> http://legacy.python.org/dev/peps/pep-0302/
>> I'm willing to go to Py3 but only if the solution to the problem is
>> simpler than getting it fixed in Py2. So some questions first:
>>
>> - does this importlib stuff you showed there apply to C++ land (I need
>> to fully drive it from C++ not Python code)?
> As I said, implementing this in Python code is much simpler than doing it
> in C/C++ code. Basically, stop where you got the C string and do the rest
> in Python. All your C code has to do is to take a module lookup request
> from your custom Python module Finder and return a byte string with the
> code. Then let your Python code wrap that in a Loader and return it to the
> import machinery.
I don't get how this is supposed to work. I'm running it as fully
embedded Python. There is no main script. The builtin modules are added
as C++ bound classes and a user made main script is loaded but not run
directly (I'm hooking into a create object). For this purpose I load the
script module using C++ code using PyImport_ImportModule(moduleName). At
this time the module loading code has to kick in already (I've added
this one by C++ too before). The problem is now that in this call I end
up in my C++ loader version where there is no Python script involved. I
came to the conclusion that I can solve this only by having the C++ end
properly load the module. I could add Python code with
PyRun_SimpleString but then I'm down to the same problem as before: how
to evaluate code so it is attached to a module or type-class? As I
understand it the problem is the same as before just pushed around a bit.
>> - is the C++ land of Py3 similar to Py2 or totally different?
> Much the same, just slight differences. As I said, Cython can help with the
> integration and would mostly cover the differences for you.
Sounds good. I'll give it a try
>
> Stefan
>
>


-- 
Yours sincerely
Plüss Roland

Leader and Head Programmer
- Game: Epsylon ( http://www.indiedb.com/games/epsylon )
- Game Engine: Drag[en]gine ( http://www.indiedb.com/engines/dragengine
, http://dragengine.rptd.ch/wiki )
- Normal Map Generator: DENormGen ( http://epsylon.rptd.ch/denormgen.php )
- As well as various Blender export scripts und game tools

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 263 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20140517/51ab0f46/attachment.sig>