Import/Create module from buffer

[Gabriel Genellina]

En Mon, 12 May 2008 08:09:45 -0300, Gruik <benjamin.fremiot at gmail.com> escribió:

> On May 12, 12:31 pm, "Gabriel Genellina" <gagsl-... at yahoo.com.ar>
> wrote:
>> En Mon, 12 May 2008 05:49:22 -0300, Gruik <benjamin.frem... at gmail.com> escribió:
>>
>> > I'm currently working on python embedding with C++. My goal is that
>> > the C++ part handle files writing/reading so that the Python part only
>> > works with buffers.
>>
>> > I succeeded in buffer exchanges. The problem is that some of the files
>> > I read/write are python files so that, before embedding, I imported
>> > them as modules when I needed them.
>>
>> > But now that the Python part only receive buffers, I can't do it
>> > anymore. So I wonder :
>> > - is it possible to import module from a buffer instead of files?
>> > - or is it possible to create a module object from my buffer?
>>
>> Yes, first compile the buffer to get a code object, then use PyImport_ExecCodeModule. See how this function is used in import.c
>>
>
> Thanks for your quick answer !
> I think I'll have no problem with that in C++ and I'm going to try it
> right after this message.
>
> But before that 1 question: what if I'm in Python ?
> Following your solution, I did that in Python :
>
>     def load_buffer(buffer) :
>         compiled_buffer = compile(buffer, "module_name", "exec")
>         exec(compiled_buffer)
>
> It works great except that I can't have a module object and that it is
> as if I did "from module import *"
> But I need the module object and not an "import *" behavior.
> Any idea about the way to do that?

Almost - you have to create a new module and exec the code into its namespace:

py> import new
py> foo = new.module("foo", "This is the foo module")
py> exec "def f(): pass" in foo.__dict__
py> foo
<module 'foo' (built-in)>
py> foo.f
<function f at 0x00A3E230>
py> dir(foo)
['__builtins__', '__doc__', '__name__', 'f']

(replace "def f(): pass" with the compiled buffer)

Two differences with a "normal" module:

- it has no __file__ attribute - you may want to add it, with the original filename, or leave it off to show that it's not loaded from a file.
- it does not exist in sys.modules, so other modules cannot import it. If you want to allow that: sys.modules['foo'] = foo (perhaps one should check previously that there is no module named "foo" before replacing it...)

-- 
Gabriel Genellina