[Chicago] exec saved compiled code

[Atul Varma]

I believe you may want to use marshal instead of pickle.  In fact,
.pyc files are actually just marshalled code objects with a bit of
header information at the front--specifically, a 4-byte "magic number"
followed by a last-modified-timestamp of the source .py file.

Going back to your original example, loading a .pyc file can be done like so:

  # Compile the .pyc file (this is unchanged from your original post)
  environment={'x':3}
  open('a.py','w').write('print x')
  import py_compile,os
  exec(open('a.py','r').read()) in environment ### prints 3 OK!
  py_compile.compile('a.py')
  os.unlink('a.py')

  # Now load the .pyc file.
  f = open( "a.pyc", "rb" )
  pycData = f.read()
  f.close()

  # Optional: read the magic number and ensure that it's what we expect
  # for our version of Python.
  import imp
  magic = pycData[:4]
  assert magic == imp.get_magic()

  # Optional: Get the last modified time of the original .py source,
to tell if we're in
  # sync with the compiled version of the latest version of the .py
file.  At least,
  # that's what I *think* this is for...
  mtime = pycData[4:8]
  # Well, actually, we can't do any comparison because we've deleted the
  # original .py file, so scratch that. :)

  # Read the actual marshalled code object and unmarshal it.
  codeBytes = pycData[8:]
  code = marshal.loads( codeBytes )

  # execute the code.
  exec code in environment

Note that more needs to be done in that last step if you actually want
to set up a real module (e.g., if you're creating a custom
importer--see PEP 302 for more information).

It's also interesting to note that the .pyc file format isn't
documented in prose anywhere, so the following source code can be used
to determine it:

  http://svn.python.org/projects/python/trunk/Python/import.c

I hope this is helpful.

- Atul

On 8/29/07, Massimo Di Pierro <mdipierro at cti.depaul.edu> wrote:
>
> almost... but I want to be able to save c and reload it before the exec. It
> appears c cannot be picked.
>
> Massimo
>
>
> On Aug 29, 2007, at 2:28 AM, Cosmin Stejerean wrote:
> I'm not sure if I understand what's going on with the environment = {} part
> so I'm not sure if my solution to your problem is correct but here is what I
> think is wrong. As far as I can tell exec can take either a string or a code
> object. exec(open(' a.py','r').read()) works because you are reading the
> code from a file (it could have been any other file extension) and exec is
> parsing it.
>
> exec(open('a.pyc','r').read()) this however reads the bytecode as a string
> and passes it to exec. This is probably not what you want (and most likely
> won't work). One solution is to simply import a, but I'm guessing that's not
> what you want either. The other option is to not use py_compile but use the
> compile function and pickle as follows
>
> c = compile("print x", '<string>', 'exec')
>
> now you have a code object that you can execute
>
> exec(c)
>
> If you pickle this code object to a file you will be able to read it and run
> it (then again why not pickle the original string??). The problem is pickle
> won't pickle code objects. But apparently you can use a little trick around
> that (see
> http://mail.python.org/pipermail/python-list/2002-August/161661.html)
>
> Is this at least close to what you were looking for?
>
> - Cosmin
>
> On 8/29/07, Massimo Di Pierro <mdipierro at cti.depaul.edu > wrote:
> >
> > Hello,
> >
> > I am hoping one of you can help me with this.
> >
> > EXAMPLE:
> >
> > environment={'x':3}
> > open('a.py','w').write('print x')
> > import py_compile,os
> > exec(open('a.py','r').read()) in environment ### prints 3 OK!
> > py_compile.compile(' a.py')
> > os.unlink('a.py')
> > exec(open('a.pyc','r').read()) in environment ### does not work
> >
> > QUESTION: how do exec a.pyc into environment so that it prints 3? Is
> > it possible to do it?
> >
> > Massimo
> > _______________________________________________
> > Chicago mailing list
> > Chicago at python.org
> > http://mail.python.org/mailman/listinfo/chicago
> >
>
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>
> _______________________________________________
> Chicago mailing list
> Chicago at python.org
> http://mail.python.org/mailman/listinfo/chicago
>
>