executing arbitrary statements

[Jonathan Hartley]

On Saturday, October 1, 2011 8:06:43 AM UTC+1, Chris Rebert wrote:
> On Fri, Sep 30, 2011 at 11:31 PM, Jason Swails <jason.... at gmail.com> wrote:
> > I'm probably missing something pretty obvious, but I was wondering if there
> > was a way of executing an arbitrary line of code somehow (such as a line of
> > code based on user-input).  There's the obvious use of "eval" that will
> > evaluate a function call, but that doesn't allow all things.
> <snip>
> > Because write is a function eval works fine for it.  But since print isn't
> > (2.7), it throws a syntax error.  Likewise, variable assignments aren't
> > allowed either as they are also not functions and lack a return value:
> <snip>
> 
> Use the `exec` statement, which is capable of executing statements (as
> opposed to just expressions):
> http://docs.python.org/reference/simple_stmts.html#the-exec-statement
> 
> > What I'm more or less looking to do is present a (limited) form of an
> > interpreter inside the application I'm writing for the advanced user.
> >
> > I'm also interested to hear if this is a particularly bad idea for any
> > reason,
> 
> It's potentially rather hacky and ad-hoc to effectively directly
> inject arbitrary statements at certain points in your code.
> 
> Assuming you were to trust the user-provided code, callbacks/hooks or
> plugin modules are typically much cleaner ways to integrate custom
> code from the user.
> 
> Depending on your particular use case, the `cmd` module might also be
> a workable alternative:
> http://docs.python.org/library/cmd.html
> 
> > and if there are security issues involved with allowing users to
> > execute their own code inside my program (keeping in mind that some people
> > may "donate" their scripts to others that may run them as black boxes).
> 
> It is *very much* a security issue!
> 
> > Is
> > it enough to disallow import statements, thereby not giving direct access to
> > the sys and os modules?
> 
> Not by a long shot! There are a bunch of known tricks that exploit
> introspection to circumvent such restrictions.
> Secure execution of untrusted Python code is a difficult problem. Some
> have accomplished it to a degree, but typically only by modifying the
> interpreter itself or imposing relatively onerous restrictions on the
> untrusted code.
> 
> > I know more or less what I want to do, but I'd also
> > appreciate any experienced input/advice/suggestions.
> 
> I additionally came across this in researching my reply:
> http://pypi.python.org/pypi/RestrictedPython/
> Apparently the speed of execution leaves something to be desired, but
> the package /supposedly/ works well otherwise.
> 
> Cheers,
> Chris
> --
> http://rebertia.com


I (and many others) entirely avoid using 'eval' in all my code for many years, based on the security concerns that Chris rightly highlights. It's worth noting though, that RaymondH's talks last year on some valid uses of 'eval' and 'exec' have opened my eyes to it somewhat. In summary, while it's dangerous to execute user-submitted code, there are no security risks associated with executing code generated by your own program. It takes a while to see how this might be useful, if (like me) you're not used to thinking about it.

Raymond's premier example was his implementation of namedtuple:
(see http://hg.python.org/cpython/file/default/Lib/collections/__init__.py)
This defines a string, the contents of which is a class definition, with some string formatting markers in it. These are replaced with known values on invocation of the namedtuple factory function, which then exec's the class-definition string and returns the resulting new type.

This results in an implementation that is both simpler and faster than trying to simply write a general-purpose class that does at runtime all the things that 'namedtuple' does.