[Python-Dev] Challenge: Please break this! (a.k.a restricted mode revisited)

[Jon Ribbens]

On Mon, Apr 11, 2016 at 11:40:05AM +0200, Victor Stinner wrote:
> 2016-04-10 18:43 GMT+02:00 Jon Ribbens <jon+python-dev at unequivocal.co.uk>:
> > That's the opposite of my approach though - I'm starting small and
> > adding things, not starting with everything and removing stuff. Even
> > if what we end up with is an extremely restricted subset of Python,
> > there are still cases where that could be a useful tool to have.
> 
> You design rely on the assumption that CPython is only pure Python.

No it doesn't. Obviously I know CPython is written in C - the clue is
in the name. I'm not sure what you mean here. 

> It means that your protections like hiding builtin functions from the
> Python scope don't work. If an attacker gets access to a C function
> giving access to the hidden builtin, the game is over.

The former is only true if you assume the latter is possible.
Is there any reason to believe it is?

> It's hard to list all features of the C code which are indirectly
> accessible from the Python scope. Some examples: warnings and
> tracebacks. These features killed the pysandbox project because they
> open directly files on the filesystem, it's not possible to control
> these features from the Python scope.

I think what you're referring to is when they show context for errors,
for which they try and find the source code lines to display by
identifying the filename, and you can subvert that process by changing
__file__ and/or __name__. If so, you can't do that within my
experiment because you're not allowed to access either of those names.

> Another example which exposes a vulnerability of your sandbox:
> str.format() gets directly object attributes without the getattr()
> builtin function, so it's possible to escape your sandbox. Example:
> "{0.__class__}".format(obj) shows the type of an object.

Yes, I'd thought of that. However getting access to a string which
contains the name or a representation of an object is not at all the
same thing as getting access to the object itself. 

> Think also about the new f-string which allows arbitrary Python
> code: f"{code}".

Obviously I can't speak to features of future versions of Python.
I'd have to see the ast generated by an f-string to know if they
pose a problem or not, but I suspect they would compile to
expression nodes and hence be caught by the existing checks.

> > However on the other hand, nobody has tried before to do what I am
> > doing (static code analysis),
> 
> You're wrong.
> 
> Zope Security ("RestrictedPython") has a similar design. Analyzing AST
> is a common design to build a sanbox. But it's not safe.

Ah, I hadn't seen that one. Yes, they are doing something similar
(but also much more complex!) I don't know why you say this is
a "common design" though, that one is the only one that appears to
use it.

> What I see is that you asked to break your sandbox, and less than 1
> hour later, a first vulnerability was found (exec called with two
> parameters). A few hours later, a second vulnerability was found
> (async generator and cr_frame).

The former was just a stupid bug, it says nothing about the viability
of the methodology. The latter was a new feature in a Python version
later than I have ever used, and again does not imply anything much
about the viability. I think now I've blocked the names of frame
object attributes it wouldn't be a vulnerability any more anyway.

> By the way, are you sure that you fixed the vulnerability? You
> blacklisted "cb_frame", not cr_frame ;-)

Ah, thanks. As above, I think this doesn't actually make any
difference, but I've updated the code regardless.

> You should look closer, pysandbox is very close to you project.

I've just looked through it all again, and I don't understand why you
are saying that. It's nothing like my experiment. It's trying to alter
the global Python environment so that arbitrary code can be executed,
whereas I am not even trying to allow execution of arbitrary code and
am not altering the global environment.