building an online judge to evaluate Python programs

[Ned Batchelder]

On 9/20/13 6:26 PM, Jabba Laci wrote:
> I just found Docker ( http://docs.docker.io/en/latest/faq/ ). It seems
> sandboxing could be done with this easily.

At edX, I wrote CodeJail (https://github.com/edx/codejail) to use 
AppArmor to run Python securely.

For grading Python programs, we use a unit-test like series of 
challenges.  The student writes problems as functions (or classes), and 
we execute them with unit tests (not literally unittest, but a similar 
idea).  We also tokenize the code to check for simple things like, did 
you use a while loop when the requirement was to write a recursive 
function.  The grading code is not open-source, unfortunately, because 
it is part of the MIT courseware.

--Ned.
> Laszlo
>
> On Fri, Sep 20, 2013 at 10:08 PM, John Gordon <gordon at panix.com> wrote:
>> In <mailman.195.1379698177.18130.python-list at python.org> Jabba Laci <jabba.laci at gmail.com> writes:
>>
>>> There are several questions:
>>> * What is someone sends an infinite loop? There should be a time limit.
>> You could run the judge as a background process, and kill it after ten
>> seconds if it hasn't finished.
>>
>>> * What is someone sends a malicious code? The script should be run in a
>>> sandbox.
>> You could run the judge from its own account that doesn't have access to
>> anything else.  For extra security, make the judge program itself owned by
>> a separate account (but readable/executable by the judge account.)
>>
>> I suppose you'd have to disable mail access from the judge account too.
>> Not sure how to easily do that.
>>
>> --
>> John Gordon                   A is for Amy, who fell down the stairs
>> gordon at panix.com              B is for Basil, assaulted by bears
>>                                  -- Edward Gorey, "The Gashlycrumb Tinies"
>>
>> --
>> https://mail.python.org/mailman/listinfo/python-list