building an online judge to evaluate Python programs

[Ned Batchelder]

On 9/21/13 3:57 PM, Jabba Laci wrote:
> Hi Ned,
>
> Could you please post here your AppArmor profile for restricted Python scripts?
Laszlo, the instructions are in the README, including the AppArmor 
profile.  It isn't much:

#include <tunables/global>

<SANDENV>/bin/python {
     #include <abstractions/base>
     #include <abstractions/python>

     <SANDENV>/** mr,
     # If you have code that the sandbox must be able to access, add lines
     # pointing to those directories:
     /the/path/to/your/sandbox-packages/** r,

     /tmp/codejail-*/ rix,
     /tmp/codejail-*/** rix,
}

Note that there are other protections beyond AppArmor, setrlimits is also used to limit some resource use.

--Ned.

BTW: Top-posting makes it harder to follow threads of conversations, better form is to add your comments below the person you're replying to.

> Thanks,
>
> Laszlo
>
> On Sat, Sep 21, 2013 at 12:46 AM, Ned Batchelder <ned at nedbatchelder.com> wrote:
>> On 9/20/13 6:26 PM, Jabba Laci wrote:
>>> I just found Docker ( http://docs.docker.io/en/latest/faq/ ). It seems
>>> sandboxing could be done with this easily.
>>
>> At edX, I wrote CodeJail (https://github.com/edx/codejail) to use AppArmor
>> to run Python securely.
>>
>> For grading Python programs, we use a unit-test like series of challenges.
>> The student writes problems as functions (or classes), and we execute them
>> with unit tests (not literally unittest, but a similar idea).  We also
>> tokenize the code to check for simple things like, did you use a while loop
>> when the requirement was to write a recursive function.  The grading code is
>> not open-source, unfortunately, because it is part of the MIT courseware.
>>
>> --Ned.
>>
>>> Laszlo
>>>
>>> On Fri, Sep 20, 2013 at 10:08 PM, John Gordon <gordon at panix.com> wrote:
>>>> In <mailman.195.1379698177.18130.python-list at python.org> Jabba Laci
>>>> <jabba.laci at gmail.com> writes:
>>>>
>>>>> There are several questions:
>>>>> * What is someone sends an infinite loop? There should be a time limit.
>>>> You could run the judge as a background process, and kill it after ten
>>>> seconds if it hasn't finished.
>>>>
>>>>> * What is someone sends a malicious code? The script should be run in a
>>>>> sandbox.
>>>> You could run the judge from its own account that doesn't have access to
>>>> anything else.  For extra security, make the judge program itself owned
>>>> by
>>>> a separate account (but readable/executable by the judge account.)
>>>>
>>>> I suppose you'd have to disable mail access from the judge account too.
>>>> Not sure how to easily do that.
>>>>
>>>> --
>>>> John Gordon                   A is for Amy, who fell down the stairs
>>>> gordon at panix.com              B is for Basil, assaulted by bears
>>>>                                   -- Edward Gorey, "The Gashlycrumb
>>>> Tinies"
>>>>
>>>> --
>>>> https://mail.python.org/mailman/listinfo/python-list
>>