on perhaps unloading modules?

[Martin Di Paola]

This may not answer your question but it may provide an alternative 
solution.

I had the same challenge that you an year ago so may be my solution will 
work for you too.

Imagine that you have a Markdown file that *documents* the expected 
results.

--8<---------------cut here---------------start------------->8---
This is the final exam, good luck!

First I'm going to load your code (the student's code):

```python
>>> import student
```

Let's see if you programmed correctly a sort algorithm

```python
>>> data = [3, 2, 1, 3, 1, 9]
>>> student.sort_numbers(data)
[1, 1, 2, 3, 3, 9]
```

Let's now if you can choose the correct answer:

```python
>>> t = ["foo", "bar", "baz"]
>>> student.question1(t)
"baz"
```
--8<---------------cut here---------------end--------------->8---

Now you can run the snippets of code with:

   byexample -l python the_markdown_file.md

What byexample does is to run the Python code, capture the output and 
compare it with the expected result.

In the above example "student.sort_numbers" must return the list sorted.  
That output is compared by byexample with the list written below.

Advantages? Each byexample run is independent of the other and the 
snippet of codes are executed in a separated Python process. byexample 
takes care of the IPC.

I don't know the details of your questions so I'm not sure if byexample 
will be the tool for you. In my case I evaluate my students giving them 
the Markdown and asking them to code the functions so they return the 
expected values.

Depending of how many students you have you may considere to complement 
this with INGInious. It is designed to run students' assignments 
assuming nothing on the untrusted code.

Links:

https://byexamples.github.io/byexample/
https://docs.inginious.org/en/v0.7/


On Sun, Aug 15, 2021 at 12:09:58PM -0300, Hope Rouselle wrote:
>Hope Rouselle <hrouselle at jevedi.com> writes:
>
>[...]
>
>> Of course, you want to see the code.  I need to work on producing a
>> small example.  Perhaps I will even answer my own question when I do.
>
>[...]
>
>Here's a small-enough case.  We have two students here.  One is called
>student.py and the other is called other.py.  They both get question 1
>wrong, but they --- by definition --- get question 2 right.  Each
>question is worth 10 points, so they both should get losses = 10.
>
>(*) Student student.py
>
>--8<---------------cut here---------------start------------->8---
>def question1(t): # right answer is t[2]
>  return t[1] # lack of attention, wrong answer
>--8<---------------cut here---------------end--------------->8---
>
>(*) Student other.py
>
>--8<---------------cut here---------------start------------->8---
>def question1(t): # right answer is t[2]
>  return t[0] # also lack of attention, wrong answer
>--8<---------------cut here---------------end--------------->8---
>
>(*) Grading
>
>All is good on first run.
>
>Python 3.5.2 [...] on win32
>[...]
>>>> reproducible_problem()
>student.py, total losses 10
>other.py, total losses 10
>
>The the problem:
>
>>>> reproducible_problem()
>student.py, total losses 0
>other.py, total losses 0
>
>They lose nothing because both modules are now permanently modified.
>
>(*) The code of grading.py
>
>--8<---------------cut here---------------start------------->8---
># -*- mode: python; python-indent-offset: 2 -*-
>def key_question1(t):
>  # Pretty simple.  Student must just return index 2 of a tuple.
>  return t[2]
>
>def reproducible_problem(): # grade all students
>  okay, m = get_student_module("student.py")
>  r = grade_student(m)
>  print("student.py, total losses", r) # should be 10
>  okay, m = get_student_module("other.py")
>  r = grade_student(m)
>  print("other.py, total losses", r) # should be 10
>
>def grade_student(m): # grades a single student
>  losses  = question1_verifier(m)
>  losses += question2_verifier(m)
>  return losses
>
>def question1_verifier(m):
>  losses = 0
>  if m.question1( (0, 1, 2, 3) ) != 2: # wrong answer
>    losses = 10
>  return losses
>
>def question2_verifier(m):
>  m.question1 = key_question1
>  # To grade question 2, we overwrite the student's module by giving
>  # it the key_question1 procedure.  This way we are able to let the
>  # student get question 2 even if s/he got question 1 incorrect.
>  losses = 0
>  return losses
>
>def get_student_module(fname):
>  from importlib import import_module
>  mod_name = basename(fname)
>  try:
>    student = import_module(mod_name)
>  except Exception as e:
>    return False, str(e)
>  return True, student
>
>def basename(fname): # drop the the .py extension
>  return "".join(fname.split(".")[ : -1])
>--8<---------------cut here---------------end--------------->8---
>-- 
>https://mail.python.org/mailman/listinfo/python-list