[Python-Dev] fixing tests on windows

[Tim Golden]

Trent Nelson wrote:
> 1.  For a given python[_d].exe, always use the same test directory, 
> but hash it against the entire python process path such that it's 
> unique only for a given python instance.

This is to guard against several build runs in parallel, presumably?

> 2.  Make sure every time a test wants a temp file, it gets a 
> unique one in this directory.  Sounds like your TESTFN 
> modification would take care of this for those tests using 
> TESTFN

Not quite; at present TESTFN is constant for one set of test runs. 
It would need to be a function call to change for every test
within the run.

; if TESTFN is the preferred approach then any other 
> tests using tempfile or hardcoding file names would then be 
> changed to use this instead.

Part of the problem (for me): I don't know if it is "the preferred
approach", merely that it's there and widely but not universally
used and is the likely cause of the locking problems we're seeing.
If anything's rolling its own tempfile solution, I'm inclined to
leave that alone. As someone's pointed out elsewhere, the temp
directories are almost certainly not monitored.

> 3.  In order to address tests that either call the 
> test_support methods for removing files/directories, 
> or those that call os.(unlink|remove), do what ever 
> needs to be done to make these no-ops on Windows if an 
> error occurs.

I'm reluctant to monkey patch os.(unlink|remove). I'd rather
provide and encourage the use of a test_support.remove_test_file
function, or even a context manager which did the same, failing
silently if needs be.

> 4.  At the end of the regrtest.py run, create a suspended arbitrary 
> process (i.e. explorer.exe), hijack the main thread context of the 
> process and inject a routine (i.e. overwrite the thread context's 
> instruction pointers) that takes care of removing the temporary 
> directory that was used for testing -- patiently re-trying if 
> any failures occur until all rogue processes also accessing the 
> file(s) stop doing so.  Resume the thread before exiting python.

OK. That was scary. (Interesting site, though).

[... snip ...]

> Guess it all depends on how much effort we want to put into cleaning 
> up our test directory really -- just ensuring tests get a clean area 
> and unique file names each run is the easy part.

Yes. I'm trying desperately hard to stick to a narrow remit of
getting tests to run consistently in the face of messy file-locking
semantics under Windows. I really don't want to wade into the minor
minefield of making all the tests run with consistent temp file
semantics. But I may have to.

TJG