python3.5 + reportlab + windows again

[Robin Becker]

On 06/10/2015 16:31, Robin Becker wrote:
.........
>
> well it seems someone can build these extensions properly. I used Christoph
> Gohlke's reportlab build and although there are 3 failures in the latest tests I
> don't see any crashes etc etc and all the failures are explainable. Last thing I
> saw from him in respect of pyRXPU was to use extra_compile_args=['/Od']. I will
> try that in the reportlab extension builds and see if the problems go away.

I figured out the problem I was having with windows vs python3.5. It seems that 
something has changed in the way extensions are loaded when using imp to 
override the normal import mechanism.

The following code works OK for python 2.7, 3.3 & 3.4, but fails fairly 
catastrophically when used in a complex way (ie running reportlab's tests). 
Simple usage appears to work ie when I run one test.


My extension loading code looks like this

######################################
def __hack__():
     import sys, os, importlib, imp
     if sys.modules['reportlab'].__path__[0].lower()!=r'c:\code\reportlab': return
     from distutils.util import get_platform

     archname = get_platform()

     _p0=r'c:\repos\reportlab\build\lib.%s-%d.%d' % 
(archname,sys.version_info[0],sys.version_info[1])
     _p1=r'c:\repos\pyRXP\build\lib.%s-%d.%d' % 
(archname,sys.version_info[0],sys.version_info[1])
     _overrides={
             'reportlab.lib._rl_accel':os.path.join(_p0,r'reportlab\lib'),
             'reportlab.graphics._renderPM':os.path.join(_p0,r'reportlab\graphics'),
             'pyRXPU':_p1,
             }
     def find_override(name):
         if name in _overrides and os.path.isdir(_overrides[name]):
             fpd = imp.find_module(name.split('.')[-1],[_overrides[name]])
             try:
                 try:
                     return imp.load_module(name,fpd[0],fpd[1],fpd[2])
                 except:
                     pass
             finally:
                 if fpd[0]: fpd[0].close()

     class RLImporter(object):
         def __init__(self):
             self.module = None

         def find_module(self, name, path=None):
             m = find_override(name)
             if m:
                 m.__loader__ = self
                 self.module = m
                 return self

         def load_module(self, name):
             if not self.module:
                 raise ImportError("Unable to load module %s" % name)
             sys.modules[name] = m = self.module
             self.module = None
             return m
     sys.meta_path.insert(0, RLImporter())
__hack__()
del __hack__
######################################

My guess is that it's not cool any more to use imp.load_module repeatedly or 
that the hack of passing the found module about using a temporary attribute is 
not a good idea.


The intent of the above code is to allow me to build extensions for multiple 
pythons in a single area under c:\repos\reportlab\build and allow loading of 
those with different python versions. I don't want to pollute my repository with 
different extensions so prefer to keep them in the build area (and they change 
very slowly compared to the python code).

Is there some simple sample code that can be used as a model for this. I've 
tried looking at the importlib docs and my brain is bursting. I also looked at
http://chimera.labs.oreilly.com/books/1230000000393/ch10.html, but that seems 
all about normal python models/packages.

I think importlib has split the loading of extensions into a different method, 
and there seem to be assertions about re-using an existing loader.
-- 
Robin Becker