[SciPy-dev] distutils and persistent system information

[Pearu Peterson]

On Thu, 29 Apr 2004, Prabhu Ramachandran wrote:

> Hi,
> 
> I have a question on how do get something optimally working for SciPy.
> Very specifically, it concerns weave's wxPython/wxWidgets capability
> and VTK Python<->C++ capabilities.  The issues:
> 
>  1. Both wxPython and VTK (and any other converters that will be added
>     in future) require knowledge of the users setup.  Namely, where
>     wx/VTK are installed, where the libraries are etc.  Expecting the
>     user to supply these at run-time is a possibility but it is a pain
>     and leads to problems if the inlined code is moved between
>     machines/platforms.
> 
>  2. The configuration needs to be on a per installation basis,
>     preferably one that is run at build time.  Detecting the paths at
>     run-time for each inlined call would be slow.
> 
>  3. It would be ideal if the configuration could be changed by a sys
>     admin (or the user) if any of the paths changed after the install.
>     The key file to change should of course be well documented in the
>     README.  Better still a script to update the data could be
>     provided.
> 
> So here are my questions:
> 
>  1. Is the above possible currently?
> 
>  2. If it isn't possible is it a good idea to add this functionality?
> 
> The functionality in system_info looks like it might do the trick but
> from what I understand it only allows for build time configuration and
> it does not save any of the discovered information that can be used
> later at run-time.  I would think that a "config.cfg" file for site
> specific information would be a good idea.  What do you folks suggest?

Except the last feature in 3., scipy_distutils has capabilities to get
what you aiming at. Namely, system_info can be used to retrive the state 
of the system software and then build_src command could generate a .py 
file for holding this information and to be installed for further 
runtime usage, and it can be latter modified of course.

Actually, I like the idea of saving the build information in installation. 
It helps debugging, users can send this information for feedback, etc.
So, I'd propose to make this feature global to scipy or weave or any other 
standalone package. For example,

  import scipy.config
  scipy.config.get_info('lapack')

would return exactly the same information that system_info.get_info 
returned during the building process.

Here's an example of using build_src to generate a config.py file 
(you'll need the latest scipy_distutils from CVS). I am using linalg setup 
file for that and I hope you'll get a general idea from it:

#File setup_linalg.py
...
def configuration(parent_package='',parent_path=None):
    ...
    lapack_opt = get_info('lapack_opt')
    ...
    def generate_config(extension, build_dir):
        filename = os.path.sep.join(extension.name.split('.'))
        target = join(build_dir,filename+'.py')
        f = open(target,'w')
        f.write('lapack_opt=%r\n' % (lapack_opt))
        f.write('def get_info(name): return globals().get(name,{})\n')
        f.close()
        return target

    ext = Extension(name=dot_join(parent_package,package,'config'),
                    sources=[generate_config])
    config['ext_modules'].append(ext)

    return config

Now, after installing linalg:

  python setup_linalg.py install

one can do in Python:

>>> import linalg.config
>>> linalg.config.get_info('lapack_opt')
{'libraries': ['f77blas', 'cblas', 'atlas', 'lapack'], 'library_dirs': 
['/usr/lib/sse2', '/usr/lib'], 'language': 'f77', 'define_macros': 
[('ATLAS_WITHOUT_LAPACK', None), ('ATLAS_INFO', '"\\"3.6.0\\""')], 
'include_dirs': ['/usr/include', 'build/src']}

Pearu