[IronPython] config files vs. env vars and site.py

[J. Merrill]

I'm somewhat surprised that neither your 1 and 2 suggest the possibility of my using compiled DLLs created from one or more Python modules.  That is, if I am going to deploy a number of Python-coded executables (your 1) and/or DLLs for use by others (your 2), and each of them references the same set of N imported Python modules (some standard ones included in the IronPython distribution, some built by me or others), why should the code of those N modules have to be included (as MSI bytecode) in each of my EXEs and DLLs?  That's not what would happen in CPython.

I want the ability to write my source code the way we always have, referencing imported modules by name (without any reference to the physical location of the source file within my code), and later have a way to group together any number of Python modules into a .NET DLL assembly that's intended for use by IronPython code and not necessarily by non-Python code.  (The pre-.NET term for that would be "to link them into a library.")  I'll call the result a "Python-module-collection DLL" or PMCD in this discussion.

When I compile something for use outside the Python environment (as an EXE or for-others DLL), I should be able to have the Python import statements reference the code of the Python modules I've included within PMCDs, and only include the MSI bytecode (in the resulting EXE/DLL) of those Python modules that aren't found in any PMCD that I tell the compiler to look at.

That's the way other .NET languages work -- you compile while referencing other assemblies using e.g. a /r commandline parameter -- but with the Python-esque feature that if your code imports a Python module that isn't found in any of the referenced assemblies, instead of failing the compile (as would happen in C# or VB.Net) it will locate the module's source code and include the resulting MSI bytecode in the result.

For full flexibility, I should be able to specify explicitly that I want to ignore the code for a particular module within a referenced PMCD (causing the source for that module to be compiled into my EXE/DLL), perhaps by placing a /r reference to particular a Python source module before a reference to a PMCD containing a module with the same name.

It's desirable to be able to build an "all-in-one" EXE or DLL in some situations, but it's also handy to be able to distribute "library" functionality separately.

Note that the manifest of any for-others EXEs/DLLs will reference the PMCDs explicitly for any externally-defined functionality, so AFAICT we're following all the normal .NET rules.  All that's different is that you need not produce a full list of "referenced stuff" when you compile, provided that Python source modules are available for anything not otherwise visible to the compiler.

At 05:03 PM 5/4/2005, Jim Hugunin wrote
>Right now, IronPython-0.7.* isn't well designed for building Python applications to be deployed.  It's primarily a tool for developer use.  Part of fixing this is making configuration nicer; however, I'm not sure that config files will be the right final answer.  Here's how I see people wanting to deploy IronPython code.
>
>1. A single pre-compiled self-contained MyPythonApp.exe.  We will be providing a static compiler to build this based on walking the graph of imported modules and including compiled forms of any other Python modules you want to pass to it explicitly.  From the outside this will be indistinguishable from any other CLI program.
>
>2. As a self-contained MyPythonLibrary.dll to include in C#/VB/F#/... applications.  This similarly wants the static compiler that produces a nicely pre-compiled self-contained bundle.
>
>3. As an embedded scripting language in a CLI application.  In this case the application will be in control through PythonEngine.  Here, I think that the application should take on the responsibility for configuration based on its own config mechanisms and can explicitly set sys.path and perform other configuration before running any scripts.  We want to make this as easy as possible, possibly with some helper methods on PythonEngine; however, I think that it's pretty easy right now for an embedding application.
>
>Unless you have a really compelling short-term need, I'd like to wait until we get the static compiler for cases 1 and 2 built and then see if there's still need for .config file support or not.  I expect this will be available in a small number of months.
>
>Thanks - Jim


J. Merrill / Analytical Software Corp