Using Installer with PyXML

[Dan Rolander]

Hi Martin,

Thanks for responding. Here are the specifics--

When I use a script with the statements:

    from xml.sax import saxexts, saxlib, saxutils

and

    parser = saxexts.make_parser("xml.sax.drivers.drv_pyexpat")

the packager (Gordon McMillan's Installer) is able to find xml.sax.saxutils,
but is not able to find xml.sax.saxexts or xml.sax.saxlib which actually
reside in _xmlplus.sax.  I can force builder.py to include the entire
_xmlplus tree by adding a packages=_xmlplus line to the [APPZLIB] section of
the .cfg file, but the exe still fails because it is looking for xml.sax.*:

    ImportError: cannot import name xml.sax.saxexts

When I rename _xmlplus to xml and then run builder again without specifying
any additional packages, the EXE fails because it can't find an available
parser:

  File "c:\program files\python20\_xmlplus\sax\saxexts.py", line 77, in
make_parser
    xml.sax._exceptions.SAXReaderNotAvailable: No parsers found

If I manually import the entire PyXML tree (now named 'xml') by adding a
packages=xml line to the .cfg file, I get a little farther but now the exe
isn't able to find pyexpat.

    ImportError: cannot import name xml.parsers.pyexpat

I then try to manually import pyexpat by adding xml.parsers.pyexpat to the
misc line in the [MYCOLLECT] section, but finder.py is not able to find it:

      File "D:\DOCUME~1\Dan\Software\Python\INSTAL~1\MEInc\Dist\finder.py",
line 121, in identify
      ValueError: xml.parsers.pyexpat.pyd not found

If I changed the .cfg line in [MYCOLLECT] to misc=pyexpat.pyd then the core
\DLLs version of pyexpat.pyd is found and put into the dist directory. Now
when the exe is run I get a Windows error stating that the xmlparse.dll
couldn't be located.

I add xmlparse.dll to the misc= line and then I get an error stating that
the xmltok.dll couldn't be found.

I add xmltok.dll to the misc= line and voila! it works!

I then start to wonder why the exe couldn't find xml.parsers.pyexpat.pyd if
I imported the entire xml tree. I study the builder.log some more and
realize that it only imported .py files and not .pyd files! I tried using
directories= instead of packages= and got the same results.

I re-read Gordon's documentation several times and tried different
combinations of .cfg statements but nothing I tried resulted in a good
import of xml.parsers.pyexpat.

I then replaced the core version of pyexpat.pyd in \DLLs with the PyXML
version and found that I could build a good exe without having to manually
include the xmlparse.dll and xmltok.dll. So my final .cfg file looks like
this:

[MYCOLLECT]
type= COLLECT
name= dist_testsax
bindepends= testsax.py
misc= MYSTANDALONE, pyexpat.pyd
debug = 0
excludes = PyWinTypes20.dll, win32api

[MYSTANDALONE]
type= STANDALONE
name= testsax.exe
script= testsax.py
zlib = APPZLIB
userunw = 0
support = 0
debug = 0

[APPZLIB]
name= testsax.pyz
dependencies= testsax.py
excludes= dospath, posixpath, macpath
directories=xml


Now, for another example...

Another test script has the statement:

    from xml.parsers import pyexpat

and

    parser = pyexpat.ParserCreate()

I start with the _xmlplus directory renamed to xml, because I know that's
necessary, and I build a new standalone installation. This time the pyexpat
file is imported to the dist directory as xml.parsers.pyexpat.pyd but the
exe won't import it:

    ImportError: cannot import name xml.parsers.pyexpat

Renaming the file to pyexpat.pyd does not help.

I add packages=xml to the .cfg file and I still have the same problem.

The only fix I can figure out is to change the import statement to:

    import pyexpat

and that works.


So in summary, my tests lead me to conclude the following...

To use Gordon McMillan's Installer to create standalone executables of
scripts that import modules from the PyXML package, the following must be
done (depending on what modules are actually being used):

1.  Replace the core xml directory with the _xmlplus directory, by renaming
_xmlplus to xml.

2.  Copy the PyXML pyexpat.pyd file from the xml.parsers directory to the
<python_root>\DLLs directory.

3.  If pyexpat is needed, either explicitly import it in your script, or
manually include it in the standalone installation by adding an entry to the
misc line in the COLLECT section of the builder .cfg file.

4.  If importing from xml.sax, manually import the entire PyXML tree (source
files only) by specifying either packages=xml or directories=xml in the PYZ
section of the builder .cfg file.

(I have not even tried using DOM yet, so I'm sure there are more issues
there to be found.)

I am by no means an expert on this, so if anybody understands this better
and can provide simpler workarounds I would appreciate hearing it.

Thanks, and I hope this helps someone!
Dan

----- Original Message -----
From: "Martin v. Loewis" <martin at mira.cs.tu-berlin.de>
To: <dan.rolander at marriott.com>
Cc: <gmcm at hypernet.com>; <python-list at python.org>; <xml-sig at python.org>
Sent: Saturday, January 20, 2001 4:55 AM
Subject: Re: [XML-SIG] Using Installer with PyXML


> > "The problem here has to be the way that the xml library tree is
replacing
> > itself with the _xmlplus tree from the later PyXML distribution.  While
> > runtime re-assigns xml to _xmlplus in the __init__ for xml, the import
> > system used by the installation package can't track that, so it still
looks
> > for the actual module tree it loaded from the Python distribution
beneath
> > the name xml."
>
> I'm not sure I understand the problem. Will the packager refuse (or
> forget) to package the xml package, or will it, at runtime, fail to
> load it?
>
> If it manages to package both xml and _xmlplus: when loading xml, will
> it execute xml/__init__.py? In there, there is an import of _xmlplus.
> Will that succeed? If so, what happens to the lines
>
>             import sys
>             sys.modules[__name__] = _xmlplus
>
> Will __name__ have a value of "xml"? Will the assignment succeed?
>
> Now, suppose we do
>
> from xml.sax import sax2exts
>
> In normal Python, this will look for sys.modules["xml"] and start from
> there. Are you saying the installer does not work that way, or that
> even if it starts from there, it still can't figure out to load
> _xmlplus.sax?
>
> > So the question is, will this adversely impact normal Python operation,
and
> > is there a better way?
>
> No, replacing the Python xml package completely with _xmlplus will
> work just fine - except perhaps for the pyexpat difference.
>
> > The other question I have is... Why are there two different pyexpat.pyd
> > files, one as part of the core 2.0 distribution (at only 25 kb) and the
> > other as part of the PyXML distribution in _xmlplus.parsers (at 124 kb).
I
> > haven't been able to get the large one to work using Installer, but the
> > small core file works fine. What is the difference?
>
> There are two differences: the one from PyXML contains a number of bug
> fixes which are not in Python 2. In addition, it contains a literal
> copy of the expat libraries, so that the expat DLLs in the Python core
> should not be needed anymore.
>
> When you say "get the large one to work", what exactly have you tried,
> and how exactly did it fail?
>
> Regards,
> Martin
>