[Python-Dev] Freezing unicode codecs.

[M.-A. Lemburg]

Finn Bock wrote:
> 
> While porting the unicode API and the encoding modules to JPython I came
> across a problem which may also (or maybe not) exists in CPython.
> 
> jpythonc is a compiler for jpython which try to track dependencies
> between modules in an attempt to detect which modules an application or
> applet uses. I have the impression that some of the freeze tools for
> CPython does something similar.
> 
> A call to unicode("abc", "cp1250") and "abc".encode("cp1250") will cause
> the encoding.cp1250 module to be loaded as a side effect. The freeze
> tools will have a hard time figuring this out by scanning the python
> source.
> 
> For JPython I'm leaning towards making it a requirement that the
> encodings must be loading explicit from somewhere in application. Adding
> 
>    import encoding.cp1250
> 
> somewhere in the application will allow jpythonc to include this python
> module in the frozen application.
> 
> How does CPython solve this?

It doesn't. The design of the codec registry is such that it
uses search functions which then locate and load the codecs.
These search function can implement whatever scheme they desire
for the lookup and also w/r to loading the codec, e.g. they
could get the data from a ZIP archive.

This design was chosen to allow drop-in configuration of the
Python codecs. Applications can easily add new codecs to the
registry by registering a new search function (and without
having to copy files into the encodings Lib subdir).
 
When it comes to making an application freezable, I'd suggest
adding explicit imports to some freeze support module in the
application. There are other occasions where this is needed
too, e.g. for packages using lazy import of modules such
as mx.DateTime.

This module would then make sure freeze.py finds the right
modules to include in its output.

> PS. The latest release of the JPython errata have full unicode support
> and includes the "sre" module and unicode codecs.
> 
>     http://sourceforge.net/project/filelist.php?group_id=1842

Cool :-)
 
> regards,
> finn
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://www.python.org/mailman/listinfo/python-dev

-- 
Marc-Andre Lemburg
______________________________________________________________________
Business:                                      http://www.lemburg.com/
Python Pages:                           http://www.lemburg.com/python/