From thomas.heller@ion-tof.com Tue Jan 30 19:23:17 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Tue, 30 Jan 2001 20:23:17 +0100 Subject: [Import-sig] Imputil Message-ID: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> I have the impression that we cannot currently import cPickle (for example) by imputil: cPickle uses PyImport_ImportModule(name) instead of PyImport_Import(name) which would then in turn use the import hook function. cPickle imports copy_reg, string, types. Is this observation correct? Regards, Thomas From mal@lemburg.com Wed Jan 31 14:25:06 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 31 Jan 2001 15:25:06 +0100 Subject: [Import-sig] Imputil References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> Message-ID: <3A782042.B845D04E@lemburg.com> Thomas Heller wrote: > > I have the impression that we cannot currently > import cPickle (for example) by imputil: > > cPickle uses PyImport_ImportModule(name) instead of > PyImport_Import(name) which would then in turn use the > import hook function. > cPickle imports copy_reg, string, types. > > Is this observation correct? >From looking at the cPickle.c code: no. cPickle comes with its own PyImport_Import() API which looks very similar to the one in Python/import.c (probably there for historic reasons). The cPickle implementation uses __import__ as well, so there should be no problem. The various uses of PyImport_ImportModule() in cPickle are only used for standard modules -- I don't think these pose much of a problem w/r to imputil and other import hooks, since these are either available using the standard import mechanism or as frozen modules compiled into the core. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/ From thomas.heller@ion-tof.com Wed Jan 31 14:52:42 2001 From: thomas.heller@ion-tof.com (Thomas Heller) Date: Wed, 31 Jan 2001 15:52:42 +0100 Subject: [Import-sig] Imputil References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> <3A782042.B845D04E@lemburg.com> Message-ID: <007d01c08b95$77d241f0$e000a8c0@thomasnotebook> "M.-A. Lemburg" wrote: > Thomas Heller wrote: > > > > I have the impression that we cannot currently > > import cPickle (for example) by imputil: > > > > cPickle uses PyImport_ImportModule(name) instead of > > PyImport_Import(name) which would then in turn use the > > import hook function. > > cPickle imports copy_reg, string, types. > > > > Is this observation correct? > > >From looking at the cPickle.c code: no. cPickle comes with its > own PyImport_Import() API which looks very similar to the > one in Python/import.c (probably there for historic reasons). > The cPickle implementation uses __import__ as well, so there > should be no problem. > > The various uses of PyImport_ImportModule() in cPickle are only > used for standard modules -- I don't think these pose much of > a problem w/r to imputil and other import hooks, since these > are either available using the standard import mechanism or > as frozen modules compiled into the core. No, these are exactly the problem (maybe only my problem?). If the standard import mechanism is NOT available (think of Gordon's installer or my py2exe project) because everything is loaded from an archive, importing cPickle will fail if you do: import cPickle import copy_reg, string, types but succeed if you do: import copy_reg, string, types import cPickle because in the latter case copy_reg, string, types are already in sys.modules when 'import cPickle' is executed. Thomas From mal@lemburg.com Wed Jan 31 15:05:39 2001 From: mal@lemburg.com (M.-A. Lemburg) Date: Wed, 31 Jan 2001 16:05:39 +0100 Subject: [Import-sig] Imputil References: <00c301c08af2$19e85870$e000a8c0@thomasnotebook> <3A782042.B845D04E@lemburg.com> <007d01c08b95$77d241f0$e000a8c0@thomasnotebook> Message-ID: <3A7829C3.488DE50A@lemburg.com> Thomas Heller wrote: > > "M.-A. Lemburg" wrote: > > Thomas Heller wrote: > > > > > > I have the impression that we cannot currently > > > import cPickle (for example) by imputil: > > > > > > cPickle uses PyImport_ImportModule(name) instead of > > > PyImport_Import(name) which would then in turn use the > > > import hook function. > > > cPickle imports copy_reg, string, types. > > > > > > Is this observation correct? > > > > >From looking at the cPickle.c code: no. cPickle comes with its > > own PyImport_Import() API which looks very similar to the > > one in Python/import.c (probably there for historic reasons). > > The cPickle implementation uses __import__ as well, so there > > should be no problem. > > > > The various uses of PyImport_ImportModule() in cPickle are only > > used for standard modules -- I don't think these pose much of > > a problem w/r to imputil and other import hooks, since these > > are either available using the standard import mechanism or > > as frozen modules compiled into the core. > > No, these are exactly the problem (maybe only my problem?). > If the standard import mechanism is NOT available (think of > Gordon's installer or my py2exe project) because everything is loaded > from an archive, importing cPickle will fail if you do: > > import cPickle > import copy_reg, string, types > > but succeed if you do: > > import copy_reg, string, types > import cPickle > > because in the latter case copy_reg, string, types are > already in sys.modules when 'import cPickle' is executed. Perhaps you should add some "boot" code to py2exe then or freeze the standard lib into the interpreter itself ?! (mxCGIPython works this way.) Changing cPickle.c doesn't help here: the PyImport_ImportModule() API would have to be made __import__ aware to fix this problem because a lot of Python code including the core itself uses this more direct import API. OTOH, it may sometimes be necessary to explicitly use the direct and builtin import mechanism (e.g. during Python startup)... so perhaps the playing with PyImport_ImportModule() isn't such a good idea after all. -- Marc-Andre Lemburg ______________________________________________________________________ Company: http://www.egenix.com/ Consulting: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/