py2exe + svn - the final drama

[David Bolen]

Timothy Smith <timothy at open-networks.net> writes:

> what i do is as soon as the update is complete i close the app, but it
> still gives the error, i tried clear() after update and before it, it
> still got the same error. it's be nice to not have to fiddle around
> with the zip file, i really think making py2exe create a dir instead
> of a zip will be much better

Well, you'd still potentially have a problem if the update changed a
file in that directory that hadn't been imported yet, but now depended
on other updated files that your application had already loaded old
versions for.  That's a general problem of updating modules beneath
the executing application, and not really specific to the zip file,
although you're getting a zip importer specific error related to that
in this case.

> here what i do anyway
> 
> if (os.name == 'nt') or (os.name == 'win32'):
>                      client = pysvn.Client()
>             #get current revision number
>             CurrentRev = client.info('').revision.number
>                      Check = client.update('')
>             sys.path_importer_cache.clear()
>                      if Check.number > CurrentRev:
>                 self.Popup('Update installed, click ok and restart
> ','Update installed')
>                 self.Destroy()
>          else:
>             InfoMsg.Update(3,'No Updates needed')

Ah, it's more devious than I thought.  Just pointed out the other
missing piece in his response.

Apparently there are two levels of caching that you've got to defeat
if you change the underlying zip:

1. A global file set of file directory cache information for any opened
   zip file (for all files in the zip).  This is held in the zipimport
   module global _zip_directory_cache.
2. Individual file cached information within the zipimporter instance
   that is kept in the path importer cache (sys.path_importer_cache).
   Technically these are just references to the same individual entries
   being held in the dictionary from (1).

So when you cleared out (2), it still found the cached directory at
the zipimport module level and re-used that information.  But if you only
clear out (1), then the reference in (2) to the directory entries for
currently imported modules remains and still gets used.

I tried testing this with a small zip file that I first built with normal
compression on the entries, then imported one from a running interpreter,
and then rebuilt the zip without compression.  I couldn't seem to get the
precise error you were getting, but doing this gave me a decompression
error upon an attempted reload of an imported module, since the cached
information still thought it was compressed.

After clearing both sys.path_importer_cache and
zipimport._zip_directory_cache, the reload went fine.

It's sort of unfortunate that you have to cheat with the "private"
cache clearing in this case.  It might be worth an enhancement request
to see if zipimport could know to update itself if the timestamp on
the zip file changes, but this is sort of a very specialized scenario.
Although maybe just a public way to cleanly flush import cache
information would be useful.

-- David