[Patches] [ python-Patches-813436 ] Scalable zipfile extension
SourceForge.net
noreply at sourceforge.net
Fri Dec 22 19:58:12 CET 2006
Patches item #813436, was opened at 2003-09-27 04:09
Message generated for change (Comment added) made by akuchling
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Performance
Group: Python 2.5
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Marc De Falco (deufeufeu)
Assigned to: Nobody/Anonymous (nobody)
Summary: Scalable zipfile extension
Initial Comment:
Playing around with large zipfiles (> 10000 files),
I've encountered big loading time, even if after having
loaded it I use only 30 files in it.
So I've introduced a differed parameter to the
Zipfile.__init__ in order to load headers on-demand.
As it's not a really good idea to activated it for all
zip it defaults to False.
I've updated the documentation too.
Thx and keep the good work ;)
P.S. : Dunno if it can be added to 2.3 or have to be
included in 2.4, so I've choosed 2.4 group.
----------------------------------------------------------------------
>Comment By: A.M. Kuchling (akuchling)
Date: 2006-12-22 13:58
Message:
Logged In: YES
user_id=11375
Originator: NO
According to
http://mail.python.org/pipermail/python-dev/2006-November/069969.html, the
author of the zipfile rewrite isn't quite happy with it. It doesn't look
like the new module will be API-compatible with zipfile, so I think this
patch should still be considered for inclusion.
----------------------------------------------------------------------
Comment By: Ronald Oussoren (ronaldoussoren)
Date: 2006-05-27 14:22
Message:
Logged In: YES
user_id=580910
Patch [1446489 ] zipfile: support for ZIP64 also addresses this as a
side-
effect of adding support ZIP64 support (for very big zipfiles).
BTW. I don't quite understand why this patch is put on hold just because a
rewrite of the zipfile module is planned.
W.r.t. this patch: why is the on-demand loading optional? Loading the
per-file
headers when the zipfile is opened is not necessary for normal operation,
the
current zipfile module is basically doing a full verify of the zipfile on
all
occassions. This isn't necessary for normal operation and I don't think
the
infozip tools do this (probably because verification is very expensive).
----------------------------------------------------------------------
Comment By: Sean Reifschneider (jafo)
Date: 2006-05-25 10:38
Message:
Logged In: YES
user_id=81797
Actually, we'll leave it open until the Summer of Code
implementation is completed and accepted.
Sean
----------------------------------------------------------------------
Comment By: Sean Reifschneider (jafo)
Date: 2006-05-25 10:36
Message:
Logged In: YES
user_id=81797
There is a summer of code project to re-write the zipfile
module, so this patch is moot.
Sean
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=305470&aid=813436&group_id=5470
More information about the Patches
mailing list