[Tutor] Shelve del not reducing file size

[Andreas Kostyrka]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

That's in the nature of the underlying database library used for the
shelve. Most dbm implemention on Unix are using a sparse file that is
used as hashmap. E.g. compare ls -l versus du of the database file.

Now if this is the case, there is no way (with the exception of
rewriting the whole file cleverly from scratch, e.g. with cp
- --sparse=always) to release the unused space. (I don't think that even
Linux detects 0 byte filled pages and frees them.)

Andreas

Barton David wrote:
> Hi,
>  
> I've hit a snag with Python's shelve module. By way of example...
>  
> _____
> from shelve import DbfilenameShelf as StoreFile
> import os
>  
> sf=StoreFile("mytest.db",writeback=False) # but same problem if
> writeback=True
> for i in range(10000):
>     sf[str(i)]="TESTOBJECT"
> sf.sync()
> print len(sf)
> sf.close()
> predeletesize=os.path.getsize("mytest.db")
> print predeletesize
>  
> sf=StoreFile("mytest.db",writeback=False) # but same problem if
> writeback=True
> for i in range(5000):
>     del sf[str(i)]
> sf.sync()
> print len(sf)
> sf.close()
> postdeletesize=os.path.getsize("mytest.db")
> print postdeletesize
> _____
>  
> So why, when I run this, does predeletesize!=postdeletesize?
> I gather that with most database types you have to invoke a special
> command like VACUUM to perform cleanups after table deletions etc
> so, since shelve uses database backends, is this related to that? Is
> there a way to tell (e.g.) a DBfilenameShelf to do this? Or am I stuck
> with having to delete the entire file and save it again?
>  
>  
> Running PythonWin 2.5.1 on XP.
>  
> thanks
> Dave
> 
> This message has been checked for viruses but the contents of an
> attachment may still contain software viruses, which could damage your
> computer system: you are advised to perform your own checks. Email
> communications with the University of Nottingham may be monitored as
> permitted by UK legislation.
> 
> 
> ------------------------------------------------------------------------
> 
> _______________________________________________
> Tutor maillist  -  Tutor at python.org
> http://mail.python.org/mailman/listinfo/tutor
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFGqb+nHJdudm4KnO0RAtxDAKDCcZUJ8uy6bGJ1mL/kUEnswUL3oACg3ihz
iQjQTZFdetdzYME6XmrfXRQ=
=Xxwv
-----END PGP SIGNATURE-----