Shelve problem: why so big?

[Geert-Jan Giezeman]

I am trying to create a shelve from a table of 1100 by 1100 floating
point numbers, which I want to store as 1100 lists of length 1100,
indexed by a key. Unfortunately, I get an error when I try to create
this:
bsddb.error; (22, Invalid argument)

When I store smaller lists (I tried up to size 1100*600), there is no
problem.

My guess is that this has to do with the file size. Even with the
shorter list the size of the file is 1.01 GB (1,090,331,648 bytes).
Continuing in guessing mode, it could be that at a certain moment the
size of the file is doubled and that this fails. (I don't know exactly
how large files can get on an NTFS file system, but a 2GB maximum does
not sound unreasonable).

Now on to the questions:

1) Are those large filesizes for shelves normal? A bare minimum for
storing the data would be 8*1100*1100 = 9,681,000 bytes. So pickling
and database overhead would be a factor 100.

2)Are there ways to circumvent those problems? Use a different
database (I like shelves, though), ...