Get the entire file in a variable - error

[John Machin]

On 14 Apr 2005 07:37:11 -0700, martijn at gamecreators.nl wrote:

>H!
>
>I'm using a database and now I want to compress a file and put it into
>the database.
>
>So I'm using gzip because php can open the gzip file's.
>The only problem is saving the file into the database.
>
>The function below does this:
>- gzip the file [oke]
>- get all the bytes with tst.getvalue() [error]
>I only get the first line.
>
>I have the same problem when I try it with file.open(), .read().
>
>"how to get all the binary data in a variable to put that in a database
>LONG field?"
>
>Thank's
>
>def compressFILE(sid,filedata):
>    tst = StringIO.StringIO()
>
>    #tmp
>    zip = gzip.GzipFile(str(sid)+'.gz','w',5,tst)

According to the docs, it will convert your 'w' into 'wb'. That
shouldn't be your problem. However just because I'm paranoid doesn't
mean someone isn't out to get you, so change it to 'wb' anyway. It's
an extremely good habit to put the 'b' on whenever you are reading or
writing binary data.

>    zip.write(filedata)
>
>    #put every byte in a database
>    print tst.getvalue()

Here's your problem, being over-eager. Compressors that allow you to
feed them their input a spoonful at a time will maintain a large
amount of "state" and won't finish the job until you call their
close() method. Just like if you have a process writing a file to
disk, a second process trying to read the file before the first
process closes it is not in general guaranteed to see *any* of the
file's content, let alone all of it.

>
>    zip.close()
>    tst.close()

OK, now you can inspect the contents of "tst", write it to your
database, and then (and only then) worry about BLOBs and embedded
nulls and other nasties.

HTH,

John