large arbitrary binary data in python

Wayne Cuddy wcuddy at lserv.ja10629.home
Tue Mar 27 11:46:19 EST 2001


I am using linux for this project..

I am currently developing a python script to upgrade an embedded system and
have run into a road block.  I need to contact an arbitrary ftp or http server
and grab a rather large (several megabyte) binary file and an accompanying md5
checksum file.  I need to verify the checksum, then expand the gzipped binary
file and write it to a device with the dd command.

Now, I don't have a filesystem on the embedded system big enough to receive
the binary file.  I do have 512Megs of memory in this system, so can I do this
all in python, and can I do this all in memory.  It is possible to do the
following?

    1) simply read the data into a python binary string
    2) pipe the string to md5sum to verify integrity
    3) pipe the data to a shell command like this "gunzip -dc | dd of=device"

    How will python handle the large amount of binary data?  Is there a limit to
    the size of a binary string in python?

Much thanks in advance,
Wayne




More information about the Python-list mailing list