optimizing memory utilization

[John Lenton]

On Tue, Sep 14, 2004 at 04:39:49AM +0000, Anon wrote:
> Hello all,
> 
> I'm hoping for some guidance here...  I am a c/c++ "expert", but a
> complete python virgin.  I'm trying to create a program that loads in the
> entire FreeDB database (excluding the CDDBID itself) and uses this
> "database" for other subsequent processing.  The problem is, I'm running
> out of memory on a linux RH8 box with 512MB.  The FreeDB database that I'm
> trying to load is presently consisting of two "CSV" files.  The first file
> contains a "list" of albums with artist name and an arbitrary sequential
> album ID an the CDDBID (an ascii-hex representation of a 32-bit value).
> The second file contains a list of all of the tracks on each of the
> albums, crossreferenced via the album ID.  When I load into memory, I
> create a python list where each entry in the list is itself a list
> representing the data for a given album.  The album data list consists of
> a small handful of text items like the album title, author, genre, and
> year, as well as a list which itself contains a list for each of the track
> on the album.
> 
> [[<Alb1ID#>, '<Alb1Artist>', '<Alb1Title>', '<Alb1Genre>','<Alb1Year>',
>   [["Track1", 1], ["Track2", 2], ["Track3", 3], ..., ["TrackN",N]],
>  [<Alb2ID#>, '<Alb2Artist>', '<Alb2Title>', '<Alb2Genre>','<Alb2Year>',
>   [["Track1", 1], ["Track2", 2], ["Track3", 3], ..., ["TrackN",N]],
>     ...
>  [<AlbNID#>, '<AlbNArtist>', '<AlbNTitle>', '<AlbNGenre>','<AlbNYear>',
>   [["Track1", 1], ["Track2", 2], ["Track3", 3], ..., ["TrackN",N]]]]

silly question: have you looked into using tuples instead of lists for
the inner objects? They're supposed to be more memory efficient,
although I have found that that isn't necessarily the case.

-- 
John Lenton (john at grulic.org.ar) -- Random fortune:
You're a card which will have to be dealt with.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 196 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/python-list/attachments/20040915/06d1f85d/attachment.sig>