creating really big lists

[Dr Mephesto]

On 6 Sep., 01:34, "Delaney, Timothy (Tim)" <tdela... at avaya.com> wrote:
> Hrvoje Niksic wrote:
> > Dr Mephesto <dnh... at googlemail.com> writes:
>
> >> I would like to create a pretty big list of lists; a list 3,000,000
> >> long, each entry containing 5 empty lists. My application will
> >> append data each of the 5 sublists, so they will be of varying
> >> lengths (so no arrays!).
>
> >> Does anyone know the most efficient way to do this? I have tried:
>
> >> list = [[[],[],[],[],[]] for _ in xrange(3000000)]
> > If you're building large data structures and don't need to reclaim
> > cyclical references, I suggest turning GC off, at least during
> > construction.
>
> This is good advice, but another question is whether you really want
> such a list. You may well be better off with a database of some kind -
> they're designed for manipulating large amounts of data.
>
> Tim Delaney

I need some real speed! a database is waaay to slow for the algorithm
im using. and because the sublists are of varying size, i dont think I
can use an array...