Large data arrays?

[Scott David Daniels]

Ole Streicher wrote:
> Nick Craig-Wood <nick at craig-wood.com> writes:
>> mmaps come out of your applications memory space, so out of that 3 GB
>> limit.  You don't need that much RAM of course but it does use up
>> address space.
> 
> Hmm. So I have no chance to use >= 2 of these arrays simultaniously?
> ... That would be a solution, but it takes twice the amount of address
> space (which seems already to be the limiting factor). In my case (1.6
> GB per array), I could even not use one array. 

Looks like you are out of luck here.  I believe looking to do this
without managing the I/O yourself is likely to involve someone else's
trade-offs, which likely fail to match your use case.  You might check
     gmane.comp.python.scientific.user
which I think is gmane's  mirror of comp.python.scipy.user, but I'm
not certain of the mailing list name.  Certainly the people over there
regularly deal with accessing large data.

--Scott David Daniels
Scott.Daniels at Acm.Org