dynamic allocation file buffer

[Steven D'Aprano]

On Wed, 10 Sep 2008 11:59:35 -0700, Aaron \"Castironpi\" Brady wrote:

> On Sep 10, 5:24 am, Steven D'Aprano
> <ste... at REMOVE.THIS.cybersource.com.au> wrote:
>> On Wed, 10 Sep 2008 09:26:20 +0200, Fredrik Lundh wrote:
>> > Steven D'Aprano wrote:
>>
>> >> You've created a solution to a problem which (probably) only affects
>> >> a very small number of people, at least judging by your use-cases.
>> >> Who has a 4GB XML file
>>
>> > Getting 4GB XML files from, say, logging processes or databases that
>> > can render their output as XML is not that uncommon.  They're usually
>> > record-oriented, and are intended to be processed as streams.  And
>> > given the right tools, doing that is no harder than doing the same to
>> > a 4GB text file.
>>
>> Fair enough, that's a good point.
>>
>> But would you expect random access to a 4GB XML file? If I've
>> understood what Castironpi is trying for, his primary use case was for
>> people wanting exactly that.
>>
>> --
>> Steven
> 
> Steven,
> 
> Are you claiming that sequential storage is sufficient for small amounts
> of data, and relational db.s are necessary for large amounts?

I'm no longer *claiming* anything, I'm *asking* whether random access to 
a 4GB XML file is something that is credible or useful. It is my 
understanding that XML is particularly ill-suited to random access once 
the amount of data is too large to fit in RAM.

I'm interested in what Fredrik has to say about this, as he's the author 
of ElementTree.



-- 
Steven