how to write a line in a text file

[James Dennett]

Calvin Spealman wrote:
> On 7/31/05, James Dennett <jdennett at acm.org> wrote:
> 
>>Peter Hansen wrote:
>>
>>
>>>Steven D'Aprano wrote:
>>>
>>>Given that ZODB and PySQLite are simply Python extension modules, which
>>>get bundled by your builder tool and are therefore installed
>>>transparently along with your app by your installer, this is a total
>>>non-issue at least with those packages.
>>>
>>>After all, it's not 1970 any more. ;-)
>>
>>Indeed; since 1970 we learned to prefer straightforward
>>file formats where possible, reserving use of databases
>>for structured data where the extra costs are justified.
>>
>>Sometime maybe databases will get better to the point
>>that we don't need to distinguish so much between them
>>and filesystems, but we're not there yet.  Managing raw
>>files, carefully, still has a place.
>>
>>-- James
> 
> 
> Filesystems are a horrible way to organize information, and even worse
> at structuring it. The mentality that there are any benefits of
> low-overhead the outweigh the benefits of minimal database layers,
> such as ZODB, BSD DB, and SQLite, is a large part of the reason we are
> still stuck with such a mess. Those "extra costs" are so minimal, that
> you wouldn't even notice them, if you hadn't convinced yourself of
> their presense before performing or researching any realy benchmarks.
> A simple RDBMS can be much easier to work with than any flat file
> format, will likely be far faster in processing the data, and has the
> benefit of years of coding making the code that actually reads and
> writes your data as fast and stable as possible.

Read again, and you'll note that I didn't say the costs were
performance related (though in some, just some, situations the
performance cost of using an RDBMS is a reason to avoid it,
and in other cases an RDBMS will be faster than naive file
access).

The reasons for using other formats are things such as better
interoperability, which extends to better future proofing,
easier installations (in some situations -- note again that
there are no hard and fast rules), and the fact that power
users can edit the files by hand (more easily than they can
tweak information in databases in many, but not all, cases).
It can also (again, in some cases) be easier to port filesystem
based storage between systems than to move database access code.

Oh, and please don't assume that I make my decisions without
measuring performance.  If you store enough millions, or
billions, of items, then performance is worth measuring.  I've
found situations where using various database engines made sense,
I've found situations where flat files made more sense, and I've
seen cases where either was used where the other might have been
a better choice.

-- James