'Lite' Databases (Re: sqlite3 and dates)

[Steven D'Aprano]

rurpy at yahoo.com wrote:

> On 02/18/2015 07:13 PM, Steven D'Aprano wrote:> Chris Angelico wrote:
>>>> SQLite misses some important features that makes it better suited as a
>>>> simple datastore, not much unlike shelve. And network use is not one
>>>> of them, since you can actually implement concurrent sqlite access by
>>>> coding an intermediate layer. Assuming of course we are talking about
>>>> a small number of concurrent users.
>>>
>>> This is what I was saying: it's fine for purposes like Firefox's
>>> bookmarks and settings and such (which I think was what it was
>>> originally developed for?). Not so fine over a network.
>> 
>> The sheer number of Firefox bugs related to its use of SQLite says
>> different.
>>
>> Once upon a time, Firefox's config, bookmarks, etc. were stored in plain
>> text files. At worst they were HTML. You could trivially read them, copy
>> them, restore them and even (if you were careful) edit them using the
>> text editor of your choice. Many a time I was on one machine, wanted to
>> know a bookmark from another machine, so I would ssh across to the other
>> machine and run grep over the bookmark file.
> 
> I agree, I prefer plain text files whenever practical.  But since
> the original discussion was about Sqlite vs Postgresql, not Sqlite
> vs text files, shouldn't the question be: would Firefox be better
> if it required you to install and configure Postgreql instead of
> using Sqlite?

Very possibly. With modern dependency management, it isn't hard to install 
Postgresql:

sudo aptitude postgresql

or equivalent should work. For primitive operating systems with no 
dependency management available, Firefox could come with a simple script 
which downloads, installs, configures and runs Postgresql. (Everything is 
simple for the guy who doesn't have to do it.)

Possible snags:

- Possibly postgresql is simply *too big*. Your 50MB(?) Firefox 
  turns into a 2GB install. I doubt it -- on Debian, postgresql 
  is 17MB installed. But I don't know what dependencies I'm not
  counting.

- Or it is impossible to configure without excessive amounts of
  tech-savvy human intervention. Again, I doubt it. I seem to 
  recall needing to create a Postgresql user and password. But 
  maybe even that is too technical for the average Firefox user.

- Maybe there are nasty interactions between Postgresql listening
  on some port and Windows firewall wanting to block that same port.

Or... and here is a radical thought... maybe Firefox could give you the 
choice of which database? By default, it might use Sqlite, to satisfy the 
desktop users who don't want to think about it. And for those who are 
disturbed by the fragility of Sqlite on a network home directory, you just 
set a config setting in about:config to point at your existing Postgresql 
instance, and never need worry about it again.

The Firefox devs surprise and confuse me. On the one hand, they have 
designed a powerful plug-in architecture, and encourage their user-base to 
use it for all sorts of amazing functionality that they don't want to build 
into the core browser. Yay for this. And on the other hand, they are 
*actively hostile* to any suggestion that using SQlite is not the best and 
*only* appropriate solution to the problem of storing config, bookmarks and 
history. A plug-in database architecture would probably work really well.

> I don't see any evidence that it is Sqlite that is the problem
> as opposed to FF's use (or misuse) of it, or other problems that
> are in FF and have nothing to do with Sqlite.

No no, even Sqlite devs recommend against using it on network drives. The 
Firefox problem is that when FF crashes, as it can do, or if you yank the 
power to the computer and everything dies, if your home directory is on a 
network drive, the database may be left in a locked state, or even 
corrupted. Nothing that the FF developers can do, given the choice of 
Sqlite.


> If Sqlite reliably
> implements ACID semantics as they claim, 

Ah, well "reliably" is a tricky word...

http://stackoverflow.com/questions/788517/sqlite-over-a-network-share


-- 
Steve