[SciPy-Dev] using sqlite for storing submissions

[Andreas Hilboll]

On 15.07.2013 18:16, Surya Kasturi wrote:
> 
> 
> 
> On Mon, Jul 15, 2013 at 6:15 PM, Andreas Hilboll <lists at hilboll.de
> <mailto:lists at hilboll.de>> wrote:
> 
>     On 15.07.2013 18:06, Surya Kasturi wrote:
>     >
>     >
>     >
>     > On Mon, Jul 15, 2013 at 5:46 PM, Andreas Hilboll <lists at hilboll.de
>     <mailto:lists at hilboll.de>
>     > <mailto:lists at hilboll.de <mailto:lists at hilboll.de>>> wrote:
>     >
>     >     On 15.07.2013 17:39, Surya Kasturi wrote:
>     >     > Hi all,
>     >     >
>     >     > [idea]
>     >     >
>     >     > Right now we are using Hg for storing submissions inside static
>     >     root..Is
>     >     > not totally good on a cloud platform with limited
>     resources..its not
>     >     > fast too (takes lot of server load)
>     >     >
>     >     > I rather would like to use SQLite separately for storing
>     >     submissions (I
>     >     > agree that using Postgre SQL for it is expensive on cloud)..
>     >     >
>     >     > What do you guys say?
>     >     >
>     >     >
>     >     > we dont have to provide all revisions as a Hg repo when
>     downloading..
>     >     > people can employ some tools to compare if they want...
>     >
>     >     I disagree. Using SQLite in a production server environment causes
>     >     nothing but pain (at least to my experience). I would assume
>     using some
>     >     DVCS for storing revisions should be the most natural way to
>     do so, and
>     >     I would hope that there's some webapp libraries availble to
>     show diff's
>     >     etc, though I don't have any direct experience with this.
>     >
>     >
>     > Showing diff is one idea I had too but we can figure out something
>     > within database too (I guess... dont know for sure).
>     > If we are not a busy site.. with very huge traffic and only using
>     SQLite
>     > partly, it should work nicely
>     >
>     >
>     http://stackoverflow.com/questions/913067/sqlite-as-a-production-database-for-a-low-traffic-site
> 
>     From my experience, it doesn't, even on low-traffic sites. SQLite
>     doesn't handle concurrency, so every so often you get some "database
>     locked" errors. SQLite is good for development stage, but for production
>     sites, don't use it.
> 
>     BTW, I don't see a problem using PostgreSQL as a database server.
>     Especially for low traffic sites, the performance should be fine.
> 
>     >
>     >
>     >     Oh, and I personally don't see the need to have spyce running on a
>     >     "cloud platform" (depending on what you mean by that, of
>     course). If you
>     >     have one single server, then you don't need all the
>     >     distributed-whatsoever and can concentrate on having
>     everything on one
>     >     machine.
>     >
>     > I didnt get you
> 
>     Never mind. Just a pun about this "cloud hype" terminology.
> 
>     Cheers, Andreas.
> 
> 
>     >
>     >
>     >     Just my 2ct.
>     >
>     >     -- Andreas.
>     >
>     >     _______________________________________________
>     >     SciPy-Dev mailing list
>     >     SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>     <mailto:SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>>
>     >     http://mail.scipy.org/mailman/listinfo/scipy-dev
>     >
>     >
>     >
>     >
>     > _______________________________________________
>     > SciPy-Dev mailing list
>     > SciPy-Dev at scipy.org <mailto:SciPy-Dev at scipy.org>
>     > http://mail.scipy.org/mailman/listinfo/scipy-dev
>     >
> 
> 
> I am not worried about current PostgreSQL.. I am thinking to do
> something with static submissions data (as Hg repos) on filesystem..

Well, in your original mail, you said

> I rather would like to use SQLite separately for storing submissions (I agree that using Postgre SQL for it is expensive on cloud)..

But also for file-system interactions with hg or git I wouldn't be
worried with a low-traffic site. After all, gitorious, bitbucket,
github, and the like are all based on DVCS which store their data in the
filesystem. I doubt that github have their repositories in some SQL
database (but might be mistaken here) ...

-- Andreas.