[IPython-dev] should we make nodb the default hub backend?

[MinRK]

On Mon, Jun 11, 2012 at 7:52 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hey guys,
>
> I'm wondering if we shouldn't make nodb the default... The new cell
> magics and the ability to combiine cython with parallel stuff make it
> so blindingly easy to parallelize existing codes (for certain
> patterns, of course!) that I bet people are *quickly* going to run
> into the problems with memory introduced by the persistent backends.
> I was just now parallelizing some Gibbs sampling code in Cython
> written by Chris Fonnesbeck for a panel at the R conference later this
> week, and all of a sudden noticed my machine grind to a halt.  I had
> only run the code a few times, and I was overwriting the same
> variables over and over, so I wasn't really thinking I'd be in memory
> trouble, but it was the hub's persistence that bit me.
>
> The DB backends are really useful only in more advanced circumstances,
> so I'm thinking that perhaps they should be activated knowingly by
> users who need their features, leaving the 'naive' patterns to behave
> a bit better.  What I'm worried is that right now, a fairly natural
> use pattern degrades badly and somewhat mysteriously (unless you
> really know what's under the hood).
>
> Thoughts?
>

I'm 50-50.  I generally think of NoDB as an optimization, which would
suggest that *it* should be the optional case for people in the know,
rather than the other way around.

I don't view delayed result retrieval as "advanced", but I think has turned
out to be rare, so I would be okay with this.  The one thing it changes is
adding big warnings to the docs, because a huge swath of features would be
totally unavailable by default.

In fact, with NoDB the Hub does very nearly nothing, and can be killed
entirely after engines and clients are connected with ~no consequences.

-MinRK


>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20120611/708115b4/attachment.html>