[IPython-dev] [IPython-user] Development plans update

[Barry Wark]

I'm very interested in helping with a Cocoa frontend. It looks like
ipython/frontend/IPReadline is soon to become my friend. A quick
related question: ipython1.frontend.PendingResult looks like it could
be a twisted.internet.Deferred. Is there a reason not to use Twisted
for this? There's already Twisted integration with all the GUI
toolkits that folks have mentioned (Qt, Wx, GTK, Cocoa). Perhaps a
switch to the IPReadline constructor that sets things up to use
synchronous (for e.g. terminal use) or Twisted InterpreterResults.

barry

On Jan 29, 2008 11:57 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> I'd like to drop a quick update on the current plans, as well as
> propose a few specific steps to take from here on.  It was great to
> hear so much enthusiasm coming from different corners, so it's obvious
> that we must now make good use of such 'capital', lest we lose it to
> other projects :)
>
> - First, an update on the HG situation: Ryan Earl, from Enthought, has
> kindly agreed to help up set up hg on their servers, with a likely
> timeline of mid- to late-February.  In the meantime we'll continue
> operating on our existing SVN, but I'm sure it will be very handy to
> have hg once we start making deeper changes and keeping more parallel
> lines of development running.
>
> The plan is to initially only set up hg without any Trac coupling, so
> we'll continue to use Trac for tickets but will simply point to the hg
> changeset URLs instead of having the convenience of integrated
> source/ticket management.    This is a small, and hopefully temporary
> loss that I'm sure we can live with.  Full Trac integration remains in
> the plans, but for later as time permits.  We may simply modify the
> Trac config to stop showing the 'browse source' part of the tree to
> avoid confusing new visitors.
>
> I spent a few hours today familiarizing myself with hg in more depth,
> and I think we can start with a work model where we serve multiple hg
> repos via hgwebdir, and set the server permissions so that current
> devs can commit to them.  We have a small and well behaved enough team
> that I don't foresee having to implement more granular controls, since
> each of us can just touch only what we're supposed to at any point in
> time.  These repos will serve as the 'official' ones for the public to
> download, contribute patches, etc.  Obviously with hg being fully
> distributed, anyone can simply set up a publicly visible repo to offer
> material for experimental development, and that type of work is
> absolutely welcome.  But I think that having an official repo (or
> group thereof) is a good idea so that newcomers can know what to use
> from the start.
>
> Some references on the matter for the curious:
>
> http://www.selenic.com/mercurial/wiki/index.cgi/PublishingRepositories
> http://www.selenic.com/mercurial/wiki/index.cgi/HgWebDirStepByStep
>
> Comments on hg-based workflows are welcome.
>
>
> - Next, what to do: we've seen many new ideas coming around and offers
> for collaboration, so I think our key priority is to organize things a
> bit for that to happen smoothly.  Up until now we've had basically two
> lines of work proceeding in parallel: the trunk, dutifully maintained
> by Ville and the ipython1 work, mostly done by Brian and Min.  I'm not
> sure what exactly I've been doing :)
>
> Our idea has always been that the work in ipython1 would eventually
> absorb the trunk, since there's no question whatsoever about ever
> dropping the command-line ipython we use today as a tool we offer.
> But with finite resources, and the trunk continuing to evolve, I think
> we might make more definitive progress by following a slightly
> different strategy.  What I'd like to see happen, if all the devs are
> interested, would be to all focus over the next few months on actually
> pushing out a 'new ipython' that would be developed off the current
> ipython1/ codebase, but trying to incorporate, as fast as possible,
> all the existing functionality.
>
> Specifically, this is my proposal:
>
> 1. We push out an 0.8.3 release with a fair amount of care.  We
> actually go through Trac and find anything we can reasonably expect to
> fix and try to close it.  This would likely be the 'end of life' of
> the series.  I'm willing to participate on this work directly, since I
> know the codebase pretty well, but obviously it's not a decision I'll
> push without the agreement of Ville, Jorgen, Walter and the rest of
> the team.
>
> 2. Once this release is out, we really pretty much stop working on the
> trunk altogether.  *All of us*. We put it in pure maintenance mode,
> and only fix critical bugs.  New feature requests get tickets made for
> them, but their fulfillment will actually happen in the new series.
>
> 3. We all start working on getting the code in ipython1/ in a usable
> form.  I expect by the time we get here, the hg repos will be up, so
> we can partition the work in logical components that various teams can
> work on.   I'll outline roughly what those pieces are below, but this
> message is getting long already, so I won't go into too much detail
> yet.  If this plan is agreeable to all, I'll probably start putting
> these ideas as reST docs in the codebase itself, so we can discuss
> them as design documents.
>
> The basic system is a core capable of doing what ipython does today in
> a terminal, but with a cleanly defined API for all of its public
> functionality, and subcomponents well separated.  The raw draft of
> this is already in place:
>
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/frontend
>
> The first order of business would be, using this, to get working:
>
> - a terminal-based, single-process ipython, much like today.
> - in-terminal GUI support like our -Xthread options of today.
> - a gui embedded one, likely Laurent's WX tool.
>
> If the above goals are met, likely it will be because the right
> abstractions will have been made (things like separating prompt and
> readline handling from stdin/out, etc).
>
> At this point, we'd have specific projects that are either already
> functioning but without a 'true ipython backend' or that don't exist
> yet:
>
> - All of the distributed computing facilities that Brian and I are
> very interested in, and for which he's already working on here, in
> addition to all the existing ipython1 machinery:
> http://projects.scipy.org/ipython/ipython/browser/ipythondistarray
>
> - A JavaScript notebook, for which we already have also prototype code:
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/notebook
>
> - The Qt/Cocoa frontends and other ideas that some of you have
> expressed interest in.
>
> - The Leo integration work that has been recently discussed.
>
> - End-user interactive functionality like a better help system (Steve
> Roger's project),
>
> - Integration with Enthought's Envisage
>
> - Etc.
>
> In order to achieve all this, the basic guidelines we have in mind are:
>
> - We'll try to keep compatibility with existing APIs (like ipapi)
> where feasible, but there are no promises.  We'll break backwards
> compatibility wherever we need to in order to do the right thing.
>
> - No untested code.  Between nose, doctests and Twisted's trial (for
> the network parts), we have more than enough for properly testing new
> code.
>
> - Good docstrings using epydoc/reST for everything.
>
> - A reST-based set of manuals from which we can produce at any time
> HTML and PDF documentation.  I'll do the work of exporting the current
> lyx user manual out to reST, but that will be just a start.
>
>
> I'd like to make a release 1.0 of ipython once the 'first order of
> business' items above are reasonably met, even if we have a few
> functionality regressions on the current terminal-based ipython.
> After that, as the various components and subprojects improve, we'll
> continue releasing.  This release would also include at least all of
> the existing ipython1 distributed computing code, since that already
> works quite well for many uses.
>
> Do these sound like reasonable steps to all of you, especially to the
> other devs?  If so, we can get going on Trac ticket triage for the
> 0.8.3 release as best we all can.  I should add that my
> moving/changing jobs will make me more useless than usual for the next
> few weeks, which is why at least I wanted to get this message out.
> I'll do my best for this period, though I'm sure that won't be much.
> But once I'm settled in I intend to commit more time to this effort,
> since the distributed computing aspects of it will be immediately
> needed in my research.
>
> Ok, I better stop.  Thanks to anyone still reading :)  That was long...
>
> Cheers,
>
> f
> _______________________________________________
> IPython-user mailing list
> IPython-user at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>