From erik.tollerud at gmail.com  Wed Feb  3 15:58:32 2010
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 3 Feb 2010 12:58:32 -0800
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com> 
	<db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
Message-ID: <fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>

I encountered some problems with isympy as well, but managed to get it
to work (at least for my purposes) by changing /usr/local/bin/isympy
(or wherever your script installs the isympy script) to just:

#!/bin/bash
ipython -p sympy


and then I put a ipython_config_sympy.py file in the .ipython
directory with the following:


c = get_config()

# This can be used at any point in a config file to load a sub config
# and merge it into the current one.
load_subconfig('ipython_config.py')

lines = """
from __future__ import division
from sympy import *
x, y, z = symbols('xyz')
k, m, n = symbols('kmn', integer=True)
f, g, h = map(Function, 'fgh')
"""

# You have to make sure that attributes that are containers already
# exist before using them.  Simple assigning a new list will override
# all previous values.
if hasattr(c.Global, 'exec_lines'):
    c.Global.exec_lines.append(lines)
else:
    c.Global.exec_lines = [lines]



On Fri, Jan 22, 2010 at 4:58 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Fri, Jan 22, 2010 at 1:20 PM, G?khan Sever <gokhansever at gmail.com> wrote:
>>
>> I can't get the isympy working with the Fernando's trunk. I solved one of
>> the missing attribute error but stuck at the other one:
>
> No, things have changed a bit. ?I'm insanely pressed for time right
> now, the best I suggest is looking at this test code file, around line
> 130:
>
> http://bazaar.launchpad.net/%7Efdo.perez/ipython/trunk-dev/annotate/head%3A/IPython/testing/globalipapp.py#L130
>
>
> That makes the kind of shell you're looking for, sort of.
>
> Sorry not to be able to give you a detailed example right now...
>
> cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Tue Feb  9 11:45:50 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 9 Feb 2010 11:45:50 -0500
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
Message-ID: <db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>

I've realized I let this linger without action, and we probably should
do something about it while it's (relatively) fresh in our minds.

On Wed, Jan 20, 2010 at 4:46 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
>> Now I could issue lp commands and upgrade the repos in lp itself, but
>> that's going to cascade to everyone else, so I don't want to do that
>> without further feedback.
>
> We did it for Leo and it didn't cause problems. People just needed to
> get a fresh copy from launchpad and merge their changes there.
>
> Remote repository upgrade did take quite a bit of time, though
> (possibly hours?).

So, as a concrete proposal, I can update the remote repo say by this
weekend, giving anyone with local changes in any branch time to merge
them.  By Sunday I will start a remote repo upgrade, and we'll all
simply have to run local upgrades before doing any further operations.

Does that sound OK, have I missed anything?

>> It's nuts that bzr would make a change like this with such a nasty
>> cascading effect...
>
> It's actually quite simple to get over this; I agree that launchpad
> admins could have made a little bit more noise about this change.

Yes, it's annoying because they more or less forced projects to deal
with the issue to some extent right away, without any clear head
warning.  But it doesn't matter, we can do it and move on.

>> Ideas from the experts? (I know, I want to switch to git too so we can
>> get off the mess that is bzr and its never ending stream of weird
>> problems, but that's probably too disruptive for now, so let's stick
>> to solutions that use bzr...)
>
> Judging by my experience with git, I don't think it would be much of
> an improvement. Mercurial, on the other hand...

That's a topic for another day, but I'd be curious about your
experiences.  The more I use git the more I like it and I've only used
hg very lightly, so I'd be interested in this.

Cheers,

f


From jorgen.stenarson at bostream.nu  Tue Feb  9 13:07:15 2010
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 09 Feb 2010 19:07:15 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
Message-ID: <4B71A453.1060406@bostream.nu>

Fernando Perez skrev 2010-02-09 17:45:
> So, as a concrete proposal, I can update the remote repo say by this
> weekend, giving anyone with local changes in any branch time to merge
> them.  By Sunday I will start a remote repo upgrade, and we'll all
> simply have to run local upgrades before doing any further operations.
>
> Does that sound OK, have I missed anything?
>
Sounds reasonable to me.

/J?rgen


From nwagner at iam.uni-stuttgart.de  Tue Feb  9 14:22:10 2010
From: nwagner at iam.uni-stuttgart.de (Nils Wagner)
Date: Tue, 09 Feb 2010 20:22:10 +0100
Subject: [IPython-dev] ImportError: No module named ipapp
Message-ID: <web-130131249@uni-stuttgart.de>

Hi all,

I have installed ipython from scratch

bzr branch lp:ipython
cd ipython
python setup.py install --prefix=$HOME/local

If I start ipython

ipython
Traceback (most recent call last):
   File "/home/nwagner/local/bin/ipython", line 4, in 
<module>
     from IPython.core.ipapp import launch_new_instance
ImportError: No module named ipapp

Any idea ?

            Nils


From ellisonbg.net at gmail.com  Tue Feb  9 23:34:48 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 9 Feb 2010 20:34:48 -0800
Subject: [IPython-dev] ImportError: No module named ipapp
In-Reply-To: <web-130131249@uni-stuttgart.de>
References: <web-130131249@uni-stuttgart.de>
Message-ID: <6ce0ac131002092034s7e6c27e6kdf34be3082af137b@mail.gmail.com>

Can you try removing all traces of ipython (both from bin dirs and
site-packages) and reinstall.  It looks like you have multiple versions on
your PATH/PYTHONPATH that are conflicting.

Cheers,

Brian



On Tue, Feb 9, 2010 at 11:22 AM, Nils Wagner
<nwagner at iam.uni-stuttgart.de>wrote:

> Hi all,
>
> I have installed ipython from scratch
>
> bzr branch lp:ipython
> cd ipython
> python setup.py install --prefix=$HOME/local
>
> If I start ipython
>
> ipython
> Traceback (most recent call last):
>   File "/home/nwagner/local/bin/ipython", line 4, in
> <module>
>     from IPython.core.ipapp import launch_new_instance
> ImportError: No module named ipapp
>
> Any idea ?
>
>            Nils
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100209/1725f181/attachment.html>

From ellisonbg.net at gmail.com  Tue Feb  9 23:37:29 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 9 Feb 2010 20:37:29 -0800
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
	<fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>
Message-ID: <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>

Erik,

This approach will also work.  With the current trunk, you shouldn't even
have
to move the ipython_config_sympy.py to .ipython.  But if you do move it
over,
you can change it to your taste.  Can you mention this on the sympy list to
see what approach they want to take?

Cheers,

Brian

I encountered some problems with isympy as well, but managed to get it
> to work (at least for my purposes) by changing /usr/local/bin/isympy
> (or wherever your script installs the isympy script) to just:
>
> #!/bin/bash
> ipython -p sympy
>
>
> and then I put a ipython_config_sympy.py file in the .ipython
> directory with the following:
>
>
> c = get_config()
>
> # This can be used at any point in a config file to load a sub config
> # and merge it into the current one.
> load_subconfig('ipython_config.py')
>
> lines = """
> from __future__ import division
> from sympy import *
> x, y, z = symbols('xyz')
> k, m, n = symbols('kmn', integer=True)
> f, g, h = map(Function, 'fgh')
> """
>
> # You have to make sure that attributes that are containers already
> # exist before using them.  Simple assigning a new list will override
> # all previous values.
> if hasattr(c.Global, 'exec_lines'):
>    c.Global.exec_lines.append(lines)
> else:
>    c.Global.exec_lines = [lines]
>
>
>
> On Fri, Jan 22, 2010 at 4:58 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
> > On Fri, Jan 22, 2010 at 1:20 PM, G?khan Sever <gokhansever at gmail.com>
> wrote:
> >>
> >> I can't get the isympy working with the Fernando's trunk. I solved one
> of
> >> the missing attribute error but stuck at the other one:
> >
> > No, things have changed a bit.  I'm insanely pressed for time right
> > now, the best I suggest is looking at this test code file, around line
> > 130:
> >
> >
> http://bazaar.launchpad.net/%7Efdo.perez/ipython/trunk-dev/annotate/head%3A/IPython/testing/globalipapp.py#L130
> >
> >
> > That makes the kind of shell you're looking for, sort of.
> >
> > Sorry not to be able to give you a detailed example right now...
> >
> > cheers,
> >
> > f
> > _______________________________________________
> > IPython-dev mailing list
> > IPython-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/ipython-dev
> >
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100209/1eb17644/attachment.html>

From fperez.net at gmail.com  Tue Feb  9 23:45:34 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 9 Feb 2010 23:45:34 -0500
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
	<fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>
	<6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
Message-ID: <db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>

On Tue, Feb 9, 2010 at 11:37 PM, Brian Granger <ellisonbg.net at gmail.com> wrote:
> Erik,
>
> This approach will also work.? With the current trunk, you shouldn't even
> have
> to move the ipython_config_sympy.py to .ipython.? But if you do move it
> over,
> you can change it to your taste.? Can you mention this on the sympy list to
> see what approach they want to take?

Erik, it's worth mentioning we've been mulling the idea of allowing
profiles to reside in third-party projects.  Something like

iptyhon -p sympy:isympy

would make IPython look for

sympy/ipython/profile/ipython_config_isympy.py

So that Sympy itself (using sympy just as an example) could ship its
profiles/config files by including a directory named ipython/ in its
own tree, with config/profile directories.

We'd like to get feedback on this idea.  It's not implemented yet,
we've just been thinking about it to minimize the coupling between
project schedules and make it really easy for projects to expose
ipython-based functionality without having to ask their users to copy
profile/config files.

Cheers,

f


From ellisonbg.net at gmail.com  Tue Feb  9 23:59:55 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 9 Feb 2010 20:59:55 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
Message-ID: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>

Hello all,

As many of you know, we have been working hard on IPython 0.11.  One of the
big new features of 0.11 is a refactored integration with GUI event loops.
We are very excited about this as it is much more stable and add new
capabilities, like being able to enable pylab *after* starting IPython (it
is a magic: %pylab).

BUT, as Fernando and I have started to use the IPython 0.11 alpha for our
daily work, we have found some subtle issue with IPython 0.11 and the
current
versions of matplotlib and Enthought tool suite (traits, chaco, mayavi).
All of these packages rely on IPython's previous way of handling GUI
integration.  Because of this, current versions of matplotlib and ets don't
work fully with IPython 0.11.  In some cases, they work OK, but there are a
whole class
of bugs that keep popping up.  We don't want to release IPython 0.11 until
these issues are resolved.

All of these bugs are related to a few aspects of creating the main GUI app
objects and starting event loops.  We think we understand the issues, but we
will
need to work with the devs of matplotlib and ets to decide on how we all
want to handle these things.  Here is where we are at...

Current situation
=============

Both matplotlib and ets have code that tries to:

* See what GUI toolkit is being used
* Get the global App object if it already exists, if not create it.
* See if the main loop is running, if not possibly start it.

All of this logic makes many assumptions about how IPython affects the
answers to these
questions.  Because IPython's GUI support has changed in significant ways,
current matplotlib
and ets make incorrect decisions about these issues (such as trying to start
the event loop
a second time, creating a second main App ojbect, etc.) under IPython 0.11.
This leads to crashes...

Description of GUI support in 0.11
==========================

IPython allows GUI event loops to be run in an interactive IPython session.
This is done using Python's PyOS_InputHook hook which Python calls
when the :func:`raw_input` function is called and is waiting for user input.
IPython has versions of this hook for wx, pyqt4 and pygtk.  When the
inputhook
is called, it iterates the GUI event loop until a user starts to type
again.  When
the user stops typing, the event loop iterates again.  This is how tk works.

When a GUI program is used interactively within IPython, the event loop of
the GUI should *not* be started. This is because, the PyOS_Inputhook itself
is responsible for iterating the GUI event loop.

IPython has facilities for installing the needed input hook for each GUI
toolkit and for creating the needed main GUI application object. Usually,
these main application objects should be created only once and for some
GUI toolkits, special options have to be passed to the application object
to enable it to function properly in IPython.

What we need to decide
===================

We need to answer the following questions:

* Who is responsible for creating the main GUI application object, IPython
  or third parties (matplotlib, enthought.traits, etc.)?

* What is the proper way for third party code to detect if a GUI application
  object has already been created?  If one has been created, how should
  the existing instance be retrieved?

* In a GUI application object has been created, how should third party code
  detect if the GUI event loop is running. It is not sufficient to call the
  relevant function methods in the GUI toolkits (like ``IsMainLoopRunning``)
  because those don't know if the GUI event loop is running through the
  input hook.

* We might need a way for third party code to determine if it is running
  in IPython or not.  Currently, the only way of running GUI code in IPython
  is by using the input hook, but eventually, GUI based versions of IPython
  will allow the GUI event loop in the more traditional manner. We will need
  a way for third party code to distinguish between these two cases.

Thanks for participating in this discussion!

Cheers,

Brian
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100209/e39a4482/attachment.html>

From hans_meine at gmx.net  Wed Feb 10 03:06:56 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 10 Feb 2010 09:06:56 +0100
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
	<db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
Message-ID: <201002100907.02504.hans_meine@gmx.net>

On Mittwoch 10 Februar 2010, Fernando Perez wrote:
> Erik, it's worth mentioning we've been mulling the idea of allowing
> profiles to reside in third-party projects.  Something like
>
> iptyhon -p sympy:isympy
>
> would make IPython look for
>
> sympy/ipython/profile/ipython_config_isympy.py
>
> [...]
> We'd like to get feedback on this idea.

What kind of feedback are you waiting for?

That idea is terrific! :-)

I am sure that *many* projects will be interested in this - I for sure will 
use it with some of my own software!  (OK, I guess it would already be 
possible to install some minimal executable which does the equivalent of 
starting ipython and loading some profile, but this seems to be easier to set 
up.)

Ciao, /  /                                                    .o.
     /--/                                                     ..o
    /  / ANS                                                  ooo

PS: I dislike the change of the executable name though. ;-P
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100210/424b640e/attachment.sig>

From hans_meine at gmx.net  Wed Feb 10 03:26:14 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 10 Feb 2010 09:26:14 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
Message-ID: <201002100926.15886.hans_meine@gmx.net>

On Mittwoch 10 Februar 2010, Brian Granger wrote:
> When a GUI program is used interactively within IPython, the event loop of
> the GUI should *not* be started. This is because, the PyOS_Inputhook itself
> is responsible for iterating the GUI event loop.

From a user's perspective (i.e. not from an MPL/ETS point of view), the above 
also rings bells: I am used to running GUI apps from within ipython, where 
(with older IPython versions) I'll get at most an exception from running 
sys.exit(), but my main window pops up and the app is running while I still 
have the interpreter available.  That's an incredibly important use case to 
me.

Now if the mainloop must not be started with 0.11, how about monkey-patching 
the application classes (i.e. QCoreApplication.exec_) in order to allow 
running unpatched programs when the GUI toolkit integration is active?

(In fact, in our own Qt3-based python terminal, we had monkeypatched 
QApplication in order to also ignore the creation of a second app object.)

-- 
Ciao, /  /                                                    .o.
     /--/                                                     ..o
    /  / ANS                                                  ooo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 197 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100210/fd70da8c/attachment.sig>

From Chris.Barker at noaa.gov  Wed Feb 10 13:15:34 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 10 Feb 2010 10:15:34 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
 Matplotlib/ETS
In-Reply-To: <201002100926.15886.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
Message-ID: <4B72F7C6.9060109@noaa.gov>

Hi folks,

I'm new to the Ipython dev list, and don't really understand the 
internals, but this thread was posted elsewhere to solicit ideas, so 
here I am:

> One of the
> big new features of 0.11 is a refactored integration with GUI event loops.
> We are very excited about this as it is much more stable and add new
> capabilities, like being able to enable pylab *after* starting IPython (it
> is a magic: %pylab).

very nice.

> Current situation
> =============
> 
> Both matplotlib and ets have code that tries to:
> 
> * See what GUI toolkit is being used
> * Get the global App object if it already exists, if not create it.
> * See if the main loop is running, if not possibly start it.

Are these done in order to support IPython? or for other general usage?

> Description of GUI support in 0.11
> ==========================
> 
> IPython allows GUI event loops to be run in an interactive IPython session.
> This is done using Python's PyOS_InputHook hook which Python calls
> when the :func:`raw_input` function is called and is waiting for user input.

So it doesn't use threads at all? that is good news.

> some
> GUI toolkits, special options have to be passed to the application object
> to enable it to function properly in IPython.

so the app itself needs to do something special? that is too bad -- so 
far, I've found IPython works pretty well with my own wxPython apps, 
with nothing changed in my app.

> * Who is responsible for creating the main GUI application object, IPython
>   or third parties (matplotlib, enthought.traits, etc.)?

I think the third parties -- i.e. you really want to be able to use 
IPython with ANY GUI app - not just one written to be used with IPython.

> * What is the proper way for third party code to detect if a GUI application
>   object has already been created?  If one has been created, how should
>   the existing instance be retrieved?

This is why it seems better for IPython NOT to create the GUI app 
object. If it doesn't, would the third party code have to detect it?

hmm, yes it would, or something -- if I want to call my program with 
"run" more than once, then a wxApp has been initialized already, and I 
think you can only do that once per process.

Could Ipython monkey-patch the GUI toolkits to override the app creation 
and mainloop starting? I'm not sure this is possible, but a typical wx 
app might look like this:

app = wx.App()
app.MainLoop()

if wx had been monkey-patched by IPython, the app creation and mainloop 
calls could be overridden with ones that check to see if they have been 
called before, and do something different if so. As for wx, I don't know 
if you can stop and re-start the mainloop anyway.

I'm still confused how this all seems to work for me with IPython 0.9.1!

I hope this is helpful,

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From erik.tollerud at gmail.com  Wed Feb 10 17:11:07 2010
From: erik.tollerud at gmail.com (Erik Tollerud)
Date: Wed, 10 Feb 2010 14:11:07 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <201002100926.15886.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com> 
	<201002100926.15886.hans_meine@gmx.net>
Message-ID: <fbab3e0d1002101411p196a4b8bwc9c88d3bda74a007@mail.gmail.com>

I wholly concur with Hans that running GUI apps simultaneously (e.g.
non-blocking) within IPython is crucial, but it's not necessarily the
case that because IPython doesn't start the GUI it doesn't get started
- as Christopher says, it could just be foisting the responsibility
onto the third party...

On the other hand, there are clearly some problems there (at least in
the current designs) - I've been running on the bzr ipython for a
while now along with traits GUIs, and I've noticed a big change in
behavior from IPython < .11: it used to be when you called
traitedobject.configure_traits() or traitedobject.edit_traits(), the
GUIs would pop up and everything would run fine.  But now,
configure_traits is blocking (and it kills the app at the end, making
edit_traits stop working), while edit_traits has the same behavior as
before, so I use edit_traits...  On the other hand, if I have a
non-interactive script (e.g. not ipython) if I run the exact same
code, edit_traits freaks out because no app object has been created.
For now I've just ignored the problem, as I mostly work in interactive
mode, but it would be way better if I could use a single function call
in either situation.  But that could be resolved in either traits or
IPython, presumably.

As for which layer that belongs on... personally, I think that
anything that requires "two-way" communication between IPython and the
third-party app is a bad idea do to the dangers of version desync.
e.g. if Traits has to know it's in IPython to know that it should
create the app, there's a huge potentially for the code getting
out-of-sync between the two projects whenever some new feature gets
implemented with the inevitable unintended consequences, causing
headaches for all involved.  So I would advocate either
monkey-patching as some of the others here have suggested, or use some
method of passing the information through the GUI toolkit instead of
directly from IPython to the third-party if possible.  I'm really only
familiar with wx, but do the other toolkits have an equivalent
wx.GetApp() function?  And are there similar mechanisms that can leave
messages embedded somewhere in the toolkit as to whether or not the
PyOS_InputHook is calling as opposed to living in the event loop?  If
that's the case, most everything should be left to the 3rd party
app...



On Wed, Feb 10, 2010 at 12:26 AM, Hans Meine <hans_meine at gmx.net> wrote:
> On Mittwoch 10 Februar 2010, Brian Granger wrote:
>> When a GUI program is used interactively within IPython, the event loop of
>> the GUI should *not* be started. This is because, the PyOS_Inputhook itself
>> is responsible for iterating the GUI event loop.
>
> From a user's perspective (i.e. not from an MPL/ETS point of view), the above
> also rings bells: I am used to running GUI apps from within ipython, where
> (with older IPython versions) I'll get at most an exception from running
> sys.exit(), but my main window pops up and the app is running while I still
> have the interpreter available. ?That's an incredibly important use case to
> me.
>
> Now if the mainloop must not be started with 0.11, how about monkey-patching
> the application classes (i.e. QCoreApplication.exec_) in order to allow
> running unpatched programs when the GUI toolkit integration is active?
>
> (In fact, in our own Qt3-based python terminal, we had monkeypatched
> QApplication in order to also ignore the creation of a second app object.)
>
> --
> Ciao, / ?/ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?.o.
> ? ? /--/ ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ..o
> ? ?/ ?/ ANS ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ?ooo
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>


From Chris.Barker at noaa.gov  Wed Feb 10 18:23:26 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Wed, 10 Feb 2010 15:23:26 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
 Matplotlib/ETS
In-Reply-To: <fbab3e0d1002101411p196a4b8bwc9c88d3bda74a007@mail.gmail.com>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
	<fbab3e0d1002101411p196a4b8bwc9c88d3bda74a007@mail.gmail.com>
Message-ID: <4B733FEE.1040003@noaa.gov>

Erik Tollerud wrote:
> As for which layer that belongs on... personally, I think that
> anything that requires "two-way" communication between IPython and the
> third-party app is a bad idea do to the dangers of version desync.

And the fact that there are an infinite number of potential apps to 
communicate with. MPL and ETS-based apps is great, but it would be so 
much greater if IPython could be used with ANY app.

 > or use some
> method of passing the information through the GUI toolkit instead of
> directly from IPython to the third-party if possible.

If monkey patching isn't possible, then this probably is, and there are 
only a handful of GUI toolkits. If IPython were to work well with PyGTK, 
PyQT, wxPython and TKinter (and maybe PyObjC), that would go a long way.

 >  I'm really only familiar with wx,

me too, but I'd be happy to try to help with it.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From termim at gmail.com  Thu Feb 11 01:16:50 2010
From: termim at gmail.com (Mikhail Terekhov)
Date: Thu, 11 Feb 2010 01:16:50 -0500
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
	<fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>
	<6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
	<db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
Message-ID: <12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com>

On Tue, Feb 9, 2010 at 11:45 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> Erik, it's worth mentioning we've been mulling the idea of allowing
> profiles to reside in third-party projects. ?Something like
>
> iptyhon -p sympy:isympy
>
> would make IPython look for
>
> sympy/ipython/profile/ipython_config_isympy.py
>
> So that Sympy itself (using sympy just as an example) could ship its
> profiles/config files by including a directory named ipython/ in its
> own tree, with config/profile directories.
>
> We'd like to get feedback on this idea. ?It's not implemented yet,
> we've just been thinking about it to minimize the coupling between
> project schedules and make it really easy for projects to expose
> ipython-based functionality without having to ask their users to copy
> profile/config files.
>

I'd suggest to allow loading profiles from _arbitrary_ place.
Why it is forbidden to have some specialized profile just somewhere?
For example, I would benefit if it was possible to have profile somewhere
on NFS and my group could use it from different computers. This would
allow centralized profile management in big organizations.

Regards,
-- 
Mikhail Terekhov


From ellisonbg.net at gmail.com  Thu Feb 11 15:07:06 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 11 Feb 2010 12:07:06 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <201002100926.15886.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
Message-ID: <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com>

Hans,

> From a user's perspective (i.e. not from an MPL/ETS point of view), the
> above
> also rings bells: I am used to running GUI apps from within ipython, where
> (with older IPython versions) I'll get at most an exception from running
> sys.exit(), but my main window pops up and the app is running while I still
> have the interpreter available.  That's an incredibly important use case to
> me.

Yes and we absolutely need to support this usage case.

> Now if the mainloop must not be started with 0.11, how about monkey-patching
> the application classes (i.e. QCoreApplication.exec_) in order to allow
> running unpatched programs when the GUI toolkit integration is active?

Fernando and I spend a full day at Scipy 2009 trying this monkey
patching approach.
Summary: it doesn't work.  Full details: each GUI toolkits has additional ways
of running the event loop (other than just calling "run") for a short
period of time.  If everyone avoided these other calls the monkey
patch approach would work.  But *many* people (including matplotlib
and traits) do need to spin the event loop by hand and call these
functions.  All of our monkeypatch approaches broke these things.

> (In fact, in our own Qt3-based python terminal, we had monkeypatched
> QApplication in order to also ignore the creation of a second app object.)

Yes, this type of monkey patch is not too difficult to handle.  But
*any* GUI app should first call getApp or its equivalent before
creating an App object to make sure a 2nd one is created.  The more
difficult one is to figure out if the event loop is running.  This is
difficult because there are so many different ways of running the
event loop and the "isMainLoopRunning" type functions don't account
for all of these.

Cheers,

Brian

> --
> Ciao, /  /                                                    .o.
>      /--/                                                     ..o
>     /  / ANS                                                  ooo
>


From ellisonbg.net at gmail.com  Thu Feb 11 15:19:04 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 11 Feb 2010 12:19:04 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <4B72F7C6.9060109@noaa.gov>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net> <4B72F7C6.9060109@noaa.gov>
Message-ID: <6ce0ac131002111219u568441f9n7d1bd431d685afe4@mail.gmail.com>

Chris,

> I'm new to the Ipython dev list, and don't really understand the
> internals, but this thread was posted elsewhere to solicit ideas, so
> here I am:

Thanks for joining the discussion, it helps alot.

>> Both matplotlib and ets have code that tries to:
>>
>> * See what GUI toolkit is being used
>> * Get the global App object if it already exists, if not create it.
>> * See if the main loop is running, if not possibly start it.
>
> Are these done in order to support IPython? or for other general usage?

Yes, as I understand it that is why this logic exists (for IPython's
-wthread or -pylab).

> So it doesn't use threads at all? that is good news.

Yep, no threads at all.  Other than the bugs we are talking about here
it is *much* more robust and stable.

>> some
>> GUI toolkits, special options have to be passed to the application object
>> to enable it to function properly in IPython.
>
> so the app itself needs to do something special? that is too bad -- so
> far, I've found IPython works pretty well with my own wxPython apps,
> with nothing changed in my app.

Yes, you are understanding the issue well.  We don't see anyway around
this.  If you want to use GUI code both in IPython and outside of
IPython, you will need to have a small amount of custom logic.
Fernando and I spent an entire day at Scipy 2009 trying to figure out
a way to prevent this...no luck.. I am afraid this is a fundamental
limitation unless all the GUI toolkits radically change their APIs.

>> * Who is responsible for creating the main GUI application object, IPython
>>   or third parties (matplotlib, enthought.traits, etc.)?
>
> I think the third parties -- i.e. you really want to be able to use
> IPython with ANY GUI app - not just one written to be used with IPython.

Again, unless qt/wx/gtk change their code, I don't think this is
possible.  Plus even if they did, Ipython still has to work with
current/older versions.  By third party here I am thinking of projects
like matplotlib/enthought that, in general, are commited to
maintaining good integration with Ipython.

>> * What is the proper way for third party code to detect if a GUI
>> application
>>   object has already been created?  If one has been created, how should
>>   the existing instance be retrieved?
>
> This is why it seems better for IPython NOT to create the GUI app
> object. If it doesn't, would the third party code have to detect it?

In general there are reliable ways for third party code to detect if
an app object has been created already.  But, creating an app is not
always trivial.  In some cases (like wx) in order to work properly
with IPython, specific arguments have to be passed (non-default ones)
to the app.  In these cases, it makes sense for IPython to handle it.

> hmm, yes it would, or something -- if I want to call my program with
> "run" more than once, then a wxApp has been initialized already, and I
> think you can only do that once per process.

Yep.

> Could Ipython monkey-patch the GUI toolkits to override the app creation
> and mainloop starting? I'm not sure this is possible, but a typical wx
> app might look like this:
>
> app = wx.App()
> app.MainLoop()

The app creation step is pretty easy to handle.  Most GUI code should
call getApp first to make sure the app has not already  been created
before creating an app from scratch.

The more difficult question is knowing if you should start the main
loop.  For example, if IPython is already running the event loop using
pyos_inputhook, calling app.MainLoop will lead to bad things.
Unfortunately, calling isMainLoopRunning() returns False, so you don't
have a good way of discerning what to do....currently.

> if wx had been monkey-patched by IPython, the app creation and mainloop
> calls could be overridden with ones that check to see if they have been
> called before, and do something different if so. As for wx, I don't know
> if you can stop and re-start the mainloop anyway.

See my other post, but the mainloop monkeypatch approach does not
work.  It breaks qt/wx/etc.

> I'm still confused how this all seems to work for me with IPython 0.9.1!

It doesn't *really* work :)  The pre 0.11 code is not thread safe and
does very subtle tricks to get cross thread signals working....which
is why it crashes often and breaks if you touch the code.

> I hope this is helpful,

Very much, thank you.

Cheers,

Brian

> -Chris
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Thu Feb 11 15:20:04 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 11 Feb 2010 12:20:04 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <4B733FEE.1040003@noaa.gov>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
	<fbab3e0d1002101411p196a4b8bwc9c88d3bda74a007@mail.gmail.com>
	<4B733FEE.1040003@noaa.gov>
Message-ID: <6ce0ac131002111220v4610b49ey4294e65cf5954fc5@mail.gmail.com>

> If monkey patching isn't possible, then this probably is, and there are
> only a handful of GUI toolkits. If IPython were to work well with PyGTK,
> PyQT, wxPython and TKinter (and maybe PyObjC), that would go a long way.

Yes, those are our targets and all of these GUI toolkits support the
pyos_inputhook approach.

Cheers,

Brian

>  >  I'm really only familiar with wx,
>
> me too, but I'd be happy to try to help with it.
>
> -Chris
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From ellisonbg.net at gmail.com  Thu Feb 11 15:27:06 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 11 Feb 2010 12:27:06 -0800
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <201002100907.02504.hans_meine@gmx.net>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
	<db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
	<201002100907.02504.hans_meine@gmx.net>
Message-ID: <6ce0ac131002111227n111df2a6s4a323c56db4621e9@mail.gmail.com>

On 2/10/10, Hans Meine <hans_meine at gmx.net> wrote:
> On Mittwoch 10 Februar 2010, Fernando Perez wrote:
>> Erik, it's worth mentioning we've been mulling the idea of allowing
>> profiles to reside in third-party projects.  Something like
>>
>> iptyhon -p sympy:isympy
>>
>> would make IPython look for
>>
>> sympy/ipython/profile/ipython_config_isympy.py
>>
>> [...]
>> We'd like to get feedback on this idea.
>
> What kind of feedback are you waiting for?
>
> That idea is terrific! :-)

This type of feedback :)

> I am sure that *many* projects will be interested in this - I for sure will
> use it with some of my own software!  (OK, I guess it would already be
> possible to install some minimal executable which does the equivalent of
> starting ipython and loading some profile, but this seems to be easier to
> set
> up.)

Thanks!

Brian

> Ciao, /  /                                                    .o.
>      /--/                                                     ..o
>     /  / ANS                                                  ooo
>
> PS: I dislike the change of the executable name though. ;-P
>


From vivainio at gmail.com  Thu Feb 11 15:27:28 2010
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 11 Feb 2010 22:27:28 +0200
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
Message-ID: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>

On Tue, Feb 9, 2010 at 6:45 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> I've realized I let this linger without action, and we probably should
> do something about it while it's (relatively) fresh in our minds.

You may be into the same entertaining chore soon again, I'm running
Lucid beta's and am seeing:

"""
Doing on-the-fly conversion from <RemoteRepositoryFormat> to
<RepositoryFormat2a>.
This may take some time. Upgrade the repositories to the same format
for better performance.
"""

>> Judging by my experience with git, I don't think it would be much of
>> an improvement. Mercurial, on the other hand...
>
> That's a topic for another day, but I'd be curious about your
> experiences. ?The more I use git the more I like it and I've only used
> hg very lightly, so I'd be interested in this.

I'm using git with gitorious myself, but basically it's just that I'm
unable to feel confident with the program at all (as in, when problems
appear, I'm not sure I can intuitively understand them, like I do with
mercurial and bzr).

I guess it has everything to do with the bad "ui" of git, cryptic
error messages etc. So far, I haven't really hit much problems, but
I've seen a lot of other people do, and on those situations they tend
to need expert support. Not so with hg, people can basically  use it
after they understand dvcs in general.

I also like the fact that it's trivial to put up a "hg serve" session
for quick collaborative hacking, and that you get time based "simple"
revision numbers instead of relying only on hashes.

-- 
Ville M. Vainio
http://tinyurl.com/vainio


From ellisonbg.net at gmail.com  Thu Feb 11 15:28:54 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Thu, 11 Feb 2010 12:28:54 -0800
Subject: [IPython-dev] isympy doesn't work with the new ipython-dev
In-Reply-To: <12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com>
References: <49d6b3501001221320x43700e51m914d366c11b7cbf3@mail.gmail.com>
	<db6b5ecc1001221658m42ff04f7h1d637fb336b85e40@mail.gmail.com>
	<fbab3e0d1002031258p36e6c009k260ebf393cd30081@mail.gmail.com>
	<6ce0ac131002092037j5183b9d0rf2dc636240f065c0@mail.gmail.com>
	<db6b5ecc1002092045m26014e38sd127dff9c716b2be@mail.gmail.com>
	<12aaa0811002102216i2b1be3e0sb6350c0ad5ce41a4@mail.gmail.com>
Message-ID: <6ce0ac131002111228w4213c16bs61ec57e8468cf421@mail.gmail.com>

Profiles can be loaded from arbitrary places currently, but Fernando's
approach would allow profiles to be found using the regular import
mechanisms in IPython.  Basically any module or package in sys.path
could declare the location of an IPython profile using this API.

Cheers,

Brian

> I'd suggest to allow loading profiles from _arbitrary_ place.
> Why it is forbidden to have some specialized profile just somewhere?
> For example, I would benefit if it was possible to have profile somewhere
> on NFS and my group could use it from different computers. This would
> allow centralized profile management in big organizations.
>
> Regards,
> --
> Mikhail Terekhov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From hans_meine at gmx.net  Fri Feb 12 04:31:57 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 10:31:57 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
Message-ID: <201002121031.57250.hans_meine@gmx.net>

On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote:
> [...] Not so with hg, people can basically  use it
> after they understand dvcs in general.
> 
> I also like the fact that it's trivial to put up a "hg serve" session
> for quick collaborative hacking, and that you get time based "simple"
> revision numbers instead of relying only on hashes.

OK, my 2 cents:
About 2 years ago, I made the choice myself.  I went with hg, because I saw 
the majority of (python) projects I watched making this choice.  I did never 
regret it, to the contrary.  From the discussions on this list and on many 
other OSS development lists, my impression is that
- bzr's strenth is launchpad, but it sometimes tries to be too clever and it's 
behavior is not completely trivial to predict, while
- git is for professionals.  Its UI is said to have improved, but still it 
seems to be the swiss army knife which you need to know very well in order to 
use the correct part of it.

hg's big strength is its simplicity, really.  Out of the box (i.e. without any 
third-party extensions), it is incredibly simple to use and understand.  After 
a while you start to become more brave and try out some extensions which offer 
dangerous commands, but I have always stayed in harmony with mercurial.

The mercurial mailing list offers help for everyone, often discussing either 
optimized ways to perform certain tasks, or general DVCS workflows, often with 
posters trying to convince their bosses to make the switch from SVN or 
(shudder) CVS.  But I have never had the impression that there's anything hard 
to understand with mercurial.

Oh, and BTW: the repository format is *incredibly* stable, with only one minor 
(filename mangling) change in all the years.  Basically, Matt chose the 
correct design from the beginning.

I do not plan to convince anybody that hg is better than bzr, but I want to 
report that I am very happy with it (and I am willing to answer hg-related 
questions if you want more input).

HTH,
  Hans


From hans_meine at gmx.net  Fri Feb 12 04:42:32 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 10:42:32 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
	<6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com>
Message-ID: <201002121042.32877.hans_meine@gmx.net>

On Thursday 11 February 2010 21:07:06 you wrote:
> > Now if the mainloop must not be started with 0.11, how about
> > monkey-patching the application classes (i.e. QCoreApplication.exec_) in
> > order to allow running unpatched programs when the GUI toolkit
> > integration is active?
> 
> Fernando and I spend a full day at Scipy 2009 trying this monkey
> patching approach.
> Summary: it doesn't work. 

That's very interesting, I'd like to know more details.

> Full details: each GUI toolkits has additional
>  ways of running the event loop (other than just calling "run") for a short
>  period of time.

Right.  For Qt, would that be qApp.processEvents() (i.e. "handle [some] 
pending events"), or QDialog.exec_ (which actually waits for user input in a 
recursively entered eventloop)?
Both are commonly used and must work indeed.

>  If everyone avoided these other calls the monkey
> patch approach would work.  But *many* people (including matplotlib
> and traits) do need to spin the event loop by hand and call these
> functions.

Right, but which of the above methods cause problems, and why exactly?

> > (In fact, in our own Qt3-based python terminal, we had monkeypatched
> > QApplication in order to also ignore the creation of a second app
> > object.)
> 
> Yes, this type of monkey patch is not too difficult to handle.  But
> *any* GUI app should first call getApp or its equivalent before
> creating an App object to make sure a 2nd one is created.

Huh?  Honestly, why should applications modify their __main__ to check for 
*existing* global objects - I think we're talking about a special case here 
which should be handled transparently, don't you agree?

(OK, I *do* occasionally check for existing objects in my __main__, in order 
be able to quickly reload a program with "%run -i", without losing the results 
of some expensive computations, but that's really a special case IMO.)

Have a nice day,
  Hans


From gael.varoquaux at normalesup.org  Fri Feb 12 05:02:52 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 12 Feb 2010 11:02:52 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11
	and	Matplotlib/ETS
In-Reply-To: <201002121042.32877.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002100926.15886.hans_meine@gmx.net>
	<6ce0ac131002111207w4c806528w58c926d6e064e8c1@mail.gmail.com>
	<201002121042.32877.hans_meine@gmx.net>
Message-ID: <20100212100252.GA20392@phare.normalesup.org>

On Fri, Feb 12, 2010 at 10:42:32AM +0100, Hans Meine wrote:
> Huh?  Honestly, why should applications modify their __main__ to check for 
> *existing* global objects - 

Because nobody should code thinking they own the main. It makes debugging
and reusing much harder.

That's my point of view, and it comes from a long experience of trying to
help people run code, debug code, or simply get the job done using third
party code.

Ga?l


From hans_meine at gmx.net  Fri Feb 12 05:08:18 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 11:08:18 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <20100212100252.GA20392@phare.normalesup.org>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121042.32877.hans_meine@gmx.net>
	<20100212100252.GA20392@phare.normalesup.org>
Message-ID: <201002121108.19174.hans_meine@gmx.net>

Hi Ga?l,

On Friday 12 February 2010 11:02:52 you wrote:
> On Fri, Feb 12, 2010 at 10:42:32AM +0100, Hans Meine wrote:
> > Huh?  Honestly, why should applications modify their __main__ to check
> > for *existing* global objects -
> 
> Because nobody should code thinking they own the main. It makes debugging
> and reusing much harder.

But I am thinking of mains like the following:

if __name__ == "__main__":
	import sys

	app = QtGui.QApplication(sys.argv)

	mw = BeverageMainWindow()
	mw.resize(800, 600)
	mw.show()
	sys.exit(app.exec_())

This is a prototypical __main__ which I'd like to be able to %run from IPython 
as well as from the commandline.  If I want to reuse code, I'll import the 
file and reuse BeverageMainWindow, but not the __main__ code.

> That's my point of view, and it comes from a long experience of trying to
> help people run code, debug code, or simply get the job done using third
> party code.

Maybe then those people put too much code into __main__?

Have a nice day,
  Hans


From gael.varoquaux at normalesup.org  Fri Feb 12 05:17:22 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 12 Feb 2010 11:17:22 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11
	and	Matplotlib/ETS
In-Reply-To: <201002121108.19174.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121042.32877.hans_meine@gmx.net>
	<20100212100252.GA20392@phare.normalesup.org>
	<201002121108.19174.hans_meine@gmx.net>
Message-ID: <20100212101722.GB20392@phare.normalesup.org>

On Fri, Feb 12, 2010 at 11:08:18AM +0100, Hans Meine wrote:
> > > Huh?  Honestly, why should applications modify their __main__ to check
> > > for *existing* global objects -

> > Because nobody should code thinking they own the main. It makes debugging
> > and reusing much harder.

> But I am thinking of mains like the following:

> if __name__ == "__main__":
> 	import sys

> 	app = QtGui.QApplication(sys.argv)

> 	mw = BeverageMainWindow()
> 	mw.resize(800, 600)
> 	mw.show()
> 	sys.exit(app.exec_())

> This is a prototypical __main__ which I'd like to be able to %run from
> IPython as well as from the commandline.  If I want to reuse code, I'll
> import the file and reuse BeverageMainWindow, but not the __main__
> code.

OK, but in the above code, you are saying that you want to own the
application object. What if you want to interact with other code that
does the same thing? There can only be one like this.

> > That's my point of view, and it comes from a long experience of trying to
> > help people run code, debug code, or simply get the job done using third
> > party code.

> Maybe then those people put too much code into __main__?

Most probably, but if you follow this logic, then you end up deciding
that nobody should ever create a new application object without checking
that one already exists, unless they want exactly that (eg for explicit
threading reasons).

What I am trying to say is that you have contradictory requirements: you
are asking for full ownership of the application, but you would also like
to play nice with other environments. This is going to fail, and not only
in IPython. I'd be interested: how does such code behave when run through
an execfile in a PyQt IDE, for instance eric4 or spyder?

Basically, whoever requires control and ownership of a global state will
be limited in its interaction with other programs, and I don't believe
there is a way around it.

Ga?l


From gael.varoquaux at normalesup.org  Fri Feb 12 05:37:31 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 12 Feb 2010 11:37:31 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <201002121031.57250.hans_meine@gmx.net>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
	<201002121031.57250.hans_meine@gmx.net>
Message-ID: <20100212103731.GC20392@phare.normalesup.org>

On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote:
> On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote:
> > [...] Not so with hg, people can basically  use it
> > after they understand dvcs in general.

> > I also like the fact that it's trivial to put up a "hg serve" session
> > for quick collaborative hacking, and that you get time based "simple"
> > revision numbers instead of relying only on hashes.

> OK, my 2 cents:

And mine...

I use heavily only bzr, and I see its limitations. I agree with Hans that
its main strength is nowadays Launchpad, although that fact that it is
easy to get going under Windows is a pro too.

Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it
strikes the right balance between power and simplicity. However, for me
hg has a killer feature: subrepositories (or the forest extension,
previously): http://mercurial.selenic.com/wiki/subrepos

I work in many projects where the natural setting is actually a group of
projects evolving together. Sub-repositories are the natural answer to
this situation and a big pro for me because the reduce technical and
social difficulties with loosely coupled projects: better than a
fully-integrated repo as projects have some independence, but better than
separate repos, as it is possible to have comon history.

Maybe git has a similar feature. I haven't seen it yet.

Ga?l


From hans_meine at gmx.net  Fri Feb 12 06:06:50 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 12:06:50 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <20100212101722.GB20392@phare.normalesup.org>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121108.19174.hans_meine@gmx.net>
	<20100212101722.GB20392@phare.normalesup.org>
Message-ID: <201002121206.51045.hans_meine@gmx.net>

On Friday 12 February 2010 11:17:22 you wrote:
> On Fri, Feb 12, 2010 at 11:08:18AM +0100, Hans Meine wrote:
> > if __name__ == "__main__":
> > 	import sys
> >
> > 	app = QtGui.QApplication(sys.argv)
> >
> > 	mw = BeverageMainWindow()
> > 	mw.resize(800, 600)
> > 	mw.show()
> > 	sys.exit(app.exec_())
> >
> > This is a prototypical __main__ which I'd like to be able to %run from
> > IPython as well as from the commandline.  If I want to reuse code, I'll
> > import the file and reuse BeverageMainWindow, but not the __main__
> > code.
> 
> OK, but in the above code, you are saying that you want to own the
> application object.

Do I?  I see it more like a minimal setup function for opening the main 
window, which is all I care about.  The QApplication line is only there to 
make the stuff below work (with Qt, as with most toolkits AFAIK, I would 
otherwise get an error because the widgets will rely on global state being 
initialized by the application constructor).

> What if you want to interact with other code that
> does the same thing? There can only be one like this.

Actually, the most important use case I see is that I repeatedly %run the same 
program.  That would even work if I called app.setApplicationName(), 
.setVersion, ... which I did not in my above minimal example.

> [...] if you follow this logic, then you end up deciding
> that nobody should ever create a new application object without checking
> that one already exists, unless they want exactly that (eg for explicit
> threading reasons).

OK, but then my __main__ becomes twice as long, because I must also not enter 
the main loop and not call sys.exit() in theory.

I would simply prefer if IPython automatically handled that for me.

I fully understand that this will not work in general, with any application.
(For instance, Qt's database handling includes global state about the "default 
database" for your convenience etc.)

OTOH, ipython mostly is about hacks for convenient interactive experiments, 
and most of my programs
- do not rely on global state at all (i.e. the main window is all I actually 
care about), or
- just set up some "more or less constant" global state (like app name & 
version) which would still work if I %run the same app several times (and 
probably degrade quite gracefully if I %run another app).

> What I am trying to say is that you have contradictory requirements: you
> are asking for full ownership of the application,

I do not mean to.

> but you would also like
> to play nice with other environments.

Not really either. ;-)

Greetings,
  Hans


From gael.varoquaux at normalesup.org  Fri Feb 12 06:11:04 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 12 Feb 2010 12:11:04 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11
	and	Matplotlib/ETS
In-Reply-To: <201002121206.51045.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121108.19174.hans_meine@gmx.net>
	<20100212101722.GB20392@phare.normalesup.org>
	<201002121206.51045.hans_meine@gmx.net>
Message-ID: <20100212111104.GA7228@phare.normalesup.org>

On Fri, Feb 12, 2010 at 12:06:50PM +0100, Hans Meine wrote:
> Do I?  I see it more like a minimal setup function for opening the main 
> window, which is all I care about.  The QApplication line is only there to 
> make the stuff below work (with Qt, as with most toolkits AFAIK, I would 
> otherwise get an error because the widgets will rely on global state being 
> initialized by the application constructor).

Use:

    app = QtGui.QApplication.instance()

That should suit your needs and avoid creating an QApplication if it
already exists.

Cheers,

Ga?l


From hans_meine at gmx.net  Fri Feb 12 06:12:22 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 12:12:22 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <20100212103731.GC20392@phare.normalesup.org>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<201002121031.57250.hans_meine@gmx.net>
	<20100212103731.GC20392@phare.normalesup.org>
Message-ID: <201002121212.22530.hans_meine@gmx.net>

On Friday 12 February 2010 11:37:31 you wrote:
> On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote:
> > On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote:
> > > [...] Not so with hg, people can basically  use it
> > > after they understand dvcs in general.
> > >
> > > I also like the fact that it's trivial to put up a "hg serve" session
> > > for quick collaborative hacking, and that you get time based "simple"
> > > revision numbers instead of relying only on hashes.
> >
> > OK, my 2 cents:
> 
> And mine...
> 
> I use heavily only bzr, and I see its limitations. I agree with Hans that
> its main strength is nowadays Launchpad, although that fact that it is
> easy to get going under Windows is a pro too.

I haven't had problems with hg's windows installer either.  Or are you talking 
about TortoiseHg?  AFAIK, that's also well-maintained and rapidly improving, 
but I don't have much experience with it (at least not on Windows *).

HTH,
  Hans

* Yes, contrary to my former belief, TortoiseHg can be used on Linux, too (for 
example as a convenient commit GUI).


From hans_meine at gmx.net  Fri Feb 12 06:25:04 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 12 Feb 2010 12:25:04 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <20100212111104.GA7228@phare.normalesup.org>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121206.51045.hans_meine@gmx.net>
	<20100212111104.GA7228@phare.normalesup.org>
Message-ID: <201002121225.05301.hans_meine@gmx.net>

On Friday 12 February 2010 12:11:04 Gael Varoquaux wrote:
> On Fri, Feb 12, 2010 at 12:06:50PM +0100, Hans Meine wrote:
> > Do I?  I see it more like a minimal setup function for opening the main
> > window, which is all I care about.  The QApplication line is only there
> > to make the stuff below work (with Qt, as with most toolkits AFAIK, I
> > would otherwise get an error because the widgets will rely on global
> > state being initialized by the application constructor).
> 
> Use:
> 
>     app = QtGui.QApplication.instance()
> 
> That should suit your needs and avoid creating an QApplication if it
> already exists.

Thanks for the hint, but the docs say:
  "If no instance has been allocated, null is returned."

Furthermore, what about app.exec_?  I thought I needed sth. like:

if __name__ == "__main__":
	hasApp = QtGui.QApplication.instance()
	if not hasApp:
		import sys
		app = QtGui.QApplication(sys.argv)

	mw = BeverageMainWindow()
	mw.resize(800, 600)
	mw.show()

	if not hasApp:
		sys.exit(app.exec_())

That's OK, I can do that, but I would still prefer not to have to change every 
application just for the occasional debugging session in ipython.

And how do you want to communicate this to all ipython users?  I mean - there 
needs to be some documentation about this boilerplate code for every toolkit, 
and every user needs to be aware of this and get it right.

I fully agree that the above code has a much better design, is more well-
behaved, but you know very well that most programs are not so well-behaved, 
and consequently you'll lock out a large fraction of existing programs from 
being %run from IPython.  And from a user's POV, this will even appear to be a 
regression, even if previous support for that was buggy - there *was* support 
for it.

I'd say not looking at other program's deficiencies, but closing the eyes and 
adding some proper hack would be much more in the spirit of IPython.

Best,
  Hans


From matthieu.brucher at gmail.com  Fri Feb 12 06:32:18 2010
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Fri, 12 Feb 2010 12:32:18 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <20100212103731.GC20392@phare.normalesup.org>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
	<201002121031.57250.hans_meine@gmx.net>
	<20100212103731.GC20392@phare.normalesup.org>
Message-ID: <e76aa17f1002120332j39db84f8i67741ba13f7a9883@mail.gmail.com>

2010/2/12 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Fri, Feb 12, 2010 at 10:31:57AM +0100, Hans Meine wrote:
>> On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote:
>> > [...] Not so with hg, people can basically ?use it
>> > after they understand dvcs in general.
>
>> > I also like the fact that it's trivial to put up a "hg serve" session
>> > for quick collaborative hacking, and that you get time based "simple"
>> > revision numbers instead of relying only on hashes.
>
>> OK, my 2 cents:
>
> And mine...

And mine ;)

I've started with Bazaar and I'm still using it for my personal
projects, but professionnally I switched to Hg, because I didn't need
Launchpad there. Multiple branches are a extraordinary thing, but it
can be messy if you have several different branches yourself.

> I use heavily only bzr, and I see its limitations. I agree with Hans that
> its main strength is nowadays Launchpad, although that fact that it is
> easy to get going under Windows is a pro too.

+1

> Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it
> strikes the right balance between power and simplicity. However, for me
> hg has a killer feature: subrepositories (or the forest extension,
> previously): http://mercurial.selenic.com/wiki/subrepos

It's not that great. Bzr also has the same feature.
The main issue is when you copy something from one subrepos to
another: you loose the history. Basically, it is useless IMHO.

> I work in many projects where the natural setting is actually a group of
> projects evolving together. Sub-repositories are the natural answer to
> this situation and a big pro for me because the reduce technical and
> social difficulties with loosely coupled projects: better than a
> fully-integrated repo as projects have some independence, but better than
> separate repos, as it is possible to have comon history.

I think the ultimate solution to this issue (that I also have) is not
there yet :(

Matthieu
-- 
Information System Engineer, Ph.D.
Blog: http://matt.eifelle.com
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From gael.varoquaux at normalesup.org  Fri Feb 12 06:39:03 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 12 Feb 2010 12:39:03 +0100
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11
	and	Matplotlib/ETS
In-Reply-To: <201002121225.05301.hans_meine@gmx.net>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121206.51045.hans_meine@gmx.net>
	<20100212111104.GA7228@phare.normalesup.org>
	<201002121225.05301.hans_meine@gmx.net>
Message-ID: <20100212113903.GB7228@phare.normalesup.org>

On Fri, Feb 12, 2010 at 12:25:04PM +0100, Hans Meine wrote:
> > Use:

> >     app = QtGui.QApplication.instance()

> > That should suit your needs and avoid creating an QApplication if it
> > already exists.

> Thanks for the hint, but the docs say:
>   "If no instance has been allocated, null is returned."

Hehe, you caught me bullshiting, I am afraid. :)

How about:

QtGui.qApp

I believe that works, but I am not sure (being cautious this time).

> And how do you want to communicate this to all ipython users?  I mean - there 
> needs to be some documentation about this boilerplate code for every toolkit, 
> and every user needs to be aware of this and get it right.

Honestly, I don't really care about how things are done. I am no longer
really involved in IPython development. All I know is that trying to
solve impossible problems leads to hair loss and disappointment. I
believe you are trying to solve an impossible problem: making sure a user
can use a main event loop without being aware it is around. IPython used
to try and solve that problem using threads. It lead to segfaults because
every single piece of code executed had to be injected in the main loop.
So what was supposed to be transparent worked 99.9% of the time, and the
rest of the time forced people to modify their code in subtle ways and
debug race conditions, which is hard. The developers have now stepped
back and decided that they would make a choice that forces users to be
aware of the problems upfront, but has a simple solution.

I am not saying this is a good or bad choice. I haven't had time to try
out the new IPython, and I am worried of the problems with Mayavi and
Traits. However, I am saying that there is some reasonning behind the
choice, and that whatever we do, we are sitting between a rock and a hard
place.

> I fully agree that the above code has a much better design, is more
> well- behaved, but you know very well that most programs are not so
> well-behaved, and consequently you'll lock out a large fraction of
> existing programs from being %run from IPython.  And from a user's POV,
> this will even appear to be a regression, even if previous support for
> that was buggy - there *was* support for it.

Well, yes, but there were threading problems. You had to be aware of how
threading works under IPython, which is much more technical.

> I'd say not looking at other program's deficiencies, but closing the eyes and 
> adding some proper hack would be much more in the spirit of IPython.

Hacks lead to what we've all been experimenting: segfaults. For somebody
like me who spends most of my day in IPython, it is nice no longer to
have any segfaults.

Ga?l


From drewm1980 at gmail.com  Fri Feb 12 12:23:48 2010
From: drewm1980 at gmail.com (Andrew Wagner)
Date: Fri, 12 Feb 2010 11:23:48 -0600
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <20100212103731.GC20392@phare.normalesup.org>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
	<201002121031.57250.hans_meine@gmx.net>
	<20100212103731.GC20392@phare.normalesup.org>
Message-ID: <795840fe1002120923y372163f4t25ab0aa7553aee03@mail.gmail.com>

Hello-

I've only been lurking for a few days, but since I use git, I can
share a bit of useful info to help your evaluation.  (I have not used
hg; went straight from SVN to git)

On Fri, Feb 12, 2010 at 4:37 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> Out of the 3 main DVCSs, I am also most tempted by hg, because I feels it
> strikes the right balance between power and simplicity. However, for me
> hg has a killer feature: subrepositories (or the forest extension,
> previously): http://mercurial.selenic.com/wiki/subrepos
>
> I work in many projects where the natural setting is actually a group of
> projects evolving together. Sub-repositories are the natural answer to
> this situation and a big pro for me because the reduce technical and
> social difficulties with loosely coupled projects: better than a
> fully-integrated repo as projects have some independence, but better than
> separate repos, as it is possible to have comon history.
>
> Maybe git has a similar feature. I haven't seen it yet.

The command you seek is called git-submodule, but I have not used it.

For those who are new to git, I recommend first reading the git
community "book":

http://book.git-scm.com/

especially the page about git's object model:

http://book.git-scm.com/1_the_git_object_model.html

I also recommend gitk for inspecting your history (usually distributed with git)

and the eg wrapper script for improved command-line help:

http://www.gnome.org/~newren/eg/

Cheers,
Drew


From JDM at MarchRay.net  Fri Feb 12 14:12:58 2010
From: JDM at MarchRay.net (Jonathan March)
Date: Fri, 12 Feb 2010 13:12:58 -0600
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <201002121031.57250.hans_meine@gmx.net>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<46cb515a1002111227m1ef430f9x54292acf8a511007@mail.gmail.com>
	<201002121031.57250.hans_meine@gmx.net>
Message-ID: <b8aa56d51002121112w50a4207cha9eb3d9086765753@mail.gmail.com>

On Fri, Feb 12, 2010 at 3:31 AM, Hans Meine <hans_meine at gmx.net> wrote:

> On Thursday 11 February 2010 21:27:28 Ville M. Vainio wrote:
> > [...] Not so with hg, people can basically  use it
> > after they understand dvcs in general.
>
> OK, my 2 cents:
>

I switched from bzr to hg over a year ago and have no regrets. FWIW I'd be
very happy if scipy went hg.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100212/bcd4940b/attachment.html>

From JDM at MarchRay.net  Fri Feb 12 14:25:07 2010
From: JDM at MarchRay.net (Jonathan March)
Date: Fri, 12 Feb 2010 13:25:07 -0600
Subject: [IPython-dev] scipy at pycon?
Message-ID: <b8aa56d51002121125pa8f8c04y8569327c9e5f3d38@mail.gmail.com>

I plan to come to scipy in Austin in June, but hoped/expected that there
would also be some scipy-related activity in Atlanta this month... but see
no sprints, almost no talks or posters... anybody going?

Jonathan March
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100212/992ee8f9/attachment.html>

From fperez.net at gmail.com  Sat Feb 13 23:55:40 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 13 Feb 2010 23:55:40 -0500
Subject: [IPython-dev] scipy at pycon?
In-Reply-To: <b8aa56d51002121125pa8f8c04y8569327c9e5f3d38@mail.gmail.com>
References: <b8aa56d51002121125pa8f8c04y8569327c9e5f3d38@mail.gmail.com>
Message-ID: <db6b5ecc1002132055y543ab758icc44b4d631ffb430@mail.gmail.com>

Hi Jonathan,

On Fri, Feb 12, 2010 at 2:25 PM, Jonathan March <JDM at marchray.net> wrote:
> I plan to come to scipy in Austin in June, but hoped/expected that there
> would also be some scipy-related activity in Atlanta this month... but see
> no sprints, almost no talks or posters... anybody going?

I won't be attending, but I think the Enthought guys are holding a
tutorial there (I saw it mentioned at the numpy/stats webinar on
Friday).  I don't know about anyone else who might be going from the
local Berkeley crowd either.

Cheers,

f


From fperez.net at gmail.com  Sun Feb 14 02:26:27 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 14 Feb 2010 02:26:27 -0500
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <20100212113903.GB7228@phare.normalesup.org>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121206.51045.hans_meine@gmx.net>
	<20100212111104.GA7228@phare.normalesup.org>
	<201002121225.05301.hans_meine@gmx.net>
	<20100212113903.GB7228@phare.normalesup.org>
Message-ID: <db6b5ecc1002132326y6b929b76uddf0bc1743333b1e@mail.gmail.com>

Hi all,

Thanks to everyone for the feedback, it's indeed very useful and we
are aware this is a hard problem.  The last thing we want to do is
cause what people will effectively see as massive regressions, no
matter how good our intentions or the reasons behind our choices.
Gael summarizes the situation very well here:

On Fri, Feb 12, 2010 at 6:39 AM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> Honestly, I don't really care about how things are done. I am no longer
> really involved in IPython development. All I know is that trying to
> solve impossible problems leads to hair loss and disappointment. I
> believe you are trying to solve an impossible problem: making sure a user
> can use a main event loop without being aware it is around. IPython used
> to try and solve that problem using threads. It lead to segfaults because
> every single piece of code executed had to be injected in the main loop.
> So what was supposed to be transparent worked 99.9% of the time, and the
> rest of the time forced people to modify their code in subtle ways and
> debug race conditions, which is hard. The developers have now stepped
> back and decided that they would make a choice that forces users to be
> aware of the problems upfront, but has a simple solution.
>
> I am not saying this is a good or bad choice. I haven't had time to try
> out the new IPython, and I am worried of the problems with Mayavi and
> Traits. However, I am saying that there is some reasonning behind the
> choice, and that whatever we do, we are sitting between a rock and a hard
> place.

As we go through this, I am pondering the possibility of eventually
adding the threading-style hacks back, but NOT as default.  This isn't
final by any means, I'm just thinking out loud here, about a plan
along these lines:

- non-threaded gui support with pyos_inputhook, that would require
some code changes.  If this can give a really robust solution and the
needed changes aren't too intrusive, I could see matplotlib, ETS and
possibly others adding the necessary adaptation (especially if we
provide it in ipython already so for the users it ends up being almost
a one-liner).

- adding back the threading hacks, which could be enabled with special
command-line flags.  These would carry the known fragility we are
trying to get away from, but would let users continue to use them in
codes that have not been modified, just like they do today.  No
better, but no worse either.

IPython is an eminently practical tool, so we'll definitely remember
the zen's stanza on purity and practicality, or at least we'll try.

Regards,

f


From fperez.net at gmail.com  Sun Feb 14 23:33:34 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 14 Feb 2010 23:33:34 -0500
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <4B71A453.1060406@bostream.nu>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<4B71A453.1060406@bostream.nu>
Message-ID: <db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>

Hi all,

[ thanks a lot for the dvcs comments, there was actually a lot of good
info there.  But for now I'm sticking to the migration topic here, I'm
a little pressed for time]

On Tue, Feb 9, 2010 at 1:07 PM, J?rgen Stenarson
<jorgen.stenarson at bostream.nu> wrote:
>> So, as a concrete proposal, I can update the remote repo say by this
>> weekend, giving anyone with local changes in any branch time to merge
>> them. ?By Sunday I will start a remote repo upgrade, and we'll all
>> simply have to run local upgrades before doing any further operations.
>>
>> Does that sound OK, have I missed anything?
>>
> Sounds reasonable to me.

OK, I tried to get this ball rolling, but the process looks more
complex than I thought, and I'm having second thoughts.  This is what
I have found so far in terms of instructions:

http://doc.bazaar.canonical.com/latest/en/upgrade-guide/index.html#data-migration

It's a very involved process, that seems to require lots of manual
checks and attention to detail, with multiple chances for screwing up.

Ville, you mentioned doing this for Leo, could you provide us with
some info on how you went about it?  Is that guide overly complicated
and do you have a simpler alternative?

I am seriously, seriously annoyed that using a version control tool
becomes a chore like this.  We have precious little time as it is for
ipython, the last thing I want to do is to waste it babysitting the
VCS (especially when it's slow and crash-prone to begin with).

This thread on the upgrade topic is another one that I found when
searching instructions for this:

http://omgili.com/mailinglist/bazaar/lists/canonical/com/20090930081159735b276dlimelightwoozorg.html

and it also has lots of things in it that give me pause.

So before I pull a trigger I might regret, should we reconsider this?
Ville, if you have tips on how to do this cleanly I'd love to hear
them, perhaps I'm being unnecessarily paranoid and fearful and in
practice it really isn't that bad.

But one thing that is true is that in the next few weeks I do not have
a full 4-6 open hours to devote to this.  I thought I could simply
activate 'repo upgrade' somewhere on LP and be done with it, but that
doesn't seem to be the case.  If it requires a full day of my time to
happen cleanly, I simply can't do it now, I'm afraid.

Advice will be much welcome.

Thanks!

f


From vivainio at gmail.com  Mon Feb 15 02:52:23 2010
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 15 Feb 2010 09:52:23 +0200
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<4B71A453.1060406@bostream.nu>
	<db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>
Message-ID: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>

On Mon, Feb 15, 2010 at 6:33 AM, Fernando Perez <fperez.net at gmail.com> wrote:

> Ville, you mentioned doing this for Leo, could you provide us with
> some info on how you went about it? ?Is that guide overly complicated
> and do you have a simpler alternative?

I did "bzr upgrade lp:leo-editor" and waited it out.

Safer alternative seems to be:

bzr branch lp:ipython
cd ipython
bzr upgrade (this takes a long time)

+ push to a new branch, and change the "development focus" to be that branch.

-- 
Ville M. Vainio
http://tinyurl.com/vainio


From fperez.net at gmail.com  Mon Feb 15 11:13:42 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 15 Feb 2010 11:13:42 -0500
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<4B71A453.1060406@bostream.nu>
	<db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>
	<46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>
Message-ID: <db6b5ecc1002150813w2b6998c1ifa3a340a3cc8b9fe@mail.gmail.com>

On Mon, Feb 15, 2010 at 2:52 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> I did "bzr upgrade lp:leo-editor" and waited it out.
>
> Safer alternative seems to be:
>
> bzr branch lp:ipython
> cd ipython
> bzr upgrade (this takes a long time)
>
> + push to a new branch, and change the "development focus" to be that branch.

OK, thanks.

I'll try to fire that process tonight then.  I'll ping with an email
to the list when I start.

Cheers,

f


From gael.varoquaux at normalesup.org  Mon Feb 15 13:20:40 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 15 Feb 2010 19:20:40 +0100
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<4B71A453.1060406@bostream.nu>
	<db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>
	<46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>
Message-ID: <20100215182040.GC20321@phare.normalesup.org>

On Mon, Feb 15, 2010 at 09:52:23AM +0200, Ville M. Vainio wrote:
> Safer alternative seems to be:

> bzr branch lp:ipython
> cd ipython
> bzr upgrade (this takes a long time)

> + push to a new branch, and change the "development focus" to be that branch.

Yes, I would advise to do this. I did this for at least one project I
work on (can't remember which).

And I agree that this is a mess...

Ga?l


From fperez.net at gmail.com  Tue Feb 16 00:56:31 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 16 Feb 2010 00:56:31 -0500
Subject: [IPython-dev] What do we do about repository formats???
In-Reply-To: <db6b5ecc1002150813w2b6998c1ifa3a340a3cc8b9fe@mail.gmail.com>
References: <db6b5ecc1001172351h14018a08rab860f4b4a1524e7@mail.gmail.com>
	<46cb515a1001201346y6e10c58ar58a99141fffde0c4@mail.gmail.com>
	<db6b5ecc1002090845u309475e4v532f7a4b3ed6be8e@mail.gmail.com>
	<4B71A453.1060406@bostream.nu>
	<db6b5ecc1002142033r6e1ae5e4w2386f16be9dc4923@mail.gmail.com>
	<46cb515a1002142352t1f38411ao85d8e0407021dae1@mail.gmail.com>
	<db6b5ecc1002150813w2b6998c1ifa3a340a3cc8b9fe@mail.gmail.com>
Message-ID: <db6b5ecc1002152156se466816xf246794ecba1b1bf@mail.gmail.com>

On Mon, Feb 15, 2010 at 11:13 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>> bzr branch lp:ipython
>> cd ipython
>> bzr upgrade (this takes a long time)
>>
>> + push to a new branch, and change the "development focus" to be that branch.
>
> OK, thanks.
>
> I'll try to fire that process tonight then. ?I'll ping with an email
> to the list when I start.

For some reason this is taking a long time on my system, and I need to
crash now.  I'll be traveling for a few days, so I won't touch
anything on the repo until next week at best, when I return.

Cheers,

f


From ondrej at certik.cz  Tue Feb 16 21:21:47 2010
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 16 Feb 2010 18:21:47 -0800
Subject: [IPython-dev] using reST for representing the notebook cells+text
Message-ID: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>

Hi,

has anyone given a deeper thought how to best implement this functionality?

Currently the Sage notebook seems to remember the worksheet using:

--------------
sdsd
system:sage


<p>Well, some text</p>

{{{id=5|
awe
///
}}}

{{{id=1|
4
///
4
}}}

{{{id=2|
5
///
5
}}}

{{{id=3|

///
}}}
---------------------

E.g. each cell is either a text cell, or a cell that can be evaluated.
Is that right?

I know that Fernando suggested here:

http://groups.google.com/group/sage-devel/browse_thread/thread/3f1f5a0c26975519/

to use reST, e.g. something like:

----------
Well, some text

::
     >>> print 3
     3

------------


I am implementing a notebook prototype using pure Python
(automatically translated to javascript using pyjamas) and I would
like to run it on the google app engine (the javascript in your
browser would then communicate either with the app engine, or your own
hardware that would host Sage/FEMhub/your own code):

http://gamma.sympy.org/nb/

and here is my development version with logins and settings implemented:

http://2.latest.sympy-gamma.appspot.com/nb/

now I need to implement the worksheets, so I am thinking how to best
represent it.


Did the syntax for input/output blocks in Sphinx settled already? I
guess internally I will implement the worksheet as a set of cells,
either text cells, or "evaluatable" cells (that contain both
input/output blocks) and then support conversion to and from
reST/Sphinx.

I would be interested in any feedback.

Ondrej


From Chris.Barker at noaa.gov  Thu Feb 18 17:24:40 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Thu, 18 Feb 2010 14:24:40 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
 Matplotlib/ETS
In-Reply-To: <db6b5ecc1002132326y6b929b76uddf0bc1743333b1e@mail.gmail.com>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121206.51045.hans_meine@gmx.net>
	<20100212111104.GA7228@phare.normalesup.org>
	<201002121225.05301.hans_meine@gmx.net>
	<20100212113903.GB7228@phare.normalesup.org>
	<db6b5ecc1002132326y6b929b76uddf0bc1743333b1e@mail.gmail.com>
Message-ID: <4B7DBE28.90702@noaa.gov>

Thanks for everyone's work on this.

I had a thought. It seems that it's really impossible (or at least 
impractical) to monkey-patch the GUI toolkits such that you can run a 
regular old unmodified app exactly the same way inside and outside of 
ipython. Gael's right -- that way lies loss of hair (and I"ve lost 
enough in the last ten years as it is).

However, as it happens, I've been using ipython for a wx GUI app of mine 
(with no ETS or MPL), and it sort-of-works, and also locks up and 
crashes fairly regularly. So the threading solution isn't really robust 
either. Also, when wx starts using Cocoa on OS-X, putting the GUI in 
another thread won't be an option anymore anyway.

So:

> - non-threaded gui support with pyos_inputhook, that would require
> some code changes.  If this can give a really robust solution and the
> needed changes aren't too intrusive, I could see matplotlib, ETS and
> possibly others adding the necessary adaptation (especially if we
> provide it in ipython already so for the users it ends up being almost
> a one-liner).

Exactly -- while you can't expect unmodified regular old wx code to 
work, we could provide alternative app startup code that a user can 
simply cut and past in, and then have an app that will work with and 
without ipython -- I think that's a fine solution.

So for now, is there a sample, minimal wx app that works with the new 
pyos_inputhook code?

-Chris





-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From noamraph at gmail.com  Sun Feb 21 04:56:35 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 11:56:35 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
Message-ID: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>

Hello,

My name is Noam Yorav-Raphael. I wanted to let you know that I wrote a
new graphical Python shell called DreamPie, and I would like to have
your feedback as the developers of a mature Python shell.

DreamPie is quite similar to IDLE, in that it uses the graphical user
interface to give you automatic completions and tooltips showing
function arguments. It's similar to IPython in that it has a result
history, and in that it supports using matplotlib.

It also offers these unique features:
* Long output can be folded, so as not to distract you from the
important results. Also, textual results can be saved to external
files or viewed in an external viewer.
* You can type any amount of code and execute it at once. The Copy
Code Only command lets you easily copy successful commands into a
Python module you're editing.
* History can be saved to HTML files and loaded back, so you can
record how you got your results and also quickly re-execute commands
from a previous session.

Would you care to give it a try, at http://dreampie.sourceforge.net/ ,
and tell me what you think about it?

Thanks,
Noam


From vivainio at gmail.com  Sun Feb 21 07:51:41 2010
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 21 Feb 2010 14:51:41 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
Message-ID: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>

On Sun, Feb 21, 2010 at 11:56 AM, Noam Yorav-Raphael <noamraph at gmail.com> wrote:

> DreamPie is quite similar to IDLE, in that it uses the graphical user
> interface to give you automatic completions and tooltips showing
> function arguments. It's similar to IPython in that it has a result
> history, and in that it supports using matplotlib.

Out of curiosity, are you familiar with Spyder:

http://packages.python.org/spyder/

That project shares some of the same aims.

-- 
Ville M. Vainio
http://tinyurl.com/vainio


From noamraph at gmail.com  Sun Feb 21 08:51:15 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 15:51:15 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com> 
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
Message-ID: <b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>

Thanks for the link! I wasn't familiar with Spyder. It looks very interesting.
It seems to me that it is less oriented toward using the shell - for
example, I couldn't find a way to recall a block (more than one line)
from history, and there was no way to interrupt either the internal or
the external console. I think DreamPie is better in that respect.

Noam


On Sun, Feb 21, 2010 at 2:51 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
>
> Out of curiosity, are you familiar with Spyder:
>
> http://packages.python.org/spyder/
>
> That project shares some of the same aims.
>
> --
> Ville M. Vainio
> http://tinyurl.com/vainio
>


From gael.varoquaux at normalesup.org  Sun Feb 21 09:04:49 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 21 Feb 2010 15:04:49 +0100
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
Message-ID: <20100221140449.GA32024@phare.normalesup.org>

On Sun, Feb 21, 2010 at 03:51:15PM +0200, Noam Yorav-Raphael wrote:
> Thanks for the link! I wasn't familiar with Spyder. It looks very interesting.
> It seems to me that it is less oriented toward using the shell - for
> example, I couldn't find a way to recall a block (more than one line)
> from history, and there was no way to interrupt either the internal or
> the external console. I think DreamPie is better in that respect.

I am sure that DreamPie is better than Spyder in some respects, and vice
versa.

However, the question is: is there an interest in collecting many
different efforts that have their own strengths and weaknesses?

Note that I am a user of neither, because they don't offer all the
functionalities that I need from IPython...

Best,

Ga?l


From noamraph at gmail.com  Sun Feb 21 09:34:32 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 16:34:32 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <20100221140449.GA32024@phare.normalesup.org>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com> 
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> 
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com> 
	<20100221140449.GA32024@phare.normalesup.org>
Message-ID: <b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>

On Sun, Feb 21, 2010 at 4:04 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
>
> I am sure that DreamPie is better than Spyder in some respects, and vice
> versa.
>
> However, the question is: is there an interest in collecting many
> different efforts that have their own strengths and weaknesses?
>
That's always a problem with projects which may have common goals.
However, in this case, I don't feel that there was a lot of redundant
work done, since Spyder seems to be more focused on visualization
while DreamPie is more focused on the shell itself.

> Note that I am a user of neither, because they don't offer all the
> functionalities that I need from IPython...

Can you say what these functionalities are? I'm interested to know
which features people are missing from DreamPie.

Thanks,
Noam


From gael.varoquaux at normalesup.org  Sun Feb 21 09:39:37 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 21 Feb 2010 15:39:37 +0100
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
Message-ID: <20100221143937.GB32024@phare.normalesup.org>

On Sun, Feb 21, 2010 at 04:34:32PM +0200, Noam Yorav-Raphael wrote:
> > Note that I am a user of neither, because they don't offer all the
> > functionalities that I need from IPython...

> Can you say what these functionalities are? I'm interested to know
> which features people are missing from DreamPie.

Most of the IPython magics. Go history management. The ability to have
several GUI event loop. Fancy tracebacks and interactive debuger support.
Output history. Shell support. Tab completion on file names. The '?' and
'??' support.

Does 'Ctrl-C' work in Dreampie?

There might be other things, but that's what I can think of.

Ga?l


From noamraph at gmail.com  Sun Feb 21 10:02:20 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 17:02:20 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <20100221143937.GB32024@phare.normalesup.org>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com> 
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> 
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com> 
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com> 
	<20100221143937.GB32024@phare.normalesup.org>
Message-ID: <b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>

On Sun, Feb 21, 2010 at 4:39 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Sun, Feb 21, 2010 at 04:34:32PM +0200, Noam Yorav-Raphael wrote:
>> > Note that I am a user of neither, because they don't offer all the
>> > functionalities that I need from IPython...
>
>> Can you say what these functionalities are? I'm interested to know
>> which features people are missing from DreamPie.
>
> Most of the IPython magics. Go history management. The ability to have
> several GUI event loop. Fancy tracebacks and interactive debuger support.
> Output history. Shell support. Tab completion on file names. The '?' and
> '??' support.

Thanks for the list!
* History management is there - the recent values computed are saved,
and all your command and output history is saved too. You can even
save it to a HTML file.
* GUI event loops are there - GTK, Tkinter, and QT event loops are
automatically supported, in a way very similar to Tkinter in the
traditional interpreter - handling events while idle.
* Tracebacks are not really fancy, but are colored in red so that it's
easy to distinguish them from normal output.
* Completion of file names is there too, using a popup list.

This leaves these features:
* Fancy tracebacks and debugger support - right now pdb works like in
the classic interpreter, but I'm sure IPython supports better
debugging.
* Shell support. (you can use os.system, but it's not the same thing)
* '?' and '??'. I guess that in a graphical environment it's best if
the help appears in a new window.

That's a great list. I hope I will be able to implement those. Thanks!

>
> Does 'Ctrl-C' work in Dreampie?
>
Works like charm... It was hard getting it right on Windows, but I managed it.

Thanks,
Noam


From gael.varoquaux at normalesup.org  Sun Feb 21 10:07:16 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 21 Feb 2010 16:07:16 +0100
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
	<20100221143937.GB32024@phare.normalesup.org>
	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
Message-ID: <20100221150716.GA8338@phare.normalesup.org>

On Sun, Feb 21, 2010 at 05:02:20PM +0200, Noam Yorav-Raphael wrote:
> * History management is there - the recent values computed are saved,
> and all your command and output history is saved too. You can even
> save it to a HTML file.

What I specificaly want is recalling based on the begining of a line.

> * GUI event loops are there - GTK, Tkinter, and QT event loops are
> automatically supported, in a way very similar to Tkinter in the
> traditional interpreter - handling events while idle.

Cool. I missed that. That's nice.

> * Tracebacks are not really fancy, but are colored in red so that it's
> easy to distinguish them from normal output.

:( . IPython's traceback really improve my productivity, because they often
allow me to pin-point the problem immediatly.

> * Completion of file names is there too, using a popup list.

Cool!

> * Fancy tracebacks and debugger support - right now pdb works like in
> the classic interpreter, but I'm sure IPython supports better
> debugging.
> * Shell support. (you can use os.system, but it's not the same thing)
> * '?' and '??'. I guess that in a graphical environment it's best if
> the help appears in a new window.

And the magics too :). 

> That's a great list. I hope I will be able to implement those. Thanks!

Good luck.

> > Does 'Ctrl-C' work in Dreampie?

> Works like charm... It was hard getting it right on Windows, but I managed it.

Good work. That's indeed hard.

By the way, DreamPie is licensed under GPL. This means that commercial
users (even those doing open source) are quite likely to reimplement it,
rather than to reuse it. That's a pitty, if you are putting so much
effort in it.

Best,

Ga?l


From Chris.Barker at noaa.gov  Sun Feb 21 15:48:55 2010
From: Chris.Barker at noaa.gov (Christopher Barker)
Date: Sun, 21 Feb 2010 12:48:55 -0800
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
	<20100221143937.GB32024@phare.normalesup.org>
	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
Message-ID: <4B819C37.9000108@noaa.gov>

Noam Yorav-Raphael wrote:
> * GUI event loops are there - GTK, Tkinter, and QT event loops are
> automatically supported, in a way very similar to Tkinter in the
> traditional interpreter - handling events while idle.

what about wxPython?

What GUI toolkit is Dreampie written in? It doesn't matter much, I 
suppose as long as it works with the others, but I'm wondering. Also, if 
it's GTK, then native MAc support isn't an option ( :-( ).

Can it run your app in a different process? Given the jython and iton 
python support, it looks like yes.

> * Shell support. (you can use os.system, but it's not the same thing)

yes -- that's really nice.

> * '?' and '??'. I guess that in a graphical environment it's best if
> the help appears in a new window.

not really -- there is a real efficiency to being able whip stuff out at 
the command line. It might be nice to have the help show up in a new 
window, but don't make me pick up the mouse to get it. typing ? is 
really easy.

Gael Varoquaux wrote:
> What I specificaly want is recalling based on the begining of a line.

yes - this is VERY handy.

I'm looking forward to giving it a real try.

-Chris



-- 
Christopher Barker, Ph.D.
Oceanographer

Emergency Response Division
NOAA/NOS/OR&R            (206) 526-6959   voice
7600 Sand Point Way NE   (206) 526-6329   fax
Seattle, WA  98115       (206) 526-6317   main reception

Chris.Barker at noaa.gov


From noamraph at gmail.com  Sun Feb 21 16:17:57 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 23:17:57 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <20100221150716.GA8338@phare.normalesup.org>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com> 
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> 
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com> 
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com> 
	<20100221143937.GB32024@phare.normalesup.org>
	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com> 
	<20100221150716.GA8338@phare.normalesup.org>
Message-ID: <b348a0851002211317i4cc6d600hcb047e218acf04dc@mail.gmail.com>

On Sun, Feb 21, 2010 at 5:07 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Sun, Feb 21, 2010 at 05:02:20PM +0200, Noam Yorav-Raphael wrote:
>> * History management is there - the recent values computed are saved,
>> and all your command and output history is saved too. You can even
>> save it to a HTML file.
>
> What I specificaly want is recalling based on the begining of a line.
>
Already there!

>> * Tracebacks are not really fancy, but are colored in red so that it's
>> easy to distinguish them from normal output.
>
> :( . IPython's traceback really improve my productivity, because they often
> allow me to pin-point the problem immediatly.
>
I think that adding an option for extended exceptions won't be too
hard. It's good to know what helps people.

>> * '?' and '??'. I guess that in a graphical environment it's best if
>> the help appears in a new window.
>
> And the magics too :).

Ok. Perhaps the best way would be to add an IPython mode...

> By the way, DreamPie is licensed under GPL. This means that commercial
> users (even those doing open source) are quite likely to reimplement it,
> rather than to reuse it. That's a pitty, if you are putting so much
> effort in it.
>
I don't think that's a real problem. Since DreamPie is a program, not
a library, LGPL isn't really applicable. It can be used for commercial
purposes, it's just that others can't add a feature and sell their
version.

Thanks for the feedback,
Noam


From noamraph at gmail.com  Sun Feb 21 16:24:02 2010
From: noamraph at gmail.com (Noam Yorav-Raphael)
Date: Sun, 21 Feb 2010 23:24:02 +0200
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <4B819C37.9000108@noaa.gov>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com> 
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com> 
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com> 
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com> 
	<20100221143937.GB32024@phare.normalesup.org>
	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com> 
	<4B819C37.9000108@noaa.gov>
Message-ID: <b348a0851002211324r759e1f12ye0b7b0f22fbf8a8c@mail.gmail.com>

On Sun, Feb 21, 2010 at 10:48 PM, Christopher Barker
<Chris.Barker at noaa.gov> wrote:
> Noam Yorav-Raphael wrote:
>> * GUI event loops are there - GTK, Tkinter, and QT event loops are
>> automatically supported, in a way very similar to Tkinter in the
>> traditional interpreter - handling events while idle.
>
> what about wxPython?
>
I haven't implemented it, but it shouldn't be a problem.

> What GUI toolkit is Dreampie written in? It doesn't matter much, I
> suppose as long as it works with the others, but I'm wondering. Also, if
> it's GTK, then native MAc support isn't an option ( :-( ).

It's PyGTK. I've heard that there is a way to use PyGTK on Mac, but
I'm sure that the support is not as good as linux and Windows. I've
chosen PyGTK because its Python bindings are MUCH better than PyQT.
>
> Can it run your app in a different process? Given the jython and iton
> python support, it looks like yes.

Yes. It actually always runs a Python subprocess which executes your
code, and the main one only handles the GUI.
>
>> * Shell support. (you can use os.system, but it's not the same thing)
>
> yes -- that's really nice.
>
>> * '?' and '??'. I guess that in a graphical environment it's best if
>> the help appears in a new window.
>
> not really -- there is a real efficiency to being able whip stuff out at
> the command line. It might be nice to have the help show up in a new
> window, but don't make me pick up the mouse to get it. typing ? is
> really easy.

Indeed. I thought of something like an extended tooltip. There's
currently a tooltip showing function arguments, and I guess it can be
extended to show more documentation.
>
> I'm looking forward to giving it a real try.
>
I will be glad to hear what you think once you do!

Noam


From warren.weckesser at enthought.com  Sun Feb 21 16:38:03 2010
From: warren.weckesser at enthought.com (Warren Weckesser)
Date: Sun, 21 Feb 2010 15:38:03 -0600
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002211324r759e1f12ye0b7b0f22fbf8a8c@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
	<20100221140449.GA32024@phare.normalesup.org>	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
	<20100221143937.GB32024@phare.normalesup.org>	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
	<4B819C37.9000108@noaa.gov>
	<b348a0851002211324r759e1f12ye0b7b0f22fbf8a8c@mail.gmail.com>
Message-ID: <4B81A7BB.5010906@enthought.com>

Noam Yorav-Raphael wrote:


<snip>
>>> * '?' and '??'. I guess that in a graphical environment it's best if
>>> the help appears in a new window.
>>>       
>> not really -- there is a real efficiency to being able whip stuff out at
>> the command line. It might be nice to have the help show up in a new
>> window, but don't make me pick up the mouse to get it. typing ? is
>> really easy.
>>     
>
> Indeed. I thought of something like an extended tooltip. There's
> currently a tooltip showing function arguments, and I guess it can be
> extended to show more documentation.
>   

Actually, while I find the ? *very* convenient and helpful, there is one 
behavior about it that often irks me.  It uses a "more"-like viewer for 
the text, but the text *goes away* when I hit 'q'.  Many times I want to 
keep the docstring visible while I proceed to enter a command.  I then 
have to resort to something like "print foo.__doc__".  (Or have I missed 
a key that exits the docstring view but leaves the text visible?)  
Having the docstring in a window would be much nicer in this case.

Warren


From listsin at integrateddevcorp.com  Sun Feb 21 16:53:44 2010
From: listsin at integrateddevcorp.com (Steve Steiner (listsin))
Date: Sun, 21 Feb 2010 16:53:44 -0500
Subject: [IPython-dev] DreamPie - a new graphical Python shell
In-Reply-To: <b348a0851002211324r759e1f12ye0b7b0f22fbf8a8c@mail.gmail.com>
References: <b348a0851002210156k22f859a3l85fd292faa462613@mail.gmail.com>
	<46cb515a1002210451q5ff3609ap897c22bdc60c4bd3@mail.gmail.com>
	<b348a0851002210551p508b2143n25990bbb2bfad48@mail.gmail.com>
	<20100221140449.GA32024@phare.normalesup.org>
	<b348a0851002210634j7a3a8ff3wead3cdf7073fcebd@mail.gmail.com>
	<20100221143937.GB32024@phare.normalesup.org>
	<b348a0851002210702j14574d08wc0941a88dce6f596@mail.gmail.com>
	<4B819C37.9000108@noaa.gov>
	<b348a0851002211324r759e1f12ye0b7b0f22fbf8a8c@mail.gmail.com>
Message-ID: <28B74AAD-3467-4768-AE1D-7B5B645BACDA@integrateddevcorp.com>


On Feb 21, 2010, at 4:24 PM, Noam Yorav-Raphael wrote:
>> What GUI toolkit is Dreampie written in? It doesn't matter much, I
>> suppose as long as it works with the others, but I'm wondering. Also, if
>> it's GTK, then native MAc support isn't an option ( :-( ).
> 
> It's PyGTK. I've heard that there is a way to use PyGTK on Mac,

I'm running it under PyGTK under the 64 bit MacPorts Python.  Assuming you have Python installed under MacPorts:  

# sudo port install py26-gtk2
# sudo port install py26-pygtksourceview 
# dreampie

And off to the races.  Found the font dialog so I could make it so I could see it and there you have it!

S



From drewm1980 at gmail.com  Mon Feb 22 01:24:21 2010
From: drewm1980 at gmail.com (Andrew Wagner)
Date: Mon, 22 Feb 2010 00:24:21 -0600
Subject: [IPython-dev] Having trouble using TaskClient;
	exception inside twisted
Message-ID: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>

Hello-

I'm trying to use the TaskClient infrastructure, and I'm hitting some
exceptions I don't understand.  My python and twisted are from debian
testing.  I believe the versions in stable are too old to support task
clients, etc, which was my main motive for switching to testing.

On importing I get the warnings:
____________
from IPython.kernel import client
   ...:

/usr/lib/pymodules/python2.5/IPython/kernel/clientconnector.py:43:
DeprecationWarning: Importing class Tub directly from 'foolscap' is
deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub
instead
  self.tub = Tub()
__________

On instantiation of a task client I get:

____________
In [7]: tc = client.TaskClient()
/usr/lib/pymodules/python2.5/IPython/kernel/taskfc.py:79:
DeprecationWarning: Importing class Referenceable1 directly from
'foolscap' is deprecated since Foolscap 0.4.3. Please import
foolscap.api.Referenceable instead
  class FCTaskControllerFromTaskController(Referenceable):
____________

Something similar for a multiengine client:

______________
In [8]: mec = client.MultiEngineClient()
/usr/lib/pymodules/python2.5/IPython/kernel/multienginefc.py:86:
DeprecationWarning: Importing class Referenceable1 directly from
'foolscap' is deprecated since Foolscap 0.4.3. Please import
foolscap.api.Referenceable instead
  class FCSynchronousMultiEngineFromMultiEngine(Referenceable):
_____________


The following example from the website and docs runs cleanly.
__________
serial_result = map(lambda x:x**10, range(200))
parallel_result = tc.map(lambda x:x**10, range(200))
assert serial_result==parallel_result
__________


However, when I run my code, at the point where I run tc.map() I'm
getting the following:

______________
Traceback (most recent call last):
  File "./objectives.py", line 165, in <module>
    e = objective_multithread(s)
  File "./objectives.py", line 150, in objective_multithread
    norms = tc.map(f, range(testingImageCount))
  File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py",
line 154, in map
    return self.mapper().map(func, *sequences)
  File "/usr/lib/pymodules/python2.5/IPython/kernel/mapper.py", line 230, in map
    task_results = [self.task_controller.get_task_result(tid) for tid
in task_ids]
  File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py",
line 93, in get_task_result
    taskid, block)
  File "/usr/lib/pymodules/python2.5/IPython/kernel/twistedutil.py",
line 72, in blockingCallFromThread
    return twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw)
  File "/usr/lib/python2.5/site-packages/twisted/internet/threads.py",
line 112, in blockingCallFromThread
    result.raiseException()
  File "/usr/lib/python2.5/site-packages/twisted/python/failure.py",
line 326, in raiseException
    raise self.type, self.value, self.tb
AttributeError: 'int' object has no attribute 'rfind'
________________

Any ideas/advice would be greatly appreciated.

Thanks,
Drew


From ellisonbg at gmail.com  Mon Feb 22 01:52:59 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Sun, 21 Feb 2010 22:52:59 -0800
Subject: [IPython-dev] Having trouble using TaskClient;
	exception inside 	twisted
In-Reply-To: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>
References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>
Message-ID: <fa8579a41002212252s735801f2yec360919235f7ee0@mail.gmail.com>

Andrew,

> I'm trying to use the TaskClient infrastructure, and I'm hitting some
> exceptions I don't understand. ?My python and twisted are from debian
> testing. ?I believe the versions in stable are too old to support task
> clients, etc, which was my main motive for switching to testing.

What version of IPython are you using?  Trunk or latest stable (0.10)?

> On importing I get the warnings:
> ____________
> from IPython.kernel import client
> ? ...:
>
> /usr/lib/pymodules/python2.5/IPython/kernel/clientconnector.py:43:
> DeprecationWarning: Importing class Tub directly from 'foolscap' is
> deprecated since Foolscap 0.4.3. Please import foolscap.api.Tub
> instead
> ?self.tub = Tub()
> __________
>
> On instantiation of a task client I get:
>
> ____________
> In [7]: tc = client.TaskClient()
> /usr/lib/pymodules/python2.5/IPython/kernel/taskfc.py:79:
> DeprecationWarning: Importing class Referenceable1 directly from
> 'foolscap' is deprecated since Foolscap 0.4.3. Please import
> foolscap.api.Referenceable instead
> ?class FCTaskControllerFromTaskController(Referenceable):
> ____________
>
> Something similar for a multiengine client:
>
> ______________
> In [8]: mec = client.MultiEngineClient()
> /usr/lib/pymodules/python2.5/IPython/kernel/multienginefc.py:86:
> DeprecationWarning: Importing class Referenceable1 directly from
> 'foolscap' is deprecated since Foolscap 0.4.3. Please import
> foolscap.api.Referenceable instead
> ?class FCSynchronousMultiEngineFromMultiEngine(Referenceable):
> _____________

Don't worry to much about these errors.  I am aware of them, but need
to make a small change
to get rid of the warning.  Could you file a bug report as a reminder for me:

https://bugs.launchpad.net/ipython

>
> The following example from the website and docs runs cleanly.
> __________
> serial_result = map(lambda x:x**10, range(200))
> parallel_result = tc.map(lambda x:x**10, range(200))
> assert serial_result==parallel_result
> __________

Great, so things are basically working...

>
> However, when I run my code, at the point where I run tc.map() I'm
> getting the following:
>
> ______________
> Traceback (most recent call last):
> ?File "./objectives.py", line 165, in <module>
> ? ?e = objective_multithread(s)
> ?File "./objectives.py", line 150, in objective_multithread
> ? ?norms = tc.map(f, range(testingImageCount))
> ?File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py",
> line 154, in map
> ? ?return self.mapper().map(func, *sequences)
> ?File "/usr/lib/pymodules/python2.5/IPython/kernel/mapper.py", line 230, in map
> ? ?task_results = [self.task_controller.get_task_result(tid) for tid
> in task_ids]
> ?File "/usr/lib/pymodules/python2.5/IPython/kernel/taskclient.py",
> line 93, in get_task_result
> ? ?taskid, block)
> ?File "/usr/lib/pymodules/python2.5/IPython/kernel/twistedutil.py",
> line 72, in blockingCallFromThread
> ? ?return twisted.internet.threads.blockingCallFromThread(reactor, f, *a, **kw)
> ?File "/usr/lib/python2.5/site-packages/twisted/internet/threads.py",
> line 112, in blockingCallFromThread
> ? ?result.raiseException()
> ?File "/usr/lib/python2.5/site-packages/twisted/python/failure.py",
> line 326, in raiseException
> ? ?raise self.type, self.value, self.tb
> AttributeError: 'int' object has no attribute 'rfind'
> ________________

Hmm, haven't seen this last one.  Could you put together a small, self
contained example that
demonstrates the error.  Then I can track it down.

Cheers,

Brian

> Any ideas/advice would be greatly appreciated.
>
> Thanks,
> Drew
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From drewm1980 at gmail.com  Mon Feb 22 10:28:28 2010
From: drewm1980 at gmail.com (Andrew Wagner)
Date: Mon, 22 Feb 2010 09:28:28 -0600
Subject: [IPython-dev] Having trouble using TaskClient;
	exception inside 	twisted
In-Reply-To: <fa8579a41002212252s735801f2yec360919235f7ee0@mail.gmail.com>
References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>
	<fa8579a41002212252s735801f2yec360919235f7ee0@mail.gmail.com>
Message-ID: <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com>

On Mon, Feb 22, 2010 at 12:52 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> What version of IPython are you using? ?Trunk or latest stable (0.10)?

IPython 0.10
Python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40)
twisted.version Version('twisted', 9, 0, 0)

> Don't worry to much about these errors. ?I am aware of them, but need
> to make a small change
> to get rid of the warning. ?Could you file a bug report as a reminder for me:
Done:
https://bugs.launchpad.net/ipython/+bug/525805

> Hmm, haven't seen this last one. ?Could you put together a small, self
> contained example that
> demonstrates the error. ?Then I can track it down.
Sure thing; I'll try to hammer one out asap.

Thanks Brian!


From ellisonbg at gmail.com  Mon Feb 22 13:37:46 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 22 Feb 2010 10:37:46 -0800
Subject: [IPython-dev] Having trouble using TaskClient;
	exception inside 	twisted
In-Reply-To: <795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com>
References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>
	<fa8579a41002212252s735801f2yec360919235f7ee0@mail.gmail.com>
	<795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com>
Message-ID: <fa8579a41002221037g51848f35me9f434c5578b6b8c@mail.gmail.com>

Andrew,

> IPython 0.10

I will be fixing these things in trunk.  We are about to release 0.11,
and it will have these fixes.

Cheers,

Brian

> Python 2.5.2 (r252:60911, Jan 24 2010, 17:44:40)
> twisted.version Version('twisted', 9, 0, 0)
>
>> Don't worry to much about these errors. ?I am aware of them, but need
>> to make a small change
>> to get rid of the warning. ?Could you file a bug report as a reminder for me:
> Done:
> https://bugs.launchpad.net/ipython/+bug/525805
>
>> Hmm, haven't seen this last one. ?Could you put together a small, self
>> contained example that
>> demonstrates the error. ?Then I can track it down.
> Sure thing; I'll try to hammer one out asap.
>
> Thanks Brian!
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From drewm1980 at gmail.com  Mon Feb 22 20:01:48 2010
From: drewm1980 at gmail.com (Andrew Wagner)
Date: Mon, 22 Feb 2010 19:01:48 -0600
Subject: [IPython-dev] Having trouble using TaskClient;
	exception inside 	twisted
In-Reply-To: <fa8579a41002221037g51848f35me9f434c5578b6b8c@mail.gmail.com>
References: <795840fe1002212224w487400c1qbda55d3f06e667d0@mail.gmail.com>
	<fa8579a41002212252s735801f2yec360919235f7ee0@mail.gmail.com>
	<795840fe1002220728m62fb2e3bvd4ef38258e1ced34@mail.gmail.com>
	<fa8579a41002221037g51848f35me9f434c5578b6b8c@mail.gmail.com>
Message-ID: <795840fe1002221701s79dcb989i8a19a895ff40a843@mail.gmail.com>

>>> Hmm, haven't seen this last one. ?Could you put together a small, self
>>> contained example that
>>> demonstrates the error. ?Then I can track it down.
>> Sure thing; I'll try to hammer one out asap.

I have replicated the error on the trunk:

IPython 0.11.alpha1.bzr.r1223

TaskClient.map is giving the same error as before.
I tried MultiEngineClient.map:

CompositeError: one or more exceptions from call to method: execute
[0:execute]: AttributeError: 'int' object has no attribute 'rfind'
[1:execute]: AttributeError: 'int' object has no attribute 'rfind'
[2:execute]: AttributeError: 'int' object has no attribute 'rfind'
[3:execute]: TypeError: map() requires at least two args
[4:execute]: TypeError: map() requires at least two args
[5:execute]: TypeError: map() requires at least two args
[6:execute]: TypeError: map() requires at least two args
[7:execute]: TypeError: map() requires at least two args

engines 0-2 did have a job to work on, and gave the same error as the TaskClient
engines 3-7 did not have a job to work on.

My network setup:
* client on machine 0
* ipcontroller on machine 1
* 8 ipengine instances on machine 2
All of the machines 0 and 2 mount my home directory from an NFS share
on machine 1, which is how the furl files get distributed.
I'm using /etc/inittab to keep the controller and engine instances
alive on machines 1 and 2.

By changing in the type of the second map parameter I figured out that
the 'int' in
"[0:execute]: AttributeError: 'int' object has no attribute 'rfind'"

is the type inside the list that I send.  i.e. for

norms = mec.map(f, [(i,) for i in range(3)])

I get:

[0:execute]: AttributeError: 'tuple' object has no attribute 'rfind'
...

It's odd that it complains about the type of my list of int's, but not
the list of int's in the example.

I'm still working on a small standalone version; just sending out info as I go.


From ellisonbg.net at gmail.com  Wed Feb 24 02:52:03 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 23 Feb 2010 23:52:03 -0800
Subject: [IPython-dev] using reST for representing the notebook
	cells+text
In-Reply-To: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
Message-ID: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>

Ondrej,

I have have thought about possible notebook formats many times.  IPython has
partially implemented at least two different notebook formats.  Both of them
have not been very successful.  The first was XML based and the second was
based on an sqlalchemy db.  Both were insanely complex.

Fernando and I have talked many times about a notebook format that is simply
restructured text.  There are many advantages of this approach:

* All of us use reST for lots of things any ways.
* There are existing parsers for reST.
* It naturally knows about code + text.
* Latex support, graphics.
* And more.

In many respects it seems almost perfect.  But, one day recently I was
thinking about how successful Mathematica's notebook continues to be.  So I
began to look more at the Mathematica notebook format.  Amazingly, the
Mathematica notebook format is a plain text file that itself is *valid
Mathematica code*.  This id documented here:

http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html

For examples a simple notebook with one text cell is just:

Notebook[{Cell['Here is my text', 'Text']}]

Everything - input cells, output cells, static images and all are
represented in this way and embedded in the plain text notebook file.  The
Python generalization of this would be the following:

* A Python notebook is plain text, importable Python code.
* That code is simply a tree of objects that declare the relevant parts of
the notebook.

This has a number of advantages:

* A notebook can be imported, manipulated and run by anyone who has the
support code (the notebook module that defines the relevant classes).
* A notebook doesn't need to be parsed.  It is valid Python and can be
imported or exec'd.  Once that is done, you have the full notebook in
memory.  You can immediately do anything you want with it.
* The various Notebook, Cell, Image, etc. classes can know about how to
output to various formats, latex, html, reST, XML, etc:

import mynotebook
mynotebook.notebook.export('rest')

* Each individual format (HTML, reST, latex) has weaknesses.  If you pick
any one to be *the* notebook format, you are building those weaknesses into
your design.  A pure python based notebook format won't suffer from that
syndrome.
* It is a clean separation of the model (Notebook, Cell, Image, etc.) and
the view (HTML, reST, etc.).  Picking HTML or reST for the notebook format
confuses (at some level) the model and view...
* Third party code can define new Notebook elements that specify how they
can be rendered in different contexts.  For example, matplotlib could ship a
Figure element that knows how to render itself as a native PyQt GUI, a
static image, a web page, etc.
* A notebook remains a single plain text file that anyone can edit - even if
it has embedded images.  Neither HTML nor reST have the ability to inline
graphics in plain text files.  While I love reST, it is a pain that I need
an entire directory of files to render a single Sphinx doc.

Anyways, that is where my brain is wandering these days.

Cheers,

Brian




On Tue, Feb 16, 2010 at 6:21 PM, Ondrej Certik <ondrej at certik.cz> wrote:

> Hi,
>
> has anyone given a deeper thought how to best implement this functionality?
>
> Currently the Sage notebook seems to remember the worksheet using:
>
> --------------
> sdsd
> system:sage
>
>
> <p>Well, some text</p>
>
> {{{id=5|
> awe
> ///
> }}}
>
> {{{id=1|
> 4
> ///
> 4
> }}}
>
> {{{id=2|
> 5
> ///
> 5
> }}}
>
> {{{id=3|
>
> ///
> }}}
> ---------------------
>
> E.g. each cell is either a text cell, or a cell that can be evaluated.
> Is that right?
>
> I know that Fernando suggested here:
>
>
> http://groups.google.com/group/sage-devel/browse_thread/thread/3f1f5a0c26975519/
>
> to use reST, e.g. something like:
>
> ----------
> Well, some text
>
> ::
>     >>> print 3
>     3
>
> ------------
>
>
> I am implementing a notebook prototype using pure Python
> (automatically translated to javascript using pyjamas) and I would
> like to run it on the google app engine (the javascript in your
> browser would then communicate either with the app engine, or your own
> hardware that would host Sage/FEMhub/your own code):
>
> http://gamma.sympy.org/nb/
>
> and here is my development version with logins and settings implemented:
>
> http://2.latest.sympy-gamma.appspot.com/nb/
>
> now I need to implement the worksheets, so I am thinking how to best
> represent it.
>
>
> Did the syntax for input/output blocks in Sphinx settled already? I
> guess internally I will implement the worksheet as a set of cells,
> either text cells, or "evaluatable" cells (that contain both
> input/output blocks) and then support conversion to and from
> reST/Sphinx.
>
> I would be interested in any feedback.
>
> Ondrej
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100223/c89fe6da/attachment.html>

From vivainio at gmail.com  Wed Feb 24 03:39:23 2010
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 24 Feb 2010 10:39:23 +0200
Subject: [IPython-dev] using reST for representing the notebook
	cells+text
In-Reply-To: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
Message-ID: <46cb515a1002240039r57a62acdh3d94d7df50c0e379@mail.gmail.com>

On Wed, Feb 17, 2010 at 4:21 AM, Ondrej Certik <ondrej at certik.cz> wrote:

> I am implementing a notebook prototype using pure Python
> (automatically translated to javascript using pyjamas) and I would

You may want to draw some inspiration from caliopy:

http://www.caliopywork.org/

-- 
Ville M. Vainio
http://tinyurl.com/vainio


From robert.kern at gmail.com  Wed Feb 24 10:49:56 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 24 Feb 2010 09:49:56 -0600
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> 
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
Message-ID: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>

On Wed, Feb 24, 2010 at 01:52, Brian Granger <ellisonbg.net at gmail.com> wrote:

> In many respects it seems almost perfect.? But, one day recently I was
> thinking about how successful Mathematica's notebook continues to be.? So I
> began to look more at the Mathematica notebook format.? Amazingly, the
> Mathematica notebook format is a plain text file that itself is *valid
> Mathematica code*.

The problem with transferring this to Python is that Mathematica's
language is a very Lispy one. ExpressionCell[] contains the actual
expression, not a string representing a Mathematica expression. You
can't do that with Python. You would have to put the Python code into
strings. You just have a "dumb" tree structure with text leaf nodes.
At that point, you might as well use XML to describe the exact same
structure. With XML, you can push it across to other languages,
JavaScript being the hugely important player here. Certainly, you are
going to have a Python API that will represent that tree of text nodes
as Python objects, but I just don't see the point of making the repr()
of that be the lingua franca format of the notebook file. It's just a
wasted opportunity.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From fperez.net at gmail.com  Wed Feb 24 11:39:13 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 24 Feb 2010 11:39:13 -0500
Subject: [IPython-dev] using reST for representing the notebook
	cells+text
In-Reply-To: <6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
Message-ID: <db6b5ecc1002240839i39791f24v9157eb24e2f12c1c@mail.gmail.com>

On Wed, Feb 24, 2010 at 2:52 AM, Brian Granger <ellisonbg.net at gmail.com> wrote:
>
> In many respects it seems almost perfect.? But, one day recently I was
> thinking about how successful Mathematica's notebook continues to be.? So I
> began to look more at the Mathematica notebook format.? Amazingly, the
> Mathematica notebook format is a plain text file that itself is *valid
> Mathematica code*.? This id documented here:
>
> http://reference.wolfram.com/mathematica/guide/LowLevelNotebookProgramming.html
>
> For examples a simple notebook with one text cell is just:
>
> Notebook[{Cell['Here is my text', 'Text']}]
>
> Everything - input cells, output cells, static images and all are
> represented in this way and embedded in the plain text notebook file.? The
> Python generalization of this would be the following:
>
> * A Python notebook is plain text, importable Python code.
> * That code is simply a tree of objects that declare the relevant parts of
> the notebook.
>
> This has a number of advantages:
>
> * A notebook can be imported, manipulated and run by anyone who has the
> support code (the notebook module that defines the relevant classes).
> * A notebook doesn't need to be parsed.? It is valid Python and can be
> imported or exec'd.? Once that is done, you have the full notebook in
> memory.? You can immediately do anything you want with it.
> * The various Notebook, Cell, Image, etc. classes can know about how to
> output to various formats, latex, html, reST, XML, etc:
>
> import mynotebook
> mynotebook.notebook.export('rest')
>
> * Each individual format (HTML, reST, latex) has weaknesses.? If you pick
> any one to be *the* notebook format, you are building those weaknesses into
> your design.? A pure python based notebook format won't suffer from that
> syndrome.
> * It is a clean separation of the model (Notebook, Cell, Image, etc.) and
> the view (HTML, reST, etc.).? Picking HTML or reST for the notebook format
> confuses (at some level) the model and view...
> * Third party code can define new Notebook elements that specify how they
> can be rendered in different contexts.? For example, matplotlib could ship a
> Figure element that knows how to render itself as a native PyQt GUI, a
> static image, a web page, etc.
> * A notebook remains a single plain text file that anyone can edit - even if
> it has embedded images.? Neither HTML nor reST have the ability to inline
> graphics in plain text files.? While I love reST, it is a pain that I need
> an entire directory of files to render a single Sphinx doc.

I had just composed a reply offline (I had no network for a bit) but I
just now saw that in the interim, Robert said pretty much what I had
written.  Since python isn't lisp(ish), I'm also not convinced that a
code-as-data approach will be all that useful for us.  I do think rest
can be the next best thing for us, since at least we have parsing
machinery (docutils), extensible document tools (sphinx), and any
tools on top of rest/sphinx would likely get a lot of reuse.

I added your notes to the ipython docs for reference though, it will
be good to have these notes all in one place.

Cheers,

f


From ellisonbg at gmail.com  Wed Feb 24 11:48:06 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 08:48:06 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
Message-ID: <fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>

[Happy to get a lively response]

> The problem with transferring this to Python is that Mathematica's
> language is a very Lispy one. ExpressionCell[] contains the actual
> expression, not a string representing a Mathematica expression. You
> can't do that with Python. You would have to put the Python code into
> strings.

Absolutely....almost.  You are right that the Notebook, Cell, etc. in
the Mathematica
notebook format are full Mathematica expressions.  But, to represent Mathematica
code, they use strings:

Here is a simple "a=10" input cell:

Cell[BoxData[
 RowBox[{"a", " ", "=", " ", "10"}]], "Input",
 CellChangeTimes->{{3.4759886492031307`*^9, 3.475988649761156*^9}}]

The reason (I am guessing) that Wolfram did this is that the Notebook
lives in the
frontend, the "code" (a=10) lives in the kernel.  Thus it doesn't make sense
to "instantiate" the code into a full expression until it is in the
backend.  Which is
*exactly* how this would work in Python/IPython.  By this, I mean that
the internal
representation of "code" in IPython is a string.

> You just have a "dumb" tree structure with text leaf nodes.

At some level I agree with you.  You would have a tree in Text, Input,
Output cells
that simply have plain text leaf nodes like Text("Here is the text").  But that
does not mean that the Cell and Notebook objects could not have powerful APIs
for working with these cells.  Things like:

* Representation (reST, HTML, latex, etc.)
* Insert, delete, move, etc.
* Get all the strings of python code to execute and inject their outputs.
* Whatever else you need.

Because the Notebook would live in the frontend, it would *not* contain
methods for code execution.  But it could have an API for working
with objects that interface with the kernel.

> At that point, you might as well use XML to describe the exact same
> structure. With XML, you can push it across to other languages,
> JavaScript being the hugely important player here. Certainly, you are
> going to have a Python API that will represent that tree of text nodes
> as Python objects, but I just don't see the point of making the repr()
> of that be the lingua franca format of the notebook file. It's just a
> wasted opportunity.

Again, I think the biggest issues with XML are:

* No one wants to edit XML by hand.
* The result is not importable in a regular Python shell.

Obviously, reST doesn't suffer from the first of these issues, but it
does share the second (as well
as not being able to embed binary data like images.

Cheers,

Brian


> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Wed Feb 24 11:55:43 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 24 Feb 2010 10:55:43 -0600
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> 
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> 
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> 
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
Message-ID: <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>

On Wed, Feb 24, 2010 at 10:48, Brian Granger <ellisonbg at gmail.com> wrote:
> [Happy to get a lively response]
>
>> The problem with transferring this to Python is that Mathematica's
>> language is a very Lispy one. ExpressionCell[] contains the actual
>> expression, not a string representing a Mathematica expression. You
>> can't do that with Python. You would have to put the Python code into
>> strings.
>
> Absolutely....almost. ?You are right that the Notebook, Cell, etc. in
> the Mathematica
> notebook format are full Mathematica expressions. ?But, to represent Mathematica
> code, they use strings:
>
> Here is a simple "a=10" input cell:
>
> Cell[BoxData[
> ?RowBox[{"a", " ", "=", " ", "10"}]], "Input",
> ?CellChangeTimes->{{3.4759886492031307`*^9, 3.475988649761156*^9}}]
>
> The reason (I am guessing) that Wolfram did this is that the Notebook
> lives in the
> frontend, the "code" (a=10) lives in the kernel. ?Thus it doesn't make sense
> to "instantiate" the code into a full expression until it is in the
> backend. ?Which is
> *exactly* how this would work in Python/IPython. ?By this, I mean that
> the internal
> representation of "code" in IPython is a string.
>
>> You just have a "dumb" tree structure with text leaf nodes.
>
> At some level I agree with you. ?You would have a tree in Text, Input,
> Output cells
> that simply have plain text leaf nodes like Text("Here is the text"). ?But that
> does not mean that the Cell and Notebook objects could not have powerful APIs
> for working with these cells.

I agree, but that doesn't imply that the main text serialization of
the objects in the API should be Python code.

>> At that point, you might as well use XML to describe the exact same
>> structure. With XML, you can push it across to other languages,
>> JavaScript being the hugely important player here. Certainly, you are
>> going to have a Python API that will represent that tree of text nodes
>> as Python objects, but I just don't see the point of making the repr()
>> of that be the lingua franca format of the notebook file. It's just a
>> wasted opportunity.
>
> Again, I think the biggest issues with XML are:
>
> * No one wants to edit XML by hand.

I don't want to write Python code in that tree style by hand, either.
Too many things to get wrong. I don't see anyone editing notebooks
outside of the notebook apps, honestly. Does anyone edit Mathematica
notebooks in a text editor? I don't think the success of Mathematica's
notebook is the file format; it's the notebook GUI itself.

> * The result is not importable in a regular Python shell.

<shrug> So you use a function call instead.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ellisonbg at gmail.com  Wed Feb 24 13:39:09 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 10:39:09 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
Message-ID: <fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>

>> * No one wants to edit XML by hand.
>
> I don't want to write Python code in that tree style by hand, either.

Yes, for the most part, but the argument for reST is that people can
edit it by hand.  Python is closer to that, but I agree that most people
won't probably do that.

> Too many things to get wrong. I don't see anyone editing notebooks
> outside of the notebook apps, honestly. Does anyone edit Mathematica
> notebooks in a text editor? I don't think the success of Mathematica's
> notebook is the file format; it's the notebook GUI itself.
>
>> * The result is not importable in a regular Python shell.
>
> <shrug> So you use a function call instead.

True, that works for getting an in memory rep of the notebook
(parsing).  But the
importability feature of a pure python notebook format has a broader
impact on the design.  This is related
to validation and extensibility.

validation: not all XML files are valid notebook files.  Thus, you
have to validate the XML at some level.  This means 1) either
developing a schema (which is very rigid) or 2) doing the validation
in Python once you parse the XML.  In the case of a pure python
notebook, validation is done by Python itself.  If the notebook can be
exec'd or imported, it is a valid notebook

extensibility: using XML requires specifying the XML schema in a
central location.  That schema is fixed and thus hard to extend.
Using Python allows the notebook format to be extended by simple
subclassing:

class MatplotlibFigureCell(Cell):


Sure, if your notebook uses the MatplotlibCell



> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Wed Feb 24 13:46:33 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 10:46:33 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
Message-ID: <fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>

Ooopps, hit keyboard shortcut for send...

> Using Python allows the notebook format to be extended by simple
> subclassing:
>
> class MatplotlibFigureCell(Cell):

This subclass could contain all the logic for representing the
matplotlib figure in different formats: html, jpeg, svg, native pyqt
gui, etc and could be distributed with matplotlib.

Sure, if your notebook uses the MatplotlibCell, you will need
matplotlib, but the validation of that aspect is handled by python
itself, not an XML schema.

Doing these types of things using XML would require all of this to be
put into the centralized XML schema, making it difficult for third
parties to extend.  Plus all of the additional logic associated with
that Cell type has to be put somewhere as well.

[I am thinking aloud here...not sure if all of this would pan out in
practice...only actual code would
tell...]

[For what it is worth, while I *love* the idea of having a reST based
notebook format, I think it is simply
too limiting in the long run.  Between reST and XML, I think XML wins...]

Cheers,

Brian


>
>
>
>> --
>> Robert Kern
>>
>> "I have come to believe that the whole world is an enigma, a harmless
>> enigma that is made terrible by our own mad attempt to interpret it as
>> though it had an underlying truth."
>> ?-- Umberto Eco
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ondrej at certik.cz  Wed Feb 24 13:55:13 2010
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 24 Feb 2010 10:55:13 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
Message-ID: <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>

On Wed, Feb 24, 2010 at 10:46 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Ooopps, hit keyboard shortcut for send...
>
>> Using Python allows the notebook format to be extended by simple
>> subclassing:
>>
>> class MatplotlibFigureCell(Cell):
>
> This subclass could contain all the logic for representing the
> matplotlib figure in different formats: html, jpeg, svg, native pyqt
> gui, etc and could be distributed with matplotlib.
>
> Sure, if your notebook uses the MatplotlibCell, you will need
> matplotlib, but the validation of that aspect is handled by python
> itself, not an XML schema.
>
> Doing these types of things using XML would require all of this to be
> put into the centralized XML schema, making it difficult for third
> parties to extend. ?Plus all of the additional logic associated with
> that Cell type has to be put somewhere as well.
>
> [I am thinking aloud here...not sure if all of this would pan out in
> practice...only actual code would
> tell...]
>
> [For what it is worth, while I *love* the idea of having a reST based
> notebook format, I think it is simply
> too limiting in the long run. ?Between reST and XML, I think XML wins...]

I will do reST import/export, but having it the default format, that I
think sucks. With xml, anyone can easily write a parsers for it in any
language.

Besides, I need to push the worksheet to the browser. In my case, I
use pyjamas, so it's pure Python, however, exec/eval are not evailable
there (as far as I know), but I am sure xml would work, here is some
code to handle xml in pyjamas:

http://pyjamas.sourceforge.net/examples/xmldoc/XMLload.py

I think, that it might work to push the whole Python objects somehow
(that's the nice thing about pyjamas, that since you have python on
both sides, you can do lots of cool stuff), but maybe not.

I like Brian's idea to be able to just import it in Python. But I
think I'll just stick to xml for the first iteration of my notebook,
to get the job done.

Ondrej


From robert.kern at gmail.com  Wed Feb 24 13:57:12 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 24 Feb 2010 12:57:12 -0600
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> 
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> 
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> 
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com> 
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com> 
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
Message-ID: <3d375d731002241057n3f7bc5b4h5e4ae3153586b083@mail.gmail.com>

On Wed, Feb 24, 2010 at 12:39, Brian Granger <ellisonbg at gmail.com> wrote:
>>> * No one wants to edit XML by hand.
>>
>> I don't want to write Python code in that tree style by hand, either.
>
> Yes, for the most part, but the argument for reST is that people can
> edit it by hand. ?Python is closer to that, but I agree that most people
> won't probably do that.

No, Python is *not* closer to reST than XML. reST is very different
from either XML or the Python object tree. The latter are much more
similar to each other than they are to reST.

>> Too many things to get wrong. I don't see anyone editing notebooks
>> outside of the notebook apps, honestly. Does anyone edit Mathematica
>> notebooks in a text editor? I don't think the success of Mathematica's
>> notebook is the file format; it's the notebook GUI itself.
>>
>>> * The result is not importable in a regular Python shell.
>>
>> <shrug> So you use a function call instead.
>
> True, that works for getting an in memory rep of the notebook
> (parsing). ?But the
> importability feature of a pure python notebook format has a broader
> impact on the design. ?This is related
> to validation and extensibility.
>
> validation: not all XML files are valid notebook files. ?Thus, you
> have to validate the XML at some level. ?This means 1) either
> developing a schema (which is very rigid) or 2) doing the validation
> in Python once you parse the XML. ?In the case of a pure python
> notebook, validation is done by Python itself. ?If the notebook can be
> exec'd or imported, it is a valid notebook

And this is what I think is most wrong with using Python to do this.
It's too flexible and too easy to create things that are incompatible
with other representations. It's fine to build things that are
incompatible (and I wouldn't want to prevent you from creating a
Notebook object that was incompatible with some of its
representations), but the main file format is a bad place to make that
limitation.

> extensibility: using XML requires specifying the XML schema in a
> central location. ?That schema is fixed and thus hard to extend.

The X is there for a reason. :-)  It stands for eXtensible. It really
isn't hard to add tags or to write systems that do sensible things
when they see tags that they don't handle. This is exactly what XML
was designed for.

> Using Python allows the notebook format to be extended by simple
> subclassing:
>
> class MatplotlibFigureCell(Cell):

<mpl_cell></mpl_cell>
<mpl:cell></mpl:cell>
<mpl:cell extends="cell"></mpl:cell>
<etc>...</etc>

> This subclass could contain all the logic for representing the
> matplotlib figure in different formats: html, jpeg, svg, native pyqt
> gui, etc and could be distributed with matplotlib.
>
> Sure, if your notebook uses the MatplotlibCell, you will need
> matplotlib, but the validation of that aspect is handled by python
> itself, not an XML schema.

*You don't need XML schemas to validate XML.* They are often entirely
superfluous. Now, you will need a Python API that builds Notebook
instances with Cells and all that jazz. And you will need a way to
build that object tree from XML. *That* is your validation. Algorithm
succeeds == valid file. The same is actually true for any other
format.

> Doing these types of things using XML would require all of this to be
> put into the centralized XML schema, making it difficult for third
> parties to extend. ?Plus all of the additional logic associated with
> that Cell type has to be put somewhere as well.

Even if you do develop an XML schema to use, these are easily made
extensible. Providing a mapping from tag names to classes is a
straightforward way to extend the parser.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ellisonbg at gmail.com  Wed Feb 24 14:15:15 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 11:15:15 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
	<85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
Message-ID: <fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>

> I will do reST import/export, but having it the default format, that I
> think sucks. With xml, anyone can easily write a parsers for it in any
> language.
>
> Besides, I need to push the worksheet to the browser. In my case, I
> use pyjamas, so it's pure Python, however, exec/eval are not evailable
> there (as far as I know), but I am sure xml would work, here is some
> code to handle xml in pyjamas:

A point of clarification.  The Notebook and Cell objects would not be
responsible
for the execution of python code.  The execution logic would simply
pull InputCells
out of the Notebook, extract the python code (as a str) and return the
OutputCell.

[Thinking out loud about what this might look like in a web
browser....more thought needed]

The Python Notebook and Cell objects would not be pyjamas entities - that
part of the python code would not be compiled to Javascript.  The
Notebook and Cell
objects would live in the web server and be capable of rendering
themselves as json
or html.

The pyjamas classes would contain the logic for the user interface,
and calling out to the
Notebook/Cell web service and pass XML/json back and forth....

> I think, that it might work to push the whole Python objects somehow
> (that's the nice thing about pyjamas, that since you have python on
> both sides, you can do lots of cool stuff), but maybe not.

Yes, that might work.  I have been thinking about the Cell objects as having
all the logic needed to render itself in various formats.  But maybe a
Cell object
itself could be lighter weight - the rendering logic could be a separate class.
Then the Cell objects could be compiled to JS using pyjamas.  But I am
not familiar
with how pyjamas works enough to see if this would work.

> I like Brian's idea to be able to just import it in Python. But I
> think I'll just stick to xml for the first iteration of my notebook,
> to get the job done.

Definitely don't wait for me to implement my idea...I do agree with
your statement about
the ease of parsing XML though.

Cheers,

Brian

> Ondrej
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ondrej at certik.cz  Wed Feb 24 15:10:11 2010
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 24 Feb 2010 12:10:11 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
	<85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
	<fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>
Message-ID: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>

On Wed, Feb 24, 2010 at 11:15 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>> I will do reST import/export, but having it the default format, that I
>> think sucks. With xml, anyone can easily write a parsers for it in any
>> language.
>>
>> Besides, I need to push the worksheet to the browser. In my case, I
>> use pyjamas, so it's pure Python, however, exec/eval are not evailable
>> there (as far as I know), but I am sure xml would work, here is some
>> code to handle xml in pyjamas:
>
> A point of clarification. ?The Notebook and Cell objects would not be
> responsible
> for the execution of python code. ?The execution logic would simply
> pull InputCells
> out of the Notebook, extract the python code (as a str) and return the
> OutputCell.
>
> [Thinking out loud about what this might look like in a web
> browser....more thought needed]
>
> The Python Notebook and Cell objects would not be pyjamas entities - that
> part of the python code would not be compiled to Javascript. ?The
> Notebook and Cell
> objects would live in the web server and be capable of rendering
> themselves as json
> or html.
>
> The pyjamas classes would contain the logic for the user interface,
> and calling out to the
> Notebook/Cell web service and pass XML/json back and forth....

That's right. I just did more studying of this and I don't want to
parse xml by hand in javascript/pyjamas. I would like to use the
builtin functionality
 for RPC, essentially, you just call python methods on both sides and
things get automatically transfered (using json/xml RPC) behind the
scene, it's very convenient.

>
>> I think, that it might work to push the whole Python objects somehow
>> (that's the nice thing about pyjamas, that since you have python on
>> both sides, you can do lots of cool stuff), but maybe not.
>
> Yes, that might work. ?I have been thinking about the Cell objects as having
> all the logic needed to render itself in various formats. ?But maybe a
> Cell object
> itself could be lighter weight - the rendering logic could be a separate class.
> Then the Cell objects could be compiled to JS using pyjamas. ?But I am
> not familiar
> with how pyjamas works enough to see if this would work.
>
>> I like Brian's idea to be able to just import it in Python. But I
>> think I'll just stick to xml for the first iteration of my notebook,
>> to get the job done.
>
> Definitely don't wait for me to implement my idea...I do agree with
> your statement about
> the ease of parsing XML though.

Here is my current code, that runs in the browser:

http://github.com/certik/sympy_gamma/blob/login/templates/nb.py

and here is the code, that runs on the server, listens at some url
(/eval_cell/...) and evaluates the cell:

http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126

it's just 15 lines, 7 of those are debug prints.


So there is no way to remember sessions, worksheets anything yet. So I
will have Notebook/Cells on the server, the eval_cell() method will
(besides the actual evaluation) keep the tree structure of
Notebook/Cells updated. This structure will know how to export/import
to/from many formats (reST, xml, ...), that part is clear.

The rendering itself is done using pyjamas, so the Notebook/Cells
classes don't need to know anything about it, they just contain the
logic.

On the browser part, I will also need some
Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for
rendering and I need somehow to initialize them from the server. I
think I will do that by implementing Notebook_pyjamas.add_cell()
method, that will be exported using RPC to the server, so the server
will call this method as many times as needed, and add there
parameters (input_text, output_text, type_of_the_cell="text"/"eval").

This would also solve my problem with passing data to the pyjamas app.
So far I passed it by saving it to the "meta" tags in the header of
the html page, and then getting it from inside pyjamas. But it sucks,
it's not very versatile. See e.g. here the code to read it:

http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L453

it reads the html elements and returns a Python dict with the options.
It's very fragile.


So this pyjamas development is a bit different to what one was used to
before (e.g. generating the html on the server), essentially the
philosophy is to initialize "Python" in the browser as soon as
possible and then do everything on the Python level. And generate all
html in the browser itself, from Python by constructing appropriate
pyjamas GUI widgets (e.g. I subclassed the TextArea widget that
implements the html textarea, with backspace/tab, shift+enter,
correctly resizing it, calculating (x, y) coordinates of the cursor
/nontrivial.../, etc, etc). It's kind of a fake, since there is no
Python in the browser, just javascript, so it took my a while to
realize that I can think of this really as having a Python interpreter
in the browser (resp. a static subset of Python).

Ondrej


From ellisonbg at gmail.com  Wed Feb 24 15:24:18 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 12:24:18 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
	<85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
	<fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>
	<85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>
Message-ID: <fa8579a41002241224p400fe46buc07624ea835f3209@mail.gmail.com>

Ondrej,

>> The pyjamas classes would contain the logic for the user interface,
>> and calling out to the
>> Notebook/Cell web service and pass XML/json back and forth....
>
> That's right. I just did more studying of this and I don't want to
> parse xml by hand in javascript/pyjamas. I would like to use the
> builtin functionality
> ?for RPC, essentially, you just call python methods on both sides and
> things get automatically transfered (using json/xml RPC) behind the
> scene, it's very convenient.

Nice, that was my impression with pyjamas.  Sounds cool.

> Here is my current code, that runs in the browser:
>
> http://github.com/certik/sympy_gamma/blob/login/templates/nb.py
>
> and here is the code, that runs on the server, listens at some url
> (/eval_cell/...) and evaluates the cell:
>
> http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126
>
> it's just 15 lines, 7 of those are debug prints.

This is super nice code.  Very easy to understand.  I really like how
pyjamas works.

>
> So there is no way to remember sessions, worksheets anything yet. So I
> will have Notebook/Cells on the server, the eval_cell() method will
> (besides the actual evaluation) keep the tree structure of
> Notebook/Cells updated. This structure will know how to export/import
> to/from many formats (reST, xml, ...), that part is clear.

Yes, you will have to add urls and handlers that know how to add
cells, delete cells,
etc.  I would follow a RESTful design of those URLS:

http://en.wikipedia.org/wiki/Representational_State_Transfer

> The rendering itself is done using pyjamas, so the Notebook/Cells
> classes don't need to know anything about it, they just contain the
> logic.

OK, this is the aspect of this model I still don't quite see - where
the representation
stuff is handled.

> On the browser part, I will also need some
> Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for
> rendering and I need somehow to initialize them from the server. I
> think I will do that by implementing Notebook_pyjamas.add_cell()
> method, that will be exported using RPC to the server, so the server
> will call this method as many times as needed, and add there
> parameters (input_text, output_text, type_of_the_cell="text"/"eval").

Yep.

> This would also solve my problem with passing data to the pyjamas app.
> So far I passed it by saving it to the "meta" tags in the header of
> the html page, and then getting it from inside pyjamas. But it sucks,
> it's not very versatile. See e.g. here the code to read it:
>
> http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L453
>
> it reads the html elements and returns a Python dict with the options.
> It's very fragile.

Right using json rpc will be much better.

> So this pyjamas development is a bit different to what one was used to
> before (e.g. generating the html on the server), essentially the
> philosophy is to initialize "Python" in the browser as soon as
> possible and then do everything on the Python level. And generate all
> html in the browser itself, from Python by constructing appropriate
> pyjamas GUI widgets (e.g. I subclassed the TextArea widget that
> implements the html textarea, with backspace/tab, shift+enter,
> correctly resizing it, calculating (x, y) coordinates of the cursor
> /nontrivial.../, etc, etc). It's kind of a fake, since there is no
> Python in the browser, just javascript, so it took my a while to
> realize that I can think of this really as having a Python interpreter
> in the browser (resp. a static subset of Python).

This is great ,I am curious to see how this evolves.

Brian

> Ondrej
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ondrej at certik.cz  Wed Feb 24 15:36:57 2010
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 24 Feb 2010 12:36:57 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
	<85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
	<fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>
	<85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>
Message-ID: <85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com>

On Wed, Feb 24, 2010 at 12:10 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> On Wed, Feb 24, 2010 at 11:15 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>> I will do reST import/export, but having it the default format, that I
>>> think sucks. With xml, anyone can easily write a parsers for it in any
>>> language.
>>>
>>> Besides, I need to push the worksheet to the browser. In my case, I
>>> use pyjamas, so it's pure Python, however, exec/eval are not evailable
>>> there (as far as I know), but I am sure xml would work, here is some
>>> code to handle xml in pyjamas:
>>
>> A point of clarification. ?The Notebook and Cell objects would not be
>> responsible
>> for the execution of python code. ?The execution logic would simply
>> pull InputCells
>> out of the Notebook, extract the python code (as a str) and return the
>> OutputCell.
>>
>> [Thinking out loud about what this might look like in a web
>> browser....more thought needed]
>>
>> The Python Notebook and Cell objects would not be pyjamas entities - that
>> part of the python code would not be compiled to Javascript. ?The
>> Notebook and Cell
>> objects would live in the web server and be capable of rendering
>> themselves as json
>> or html.
>>
>> The pyjamas classes would contain the logic for the user interface,
>> and calling out to the
>> Notebook/Cell web service and pass XML/json back and forth....
>
> That's right. I just did more studying of this and I don't want to
> parse xml by hand in javascript/pyjamas. I would like to use the
> builtin functionality
> ?for RPC, essentially, you just call python methods on both sides and
> things get automatically transfered (using json/xml RPC) behind the
> scene, it's very convenient.
>
>>
>>> I think, that it might work to push the whole Python objects somehow
>>> (that's the nice thing about pyjamas, that since you have python on
>>> both sides, you can do lots of cool stuff), but maybe not.
>>
>> Yes, that might work. ?I have been thinking about the Cell objects as having
>> all the logic needed to render itself in various formats. ?But maybe a
>> Cell object
>> itself could be lighter weight - the rendering logic could be a separate class.
>> Then the Cell objects could be compiled to JS using pyjamas. ?But I am
>> not familiar
>> with how pyjamas works enough to see if this would work.
>>
>>> I like Brian's idea to be able to just import it in Python. But I
>>> think I'll just stick to xml for the first iteration of my notebook,
>>> to get the job done.
>>
>> Definitely don't wait for me to implement my idea...I do agree with
>> your statement about
>> the ease of parsing XML though.
>
> Here is my current code, that runs in the browser:
>
> http://github.com/certik/sympy_gamma/blob/login/templates/nb.py
>
> and here is the code, that runs on the server, listens at some url
> (/eval_cell/...) and evaluates the cell:
>
> http://github.com/certik/sympy_gamma/blob/login/app/views.py#L126
>
> it's just 15 lines, 7 of those are debug prints.
>
>
> So there is no way to remember sessions, worksheets anything yet. So I
> will have Notebook/Cells on the server, the eval_cell() method will
> (besides the actual evaluation) keep the tree structure of
> Notebook/Cells updated. This structure will know how to export/import
> to/from many formats (reST, xml, ...), that part is clear.
>
> The rendering itself is done using pyjamas, so the Notebook/Cells
> classes don't need to know anything about it, they just contain the
> logic.
>
> On the browser part, I will also need some
> Notebook_pyjamas/Cell_pyjamas classes, that are purely responsible for
> rendering and I need somehow to initialize them from the server. I
> think I will do that by implementing Notebook_pyjamas.add_cell()
> method, that will be exported using RPC to the server, so the server
> will call this method as many times as needed, and add there
> parameters (input_text, output_text, type_of_the_cell="text"/"eval").

The browser part is actually already implemented (see the nb.py file
above), here is how I call it:

CellWidget
Worksheet

I think I will rename Worksheet to WorksheetWidget, then it will be
clear that it is just a widget.

Hm, so it is more complex. The WorksheetWidget currently contains
methods to join cells, etc. Currently all this "technical" logic is
determined in pyjamas (e.g. the cells can only by joined sometimes,
etc.). So I think the WorksheetWidget would just call
Notebook.join_cell() RPC call on the server, when the cells were
joined.

Note sure now, what to do with synchronization. E.g. let's say the RPC
call fails (no net connection), should the user wait until the RPC
call finishes to make sure the cells can be joined, or should they be
joined in the browser immediately and then somehow remember all the
pending changes to push them to the server? I will ignore this problem
for now.

I will start by implementing this RPC logic --- currently I don't use
RPC to evaluate cells, here is what I do now:

http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L335

I simply do an asynchronous html request, pass in my Loader()
instance, and its onCompletion() method gets called, I decode the JSON
data and that's it. So I will start by rewriting this part of the code
to use RPC, and then slowly increase the RPC interface on both sides
and implement Notebook and other things on the server and simply
communicate between the server and the browser more often.

Here is a demo how to use RPC in pyjamas:

http://pyjs.org/examples/jsonrpc/output/JSONRPCExample.html

and here is the code:

http://pyjs.org/examples/jsonrpc/JSONRPCExample.py

And more documentation is here:

http://pyjs.org/book/output/Bookreader.html#Rest%20of%20the%20World


So far this is how to call methods (implemented on the server) in
pyjamas, e.g. stuff like for evaluating cells. So after pyjamas
starts, I will call the server method "give_me_the_cells", and I guess
it can just pass me a python dictionary with the cells data, and I
initialize it according to it.

So it seems the RPC is always initiated from the browser.

> Yes, you will have to add urls and handlers that know how to add
> cells, delete cells,
> etc. ?I would follow a RESTful design of those URLS:
>
> http://en.wikipedia.org/wiki/Representational_State_Transfer

Actually, from the pyjamas side, I will just call methods, no need to
bother with URLs, and on the server side (django), there is also a
functionality in django, so that I don't have to bother with URLs
either. It's explained in the pyjamas docs link, above.

Ok, I think I know what to do now, there are still couple details that
are not clear to me, but it will sort itself out, once I find some
time to implement it.

Ondrej


From ellisonbg at gmail.com  Wed Feb 24 15:48:35 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 24 Feb 2010 12:48:35 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<fa8579a41002240848i7fdea02fxb66a930816b947b9@mail.gmail.com>
	<3d375d731002240855j19aeb233q39ae1328bc37d67b@mail.gmail.com>
	<fa8579a41002241039p73f327dci702e30bfc85dd8c3@mail.gmail.com>
	<fa8579a41002241046p62652bc5ifa01f6214e9c2428@mail.gmail.com>
	<85b5c3131002241055p1fbbd464u20ec20851bda14cc@mail.gmail.com>
	<fa8579a41002241115qa61849fgde1a3e627ecb8e4a@mail.gmail.com>
	<85b5c3131002241210y2c27d3b0w80afc77bcde4f5d1@mail.gmail.com>
	<85b5c3131002241236o257efd87v9f3d1c6f37bd105c@mail.gmail.com>
Message-ID: <fa8579a41002241248v74baed07y455e4205a7956576@mail.gmail.com>

Ondrej,

> The browser part is actually already implemented (see the nb.py file
> above), here is how I call it:
>
> CellWidget
> Worksheet
>
> I think I will rename Worksheet to WorksheetWidget, then it will be
> clear that it is just a widget.
>
> Hm, so it is more complex. The WorksheetWidget currently contains
> methods to join cells, etc. Currently all this "technical" logic is
> determined in pyjamas (e.g. the cells can only by joined sometimes,
> etc.). So I think the WorksheetWidget would just call
> Notebook.join_cell() RPC call on the server, when the cells were
> joined.

Yes.

> Note sure now, what to do with synchronization. E.g. let's say the RPC
> call fails (no net connection), should the user wait until the RPC
> call finishes to make sure the cells can be joined, or should they be
> joined in the browser immediately and then somehow remember all the
> pending changes to push them to the server? I will ignore this problem
> for now.

I think the browser should wait until it gets the server response - obviously
doing the browser side pending changes might speed it up, but it more work...

> I will start by implementing this RPC logic --- currently I don't use
> RPC to evaluate cells, here is what I do now:
>
> http://github.com/certik/sympy_gamma/blob/login/templates/nb.py#L335
>
> I simply do an asynchronous html request, pass in my Loader()
> instance, and its onCompletion() method gets called, I decode the JSON
> data and that's it. So I will start by rewriting this part of the code
> to use RPC, and then slowly increase the RPC interface on both sides
> and implement Notebook and other things on the server and simply
> communicate between the server and the browser more often.

The RPC will definitely be an improvement.

> Here is a demo how to use RPC in pyjamas:
>
> http://pyjs.org/examples/jsonrpc/output/JSONRPCExample.html
>
> and here is the code:
>
> http://pyjs.org/examples/jsonrpc/JSONRPCExample.py
>
> And more documentation is here:
>
> http://pyjs.org/book/output/Bookreader.html#Rest%20of%20the%20World

Nice, thanks for the links.

>
> So far this is how to call methods (implemented on the server) in
> pyjamas, e.g. stuff like for evaluating cells. So after pyjamas
> starts, I will call the server method "give_me_the_cells", and I guess
> it can just pass me a python dictionary with the cells data, and I
> initialize it according to it.
>
> So it seems the RPC is always initiated from the browser.

Yes, that is the traditional "AJAX" limitation.  Allowing the server
to initiate the RPC
goes under the name "Comet" and is much more difficult.

>> Yes, you will have to add urls and handlers that know how to add
>> cells, delete cells,
>> etc. ?I would follow a RESTful design of those URLS:
>>
>> http://en.wikipedia.org/wiki/Representational_State_Transfer
>
> Actually, from the pyjamas side, I will just call methods, no need to
> bother with URLs, and on the server side (django), there is also a
> functionality in django, so that I don't have to bother with URLs
> either. It's explained in the pyjamas docs link, above.

This is very nice.  I wish I had some time to play with pyjamas...

> Ok, I think I know what to do now, there are still couple details that
> are not clear to me, but it will sort itself out, once I find some
> time to implement it.

Cheers,

Brian

-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From termim at gmail.com  Wed Feb 24 18:29:11 2010
From: termim at gmail.com (Mikhail Terekhov)
Date: Wed, 24 Feb 2010 18:29:11 -0500
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
Message-ID: <12aaa0811002241529k76b509f3g7b9f1e4230e39452@mail.gmail.com>

On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern <robert.kern at gmail.com> wrote:
> [Did you mean to post this to one of the lists?]
>

Robert, yes I meant to send this to the list, sorry - hit the wrong button.
Apology for the list members if this discussion is a closed matter
already, I just thought it is interesting. I'm attaching our
conversation with Robert in hope it will be somehow useful.



On Wed, Feb 24, 2010 at 14:31, Mikhail Terekhov <termim at gmail.com>  wrote:
> On Wed, Feb 24, 2010 at 10:49 AM, Robert Kern <robert.kern at gmail.com> wrote:
>> On Wed, Feb 24, 2010 at 01:52, Brian Granger <ellisonbg.net at gmail.com> wrote:
>>
>>> In many respects it seems almost perfect.  But, one day recently I was
>>> thinking about how successful Mathematica's notebook continues to be.  So I
>>> began to look more at the Mathematica notebook format.  Amazingly, the
>>> Mathematica notebook format is a plain text file that itself is *valid
>>> Mathematica code*.
>>
>> The problem with transferring this to Python is that Mathematica's
>> language is a very Lispy one. ExpressionCell[] contains the actual
>> expression, not a string representing a Mathematica expression. You
>
> Not quite so, see for example:
> http://reference.wolfram.com/mathematica/ref/format/NB.html
>
> It is true that nb format contains a valid Mathematica's code but it is not
> the code user usually deals with but a lower level one where expressions
> are constructed from strings and API calls. This is exactly as Brian suggests
> AFAICS.
>
>> can't do that with Python. You would have to put the Python code into
>> strings. You just have a "dumb" tree structure with text leaf nodes.
>
> That is exactly as Mathematica does that, at least as it is described in
> the link above.

Quite so.

>> At that point, you might as well use XML to describe the exact same
>> structure.
>
> Sure you can but you have to pay a price for that and the price is XML
> parsing. While it looks negligible when your nb is just a couple of dozen
> lines it will be huge when you have to load many big nbs at once (think
> library) and when performance and/or memory are important. Subversion
> actually is a good example here. They started to use XML to store their
> internal workspace data but then were forced to switch to some simple
> text format due to memory/performance issues.

I am almost certain that their use cases and workloads are much
different than the notebook's would be. Python's parser isn't exactly
a speed demon, either. A general statement like "XML is slow" followed
by an unrelated anecdote is not terribly convincing. Show me
experiments. I've attached mine. Python ends up being about 3 times
slower than the equivalent XML for a variety of file sizes.

>> With XML, you can push it across to other languages,
>
> This is very ambitious and most of the time is just an illusion :(
> The chances that other projects throw away their's own formats (XML
> or not) and embrace this one are quite slim IMHO.

I'm not talking about other projects adopting anything. I'm talking
about basic capabilities of other languages, like JavaScript's builtin
support for parsing XML. That enables *us* to build things in
JavaScript.

> BTW the fact that
> everyone can parse XML doesn't mean that every one can _use_ the
> data right away.

Nor am I saying that. I am saying that it is enormously easier to
build the JavaScript parser for the XML representation rather than the
Python one.

> One have to have an internal logic/library/API specific
> to the data represented by some particular XML document. If you take
> this into account then the value of the exchange document format
> somewhat reduces. It is still not zero though and IMHO it is easy to
> teach classes proposed by Brian to produce XML representation just
> for the mythical interchange with something :)

The need for interchange is not at all mythical. Web frontends are
exactly what we are talking about in this thread.

>> JavaScript being the hugely important player here. Certainly, you are
>
> Again, it is important to define to what degree the interoperability with
> something like JavaScript is needed. If you plan to work on/modify/execute
> the same nbs in Python and in JavaScript then you have to implement
> compatible engine/API in Python _and_ in JavaScript. Are you sure you
> want to do that? If only the representation or "computed" notebook is
> needed for display purposes by JavaScript, then it is something different
> and could be implemented through specialized repr methods.

Or you could use the same mechanism for both instead of duplicating efforts.

>> going to have a Python API that will represent that tree of text nodes
>> as Python objects, but I just don't see the point of making the repr()
>> of that be the lingua franca format of the notebook file. It's just a
>> wasted opportunity.
>
> The point is that nb became a first class python object - just a module,
> no need for specialized parser and you can work with it as with regular
> Python module - just import and use it. The only difference is that nb is
> mutable - if you modified it then you have to save it.

I really don't see why having the file format be Python code makes it
any more of a first class object. The objects are the first class
objects. As long as loading to those objects is easy, the format just
doesn't matter. Loading an object by importing is actually a very
inflexible and difficult to work with method compared to a function
call.
- Show quoted text -

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
notebook.py	notebook.py
2K   Download
-------------- next part --------------
A non-text attachment was scrubbed...
Name: notebook.py
Type: text/x-python
Size: 1558 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100224/50f5414c/attachment.py>

From termim at gmail.com  Wed Feb 24 18:35:38 2010
From: termim at gmail.com (Mikhail Terekhov)
Date: Wed, 24 Feb 2010 18:35:38 -0500
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
Message-ID: <12aaa0811002241535w4df29502vefde93842dd92e10@mail.gmail.com>

On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern <robert.kern at gmail.com> wrote:
> [Did you mean to post this to one of the lists?]
>

Robert, yes I meant to send this to the list, sorry - hit the wrong button.
Apology for the list members if this discussion is a closed matter
already, I just thought it is interesting. I'm attaching our
conversation with Robert in hope it will be somehow useful.



On Wed, Feb 24, 2010 at 14:31, Mikhail Terekhov <termim at gmail.com>  wrote:
> On Wed, Feb 24, 2010 at 10:49 AM, Robert Kern <robert.kern at gmail.com> wrote:
>> On Wed, Feb 24, 2010 at 01:52, Brian Granger <ellisonbg.net at gmail.com> wrote:
>>
>>> In many respects it seems almost perfect.  But, one day recently I was
>>> thinking about how successful Mathematica's notebook continues to be.  So I
>>> began to look more at the Mathematica notebook format.  Amazingly, the
>>> Mathematica notebook format is a plain text file that itself is *valid
>>> Mathematica code*.
>>
>> The problem with transferring this to Python is that Mathematica's
>> language is a very Lispy one. ExpressionCell[] contains the actual
>> expression, not a string representing a Mathematica expression. You
>
> Not quite so, see for example:
> http://reference.wolfram.com/mathematica/ref/format/NB.html
>
> It is true that nb format contains a valid Mathematica's code but it is not
> the code user usually deals with but a lower level one where expressions
> are constructed from strings and API calls. This is exactly as Brian suggests
> AFAICS.
>
>> can't do that with Python. You would have to put the Python code into
>> strings. You just have a "dumb" tree structure with text leaf nodes.
>
> That is exactly as Mathematica does that, at least as it is described in
> the link above.

Quite so.

>> At that point, you might as well use XML to describe the exact same
>> structure.
>
> Sure you can but you have to pay a price for that and the price is XML
> parsing. While it looks negligible when your nb is just a couple of dozen
> lines it will be huge when you have to load many big nbs at once (think
> library) and when performance and/or memory are important. Subversion
> actually is a good example here. They started to use XML to store their
> internal workspace data but then were forced to switch to some simple
> text format due to memory/performance issues.

I am almost certain that their use cases and workloads are much
different than the notebook's would be. Python's parser isn't exactly
a speed demon, either. A general statement like "XML is slow" followed
by an unrelated anecdote is not terribly convincing. Show me
experiments. I've attached mine. Python ends up being about 3 times
slower than the equivalent XML for a variety of file sizes.

>> With XML, you can push it across to other languages,
>
> This is very ambitious and most of the time is just an illusion :(
> The chances that other projects throw away their's own formats (XML
> or not) and embrace this one are quite slim IMHO.

I'm not talking about other projects adopting anything. I'm talking
about basic capabilities of other languages, like JavaScript's builtin
support for parsing XML. That enables *us* to build things in
JavaScript.

> BTW the fact that
> everyone can parse XML doesn't mean that every one can _use_ the
> data right away.

Nor am I saying that. I am saying that it is enormously easier to
build the JavaScript parser for the XML representation rather than the
Python one.

> One have to have an internal logic/library/API specific
> to the data represented by some particular XML document. If you take
> this into account then the value of the exchange document format
> somewhat reduces. It is still not zero though and IMHO it is easy to
> teach classes proposed by Brian to produce XML representation just
> for the mythical interchange with something :)

The need for interchange is not at all mythical. Web frontends are
exactly what we are talking about in this thread.

>> JavaScript being the hugely important player here. Certainly, you are
>
> Again, it is important to define to what degree the interoperability with
> something like JavaScript is needed. If you plan to work on/modify/execute
> the same nbs in Python and in JavaScript then you have to implement
> compatible engine/API in Python _and_ in JavaScript. Are you sure you
> want to do that? If only the representation or "computed" notebook is
> needed for display purposes by JavaScript, then it is something different
> and could be implemented through specialized repr methods.

Or you could use the same mechanism for both instead of duplicating efforts.

>> going to have a Python API that will represent that tree of text nodes
>> as Python objects, but I just don't see the point of making the repr()
>> of that be the lingua franca format of the notebook file. It's just a
>> wasted opportunity.
>
> The point is that nb became a first class python object - just a module,
> no need for specialized parser and you can work with it as with regular
> Python module - just import and use it. The only difference is that nb is
> mutable - if you modified it then you have to save it.

I really don't see why having the file format be Python code makes it
any more of a first class object. The objects are the first class
objects. As long as loading to those objects is easy, the format just
doesn't matter. Loading an object by importing is actually a very
inflexible and difficult to work with method compared to a function
call.
- Show quoted text -

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
 -- Umberto Eco
notebook.py     notebook.py
2K   Download
-------------- next part --------------
A non-text attachment was scrubbed...
Name: notebook.py
Type: text/x-python
Size: 1558 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100224/ee6d2613/attachment.py>

From termim at gmail.com  Wed Feb 24 18:35:50 2010
From: termim at gmail.com (Mikhail Terekhov)
Date: Wed, 24 Feb 2010 18:35:50 -0500
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
Message-ID: <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com>

On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern <robert.kern at gmail.com> wrote:
>
> I am almost certain that their use cases and workloads are much
> different than the notebook's would be. Python's parser isn't exactly
> a speed demon, either. A general statement like "XML is slow" followed
> by an unrelated anecdote is not terribly convincing. Show me
> experiments. I've attached mine. Python ends up being about 3 times
> slower than the equivalent XML for a variety of file sizes.
>

Believe it or not, I can't find any example of a python project that started
implementing scientific notebook as an XML document and then switched to
something else :) I've used a "scientific analogy" principle. Seriously,
the Subversion is a real project and they really suffered from the decision to
use XML as a storage for the workspace meta data and they really switched away
from XML. No anecdotes.

IMHO the relation is quite simple - the things like Mathematica's
notebooks tend to
multiply and form libraries or collections. In this case XML parsing
could became a
problem.

Your example is not quite correct but it is a good start :} It
actually illustrates two
important points. First is that writing serializer that produces XML
representation
is easy. The second and more important is that after parsing XML you've got
nothing but internal XML representation and the only thing you can do with it is
to write back to a file, you still need to implement all the
functionality as a python
objects and convert the XML tree into python objects tree. Only after
that there
will be any point in benchmarking. At the contrary, the python's
version is complete
and if you had the real Notebook implementation then you were just
ready to use it.
Also note that there is no need to write, debug and support _any_
parser/reader,
python provides that for free.

>
> I'm not talking about other projects adopting anything. I'm talking
> about basic capabilities of other languages, like JavaScript's builtin
> support for parsing XML. That enables *us* to build things in
> JavaScript.
>
>> BTW the fact that
>> everyone can parse XML doesn't mean that every one can _use_ the
>> data right away.
>
> Nor am I saying that. I am saying that it is enormously easier to
> build the JavaScript parser for the XML representation rather than the
> Python one.
>

That is the real question - why JavaScript needs to read _interenal_
representation of the nb if it is not going to implement all the
needed functionality
to use it?

>> One have to have an internal logic/library/API specific
>> to the data represented by some particular XML document. If you take
>> this into account then the value of the exchange document format
>> somewhat reduces. It is still not zero though and IMHO it is easy to
>> teach classes proposed by Brian to produce XML representation just
>> for the mythical interchange with something :)
>
> The need for interchange is not at all mythical. Web front ends are
> exactly what we are talking about in this thread.
>

Sure, and It looks like in his very interesting approach the
JavaScript part is a
client that queries python server for information about nb it needs and there is
no need for JS to read nb or even know how it is stored on disk.

More general: internal representation does not have to be tightly coupled to
interfaces to external systems. Simplicity and reliability of the
internal representation
(in this case - just a regular python compiler versus custom XML
parser) outweighs
the need to write relatively simple export/interface functions that
give a view on the nb.
As Ondrej's work shows they are needed anyway and of coarse they can use XML if
it is easy for the client.

>>> JavaScript being the hugely important player here. Certainly, you are
>>
>> Again, it is important to define to what degree the interoperability with
>> something like JavaScript is needed. If you plan to work on/modify/execute
>> the same nbs in Python and in JavaScript then you have to implement
>> compatible engine/API in Python _and_ in JavaScript. Are you sure you
>> want to do that? If only the representation or "computed" notebook is
>> needed for display purposes by JavaScript, then it is something different
>> and could be implemented through specialized repr methods.
>
> Or you could use the same mechanism for both instead of duplicating efforts.
>

Unfortunately one have to duplicate something in either case. nb->XML would
duplicate nb->repr, but as your example shows the nb->XML is quite
straightforward.
In case XML->nb one have to duplicate python compiler which is unnecessary
in case repr->nb.

>>> going to have a Python API that will represent that tree of text nodes
>>> as Python objects, but I just don't see the point of making the repr()
>>> of that be the lingua franca format of the notebook file. It's just a
>>> wasted opportunity.
>>
>> The point is that nb became a first class python object - just a module,
>> no need for specialized parser and you can work with it as with regular
>> Python module - just import and use it. The only difference is that nb is
>> mutable - if you modified it then you have to save it.
>
> I really don't see why having the file format be Python code makes it
> any more of a first class object. The objects are the first class

You are right - not the first class, just a native python object.

> objects. As long as loading to those objects is easy, the format just

In a sense I agree, the only difference is that from the programming POV
loading cost for repr->nb is zero (all is done by the regular python compiler)
and XML->nb requires a special loader that should be maintained and updated
when the application changes.

> doesn't matter. Loading an object by importing is actually a very
> inflexible and difficult to work with method compared to a function
> call.
>

If one prefers functions one can always to use __import__() or imp.load_module()
functions instead of import statement.


Regards,
--
Mikhail Terekhov


From robert.kern at gmail.com  Wed Feb 24 18:47:24 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 24 Feb 2010 17:47:24 -0600
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> 
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> 
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> 
	<12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com>
Message-ID: <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com>

On Wed, Feb 24, 2010 at 17:04, Mikhail Terekhov <termim at gmail.com> wrote:
> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>
>> I am almost certain that their use cases and workloads are much
>> different than the notebook's would be. Python's parser isn't exactly
>> a speed demon, either. A general statement like "XML is slow" followed
>> by an unrelated anecdote is not terribly convincing. Show me
>> experiments. I've attached mine. Python ends up being about 3 times
>> slower than the equivalent XML for a variety of file sizes.
>
> Believe it or not, I can't find any example of a python project that started
> implementing scientific notebook as an XML document and then switched to
> something else :) I've used a "scientific analogy" principle. Seriously,
> the Subversion is a real project and they really suffered from the decision to
> use XML as a storage for the workspace meta data and they really switched away
> from XML. No anecdotes.

Yes, that is an anecdote. Anecdotes are true stories, but they are not
convincing data. This comparison is by no means scientific. There are
large differences between the use cases here. The simple text format
that Subversion moved to is not comparable to the Python format being
discussed here. Subversion faced different read/write loads than a
notebook will. I'm happy to consider other examples of projects
finding that their XML parsers were too slow, but you need to make a
more considered argument that the circumstances are similar enough to
the one we are talking about.

XML is not slow. XML is not fast. It cannot be either thing because it
is a file format, not an implementation. Parsers can be slow or fast.
cElementTree is particularly fast and rather faster than the Python
parser on equivalent data.

> IMHO the relation is quite simple - the things like Mathematica's
> notebooks tend to
> multiply and form libraries or collections. In this case XML parsing
> could became a
> problem.

I'm sorry, but this does not follow.

> Your example is not quite correct but it is a good start :} It
> actually illustrates two
> important points. First is that writing serializer that produces XML
> representation
> is easy. The second and more important is that after parsing XML you've got
> nothing but internal XML representation and the only thing you can do with it is
> to write back to a file, you still need to implement all the
> functionality as a python
> objects and convert the XML tree into python objects tree. Only after
> that there
> will be any point in benchmarking. At the contrary, the python's
> version is complete
> and if you had the real Notebook implementation then you were just
> ready to use it.
> Also note that there is no need to write, debug and support _any_
> parser/reader,
> python provides that for free.

I intended to measure the performance of the parsing, not the
difficulty of implementation of anything else. The performance of
constructing the objects is rather smaller, at least for XML. It is a
negligible cost on top of the XML parsing, but executing the Python
code actually imposes a much larger burden on the Python
implementation. See the attached updated benchmark. The Python method
now takes about 4x as much time to construct the Notebook instance
than the XML.

>> I'm not talking about other projects adopting anything. I'm talking
>> about basic capabilities of other languages, like JavaScript's builtin
>> support for parsing XML. That enables *us* to build things in
>> JavaScript.
>>
>>> BTW the fact that
>>> everyone can parse XML doesn't mean that every one can _use_ the
>>> data right away.
>>
>> Nor am I saying that. I am saying that it is enormously easier to
>> build the JavaScript parser for the XML representation rather than the
>> Python one.
>
> That is the real question - why JavaScript needs to read _interenal_
> representation of the nb if it is not going to implement all the
> needed functionality
> to use it?

File formats are not internal. They are external. They are the primary
method of interchange.

>>> One have to have an internal logic/library/API specific
>>> to the data represented by some particular XML document. If you take
>>> this into account then the value of the exchange document format
>>> somewhat reduces. It is still not zero though and IMHO it is easy to
>>> teach classes proposed by Brian to produce XML representation just
>>> for the mythical interchange with something :)
>>
>> The need for interchange is not at all mythical. Web front ends are
>> exactly what we are talking about in this thread.
>
> Sure, and It looks like in his very interesting approach the
> JavaScript part is a
> client that queries python server for information about nb it needs and there is
> no need for JS to read nb or even know how it is stored on disk.
>
> More general: internal representation does not have to be tightly coupled to
> interfaces to external systems.

Exactly. The Python code file format is a greater coupling to the
internal implementation, not a looser one.

> Simplicity and reliability of the
> internal representation
> (in this case - just a regular python compiler versus custom XML
> parser) outweighs
> the need to write relatively simple export/interface functions that
> give a view on the nb.

I disagree with that judgement and with the characterization of the
Python format as more simple and reliable.

> As Ondrej's work shows they are needed anyway and of coarse they can use XML if
> it is easy for the client.
>
>>>> JavaScript being the hugely important player here. Certainly, you are
>>>
>>> Again, it is important to define to what degree the interoperability with
>>> something like JavaScript is needed. If you plan to work on/modify/execute
>>> the same nbs in Python and in JavaScript then you have to implement
>>> compatible engine/API in Python _and_ in JavaScript. Are you sure you
>>> want to do that? If only the representation or "computed" notebook is
>>> needed for display purposes by JavaScript, then it is something different
>>> and could be implemented through specialized repr methods.
>>
>> Or you could use the same mechanism for both instead of duplicating efforts.
>
> Unfortunately one have to duplicate something in either case. nb->XML would
> duplicate nb->repr, but as your example shows the nb->XML is quite
> straightforward.
> In case XML->nb one have to duplicate python compiler which is unnecessary
> in case repr->nb.

Honestly, it's pretty trivial stuff.

>>>> going to have a Python API that will represent that tree of text nodes
>>>> as Python objects, but I just don't see the point of making the repr()
>>>> of that be the lingua franca format of the notebook file. It's just a
>>>> wasted opportunity.
>>>
>>> The point is that nb became a first class python object - just a module,
>>> no need for specialized parser and you can work with it as with regular
>>> Python module - just import and use it. The only difference is that nb is
>>> mutable - if you modified it then you have to save it.
>>
>> I really don't see why having the file format be Python code makes it
>> any more of a first class object. The objects are the first class
>
> You are right - not the first class, just a native python object.

A file with Python code is not a native Python object. From the API
user's perspective, they call a function or execute a statement and
then they have an object. It's exactly the same for any format. There
is no benefit for either case.

>> objects. As long as loading to those objects is easy, the format just
>
> In a sense I agree, the only difference is that from the programming POV
> loading cost for repr->nb is zero (all is done by the regular python compiler)
> and XML->nb requires a special loader that should be maintained and updated
> when the application changes.

Actually, that raises another objection I have to using Python code as
the file format. The file format is intimately tied to the internal
implementation. What if we want to change the internal implementation?
It *will* happen. With a more neutral representation, you can read
older files as long as you update the reader appropriately. If you are
just executing code, you are stuck with maintaining the classes with
backwards compatible argument specs forever and won't be able to make
some of the changes that you want down the road. Format versioning is
a *huge* issue whenever you design file formats. XML and other generic
representations permit this.

The reason that Mathematica can get away with this is that it is a
Lispish language. Although you correctly point out that being able to
do things like ExpressionCell aren't particularly important, being
able to load the code into a neutral tree structure and manipulate it
before actually instantiating your API classes it is.

>> doesn't matter. Loading an object by importing is actually a very
>> inflexible and difficult to work with method compared to a function
>> call.
>
> If one prefers functions one can always to use __import__() or imp.load_module()
> functions instead of import statement.

Abusing a Python-internal API as your main file loading API is just
not a good practice.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco
-------------- next part --------------
A non-text attachment was scrubbed...
Name: notebook.py
Type: application/octet-stream
Size: 1620 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100224/cabf73ec/attachment.obj>

From dsdale24 at gmail.com  Thu Feb 25 12:51:46 2010
From: dsdale24 at gmail.com (Darren Dale)
Date: Thu, 25 Feb 2010 12:51:46 -0500
Subject: [IPython-dev] question about generics.complete_object
Message-ID: <a08e5f81002250951h18178bd5y568c69f14ac117c9@mail.gmail.com>

How is generics.complete_object supposed to work in the trunk? I have
a custom completer that uses it, and it used to work, but now
complete_object keeps returning TryNext:

In [12]: generics.complete_object('foo', dir('foo'))
---------------------------------------------------------------------------
TryNext                                   Traceback (most recent call last)

/Users/darren/Projects/phynx.git/<ipython console> in <module>()

/Users/darren/Projects/ipython/IPython/external/simplegeneric.pyc in
dispatch(*args, **kw)
     73                 f = _gbt(t, _sentinel)
     74                 if f is not _sentinel:
---> 75                     return f(*args, **kw)
     76             else:
     77                 return func(*args, **kw)

/Users/darren/Projects/ipython/IPython/utils/generics.pyc in
complete_object(obj, prev_completions)
     60     own_attrs + prev_completions.
     61     """
---> 62     raise TryNext
     63
     64

TryNext:


Is this a bug, or am I expecting functionality that no longer exists?

Thanks,
Darren


From ellisonbg at gmail.com  Thu Feb 25 12:56:26 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 25 Feb 2010 09:56:26 -0800
Subject: [IPython-dev] question about generics.complete_object
In-Reply-To: <a08e5f81002250951h18178bd5y568c69f14ac117c9@mail.gmail.com>
References: <a08e5f81002250951h18178bd5y568c69f14ac117c9@mail.gmail.com>
Message-ID: <fa8579a41002250956l7c6a642due44a65402a53c548@mail.gmail.com>

Darren,

> How is generics.complete_object supposed to work in the trunk? I have
> a custom completer that uses it, and it used to work, but now
> complete_object keeps returning TryNext:

It should still work.  This looks like a bug.  If you don't have time
to look into this
yourself, could you submit a bug report.

Thanks!

Brian

> In [12]: generics.complete_object('foo', dir('foo'))
> ---------------------------------------------------------------------------
> TryNext ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? ? Traceback (most recent call last)
>
> /Users/darren/Projects/phynx.git/<ipython console> in <module>()
>
> /Users/darren/Projects/ipython/IPython/external/simplegeneric.pyc in
> dispatch(*args, **kw)
> ? ? 73 ? ? ? ? ? ? ? ? f = _gbt(t, _sentinel)
> ? ? 74 ? ? ? ? ? ? ? ? if f is not _sentinel:
> ---> 75 ? ? ? ? ? ? ? ? ? ? return f(*args, **kw)
> ? ? 76 ? ? ? ? ? ? else:
> ? ? 77 ? ? ? ? ? ? ? ? return func(*args, **kw)
>
> /Users/darren/Projects/ipython/IPython/utils/generics.pyc in
> complete_object(obj, prev_completions)
> ? ? 60 ? ? own_attrs + prev_completions.
> ? ? 61 ? ? """
> ---> 62 ? ? raise TryNext
> ? ? 63
> ? ? 64
>
> TryNext:
>
>
> Is this a bug, or am I expecting functionality that no longer exists?
>
> Thanks,
> Darren
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Thu Feb 25 13:32:54 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 25 Feb 2010 10:32:54 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com>
	<3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com>
Message-ID: <fa8579a41002251032l1cf23fa5u4b00268e903cf7b4@mail.gmail.com>

I would expect that parsing an XML notebook would be faster than
importing/exec'ing a pure python based
notebook (as Robert's tests shows).  It is not clear to me that
performance is the central concern, but it could be
in some situations.  We all know that long import times are an issue
for some projects.

Robert, do you have any thoughts about the validation/extensibility
issues related to XML that I brought up?

I think at this point, those are my main concerns with XML versus pure python.

Cheers,

Brian

On Wed, Feb 24, 2010 at 3:47 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On Wed, Feb 24, 2010 at 17:04, Mikhail Terekhov <termim at gmail.com> wrote:
>> On Wed, Feb 24, 2010 at 4:04 PM, Robert Kern <robert.kern at gmail.com> wrote:
>>>
>>> I am almost certain that their use cases and workloads are much
>>> different than the notebook's would be. Python's parser isn't exactly
>>> a speed demon, either. A general statement like "XML is slow" followed
>>> by an unrelated anecdote is not terribly convincing. Show me
>>> experiments. I've attached mine. Python ends up being about 3 times
>>> slower than the equivalent XML for a variety of file sizes.
>>
>> Believe it or not, I can't find any example of a python project that started
>> implementing scientific notebook as an XML document and then switched to
>> something else :) I've used a "scientific analogy" principle. Seriously,
>> the Subversion is a real project and they really suffered from the decision to
>> use XML as a storage for the workspace meta data and they really switched away
>> from XML. No anecdotes.
>
> Yes, that is an anecdote. Anecdotes are true stories, but they are not
> convincing data. This comparison is by no means scientific. There are
> large differences between the use cases here. The simple text format
> that Subversion moved to is not comparable to the Python format being
> discussed here. Subversion faced different read/write loads than a
> notebook will. I'm happy to consider other examples of projects
> finding that their XML parsers were too slow, but you need to make a
> more considered argument that the circumstances are similar enough to
> the one we are talking about.
>
> XML is not slow. XML is not fast. It cannot be either thing because it
> is a file format, not an implementation. Parsers can be slow or fast.
> cElementTree is particularly fast and rather faster than the Python
> parser on equivalent data.
>
>> IMHO the relation is quite simple - the things like Mathematica's
>> notebooks tend to
>> multiply and form libraries or collections. In this case XML parsing
>> could became a
>> problem.
>
> I'm sorry, but this does not follow.
>
>> Your example is not quite correct but it is a good start :} It
>> actually illustrates two
>> important points. First is that writing serializer that produces XML
>> representation
>> is easy. The second and more important is that after parsing XML you've got
>> nothing but internal XML representation and the only thing you can do with it is
>> to write back to a file, you still need to implement all the
>> functionality as a python
>> objects and convert the XML tree into python objects tree. Only after
>> that there
>> will be any point in benchmarking. At the contrary, the python's
>> version is complete
>> and if you had the real Notebook implementation then you were just
>> ready to use it.
>> Also note that there is no need to write, debug and support _any_
>> parser/reader,
>> python provides that for free.
>
> I intended to measure the performance of the parsing, not the
> difficulty of implementation of anything else. The performance of
> constructing the objects is rather smaller, at least for XML. It is a
> negligible cost on top of the XML parsing, but executing the Python
> code actually imposes a much larger burden on the Python
> implementation. See the attached updated benchmark. The Python method
> now takes about 4x as much time to construct the Notebook instance
> than the XML.
>
>>> I'm not talking about other projects adopting anything. I'm talking
>>> about basic capabilities of other languages, like JavaScript's builtin
>>> support for parsing XML. That enables *us* to build things in
>>> JavaScript.
>>>
>>>> BTW the fact that
>>>> everyone can parse XML doesn't mean that every one can _use_ the
>>>> data right away.
>>>
>>> Nor am I saying that. I am saying that it is enormously easier to
>>> build the JavaScript parser for the XML representation rather than the
>>> Python one.
>>
>> That is the real question - why JavaScript needs to read _interenal_
>> representation of the nb if it is not going to implement all the
>> needed functionality
>> to use it?
>
> File formats are not internal. They are external. They are the primary
> method of interchange.
>
>>>> One have to have an internal logic/library/API specific
>>>> to the data represented by some particular XML document. If you take
>>>> this into account then the value of the exchange document format
>>>> somewhat reduces. It is still not zero though and IMHO it is easy to
>>>> teach classes proposed by Brian to produce XML representation just
>>>> for the mythical interchange with something :)
>>>
>>> The need for interchange is not at all mythical. Web front ends are
>>> exactly what we are talking about in this thread.
>>
>> Sure, and It looks like in his very interesting approach the
>> JavaScript part is a
>> client that queries python server for information about nb it needs and there is
>> no need for JS to read nb or even know how it is stored on disk.
>>
>> More general: internal representation does not have to be tightly coupled to
>> interfaces to external systems.
>
> Exactly. The Python code file format is a greater coupling to the
> internal implementation, not a looser one.
>
>> Simplicity and reliability of the
>> internal representation
>> (in this case - just a regular python compiler versus custom XML
>> parser) outweighs
>> the need to write relatively simple export/interface functions that
>> give a view on the nb.
>
> I disagree with that judgement and with the characterization of the
> Python format as more simple and reliable.
>
>> As Ondrej's work shows they are needed anyway and of coarse they can use XML if
>> it is easy for the client.
>>
>>>>> JavaScript being the hugely important player here. Certainly, you are
>>>>
>>>> Again, it is important to define to what degree the interoperability with
>>>> something like JavaScript is needed. If you plan to work on/modify/execute
>>>> the same nbs in Python and in JavaScript then you have to implement
>>>> compatible engine/API in Python _and_ in JavaScript. Are you sure you
>>>> want to do that? If only the representation or "computed" notebook is
>>>> needed for display purposes by JavaScript, then it is something different
>>>> and could be implemented through specialized repr methods.
>>>
>>> Or you could use the same mechanism for both instead of duplicating efforts.
>>
>> Unfortunately one have to duplicate something in either case. nb->XML would
>> duplicate nb->repr, but as your example shows the nb->XML is quite
>> straightforward.
>> In case XML->nb one have to duplicate python compiler which is unnecessary
>> in case repr->nb.
>
> Honestly, it's pretty trivial stuff.
>
>>>>> going to have a Python API that will represent that tree of text nodes
>>>>> as Python objects, but I just don't see the point of making the repr()
>>>>> of that be the lingua franca format of the notebook file. It's just a
>>>>> wasted opportunity.
>>>>
>>>> The point is that nb became a first class python object - just a module,
>>>> no need for specialized parser and you can work with it as with regular
>>>> Python module - just import and use it. The only difference is that nb is
>>>> mutable - if you modified it then you have to save it.
>>>
>>> I really don't see why having the file format be Python code makes it
>>> any more of a first class object. The objects are the first class
>>
>> You are right - not the first class, just a native python object.
>
> A file with Python code is not a native Python object. From the API
> user's perspective, they call a function or execute a statement and
> then they have an object. It's exactly the same for any format. There
> is no benefit for either case.
>
>>> objects. As long as loading to those objects is easy, the format just
>>
>> In a sense I agree, the only difference is that from the programming POV
>> loading cost for repr->nb is zero (all is done by the regular python compiler)
>> and XML->nb requires a special loader that should be maintained and updated
>> when the application changes.
>
> Actually, that raises another objection I have to using Python code as
> the file format. The file format is intimately tied to the internal
> implementation. What if we want to change the internal implementation?
> It *will* happen. With a more neutral representation, you can read
> older files as long as you update the reader appropriately. If you are
> just executing code, you are stuck with maintaining the classes with
> backwards compatible argument specs forever and won't be able to make
> some of the changes that you want down the road. Format versioning is
> a *huge* issue whenever you design file formats. XML and other generic
> representations permit this.
>
> The reason that Mathematica can get away with this is that it is a
> Lispish language. Although you correctly point out that being able to
> do things like ExpressionCell aren't particularly important, being
> able to load the code into a neutral tree structure and manipulate it
> before actually instantiating your API classes it is.
>
>>> doesn't matter. Loading an object by importing is actually a very
>>> inflexible and difficult to work with method compared to a function
>>> call.
>>
>> If one prefers functions one can always to use __import__() or imp.load_module()
>> functions instead of import statement.
>
> Abusing a Python-internal API as your main file loading API is just
> not a good practice.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From robert.kern at gmail.com  Thu Feb 25 13:37:24 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 25 Feb 2010 12:37:24 -0600
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <fa8579a41002251032l1cf23fa5u4b00268e903cf7b4@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com> 
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com> 
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com> 
	<12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com> 
	<3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com> 
	<fa8579a41002251032l1cf23fa5u4b00268e903cf7b4@mail.gmail.com>
Message-ID: <3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com>

On Thu, Feb 25, 2010 at 12:32, Brian Granger <ellisonbg at gmail.com> wrote:

> Robert, do you have any thoughts about the validation/extensibility
> issues related to XML that I brought up?

Yes:

http://mail.scipy.org/pipermail/ipython-dev/2010-February/005830.html

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ellisonbg at gmail.com  Thu Feb 25 13:54:41 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 25 Feb 2010 10:54:41 -0800
Subject: [IPython-dev] [sympy] Re: using reST for representing the
	notebook cells+text
In-Reply-To: <3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com>
References: <85b5c3131002161821i43fb2f95u66f9182eb2c223a4@mail.gmail.com>
	<6ce0ac131002232352r309e679aq1084a9122d2e6115@mail.gmail.com>
	<3d375d731002240749o455f315dt44c5a392af34b1af@mail.gmail.com>
	<12aaa0811002241535i57ac2503q2bbb7bbcd5ab2ba@mail.gmail.com>
	<3d375d731002241547m24d710b7y88db30e694946da3@mail.gmail.com>
	<fa8579a41002251032l1cf23fa5u4b00268e903cf7b4@mail.gmail.com>
	<3d375d731002251037x68dd56c3u707bb00534d312c6@mail.gmail.com>
Message-ID: <fa8579a41002251054s3fba1942p47aeb4396d1956b7@mail.gmail.com>

Opps, I missed that.  Thanks.

Brian

On Thu, Feb 25, 2010 at 10:37 AM, Robert Kern <robert.kern at gmail.com> wrote:
> On Thu, Feb 25, 2010 at 12:32, Brian Granger <ellisonbg at gmail.com> wrote:
>
>> Robert, do you have any thoughts about the validation/extensibility
>> issues related to XML that I brought up?
>
> Yes:
>
> http://mail.scipy.org/pipermail/ipython-dev/2010-February/005830.html
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From dsdale24 at gmail.com  Thu Feb 25 14:43:21 2010
From: dsdale24 at gmail.com (Darren Dale)
Date: Thu, 25 Feb 2010 14:43:21 -0500
Subject: [IPython-dev] question about generics.complete_object
In-Reply-To: <fa8579a41002250956l7c6a642due44a65402a53c548@mail.gmail.com>
References: <a08e5f81002250951h18178bd5y568c69f14ac117c9@mail.gmail.com>
	<fa8579a41002250956l7c6a642due44a65402a53c548@mail.gmail.com>
Message-ID: <a08e5f81002251143u12124844s5e09d1696a42756c@mail.gmail.com>

On Thu, Feb 25, 2010 at 12:56 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> Darren,
>
>> How is generics.complete_object supposed to work in the trunk? I have
>> a custom completer that uses it, and it used to work, but now
>> complete_object keeps returning TryNext:
>
> It should still work. ?This looks like a bug. ?If you don't have time
> to look into this
> yourself, could you submit a bug report.
>
> Thanks!

I tried to locate the problem, but was unsuccessful. The bug report is
at https://bugs.edge.launchpad.net/ipython/+bug/527968

Darren


From ellisonbg at gmail.com  Thu Feb 25 15:19:44 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 25 Feb 2010 12:19:44 -0800
Subject: [IPython-dev] question about generics.complete_object
In-Reply-To: <a08e5f81002251143u12124844s5e09d1696a42756c@mail.gmail.com>
References: <a08e5f81002250951h18178bd5y568c69f14ac117c9@mail.gmail.com>
	<fa8579a41002250956l7c6a642due44a65402a53c548@mail.gmail.com>
	<a08e5f81002251143u12124844s5e09d1696a42756c@mail.gmail.com>
Message-ID: <fa8579a41002251219n7a85328p7d1aebd1395f2842@mail.gmail.com>

Darren,

Thanks, we will make sure that is fixed before 0.11 is released.

Cheers,

Brian

On Thu, Feb 25, 2010 at 11:43 AM, Darren Dale <dsdale24 at gmail.com> wrote:
> On Thu, Feb 25, 2010 at 12:56 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>> Darren,
>>
>>> How is generics.complete_object supposed to work in the trunk? I have
>>> a custom completer that uses it, and it used to work, but now
>>> complete_object keeps returning TryNext:
>>
>> It should still work. ?This looks like a bug. ?If you don't have time
>> to look into this
>> yourself, could you submit a bug report.
>>
>> Thanks!
>
> I tried to locate the problem, but was unsuccessful. The bug report is
> at https://bugs.edge.launchpad.net/ipython/+bug/527968
>
> Darren
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg.net at gmail.com  Fri Feb 26 12:47:00 2010
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Fri, 26 Feb 2010 09:47:00 -0800
Subject: [IPython-dev] GUI support: conflicts between IPython 0.11 and
	Matplotlib/ETS
In-Reply-To: <4B7DBE28.90702@noaa.gov>
References: <6ce0ac131002092059h6bfad66agf4c01d125caa075a@mail.gmail.com>
	<201002121206.51045.hans_meine@gmx.net>
	<20100212111104.GA7228@phare.normalesup.org>
	<201002121225.05301.hans_meine@gmx.net>
	<20100212113903.GB7228@phare.normalesup.org>
	<db6b5ecc1002132326y6b929b76uddf0bc1743333b1e@mail.gmail.com>
	<4B7DBE28.90702@noaa.gov>
Message-ID: <6ce0ac131002260947y52aa366dgf9c33dd15554dee1@mail.gmail.com>

All,

I had a thought. It seems that it's really impossible (or at least
> impractical) to monkey-patch the GUI toolkits such that you can run a
> regular old unmodified app exactly the same way inside and outside of
> ipython. Gael's right -- that way lies loss of hair (and I"ve lost
> enough in the last ten years as it is).
>
> However, as it happens, I've been using ipython for a wx GUI app of mine
> (with no ETS or MPL), and it sort-of-works, and also locks up and
> crashes fairly regularly. So the threading solution isn't really robust
> either. Also, when wx starts using Cocoa on OS-X, putting the GUI in
> another thread won't be an option anymore anyway.
>
>
The old threaded code is a train wreck - it is not thread safe in the least
bit
and we have to do crazy hack to get Ctrl-C to work across threads.  The
result is, well, an eventual train wreck.

So:
>
> > - non-threaded gui support with pyos_inputhook, that would require
> > some code changes.  If this can give a really robust solution and the
> > needed changes aren't too intrusive, I could see matplotlib, ETS and
> > possibly others adding the necessary adaptation (especially if we
> > provide it in ipython already so for the users it ends up being almost
> > a one-liner).
>
> Exactly -- while you can't expect unmodified regular old wx code to
> work, we could provide alternative app startup code that a user can
> simply cut and past in, and then have an app that will work with and
> without ipython -- I think that's a fine solution.
>
>
Yes, this should be fairly easy to do.

Cheers,

Brian


> So for now, is there a sample, minimal wx app that works with the new
> pyos_inputhook code?
>
> -Chris
>
>
>
>
>
> --
> Christopher Barker, Ph.D.
> Oceanographer
>
> Emergency Response Division
> NOAA/NOS/OR&R            (206) 526-6959   voice
> 7600 Sand Point Way NE   (206) 526-6329   fax
> Seattle, WA  98115       (206) 526-6317   main reception
>
> Chris.Barker at noaa.gov
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100226/28eba64c/attachment.html>