From ondrej at certik.cz  Mon Jan  7 09:02:48 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 7 Jan 2008 15:02:48 +0100
Subject: [IPython-dev] what is the best way of doing pretty print with
	ipython
Message-ID: <85b5c3130801070602n73d080ffjf28211627f9feb3@mail.gmail.com>

Hi,

in SymPy, we use ipython as an easy to use shell and generally it
works incredibly well:

$ bin/isympy
Python 2.4.4 console for SymPy 0.5.11-hg. These commands were executed:
>>> from __future__ import division
>>> from sympy import *
>>> x, y, z = symbols('xyz')
>>> k, m, n = symbols('kmn', integer=True)
>>> f = Function("f")
>>> Basic.set_repr_level(2)     # pretty print output; Use "1" for python output
>>> pprint_try_use_unicode()    # use unicode pretty print when available

Documentation can be found at http://sympy.org/


In [1]: integr
integrals  integrate

In [1]: integrate(x**2 * cos(x), x)
Out[1]:
             2
-2*sin(x) + x *sin(x) + 2*x*cos(x)

In [2]: x**2
Out[2]:
 2
x

In [3]:


However, we are abusing python repr() function as shown here:

In [3]: repr(x**2)
Out[3]: ' 2\nx '

In [4]: str(x**2)
Out[4]: 'x**2'

The repr() should return just one line and be something that can be
"almost" parsed back to python.

We can control this behavior:

In [1]: x**2
Out[1]:
 2
x

In [2]: Basic.set_repr_level(1)
Out[2]: 2

In [3]: x**2
Out[3]: x**2


But it doesn't help always, consider:

In [1]: (x**2, x**3, y)
Out[1]:
( 2
x ,  3
x , y)

In [2]: Basic.set_repr_level(1)
Out[2]: 2

In [3]: (x**2, x**3, y)
Out[3]: (x**2, x**3, y)


the [3] is fine, but [1] is completely messed up.

OK, how to fix this? My idea is to use repr() just for the 1D
printing, like in [3] and force ipython to use maybe str()? Or our
custom method.

What would you recommend as the best way to approach this in ipython?

Thanks,
Ondrej


From ondrej at certik.cz  Mon Jan  7 09:05:48 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 7 Jan 2008 15:05:48 +0100
Subject: [IPython-dev] what is the best way of doing pretty print with
	ipython
In-Reply-To: <85b5c3130801070602n73d080ffjf28211627f9feb3@mail.gmail.com>
References: <85b5c3130801070602n73d080ffjf28211627f9feb3@mail.gmail.com>
Message-ID: <85b5c3130801070605x15601ae6s2a5bbc7c1f47e5f4@mail.gmail.com>

On Jan 7, 2008 3:02 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> Hi,
>
> in SymPy, we use ipython as an easy to use shell and generally it
> works incredibly well:
>
> $ bin/isympy
> Python 2.4.4 console for SymPy 0.5.11-hg. These commands were executed:
> >>> from __future__ import division
> >>> from sympy import *
> >>> x, y, z = symbols('xyz')
> >>> k, m, n = symbols('kmn', integer=True)
> >>> f = Function("f")
> >>> Basic.set_repr_level(2)     # pretty print output; Use "1" for python output
> >>> pprint_try_use_unicode()    # use unicode pretty print when available
>
> Documentation can be found at http://sympy.org/
>
>
> In [1]: integr
> integrals  integrate
>
> In [1]: integrate(x**2 * cos(x), x)
> Out[1]:
>              2
> -2*sin(x) + x *sin(x) + 2*x*cos(x)
>
> In [2]: x**2
> Out[2]:
>  2
> x
>
> In [3]:
>
>
> However, we are abusing python repr() function as shown here:
>
> In [3]: repr(x**2)
> Out[3]: ' 2\nx '
>
> In [4]: str(x**2)
> Out[4]: 'x**2'
>
> The repr() should return just one line and be something that can be
> "almost" parsed back to python.
>
> We can control this behavior:
>
> In [1]: x**2
> Out[1]:
>  2
> x
>
> In [2]: Basic.set_repr_level(1)
> Out[2]: 2
>
> In [3]: x**2
> Out[3]: x**2
>
>
> But it doesn't help always, consider:
>
> In [1]: (x**2, x**3, y)
> Out[1]:
> ( 2
> x ,  3
> x , y)
>
> In [2]: Basic.set_repr_level(1)
> Out[2]: 2
>
> In [3]: (x**2, x**3, y)
> Out[3]: (x**2, x**3, y)
>
>
> the [3] is fine, but [1] is completely messed up.

BTW, here we discovered a more serious issue with ipython and repr
returning several lines with unicode:

http://code.google.com/p/sympy/issues/detail?id=484

Is the problem in ipython or sympy? I guess there is some bug in
ipython too, because it messes up the terminal when I return from
ipython.
So whatever broken program ipython executes (sympy), it should return
the terminal in a sane state.

Ondrej


From fperez.net at gmail.com  Thu Jan 10 01:44:39 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 9 Jan 2008 23:44:39 -0700
Subject: [IPython-dev] Visual programming with ipython1/Vision and Django
	news...
Message-ID: <db6b5ecc0801092244p54694598yabd31762cddc027a@mail.gmail.com>

Howdy,

just a blip of news from the front page, two recent IPython-related items:

== IPython1 and the Scripps Institute's Vision ==

[http://www.osc.edu/~unpingco Jose Unpingco] made this really neat
[http://www.osc.edu/~unpingco/Tutorial_11Dec.html screencast] showing
how to couple IPython1 with the [http://mgltools.scripps.edu Vision]
environment.  Vision is an extremely impressive visual programming
environment developed by [http://www.scripps.edu/~sanner Michel
Sanner's] team at the Scripps Institute in La Jolla, CA.

== IPython and Django ==

A [http://blog.petersheats.com/2008/01/09/autoloading-your-django-models/
blog entry] by Peter Sheats describing how to use IPython as the
interactive shell for Django.

Cheers,

f


From anand.prabhakar.patil at gmail.com  Thu Jan 10 16:21:08 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Thu, 10 Jan 2008 13:21:08 -0800
Subject: [IPython-dev] ConnectionRefusedError when trying to start ipengines
Message-ID: <2bc7a5a50801101321t7becd422ye0f01ac372eb4a57@mail.gmail.com>

Hi all,

Has anyone seen an error like this when trying to start an ipengine?
Possibly useful information: I have denyhosts running but can ssh to my own
machine without getting blocked.

Thanks,
Anand Patil

anand$ ipengine
2008/01/10 14:15 -0700 [-] Log opened.
2008/01/10 14:15 -0700 [-] Starting factory <
ipython1.kernel.enginepb.PBEngineClientFactory object at 0x13c3350>
2008/01/10 14:15 -0700 [Uninitialized] Unhandled Error
        Traceback (most recent call last):
        Failure: twisted.internet.error.ConnectionRefusedError: Connection
was refused by other side: 111: Connection refused.

2008/01/10 14:15 -0700 [Uninitialized] Stopping factory <
ipython1.kernel.enginepb.PBEngineClientFactory object at 0x13c3350>


< I push ctrl-C>
2008/01/10 14:20 -0700 [-] Received SIGINT, shutting down.
2008/01/10 14:20 -0700 [-] Main loop terminated.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/0c724925/attachment.html>

From anand.prabhakar.patil at gmail.com  Thu Jan 10 16:39:42 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Thu, 10 Jan 2008 13:39:42 -0800
Subject: [IPython-dev] ConnectionRefusedError when trying to start
	ipengines
In-Reply-To: <2bc7a5a50801101321t7becd422ye0f01ac372eb4a57@mail.gmail.com>
References: <2bc7a5a50801101321t7becd422ye0f01ac372eb4a57@mail.gmail.com>
Message-ID: <2bc7a5a50801101339t265dfbf8mc79f7d13cef3691f@mail.gmail.com>

On Jan 10, 2008 1:21 PM, Anand Patil <anand.prabhakar.patil at gmail.com>
wrote:

> Hi all,
>
> Has anyone seen an error like this when trying to start an ipengine?
> Possibly useful information: I have denyhosts running but can ssh to my own
> machine without getting blocked.
>
> Thanks,
> Anand Patil
>
> anand$ ipengine
> 2008/01/10 14:15 -0700 [-] Log opened.
> 2008/01/10 14:15 -0700 [-] Starting factory <
> ipython1.kernel.enginepb.PBEngineClientFactory object at 0x13c3350>
> 2008/01/10 14:15 -0700 [Uninitialized] Unhandled Error
>         Traceback (most recent call last):
>         Failure: twisted.internet.error.ConnectionRefusedError: Connection
> was refused by other side: 111: Connection refused.
>
> 2008/01/10 14:15 -0700 [Uninitialized] Stopping factory <
> ipython1.kernel.enginepb.PBEngineClientFactory object at 0x13c3350>
>
>
> < I push ctrl-C>
> 2008/01/10 14:20 -0700 [-] Received SIGINT, shutting down.
> 2008/01/10 14:20 -0700 [-] Main loop terminated.
>

Another note: I can run ipcluster and use the resulting engines without
problems. The reason I want to get ipengine working is so that I can use
mpi4py, since I'm passing large arrays around.

Thanks,
AnandPatil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/4d1a3a39/attachment.html>

From fperez.net at gmail.com  Thu Jan 10 16:40:24 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Jan 2008 14:40:24 -0700
Subject: [IPython-dev] ConnectionRefusedError when trying to start
	ipengines
In-Reply-To: <2bc7a5a50801101321t7becd422ye0f01ac372eb4a57@mail.gmail.com>
References: <2bc7a5a50801101321t7becd422ye0f01ac372eb4a57@mail.gmail.com>
Message-ID: <db6b5ecc0801101340t3ec52947u70192870ba4f4470@mail.gmail.com>

Hi Anand,

On Jan 10, 2008 2:21 PM, Anand Patil <anand.prabhakar.patil at gmail.com> wrote:
> Hi all,
>
> Has anyone seen an error like this when trying to start an ipengine?
> Possibly useful information: I have denyhosts running but can ssh to my own
> machine without getting blocked.

First question: is the controller already running?  And on the same
machine?  Because you aren't telling the engine where to find the
controller, so it will look for it on the same host. That error means
the engine can't connect to the controller.

If not, are the engine and the controller on the same subnet?  Is the
controller running a firewall? The engine tries to connect to the
controller, by default, on ports > 10000, though you can control this
via:

ipcontroller --engine-port=10001              # listen on 10001
ipengine --controller-port=10001              # connect to 10001

as indicated in

http://ipython.scipy.org/moin/Parallel_Computing/Tutorial

But it's possible that either a firewall on your controller, or
tcpwrappers, is blocking access to those ports from anything but
localhost.

One thing you can try is to make the 'cluster' entirely local to the
controller, by running on it

ipcluster -n 2

That will make one controller and two engines on the same box, and
will sort out if connections to that port are only being blocked when
coming from outside of localhost.  In fact, once it's running, you can
try adding another engine to it.

Let us know if this helps any...

Cheers,

f


From laurent.dufrechou at gmail.com  Thu Jan 10 16:55:04 2008
From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=)
Date: Thu, 10 Jan 2008 22:55:04 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
Message-ID: <47869441.0437560a.77de.3616@mx.google.com>

Hello there,

Here is finally a new version with a completely refactored display widget.

I've switched to scintilla because of the difference in behavior between
windows and linux.

The benefit is that now history widget got syntax highlighting J

 

You can change background color with line 37 in wxIPython.py.(if you prefer
white, also if you've got a better color layout proposal for white I'm ok !)

 

So linux user.. is this ok now??? (I've treid under ubuntu so should be ok!)

Solved performances bugs discovered by Prabhu and a bug in the pager too
thanks to Stefan.

Supports also 'exit()' and '?'

For pyreadline, well no update on this side, I still don't see where
pyreadline could integrate with scintilla there.

 

On the other side I've discovered pida, they've got a cool console widget
where if you try to launch ipython you've good a beautifull ipython widget !

The main difference is that the thread approach I use permits IPython/GUi
interaction while pida process approach can't interact (or it's more
difficult) with the GUI.

 

How and last news , as I use scintilla now, and that Ipython thread class is
GUI independent, I can port it to pyQT ;)

More news next month (got lot of work in my job right now!)

 

Regards,

Laurent

 

Ps:

I've commited all of this under ipython/trunk/ipython/gui.  Don't know if
it's ok as it the first time I do this. J

 

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/76d360fe/attachment.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: wxIpython0.8.zip
Type: application/x-zip-compressed
Size: 16741 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/76d360fe/attachment.bin>

From vivainio at gmail.com  Thu Jan 10 17:09:42 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 11 Jan 2008 00:09:42 +0200
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47869441.0437560a.77de.3616@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
Message-ID: <46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>

On Jan 10, 2008 11:55 PM, Laurent Dufrechou <laurent.dufrechou at gmail.com> wrote:

> I've commited all of this under ipython/trunk/ipython/gui.  Don't know if
> it's ok as it the first time I do this? J

Have you considered how this will be distributed? Should we create a
separate installer?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From anand.prabhakar.patil at gmail.com  Thu Jan 10 17:25:11 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Thu, 10 Jan 2008 14:25:11 -0800
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter 5000X1000
	float array to 7 engines
Message-ID: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>

Hi all,

Just trying to figure out whether something is wrong with my installation:
I'm using mpi4py with mpich, on a machine with two quad-core 3.0Ghz intel
processors. The following script takes 7.21s (wall time) with seven
IPEngines, with almost all of the time spent in the scatterAll() operation:


from numpy import *
from ipython1 import *
import ipython1.kernel.api as kernel
rc = kernel.RemoteController(('127.0.0.1',10105))

C=ones((5000,1000),dtype=float)
rc.scatterAll('C',C)

rc.resetAll()


Is this fairly typical, and if not is there anything I can do to speed up
the pushing and pulling?

Thanks,
Anand Patil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/a4dd9983/attachment.html>

From laurent.dufrechou at free.fr  Thu Jan 10 17:43:57 2008
From: laurent.dufrechou at free.fr (Laurent Dufrechou)
Date: Thu, 10 Jan 2008 23:43:57 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>
Message-ID: <00b401c853da$49e61310$ddb23930$@dufrechou@free.fr>

Hum not at all.
I've designed this to be reusable in any wx gui doing something like import
ipython.gui.wx.ipython_view
(with this you get the ipython shell inside a wx.panel)
The wxIpython.py is just some sort of demo on how to use it and
ipython_history a demo widget to show how this can interact with the GUI.
Perhaps if you find the app usefull we can bundle it alone, but that 's not
what it was done for :)
I'm would greatly prefer this attached to ipython installer and if you find
it usefull, just add a link into windos/kde menu?
Do you think it could be usefull for you an app more polished like idle?
Or do you prefer a port to QT (I mean the same demo app but QT based to show
to people that they can use Ipython in QT)?
(perhaps the two ;) )

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de Ville M. Vainio
Envoy??: jeudi 10 janvier 2008 23:10
??: Laurent Dufrechou
Cc?: IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

On Jan 10, 2008 11:55 PM, Laurent Dufrechou <laurent.dufrechou at gmail.com>
wrote:

> I've commited all of this under ipython/trunk/ipython/gui.  Don't know if
> it's ok as it the first time I do this
 J

Have you considered how this will be distributed? Should we create a
separate installer?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev



From fperez.net at gmail.com  Thu Jan 10 18:09:53 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Jan 2008 16:09:53 -0700
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <-6709300660534023315@unknownmsgid>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>
	<-6709300660534023315@unknownmsgid>
Message-ID: <db6b5ecc0801101509v540b3f34vdb008399d23d68c2@mail.gmail.com>

On Jan 10, 2008 3:43 PM, Laurent Dufrechou <laurent.dufrechou at free.fr> wrote:
> Hum not at all.
> I've designed this to be reusable in any wx gui doing something like import
> ipython.gui.wx.ipython_view
> (with this you get the ipython shell inside a wx.panel)
> The wxIpython.py is just some sort of demo on how to use it and
> ipython_history a demo widget to show how this can interact with the GUI.
> Perhaps if you find the app usefull we can bundle it alone, but that 's not
> what it was done for :)
> I'm would greatly prefer this attached to ipython installer and if you find
> it usefull, just add a link into windos/kde menu?
> Do you think it could be usefull for you an app more polished like idle?
> Or do you prefer a port to QT (I mean the same demo app but QT based to show
> to people that they can use Ipython in QT)?
> (perhaps the two ;) )

This should be shipped as part of ipython.  WX won't be a dependency
for ipython, but if you have wx you can use this.  But having it
always there is good (just like ipipes and their wx support, or our
multi-gui support).

I actually will  start playing with this for the shell refactoring in
ipython1.  In a few weeks I'll be switching jobs and shortly after
that I should have a bit more time to devote to some of our
infrastructure.

Cheers,

f


From anand.prabhakar.patil at gmail.com  Thu Jan 10 22:28:29 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Thu, 10 Jan 2008 19:28:29 -0800
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
Message-ID: <2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>

On Jan 10, 2008 2:25 PM, Anand Patil <anand.prabhakar.patil at gmail.com>
wrote:

> Hi all,
>
> Just trying to figure out whether something is wrong with my installation:
> I'm using mpi4py with mpich, on a machine with two quad-core 3.0Ghz intel
> processors. The following script takes 7.21s (wall time) with seven
> IPEngines, with almost all of the time spent in the scatterAll() operation:
>
>
> from numpy import *
> from ipython1 import *
> import ipython1.kernel.api as kernel
> rc = kernel.RemoteController(('127.0.0.1',10105))
>
> C=ones((5000,1000),dtype=float)
> rc.scatterAll('C',C)
>
> rc.resetAll()
>
>
> Is this fairly typical, and if not is there anything I can do to speed up
> the pushing and pulling?
>
> Thanks,
> Anand Patil
>


I got similar results with OpenMPI. Here are the profiler results:

In [11]: run -p test
         2809230 function calls in 9.891 CPU seconds

   Ordered by: internal time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
      618    6.075    0.010    6.075    0.010 {method 'recv' of
'_socket.socket' objects}
        6    1.484    0.247    1.484    0.247 <string>:1(sendall)
        1    0.980    0.980    1.855    1.855 base64.py:285(encode)
   701757    0.371    0.000    0.371    0.000 {binascii.b2a_base64}
        1    0.216    0.216    0.216    0.216 {cPickle.dumps}
   701759    0.205    0.000    0.205    0.000 {method 'read' of '
cStringIO.StringI' objects}
       60    0.183    0.003    0.183    0.003 {method 'join' of 'str'
objects}
   702494    0.173    0.000    0.173    0.000 {method 'append' of 'list'
objects}
   701798    0.125    0.000    0.125    0.000 {len}
        3    0.040    0.013    2.079    0.693 xmlrpclib.py:1041(dumps)
        1    0.029    0.029    0.029    0.029 {method 'fill' of '
numpy.ndarray' objects}
        3    0.002    0.001    9.642    3.214 xmlrpclib.py:1427(__request)
        3    0.002    0.001    9.644    3.215 xmlrpclib.py:1146(__call__)
        1    0.001    0.001    9.815    9.815 multienginexmlrpc.py
:946(scatterAll)
       21    0.000    0.000    6.076    0.289 socket.py:321(readline)
        1    0.000    0.000    9.891    9.891 test.py:2(<module>)

Looks like it's pickling the array and sending it as a string. I think
mpi4py can send numerical arrays without pickling, so perhaps I don't have
it installed properly?

Thanks,
Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080110/e9aaabde/attachment.html>

From fperez.net at gmail.com  Thu Jan 10 22:52:59 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 10 Jan 2008 20:52:59 -0700
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
Message-ID: <db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>

Hi Anand,

On Jan 10, 2008 8:28 PM, Anand Patil <anand.prabhakar.patil at gmail.com> wrote:
>
> On Jan 10, 2008 2:25 PM, Anand Patil <anand.prabhakar.patil at gmail.com>
> wrote:
>
> > Hi all,
> >
> > Just trying to figure out whether something is wrong with my installation:
> I'm using mpi4py with mpich, on a machine with two quad-core 3.0Ghz intel
> processors. The following script takes 7.21s (wall time) with seven
> IPEngines, with almost all of the time spent in the scatterAll() operation:
> >
> >
> > from numpy import *
> > from ipython1 import *
> > import ipython1.kernel.api as kernel
> > rc = kernel.RemoteController(('127.0.0.1',10105))
> >
> > C=ones((5000,1000),dtype=float)
> > rc.scatterAll('C',C)
> >
> > rc.resetAll()
> >
> >
> > Is this fairly typical, and if not is there anything I can do to speed up
> the pushing and pulling?
> >
> > Thanks,
> > Anand Patil
> >
>
>
> I got similar results with OpenMPI. Here are the profiler results:
>
>  In [11]: run -p test
>           2809230 function calls in 9.891 CPU seconds
>
>     Ordered by: internal time
>
>     ncalls  tottime  percall  cumtime  percall filename:lineno(function)
>        618    6.075    0.010    6.075    0.010 {method 'recv' of
> '_socket.socket' objects}
>          6    1.484    0.247    1.484    0.247 <string>:1(sendall)
>          1    0.980    0.980    1.855    1.855 base64.py:285(encode)
>     701757    0.371    0.000    0.371    0.000 {binascii.b2a_base64}
>          1    0.216    0.216    0.216    0.216 {cPickle.dumps}
>     701759    0.205    0.000    0.205    0.000 {method 'read' of
> 'cStringIO.StringI' objects}
>         60    0.183    0.003    0.183    0.003 {method 'join' of 'str'
> objects}
>     702494    0.173    0.000    0.173    0.000 {method 'append' of 'list'
> objects}
>     701798    0.125    0.000    0.125    0.000 {len}
>          3    0.040    0.013    2.079    0.693 xmlrpclib.py:1041(dumps)
>          1    0.029    0.029    0.029    0.029 {method 'fill' of
> 'numpy.ndarray' objects}
>          3    0.002    0.001    9.642    3.214 xmlrpclib.py:1427(__request)
>          3    0.002    0.001    9.644    3.215 xmlrpclib.py:1146(__call__)
>          1    0.001    0.001    9.815    9.815
> multienginexmlrpc.py:946(scatterAll)
>         21    0.000    0.000    6.076    0.289 socket.py:321(readline)
>          1    0.000    0.000    9.891    9.891 test.py:2(<module>)
>
>  Looks like it's pickling the array and sending it as a string. I think
> mpi4py can send numerical arrays without pickling, so perhaps I don't have
> it installed properly?

No, it's not an installation problem, it's simply that currently,
ipython doesn't have any special mpi-optimized scatter/gathers.  We've
been talking about it, and it's something that hopefully soon we'll
get to work on, but currently the controller will be a serious
bottleneck with the usage pattern you have (sending largish arrays
from the controller to every engine).

In this usage mode, the engine also bottlenecks because it has to
serially contact all the engines.

With the existing infrastructure, the ideal usage pattern for ipython
is to be sending small messages and having your engines do significant
computation.  But anything that involves sending something big to
everybody will unfortunately be slow right now.

One question: is it possible for you to organize your code runs so
that the engines get their local data via other small parameters?
Sometimes you can instead of generating a large random array and
scattering, seed the local RNGs differently and do the generation
locally (on the engines), or you can have each engine read its data
over a network filesystem, etc.

Another possibility is to start your engine group via mpirun and then
have say engine 0 do an *MPI* scatter of an array, which is then used
by the others.

Let us know if any of this is useful...

Cheers,

f


From prabhu at aero.iitb.ac.in  Fri Jan 11 07:41:51 2008
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Fri, 11 Jan 2008 18:11:51 +0530
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47869441.0437560a.77de.3616@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
Message-ID: <4787640F.10605@aero.iitb.ac.in>

Laurent Dufrechou wrote:
> Here is finally a new version with a completely refactored display widget.
[...]
> So linux user.. is this ok now??? (I?ve treid under ubuntu so should be ok!)
> 
> Solved performances bugs discovered by Prabhu and a bug in the pager too 
> thanks to Stefan.

This seems to work great although I can't popup a wx.Frame object.  So 
if I try this:

  import wx
  f = wx.Frame(None, -1)
  f.Show()

I get nasty errors.  Perhaps this won't work for other reasons.  Anyhow 
the widget does look great and seems to work perfectly well under linux. 
  Great work!  Thanks!

cheers,
prabhu


From laurent.dufrechou at gmail.com  Fri Jan 11 07:59:26 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Fri, 11 Jan 2008 13:59:26 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <4787640F.10605@aero.iitb.ac.in>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<4787640F.10605@aero.iitb.ac.in>
Message-ID: <47876836.01b7660a.754b.ffffeab1@mx.google.com>

Ouch I see want you want to do :)
Well it is not designed for this use case.
In fact there is a problem because wx is an event driven GUI so if you do
what you wrote you ask Ipython (that is in a thread of my wxApp instance
that to create a frame). But then the Gui loop is no more run, because I
call Ipython thread only when new command is called.
I have to do some test because it's a funny thing If we should be able to
design a gui on th fly, but I'm pretty sure It will not be able...)
Where you used to do this with classical Ipython??
How about performance of wx.+'tab' I this ok for your sempron???

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de Prabhu Ramachandran
Envoy??: vendredi 11 janvier 2008 13:42
??: Laurent Dufrechou
Cc?: IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

Laurent Dufrechou wrote:
> Here is finally a new version with a completely refactored display widget.
[...]
> So linux user.. is this ok now??? (I?ve treid under ubuntu so should be
ok!)
> 
> Solved performances bugs discovered by Prabhu and a bug in the pager too 
> thanks to Stefan.

This seems to work great although I can't popup a wx.Frame object.  So 
if I try this:

  import wx
  f = wx.Frame(None, -1)
  f.Show()

I get nasty errors.  Perhaps this won't work for other reasons.  Anyhow 
the widget does look great and seems to work perfectly well under linux. 
  Great work!  Thanks!

cheers,
prabhu
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev



From prabhu at aero.iitb.ac.in  Fri Jan 11 08:09:25 2008
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Fri, 11 Jan 2008 18:39:25 +0530
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47876836.01b7660a.754b.ffffeab1@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	<4787640F.10605@aero.iitb.ac.in>
	<47876836.01b7660a.754b.ffffeab1@mx.google.com>
Message-ID: <47876A85.4040500@aero.iitb.ac.in>

Laurent Dufr?chou wrote:
> Ouch I see want you want to do :)
> Well it is not designed for this use case.

I guessed as much. :)

> In fact there is a problem because wx is an event driven GUI so if you do
> what you wrote you ask Ipython (that is in a thread of my wxApp instance
> that to create a frame). But then the Gui loop is no more run, because I
> call Ipython thread only when new command is called.
> I have to do some test because it's a funny thing If we should be able to
> design a gui on th fly, but I'm pretty sure It will not be able...)
> Where you used to do this with classical Ipython??

ipython -wthread handles this fine but thats a special flag.

> How about performance of wx.+'tab' I this ok for your sempron???

This is on a different machine (PIV, 3Ghz) but yeah performance is great!

cheers,
prabhu


From vivainio at gmail.com  Fri Jan 11 08:11:15 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 11 Jan 2008 15:11:15 +0200
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <-3625673335240607875@unknownmsgid>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>
	<-3625673335240607875@unknownmsgid>
Message-ID: <46cb515a0801110511v7071e34frae5be28b5347eb53@mail.gmail.com>

On Jan 11, 2008 12:43 AM, Laurent Dufrechou <laurent.dufrechou at free.fr> wrote:

> Do you think it could be usefull for you an app more polished like idle?

Yes, I can definitely see it could be useful.

Here is a feature suggestion:

Have an "IPython" menu.

Provide a way to create menu items and hook them to callables. I'm
thinking of something like:

_ip.create_menu_item('Kill current process", kill_current_process).

This would be VERY useful.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Fri Jan 11 08:38:19 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Fri, 11 Jan 2008 14:38:19 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <46cb515a0801110511v7071e34frae5be28b5347eb53@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>	
	<-3625673335240607875@unknownmsgid>
	<46cb515a0801110511v7071e34frae5be28b5347eb53@mail.gmail.com>
Message-ID: <47877152.17945e0a.479d.70ec@mx.google.com>

> _ip.create_menu_item('Kill current process", kill_current_process).
Yeah I see ,interesting, and where does 'kill_current_process' comes from?
It would have been defined in the ipython instance or it is something
plugged into the GUI?

Has I said before I'm not really used to do advcanced ipython scripting. So
if you can for example write me a 10 lines
example of a complete session where you define kill_current_process that I
could test in ipython shell, I will be able to add it to the GUI.



-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: vendredi 11 janvier 2008 14:11
??: Laurent Dufrechou
Cc?: Laurent Dufrechou; IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

On Jan 11, 2008 12:43 AM, Laurent Dufrechou <laurent.dufrechou at free.fr>
wrote:

> Do you think it could be usefull for you an app more polished like idle?

Yes, I can definitely see it could be useful.

Here is a feature suggestion:

Have an "IPython" menu.

Provide a way to create menu items and hook them to callables. I'm
thinking of something like:

_ip.create_menu_item('Kill current process", kill_current_process).

This would be VERY useful.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From laurent.dufrechou at gmail.com  Fri Jan 11 08:38:58 2008
From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=)
Date: Fri, 11 Jan 2008 14:38:58 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47876A85.4040500@aero.iitb.ac.in>
References: <47869441.0437560a.77de.3616@mx.google.com>	<4787640F.10605@aero.iitb.ac.in>	<47876836.01b7660a.754b.ffffeab1@mx.google.com>
	<47876A85.4040500@aero.iitb.ac.in>
Message-ID: <47877179.1d255e0a.7672.4abc@mx.google.com>

> ipython -wthread handles this fine but thats a special flag.
Ok so will take a look at it.
I haven't called Ipython instance with wthread support. Perhaps it is
possible.. who know ;)
I've just tried ipython with -wthread and I see what you want. Will dig this
because it's really interesting for GUI dev!

If see any other use case of ipython that is not covered tell me it because
I'm used to play with ipython simple mode :)
(like '?' only etc...)

Bye,
Laurent



From vivainio at gmail.com  Fri Jan 11 08:46:11 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 11 Jan 2008 15:46:11 +0200
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47877152.17945e0a.479d.70ec@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>
	<-3625673335240607875@unknownmsgid>
	<46cb515a0801110511v7071e34frae5be28b5347eb53@mail.gmail.com>
	<47877152.17945e0a.479d.70ec@mx.google.com>
Message-ID: <46cb515a0801110546u5f59d640w9bdb6934a304fa01@mail.gmail.com>

On Jan 11, 2008 3:38 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:

> > _ip.create_menu_item('Kill current process", kill_current_process).

> Yeah I see ,interesting, and where does 'kill_current_process' comes from?
> It would have been defined in the ipython instance or it is something
> plugged into the GUI?

It's just a callable that is visible where the create_menu_item is run.

> if you can for example write me a 10 lines
> example of a complete session where you define kill_current_process that I
> could test in ipython shell, I will be able to add it to the GUI.

Suppose that you created a wxapi module that implements this create_menu_item

Add to ipy_user_conf.py:

import wxapi
def sayhi():
  print "Hello!"

wxapi.create_menu_item('Do stuff!', sayhi)

Then, when you start wxipython, it will run ipy_user_conf.py when
IPython starts up, and register that menu command.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Fri Jan 11 09:24:10 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Fri, 11 Jan 2008 15:24:10 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <46cb515a0801110546u5f59d640w9bdb6934a304fa01@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	
	<46cb515a0801101409u4a5377e9y6bcb82379f31a538@mail.gmail.com>	
	<-3625673335240607875@unknownmsgid>	
	<46cb515a0801110511v7071e34frae5be28b5347eb53@mail.gmail.com>	
	<47877152.17945e0a.479d.70ec@mx.google.com>
	<46cb515a0801110546u5f59d640w9bdb6934a304fa01@mail.gmail.com>
Message-ID: <47877c11.1f145e0a.2e7b.45dd@mx.google.com>

OK i see now.
Should be easy to do.
Will check this too.

-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: vendredi 11 janvier 2008 14:46
??: Laurent Dufr?chou
Cc?: IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

On Jan 11, 2008 3:38 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com>
wrote:

> > _ip.create_menu_item('Kill current process", kill_current_process).

> Yeah I see ,interesting, and where does 'kill_current_process' comes from?
> It would have been defined in the ipython instance or it is something
> plugged into the GUI?

It's just a callable that is visible where the create_menu_item is run.

> if you can for example write me a 10 lines
> example of a complete session where you define kill_current_process that I
> could test in ipython shell, I will be able to add it to the GUI.

Suppose that you created a wxapi module that implements this
create_menu_item

Add to ipy_user_conf.py:

import wxapi
def sayhi():
  print "Hello!"

wxapi.create_menu_item('Do stuff!', sayhi)

Then, when you start wxipython, it will run ipy_user_conf.py when
IPython starts up, and register that menu command.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From fperez.net at gmail.com  Fri Jan 11 14:09:54 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 11 Jan 2008 12:09:54 -0700
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
Message-ID: <db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>

Hi Anand,

I'm keeping the list CC'd here so that this is archived for the
benefit of others.

On Jan 11, 2008 11:43 AM, Anand Patil <anand.prabhakar.patil at gmail.com> wrote:
> Thanks Fernando,
>
>
> > One question: is it possible for you to organize your code runs so
> > that the engines get their local data via other small parameters?
> > Sometimes you can instead of generating a large random array and
> > scattering, seed the local RNGs differently and do the generation
> > locally (on the engines), or you can have each engine read its data
> > over a network filesystem, etc.
> >
>
> Unfortunately I'm using the engines to fill in the entries of a matrix,
> which I then need to gather for some linear algebra that would be really
> hard to do in pieces without passing around large amounts of data.
>
>
> > Another possibility is to start your engine group via mpirun and then
> > have say engine 0 do an *MPI* scatter of an array, which is then used
> > by the others.
>
> This would solve my problem. I actually tried doing like this in two
> different ways (with OpenMPI):
>
> 1) I started the IPengines using mpirun, then started an IPython session and
> imported mpi4py. MPI.COMM_WORLD.size was 1, which I took to mean that mpi4py
> couldn't see the IPengines, although the IPython1 remote controller instance
> was able to work with them.
>
> 2) I started several instances of IPython with mpirun:
>
> mpirun -n 4 ipython
>
> However, the IPython shell acted really weird, to the point of being
> unusable. I can make a video of it if it would help.
>
> What would be the correct way to use mpi4py with IPython and/or the
> IPengines?

I'm a bit swamped right now so I'll only give you a very rough idea.
I'll try, tonight, to put together a small document/mini tutorial on
how to do this type of work.

The basic idea is to start your engine group as an mpi world, then
have them connect to the controller, and from the controller, tell
engine 0 to do the array creation and scatter.  So the engines are an
MPI world, but the controller and client don't need to be.  Is that
clear enough?  If not, I'll provide step by step (with code)
instructions later...

regards,


f


From anand.prabhakar.patil at gmail.com  Fri Jan 11 14:49:05 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Fri, 11 Jan 2008 11:49:05 -0800
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
Message-ID: <2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>

Hi Fernando,

The basic idea is to start your engine group as an mpi world, then
> have them connect to the controller, and from the controller, tell
> engine 0 to do the array creation and scatter.  So the engines are an
> MPI world, but the controller and client don't need to be.  Is that
> clear enough?  If not, I'll provide step by step (with code)
> instructions later...
>

It makes sense, but I don't seem to be getting the details. I start the
engines with

mpirun -n 7 ipengine --mpi=mpi4py

then do the following from IPython:

In [2]: rc.executeAll('from mpi4py import MPI')
Out[2]:
<Results List>
[0] In [1]: from mpi4py import MPI
[1] In [1]: from mpi4py import MPI
[2] In [1]: from mpi4py import MPI
[3] In [1]: from mpi4py import MPI
[4] In [1]: from mpi4py import MPI
[5] In [1]: from mpi4py import MPI
[6] In [1]: from mpi4py import MPI


In [3]: rc.execute(1, 'print MPI.COMM_WORLD.size')
Out[3]:
<Results List>
[1] In [2]: print MPI.COMM_WORLD.size
[1] Out[2]: 1

so it looks like the IPEngines don't know they're in the same world...

Anyway, I look forward to seeing the mini-tutorial later on, but please
don't rush on my account.

Thanks,
Anand
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080111/86ab871b/attachment.html>

From fperez.net at gmail.com  Fri Jan 11 22:12:52 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 11 Jan 2008 20:12:52 -0700
Subject: [IPython-dev] Adding a nice progressbar to scipy
Message-ID: <db6b5ecc0801111912m5243492bt8c261f4e26ebe25a@mail.gmail.com>

[ Note: this is continued from scipy-dev:
http://projects.scipy.org/pipermail/scipy-dev/2008-January/008200.html

The code in question is:
http://pypi.python.org/pypi/progressbar

I'm moving the discussion here b/c now it's an ipython thing. ]

On Jan 11, 2008 7:54 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> On Jan 12, 2008 3:26 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> >
> > On Jan 11, 2008 6:52 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> > > So what is the conclusion?
> > >
> > > Maybe I should stress, that the progress bar is just one file with 300
> > > lines of code.
> > >
> > > So it's not really a package, just a feature. I was thinking of
> > > scipy.io, but ipython
> > > is maybe better. If it can contain interaction with gnuplot, why not a simple
> > > progressbar? :)

BTW, the gnuplot and numeric support are old and deprecated, code from
2001 when Numeric was much more limited than numpy and matplotlib
didn't exist :)  We've precisely moved into a world where we try to
have better defined responsibilities (scipy used to have plotting and
that got taken out, matplotlib used to have a shell and that got taken
out, etc.).  Having said that, an interactive progress bar *does* make
sense for ipython, which is why I'm willing to entertain this
discussion.

> > A few questions:
> >
> > - Will it run *correctly* under Windows?  I don't use it, but I don't
> > want the text-only core of ipython to start sprouting functionality
> > that breaks under windows (or any OS for that matter).  Same thing for
> > Linux/OSX.  I know it says 'os independent', but I want independent
> > confirmation of that statement.
>
> Right, I don't know, since I don't use windows and I cannot test it.

I don't want to sound harsh, but since you're the one asking for the
feature, you'll have to do a bit of the work.  Either a windows user
from the list can help you out with testing it, or you can borrow a
friend's windows computer for 20 minutes for testing it.

I'm sorry to say this, but even as much as I dislike windows myself
for personal use, we do our best with ipython to be good windows
citizens. Ville and Jorgen have put a LOT of work into making ipython
behave well for windows, and I'm not about to throw on their lap a
feature  that could cause them grief, especially since they didn't
even ask for it in the first place.

So if you want it, you'll need to first satisfy these concerns and
find a way to get it checked.  Now, it looks like it's doable, so over
a few days I'm sure you can find a way to get the code checked out and
cleaned up; if it makes your life easier we can drop it in for SVN and
you can work on the cleanup. We recently had a WX ipython committed
that had cross-platform issues, and the developer has been gradually
working on them.

So I'm not asking you for perfect code out of the box, but I can't
drop code in that may have cross-platform problems if you don't commit
to fixing them if they arise.

> > - It makes calls to ioctl() without checks that the import at the top
> > actually worked.  That will break somewhere, I guarantee it.
>
> It's used in handle_resize only, which is only used in
>
>             try:
>                 self.handle_resize(None,None)
>                 signal.signal(signal.SIGWINCH, self.handle_resize)
>                 self.signal_set = True
>             except:
>                 self.term_width = 79
>
> I am not saying it's the best programming practice, but I think
> it will work without ioctl quite well.
>
> > - Is signal.SIGWINCH available on all OSes?
>
> I don't know.
>
> > - termios is also used without checks.  That will likely throw an
> > exception under windows.
>
> Again only in handle_resize(), which is only called in the code above,
> so it should be safe.
>
> > - Etc.  This code doesn't look like it's ever been properly tested for
> > cross-platform compatibility.  Welcome to the real world.
>
> Right. I can only say that it works well in Debian.
>
> > If all of the above were to be addressed, it's small enough that I'm
> > willing to throw it into ipython.externals for convenience.  300 lines
> > is a drop in the bucket at this point, and for interactive work it can
> > come in very handy, I agree.  No sysadmin will notice it's hiding in
> > there :)
> >
> > In this case, a few things:
> >
> > - You said that the author would BSD license it, but the current code
> > http://pypi.python.org/pypi/progressbar still shows LGPL-only.  I
> > won't put it in until we get from him a BSD-licensed file.
>
> Yes, he only sent an email to me, I can ask him to send an email to
> this list too.
>
> > - Please send us the patch you suggested writing for the more comon case.
>
> I'll send it to the ipython list.
>
> > - You commit to letting us know when an update is warranted and
> > sending us the corresponding patch, so that the ipython developers can
> > apply it with minimal effort.
>
> I commit. The final decision needs to be made by ipython developers though,
> if they are willing to maintain such a code if I won't be available
> (for any reason). But except
> the points you raised, I think it's very readable, so there shouldn't
> be a problem with that.

I hope the above is clear enough.  If one of our core windows people
finds this code useful and is willing to help Ondrej test that part
and clean it up, that would be great.  I think this feature can be
useful, but I'm just making sure that Ville and Jorgen don't end up
with a problem on their hands they didn't bargain for.

Cheers,

f


From ondrej at certik.cz  Fri Jan 11 23:07:25 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Sat, 12 Jan 2008 05:07:25 +0100
Subject: [IPython-dev] Adding a nice progressbar to scipy
In-Reply-To: <db6b5ecc0801111912m5243492bt8c261f4e26ebe25a@mail.gmail.com>
References: <db6b5ecc0801111912m5243492bt8c261f4e26ebe25a@mail.gmail.com>
Message-ID: <85b5c3130801112007u27d2de1du5aee307bcf91ac8d@mail.gmail.com>

On Jan 12, 2008 4:12 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> [ Note: this is continued from scipy-dev:
> http://projects.scipy.org/pipermail/scipy-dev/2008-January/008200.html
>
> The code in question is:
> http://pypi.python.org/pypi/progressbar
>
> I'm moving the discussion here b/c now it's an ipython thing. ]
>
> On Jan 11, 2008 7:54 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> > On Jan 12, 2008 3:26 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> > >
> > > On Jan 11, 2008 6:52 PM, Ondrej Certik <ondrej at certik.cz> wrote:
> > > > So what is the conclusion?
> > > >
> > > > Maybe I should stress, that the progress bar is just one file with 300
> > > > lines of code.
> > > >
> > > > So it's not really a package, just a feature. I was thinking of
> > > > scipy.io, but ipython
> > > > is maybe better. If it can contain interaction with gnuplot, why not a simple
> > > > progressbar? :)
>
> BTW, the gnuplot and numeric support are old and deprecated, code from
> 2001 when Numeric was much more limited than numpy and matplotlib
> didn't exist :)  We've precisely moved into a world where we try to
> have better defined responsibilities (scipy used to have plotting and
> that got taken out, matplotlib used to have a shell and that got taken
> out, etc.).  Having said that, an interactive progress bar *does* make
> sense for ipython, which is why I'm willing to entertain this
> discussion.
>
> > > A few questions:
> > >
> > > - Will it run *correctly* under Windows?  I don't use it, but I don't
> > > want the text-only core of ipython to start sprouting functionality
> > > that breaks under windows (or any OS for that matter).  Same thing for
> > > Linux/OSX.  I know it says 'os independent', but I want independent
> > > confirmation of that statement.
> >
> > Right, I don't know, since I don't use windows and I cannot test it.
>
> I don't want to sound harsh, but since you're the one asking for the
> feature, you'll have to do a bit of the work.  Either a windows user
> from the list can help you out with testing it, or you can borrow a
> friend's windows computer for 20 minutes for testing it.
>
> I'm sorry to say this, but even as much as I dislike windows myself
> for personal use, we do our best with ipython to be good windows
> citizens. Ville and Jorgen have put a LOT of work into making ipython
> behave well for windows, and I'm not about to throw on their lap a
> feature  that could cause them grief, especially since they didn't
> even ask for it in the first place.
>
> So if you want it, you'll need to first satisfy these concerns and
> find a way to get it checked.  Now, it looks like it's doable, so over
> a few days I'm sure you can find a way to get the code checked out and
> cleaned up; if it makes your life easier we can drop it in for SVN and
> you can work on the cleanup. We recently had a WX ipython committed
> that had cross-platform issues, and the developer has been gradually
> working on them.
>
> So I'm not asking you for perfect code out of the box, but I can't
> drop code in that may have cross-platform problems if you don't commit
> to fixing them if they arise.

Agree.

>
> > > - It makes calls to ioctl() without checks that the import at the top
> > > actually worked.  That will break somewhere, I guarantee it.
> >
> > It's used in handle_resize only, which is only used in
> >
> >             try:
> >                 self.handle_resize(None,None)
> >                 signal.signal(signal.SIGWINCH, self.handle_resize)
> >                 self.signal_set = True
> >             except:
> >                 self.term_width = 79
> >
> > I am not saying it's the best programming practice, but I think
> > it will work without ioctl quite well.
> >
> > > - Is signal.SIGWINCH available on all OSes?
> >
> > I don't know.
> >
> > > - termios is also used without checks.  That will likely throw an
> > > exception under windows.
> >
> > Again only in handle_resize(), which is only called in the code above,
> > so it should be safe.
> >
> > > - Etc.  This code doesn't look like it's ever been properly tested for
> > > cross-platform compatibility.  Welcome to the real world.
> >
> > Right. I can only say that it works well in Debian.
> >
> > > If all of the above were to be addressed, it's small enough that I'm
> > > willing to throw it into ipython.externals for convenience.  300 lines
> > > is a drop in the bucket at this point, and for interactive work it can
> > > come in very handy, I agree.  No sysadmin will notice it's hiding in
> > > there :)
> > >
> > > In this case, a few things:
> > >
> > > - You said that the author would BSD license it, but the current code
> > > http://pypi.python.org/pypi/progressbar still shows LGPL-only.  I
> > > won't put it in until we get from him a BSD-licensed file.
> >
> > Yes, he only sent an email to me, I can ask him to send an email to
> > this list too.
> >
> > > - Please send us the patch you suggested writing for the more comon case.
> >
> > I'll send it to the ipython list.
> >
> > > - You commit to letting us know when an update is warranted and
> > > sending us the corresponding patch, so that the ipython developers can
> > > apply it with minimal effort.
> >
> > I commit. The final decision needs to be made by ipython developers though,
> > if they are willing to maintain such a code if I won't be available
> > (for any reason). But except
> > the points you raised, I think it's very readable, so there shouldn't
> > be a problem with that.
>
> I hope the above is clear enough.  If one of our core windows people
> finds this code useful and is willing to help Ondrej test that part
> and clean it up, that would be great.  I think this feature can be
> useful, but I'm just making sure that Ville and Jorgen don't end up
> with a problem on their hands they didn't bargain for.

Of course. Attached find the first draft of the patch.

Apply like this:

$ patch -p1 < /tmp/progressbar.patch

in the trunk. Could someone plese test this example on windows?

from progressbar import progressbar
pbar=progressbar(maxval=10000)
for i in range(10000):
    pbar.update(i)
    #do some heavy calculation in each step
pbar.finish()

(you can increase the number of iterations from 10000 to something
bigger, to see the progress).

If you have some comments, I'll by happy to improve the patch.

Thanks a lot,
Ondrej
-------------- next part --------------
A non-text attachment was scrubbed...
Name: progressbar.patch
Type: text/x-patch
Size: 14838 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080112/15e4e329/attachment.bin>

From prabhu at aero.iitb.ac.in  Sat Jan 12 00:30:26 2008
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Sat, 12 Jan 2008 11:00:26 +0530
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <47877179.1d255e0a.7672.4abc@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	<4787640F.10605@aero.iitb.ac.in>	<47876836.01b7660a.754b.ffffeab1@mx.google.com>
	<47876A85.4040500@aero.iitb.ac.in>
	<47877179.1d255e0a.7672.4abc@mx.google.com>
Message-ID: <47885072.7010701@aero.iitb.ac.in>

Laurent Dufrechou wrote:
> If see any other use case of ipython that is not covered tell me it because
> I'm used to play with ipython simple mode :)
> (like '?' only etc...)

Also see -qthread -q4thread and -pylab (which is a must have, you may 
ask that users use the wx backends of pylab).


cheers,
prabhu


From fperez.net at gmail.com  Sat Jan 12 01:36:30 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 11 Jan 2008 23:36:30 -0700
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
	<2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
Message-ID: <db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>

On Jan 11, 2008 12:49 PM, Anand Patil <anand.prabhakar.patil at gmail.com> wrote:
> Hi Fernando,
>
>
> > The basic idea is to start your engine group as an mpi world, then
> > have them connect to the controller, and from the controller, tell
> > engine 0 to do the array creation and scatter.  So the engines are an
> > MPI world, but the controller and client don't need to be.  Is that
> > clear enough?  If not, I'll provide step by step (with code)
> > instructions later...
> >
>
> It makes sense, but I don't seem to be getting the details. I start the
> engines with
>
> mpirun -n 7 ipengine --mpi=mpi4py
>
> then do the following from IPython:
>
> In [2]: rc.executeAll('from mpi4py import MPI')
> Out[2]:
> <Results List>
> [0] In [1]: from mpi4py import MPI
> [1] In [1]: from mpi4py import MPI
>  [2] In [1]: from mpi4py import MPI
> [3] In [1]: from mpi4py import MPI
> [4] In [1]: from mpi4py import MPI
> [5] In [1]: from mpi4py import MPI
> [6] In [1]: from mpi4py import MPI
>
>
> In [3]: rc.execute(1, 'print MPI.COMM_WORLD.size')
> Out[3]:
> <Results List>
> [1] In [2]: print MPI.COMM_WORLD.size
> [1] Out[2]: 1
>
> so it looks like the IPEngines don't know they're in the same world...
>
> Anyway, I look forward to seeing the mini-tutorial later on, but please
> don't rush on my account.

What platform are you on?  I've just wasted a few hours chasing the
same behavior you're seeing, only to find out that openmpi on ubuntu
gutsy is completely #$@^* broken!!!

https://bugs.launchpad.net/ubuntu/+source/openmpi/+bug/152273

<rant> As much of an ubuntu fan as I've been over the years, lately
I've really had a few encounters like this that are annoying the hell
out of me.  We all have bugs, but a distro with fully
doesn't-work-at-all packages, and bugs that make the kernel *fully
unusable in text mode* (with certain configurations that have worked
for only, oh, 10 years) and that linger for months unfixed despite
dozens of confirmed reports... AGGGGHHHH.
</rant>

There, I feel much better.

So, solution: if you're having MPI problems, try first a simple test:

tlon[~]> cat mpitest.py
import mpi4py.MPI as mpi
print mpi.COMM_WORLD.size

Run this as

tlon[~]> mpirun -n 4 python mpitest.py
[tlon:18863] mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)
[tlon:18864] mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)
[tlon:18866] mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)
[tlon:18869] mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)
4
4
4
4

(ignore the top 'pt2pt' warnings, it's OpenMPI line noise).

I was also getting '1' before with mpi4py installed via easy_install.
I went back and:

0. Nuked all my MPI-related installs from Ubuntu (MPICH, LAM,
OpenMPI).  Triple-checked that mpirun, mpicc and friends were NOT
available.

1. Built OpenMPI from scratch from

http://www.open-mpi.org/software/ompi/v1.2/

into /usr/local.

2. Re-installed MPI4Py again, to make sure it got built against the
above, clean OpenMPI.

Now it's working fine:

In [23]: rc.getIDs()
Out[23]: [0, 1, 2, 3]

In [24]: autopx
Auto Parallel Enabled
Type %autopx to disable

In [25]: import mpi4py.MPI as mpi
<Results List>
[0] In [2]: import mpi4py.MPI as mpi
[1] In [2]: import mpi4py.MPI as mpi
[2] In [2]: import mpi4py.MPI as mpi
[3] In [2]: import mpi4py.MPI as mpi


In [26]: print mpi.COMM_WORLD.size
<Results List>
[0] In [3]: print mpi.COMM_WORLD.size
[0] Out[3]: 4

[1] In [3]: print mpi.COMM_WORLD.size
[1] Out[3]: 4

[2] In [3]: print mpi.COMM_WORLD.size
[2] Out[3]: 4

[3] In [3]: print mpi.COMM_WORLD.size
[3] Out[3]: 4



In [27]: print mpi.COMM_WORLD.rank
<Results List>
[0] In [4]: print mpi.COMM_WORLD.rank
[0] Out[4]: 0

[1] In [4]: print mpi.COMM_WORLD.rank
[1] Out[4]: 2

[2] In [4]: print mpi.COMM_WORLD.rank
[2] Out[4]: 3

[3] In [4]: print mpi.COMM_WORLD.rank
[3] Out[4]: 1



Let me know if that helps...

Cheers,

f


From anand.prabhakar.patil at gmail.com  Sat Jan 12 18:57:25 2008
From: anand.prabhakar.patil at gmail.com (Anand Patil)
Date: Sat, 12 Jan 2008 15:57:25 -0800
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
	<2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
	<db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
Message-ID: <2bc7a5a50801121557hf3ec73ft7aeeb81a1e6dc52c@mail.gmail.com>

Hi Fernando,

 What platform are you on?  I've just wasted a few hours chasing the
> same behavior you're seeing, only to find out that openmpi on ubuntu
> gutsy is completely #$@^* broken!!!

Yep, it was Ubuntu Gutsy. Thanks for tracking this down, as
frustrating it is to find that a core package was unreliable I'm glad
to know the problem is not just me. This isn't the first problem I've
had with Ubuntu packages, MPI in particular: as far as I could tell
the LAM package was missing mpirun altogether. I was pretty mad at
Ubuntu for a while... but then I looked up what Ubuntu means out of
curiosity, http://en.wikipedia.org/wiki/Ubuntu_%28philosophy%29 , and
was like 'Way for an operating system to be!'

I ran into a rough spot compiling mpi4py related to 'osc pt2pt' and
managed to figure out by Googling that the subversion head of mpi4py
fixes the problem... but then had to google harder to find the actual
subversion repository in a newsgroup message from 2006. When the
mpi4py page finishes migrating it would be helpful to include a clear
link.

When I did get mpi4py up and running, I ran this script:

from numpy import *
C=ones((100,10),dtype=float)

from ipython1 import *
import ipython1.kernel.api as kernel
rc = kernel.RemoteController(('127.0.0.1',10105))
rc.resetAll()
rc.executeAll('from mpi4py import MPI as mpi')
rc.executeAll('from numpy import *')
rc.push(0,C=C)
rc.execute(0,'mpi.COMM_WORLD.Send(C,1)')

When C was 10 by 10, the last line sent it like a champ, but when C
was 10 by 100 or larger the ipengines hung up altogether, I had to
kill them with KILL (though mysteriously the log says they received
TERM). Most of the log follows this email, I lost the top bit because
I was running them in a screen.

I've found an alternative solution to my parallelization problem based
on OpenMP. It's less nice than an IPython based solution would be
assuming data passing is fast enough, but I can move forward on my own
now if you want to drop this case and move on to other things. If
you'd like to pursue the bug, on the other hand, I'd be happy to keep
iterating.

Cheers,
Anand


Log from ipengines:

mpirun -n 7 ipengine --mpi=mpi4py

...

2008/01/12 16:39 -0700 [-] Log opened.
2008/01/12 16:39 -0700 [-] MPI started with rank = 7 and size = 2
2008/01/12 16:39 -0700 [-] Log opened.
2008/01/12 16:39 -0700 [-] MPI started with rank = 7 and size = 3
2008/01/12 16:39 -0700 [-] Log opened.
2008/01/12 16:39 -0700 [-] MPI started with rank = 7 and size = 4
2008/01/12 16:39 -0700 [-] Log opened.
2008/01/12 16:39 -0700 [-] MPI started with rank = 7 and size = 5
2008/01/12 16:39 -0700 [-] Log opened.
2008/01/12 16:39 -0700 [-] MPI started with rank = 7 and size = 6
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [-] Starting factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:39 -0700 [Broker,client] got ID: 7
2008/01/12 16:39 -0700 [Broker,client] got ID: 0
2008/01/12 16:39 -0700 [Broker,client] got ID: 1
2008/01/12 16:39 -0700 [Broker,client] got ID: 2
2008/01/12 16:39 -0700 [Broker,client] got ID: 5
2008/01/12 16:39 -0700 [Broker,client] got ID: 3
2008/01/12 16:39 -0700 [Broker,client] got ID: 4
[hokhmah:21241] [0,0,0]-[0,1,1] mca_oob_tcp_msg_recv: readv failed:
Connection reset by peer (104)
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
[hokhmah:21245] *** Process received signal ***
[hokhmah:21245] Signal: Segmentation fault (11)
[hokhmah:21245] Signal code: Address not mapped (1)
[hokhmah:21245] Failing at address: 0x100
[hokhmah:21245] [ 0] /lib/libpthread.so.0 [0x2adace056100]
[hokhmah:21245] [ 1] /usr/local/lib/openmpi/mca_pml_ob1.so [0x2adadcac0202]
[hokhmah:21245] [ 2]
/usr/local/lib/openmpi/mca_btl_sm.so(mca_btl_sm_component_progress+0x797)
[0x2adadd0d7e57]
[hokhmah:21245] [ 3]
/usr/local/lib/openmpi/mca_bml_r2.so(mca_bml_r2_progress+0x2a)
[0x2adadcccc16a]
[hokhmah:21245] [ 4]
/usr/local/lib/libopen-pal.so.0(opal_progress+0x4a) [0x2adad7976c4a]
[hokhmah:21245] [ 5]
/usr/local/lib/openmpi/mca_oob_tcp.so(mca_oob_tcp_msg_wait+0x1a)
[0x2adad94302ca]
[hokhmah:21245] [ 6]
/usr/local/lib/openmpi/mca_oob_tcp.so(mca_oob_tcp_recv+0x36d)
[0x2adad94340cd]
[hokhmah:21245] [ 7]
/usr/local/lib/libopen-rte.so.0(mca_oob_recv_packed+0x33)
[0x2adad773a553]
[hokhmah:21245] [ 8]
/usr/local/lib/openmpi/mca_gpr_proxy.so(orte_gpr_proxy_put+0x21a)
[0x2adad9843cda]
[hokhmah:21245] [ 9]
/usr/local/lib/libopen-rte.so.0(orte_smr_base_set_proc_state+0x2d9)
[0x2adad7750a79]
[hokhmah:21245] [10]
/usr/local/lib/libmpi.so.0(ompi_mpi_finalize+0x13f) [0x2adad74abc1f]
[hokhmah:21245] [11] /usr/lib/python2.5/site-packages/mpi4py/_mpi.so
[0x2adad7267295]
[hokhmah:21245] [12] /usr/bin/python(Py_Finalize+0x135) [0x4ab225]
[hokhmah:21245] [13] /usr/bin/python [0x4aacec]
[hokhmah:21245] [14] /usr/bin/python(PyErr_PrintEx+0x19a) [0x4aaeea]
[hokhmah:21245] [15] /usr/bin/python(PyRun_SimpleFileExFlags+0x107) [0x4ab6f7]
[hokhmah:21245] [16] /usr/bin/python(Py_Main+0x935) [0x414725]
[hokhmah:21245] [17] /lib/libc.so.6(__libc_start_main+0xf4) [0x2adace909b44]
[hokhmah:21245] [18] /usr/bin/python [0x413c69]
[hokhmah:21245] *** End of error message ***
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
2008/01/12 16:43 -0700 [-] Received SIGTERM, shutting down.
2008/01/12 16:43 -0700 [Broker,client] Stopping factory
<ipython1.kernel.enginepb.PBEngineClientFactory object at 0x10b9610>
2008/01/12 16:43 -0700 [-] Main loop terminated.
7 processes killed (possibly by Open MPI)


From fperez.net at gmail.com  Sun Jan 13 01:53:17 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 12 Jan 2008 23:53:17 -0700
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <2bc7a5a50801121557hf3ec73ft7aeeb81a1e6dc52c@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
	<2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
	<db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
	<2bc7a5a50801121557hf3ec73ft7aeeb81a1e6dc52c@mail.gmail.com>
Message-ID: <db6b5ecc0801122253l55b9bd3aof114074716865d4f@mail.gmail.com>

Hi Anand,

On Jan 12, 2008 4:57 PM, Anand Patil <anand.prabhakar.patil at gmail.com> wrote:

> When I did get mpi4py up and running, I ran this script:
>
> from numpy import *
> C=ones((100,10),dtype=float)
>
> from ipython1 import *
> import ipython1.kernel.api as kernel
> rc = kernel.RemoteController(('127.0.0.1',10105))
> rc.resetAll()
> rc.executeAll('from mpi4py import MPI as mpi')
> rc.executeAll('from numpy import *')
> rc.push(0,C=C)
> rc.execute(0,'mpi.COMM_WORLD.Send(C,1)')
>
> When C was 10 by 10, the last line sent it like a champ, but when C
> was 10 by 100 or larger the ipengines hung up altogether, I had to
> kill them with KILL (though mysteriously the log says they received
> TERM). Most of the log follows this email, I lost the top bit because
> I was running them in a screen.

Well, that code is unfortunately incorrect MPI: you have a send
without a matching receive, so it's no surprise (and has nothing to do
with ipython or mpi4py) that it got wedged.

Here's a simple example of a session doing a trivial scatter:

In [1]: import mpi4py.MPI as mpi
[tlon:28505] mca: base: component_find: unable to open osc pt2pt: file
not found (ignored)

In [2]: import ipython1.kernel.api as kernel

In [3]: rc = kernel.RemoteController(('127.0.0.1',10105))

In [4]: rc.getIDs()
Out[4]: [0, 1, 2, 3]

In [5]: rc.activate
------> rc.activate()

In [6]: autopx
Auto Parallel Enabled
Type %autopx to disable

In [7]: import mpi4py.MPI as MPI
<Results List>
[0] In [1]: import mpi4py.MPI as MPI
[1] In [1]: import mpi4py.MPI as MPI
[2] In [1]: import mpi4py.MPI as MPI
[3] In [1]: import mpi4py.MPI as MPI


In [8]: rank,size = MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size
<Results List>
[0] In [2]: rank,size = MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size
[1] In [2]: rank,size = MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size
[2] In [2]: rank,size = MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size
[3] In [2]: rank,size = MPI.COMM_WORLD.rank, MPI.COMM_WORLD.size

In [9]: root = size//2
<Results List>
[0] In [3]: root = size//2
[1] In [3]: root = size//2
[2] In [3]: root = size//2
[3] In [3]: root = size//2

In [21]: sendbuf=None
<Results List>
[0] In [14]: sendbuf=None
[1] In [14]: sendbuf=None
[2] In [14]: sendbuf=None
[3] In [14]: sendbuf=None


In [22]: if rank==root:
    sendbuf = np.random.rand(size,10)
   ....:
<Results List>
[0] In [15]: if rank==root:
    sendbuf = np.random.rand(size,10)

[1] In [15]: if rank==root:
    sendbuf = np.random.rand(size,10)

[2] In [15]: if rank==root:
    sendbuf = np.random.rand(size,10)

[3] In [15]: if rank==root:
    sendbuf = np.random.rand(size,10)



In [24]: print sendbuf
<Results List>
[0] In [16]: print sendbuf
[0] Out[16]: [[ 0.59273955  0.91941819  0.18332745  0.50236023
0.23186433  0.40603873
   0.60863188  0.79174977  0.41726042  0.25742354]
 [ 0.45079829  0.74095739  0.50687041  0.45614561  0.80468414  0.82021551
   0.78716086  0.93041007  0.02055786  0.39692305]
 [ 0.6522603   0.38565446  0.00305974  0.4883121   0.91963356  0.93035331
   0.16671677  0.695877    0.88859014  0.01461159]
 [ 0.56898343  0.13195333  0.56278637  0.70708685  0.65832335  0.26670947
   0.17980937  0.34002591  0.36724169  0.36621309]]

[1] In [16]: print sendbuf
[1] Out[16]: None

[2] In [16]: print sendbuf
[2] Out[16]: None

[3] In [16]: print sendbuf
[3] Out[16]: None



In [25]: recvbuf = p.Scatter(sendbuf)
<Results List>
[0] In [17]: recvbuf = p.Scatter(sendbuf)
[1] In [17]: recvbuf = p.Scatter(sendbuf)
[2] In [17]: recvbuf = p.Scatter(sendbuf)
[3] In [17]: recvbuf = p.Scatter(sendbuf)


In [26]: print rank, recvbuf
<Results List>
[0] In [18]: print rank, recvbuf
[0] Out[18]: 2 [ 0.6522603   0.38565446  0.00305974  0.4883121
0.91963356  0.93035331
  0.16671677  0.695877    0.88859014  0.01461159]

[1] In [18]: print rank, recvbuf
[1] Out[18]: 0 [ 0.59273955  0.91941819  0.18332745  0.50236023
0.23186433  0.40603873
  0.60863188  0.79174977  0.41726042  0.25742354]

[2] In [18]: print rank, recvbuf
[2] Out[18]: 3 [ 0.56898343  0.13195333  0.56278637  0.70708685
0.65832335  0.26670947
  0.17980937  0.34002591  0.36724169  0.36621309]

[3] In [18]: print rank, recvbuf
[3] Out[18]: 1 [ 0.45079829  0.74095739  0.50687041  0.45614561
0.80468414  0.82021551
  0.78716086  0.93041007  0.02055786  0.39692305]


The above is a slightly modified version from the mpi4py tests.

MPI isn't the most pleasant API to work with, so I'd recommend having
a read of one of the MPI tutorials on the net if you want to use it,
I'm pretty sure it will save you a lot of time.

Cheers,

f


From vivainio at gmail.com  Sun Jan 13 06:55:32 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 13 Jan 2008 13:55:32 +0200
Subject: [IPython-dev] IPython, xemacs,
	win32 - raw_input does not work at all
Message-ID: <46cb515a0801130355j7a912fccpb562a94789ad680b@mail.gmail.com>

I figured I should start experimenting with win32 + IPython + xemacs
integration a little bit, but bumped into problems immediately.

- I had to create "ipython.bat" to launch ipython.py, trying to launch
the .py directly caused "Exec format error".

- When IPython starts up, it tries to exit immediately because raw_input causes:

EOFError: EOF when reading a line

This is not a fault of readline or IPython, because any odd script
that tries to use raw_input causes EOFError. Anyone have any clue on
how to go from here?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Sun Jan 13 11:37:50 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 09:37:50 -0700
Subject: [IPython-dev] IPython, xemacs,
	win32 - raw_input does not work at all
In-Reply-To: <46cb515a0801130355j7a912fccpb562a94789ad680b@mail.gmail.com>
References: <46cb515a0801130355j7a912fccpb562a94789ad680b@mail.gmail.com>
Message-ID: <db6b5ecc0801130837v6b3d2ddfgc9acad20e33d13ff@mail.gmail.com>

On Jan 13, 2008 4:55 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> I figured I should start experimenting with win32 + IPython + xemacs
> integration a little bit, but bumped into problems immediately.
>
> - I had to create "ipython.bat" to launch ipython.py, trying to launch
> the .py directly caused "Exec format error".
>
> - When IPython starts up, it tries to exit immediately because raw_input causes:
>
> EOFError: EOF when reading a line
>
> This is not a fault of readline or IPython, because any odd script
> that tries to use raw_input causes EOFError. Anyone have any clue on
> how to go from here?

Do you mean this error occurs when any little  python script of the type

x = raw_input('enter something')

EOFs if run inside xemacs/win32?  Just trying to understand here...

Though I'm afraid I'll have precious little to offer on this front.
It's a bummer to see how much variation there is in behavior in the
Emacses across platforms: while things seem to more or less 'just
work' under linux, they've proven surprisingly tricky both under OSX
and win32...

Cheers,

f


From fperez.net at gmail.com  Sun Jan 13 13:02:47 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 11:02:47 -0700
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>
Message-ID: <db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>

[ Note: the screenshot has been removed because the scipy server is
having problems and I can't approve the submission.

It basically showed no highlighting, and weird artifacts for commands
like 'ls' in the output.  Laurent, I can send it to you privately if
you want ]

Hi Laurent,

On Jan 10, 2008 2:55 PM, Laurent Dufrechou <laurent.dufrechou at gmail.com> wrote:
>
>
>
>
> Hello there,
>
> Here is finally a new version with a completely refactored display widget.

Great, many thanks.

> I've switched to scintilla because of the difference in behavior between
> windows and linux.
>
> The benefit is that now history widget got syntax highlighting J

Mmh, I don't see that yet, at least not on my linux box.  I'm
attaching a screenshot so you can see:

- no highlighting
- strange ESC]0; artifacts for some output.

I also noticed that anything that is printed out goes to the terminal.
 Try for example

magic

and you'll see the output paged in the terminal where wxIPython was
started.  After that, I had to kill the whole thing because I couldn't
regain control of the WX shell at all, no matter what I tried.

So it still seems to have a few rough edges, but I see a lot of
potential here.  Thanks!

> I've commited all of this under ipython/trunk/ipython/gui.  Don't know if
> it's ok as it the first time I do this? J

That's OK, it gives you a self-contained area to work in.  It would be
good if you could add a README.txt file that indicates that this is
still *purely experimental* code, so that people aren't misled yet
into thinking that it has the full ipython capabilities.  I'm very
happy to have you on board and contributing a real GUI ipython, but we
need to be careful not to confuse users as to the robustness of the
various pieces.

Cheers,

f


From strawman at astraw.com  Sun Jan 13 13:16:29 2008
From: strawman at astraw.com (Andrew Straw)
Date: Sun, 13 Jan 2008 10:16:29 -0800
Subject: [IPython-dev] Performance sanity check: 7.21s to
 scatter	5000X1000 float array to 7 engines
In-Reply-To: <db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>	<2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
	<db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
Message-ID: <478A557D.1040702@astraw.com>

Which bug makes the kernel unusable in text mode? (Sorry if it's the 
openmpi bug you linked below, I don't see a connection there.)

-Andrew

Fernando Perez wrote:
> What platform are you on?  I've just wasted a few hours chasing the
> same behavior you're seeing, only to find out that openmpi on ubuntu
> gutsy is completely #$@^* broken!!!
>
> https://bugs.launchpad.net/ubuntu/+source/openmpi/+bug/152273
>
> <rant> As much of an ubuntu fan as I've been over the years, lately
> I've really had a few encounters like this that are annoying the hell
> out of me.  We all have bugs, but a distro with fully
> doesn't-work-at-all packages, and bugs that make the kernel *fully
> unusable in text mode* (with certain configurations that have worked
> for only, oh, 10 years) and that linger for months unfixed despite
> dozens of confirmed reports... AGGGGHHHH.
> </rant>



From fperez.net at gmail.com  Sun Jan 13 13:36:15 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 11:36:15 -0700
Subject: [IPython-dev] Performance sanity check: 7.21s to scatter
	5000X1000 float array to 7 engines
In-Reply-To: <478A557D.1040702@astraw.com>
References: <2bc7a5a50801101425w2d9de9dcq65e89141474858cb@mail.gmail.com>
	<2bc7a5a50801101928t92db991tfd747ccd93e58c10@mail.gmail.com>
	<db6b5ecc0801101952t277197a1jc1bbdbb739e2db69@mail.gmail.com>
	<2bc7a5a50801111043w3e624764i60dbb24f52a555ab@mail.gmail.com>
	<db6b5ecc0801111109m1da86699o92b1d57a59df4649@mail.gmail.com>
	<2bc7a5a50801111149mad1d756yae3f301c74070f92@mail.gmail.com>
	<db6b5ecc0801112236t3c2e5cb8gd5d3f28c3f5ebe72@mail.gmail.com>
	<478A557D.1040702@astraw.com>
Message-ID: <db6b5ecc0801131036i67b3a52evd5c49bcfc297b8d1@mail.gmail.com>

[Off-topic on ubuntu stuff, feel free to ignore.  No new MPI info here]

On Jan 13, 2008 11:16 AM, Andrew Straw <strawman at astraw.com> wrote:
> Which bug makes the kernel unusable in text mode? (Sorry if it's the
> openmpi bug you linked below, I don't see a connection there.)

https://bugs.launchpad.net/ubuntu/+source/initramfs-tools/+bug/129910

Now I see it's been marked as "Declined for gutsy", annoying the hell
out of quite a few people (myself included, now).

And the cavalier attitude of the devs on this bug is annoying: if it
hits you on a box with a problem, it can be *very* tricky to get out
of.  Imagine a machine with a broken X, now you have no text console
either, and no means of finding out that a special vga=X line is the
culprit (since that's been working for say 10 years!).

This particular bug cost me a good 3 hours of hunting until I stumbled
upon the actual origin of the problem and found this long thread.

But I'm happy to see the problem remains in Hardy, which will be LTS.
We can look forward to 5 years of non-working text consoles on
servers, or being stuck with 80x25 mode in 2012 (like I am now: a
gorgeous 24 inch, 1900x1200 screen rendering an 80x25 terminal).
Yipee.

Cheers,

f


From vivainio at gmail.com  Sun Jan 13 15:04:38 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 13 Jan 2008 22:04:38 +0200
Subject: [IPython-dev] IPython, xemacs,
	win32 - raw_input does not work at all
In-Reply-To: <db6b5ecc0801130837v6b3d2ddfgc9acad20e33d13ff@mail.gmail.com>
References: <46cb515a0801130355j7a912fccpb562a94789ad680b@mail.gmail.com>
	<db6b5ecc0801130837v6b3d2ddfgc9acad20e33d13ff@mail.gmail.com>
Message-ID: <46cb515a0801131204y22d530e3k7141577b061de452@mail.gmail.com>

On Jan 13, 2008 6:37 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> Do you mean this error occurs when any little  python script of the type
>
> x = raw_input('enter something')
>
> EOFs if run inside xemacs/win32?  Just trying to understand here...

Exactly.

> Though I'm afraid I'll have precious little to offer on this front.
> It's a bummer to see how much variation there is in behavior in the
> Emacses across platforms: while things seem to more or less 'just
> work' under linux, they've proven surprisingly tricky both under OSX
> and win32...

I'm willing to devote a bit of time on this, but would appreciate some
pointers, or input from other win32 users. After all, raw_input works
with py-shell & standard python-mode.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Sun Jan 13 16:01:18 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sun, 13 Jan 2008 22:01:18 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	
	<db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>
	<db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>
Message-ID: <478a7c2e.0b38560a.5c39.ffff80cd@mx.google.com>

Hi Fernando,

Ok got the problem with 'magic'
In fact the problem I got with ipython is that every time a call to pager is
done depending from where it is called it can appear that I haven't
redirected the function.

I had to do this weird thing in my code...:

#we replace the ipython default pager by our pager
#FIXME: add a pager callback to IPython
IPython.OInspect.page = self._pager
IPython.genutils.page = self._pager
IPython.iplib.page = self._pager
(any better idea welcome!!)
        
So 'magic' function is one of these function that need to be redirected, to
my pager.
Fernando, do you think it could be possible to add a hook to ipython to
avoid this bad coding method??

For the syntax highlighting problem can you send a private mail @gmail? +
tell me witch linux distrib you use and the version of wx too. Will take a
look at it and add Experimental warning inside it :).

By the way, can you give me the commands that didn't work as expected??
(should only be ipython one...)

Cheers,

Laurent

-----Message d'origine-----
De?: Fernando Perez [mailto:fperez.net at gmail.com] 
Envoy??: dimanche 13 janvier 2008 19:03
??: Laurent Dufrechou
Cc?: IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

[ Note: the screenshot has been removed because the scipy server is
having problems and I can't approve the submission.

It basically showed no highlighting, and weird artifacts for commands
like 'ls' in the output.  Laurent, I can send it to you privately if
you want ]

Hi Laurent,

On Jan 10, 2008 2:55 PM, Laurent Dufrechou <laurent.dufrechou at gmail.com>
wrote:
>
>
>
>
> Hello there,
>
> Here is finally a new version with a completely refactored display widget.

Great, many thanks.

> I've switched to scintilla because of the difference in behavior between
> windows and linux.
>
> The benefit is that now history widget got syntax highlighting J

Mmh, I don't see that yet, at least not on my linux box.  I'm
attaching a screenshot so you can see:

- no highlighting
- strange ESC]0; artifacts for some output.

I also noticed that anything that is printed out goes to the terminal.
 Try for example

magic

and you'll see the output paged in the terminal where wxIPython was
started.  After that, I had to kill the whole thing because I couldn't
regain control of the WX shell at all, no matter what I tried.

So it still seems to have a few rough edges, but I see a lot of
potential here.  Thanks!

> I've commited all of this under ipython/trunk/ipython/gui.  Don't know if
> it's ok as it the first time I do this
 J

That's OK, it gives you a self-contained area to work in.  It would be
good if you could add a README.txt file that indicates that this is
still *purely experimental* code, so that people aren't misled yet
into thinking that it has the full ipython capabilities.  I'm very
happy to have you on board and contributing a real GUI ipython, but we
need to be careful not to confuse users as to the robustness of the
various pieces.

Cheers,

f



From fperez.net at gmail.com  Sun Jan 13 16:12:43 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 14:12:43 -0700
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <478a7c2e.0b38560a.5c39.ffff80cd@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>
	<db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>
	<478a7c2e.0b38560a.5c39.ffff80cd@mx.google.com>
Message-ID: <db6b5ecc0801131312x316e4e87nbfc440493f111389@mail.gmail.com>

On Jan 13, 2008 2:01 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:
> Hi Fernando,
>
> Ok got the problem with 'magic'
> In fact the problem I got with ipython is that every time a call to pager is
> done depending from where it is called it can appear that I haven't
> redirected the function.
>
> I had to do this weird thing in my code...:
>
> #we replace the ipython default pager by our pager
> #FIXME: add a pager callback to IPython
> IPython.OInspect.page = self._pager
> IPython.genutils.page = self._pager
> IPython.iplib.page = self._pager
> (any better idea welcome!!)

Mmh, I can't recall at the moment, but we should have a cleaner API for this...

> So 'magic' function is one of these function that need to be redirected, to
> my pager.
> Fernando, do you think it could be possible to add a hook to ipython to
> avoid this bad coding method??

Certaily we should :)

> For the syntax highlighting problem can you send a private mail @gmail? +
> tell me witch linux distrib you use and the version of wx too. Will take a
> look at it and add Experimental warning inside it :).

I'm on ubuntu gutsy, up to date.  I'll send you the screenshot off-list.

>
> By the way, can you give me the commands that didn't work as expected??
> (should only be ipython one...)

ls
cd

They worked, but produced junk.  You'll see it in the screenshot.

Cheers,
f


From laurent.dufrechou at gmail.com  Sun Jan 13 17:03:34 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sun, 13 Jan 2008 23:03:34 +0100
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <db6b5ecc0801131312x316e4e87nbfc440493f111389@mail.gmail.com>
References: <47869441.0437560a.77de.3616@mx.google.com>	
	<db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>	
	<db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>	
	<478a7c2e.0b38560a.5c39.ffff80cd@mx.google.com>
	<db6b5ecc0801131312x316e4e87nbfc440493f111389@mail.gmail.com>
Message-ID: <478a8ac6.0407560a.61e1.7883@mx.google.com>

Ok got the screenshot.
For syntax highlighting, it is for history only (look the little text
written in blue ;) )
Sadly I couldn't activate it for Ipython shell widget because it messed up
the colors of In[,Out[ and ipython ouput (which still colored as normal
ipython shell indeed).
Parhaps will add it to my todo list, but I was so happy to have a similar
working behavior on liux and windows that I let the things like this :)
For the ESC bell etc...
Ouch this is a big bug!
ls,cd works well here under liux and windows.

Please can you tell wich version of wx you use?
Sudo synaptic, search wx: I've got python-wxgtk-2.8.4.0-0ubunutu3


I suspect a locales error.
Can you do this?:

Around line 424 in ipython_view.py:
Replace line 
result =
stdout.read()#.decode('cp437').encode(locale.getpreferredencoding())
by            
result = stdout.read()
#we use print command because the shell command is called inside IPython
instance and thus is
#redirected to thread cout
#"\x01\x1b[1;36m\x02" <-- add colour to the text...
print "\x01\x1b[1;36m\x02"+result
            
I suspect that your terminal is not cp437 encoded.
If it doesn't works try also to remove 
print "\x01\x1b[1;36m\x02"+result
by:
print result

by the way I'm upgrading my ubuntu gutsy or gusty perhaps I will have same
bug as yours :)
Laurent

-----Message d'origine-----
De?: Fernando Perez [mailto:fperez.net at gmail.com] 
Envoy??: dimanche 13 janvier 2008 22:13
??: Laurent Dufr?chou
Cc?: IPython-dev at scipy.org
Objet?: Re: [IPython-dev] Brand new refactored version for linux users!

On Jan 13, 2008 2:01 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com>
wrote:
> Hi Fernando,
>
> Ok got the problem with 'magic'
> In fact the problem I got with ipython is that every time a call to pager
is
> done depending from where it is called it can appear that I haven't
> redirected the function.
>
> I had to do this weird thing in my code...:
>
> #we replace the ipython default pager by our pager
> #FIXME: add a pager callback to IPython
> IPython.OInspect.page = self._pager
> IPython.genutils.page = self._pager
> IPython.iplib.page = self._pager
> (any better idea welcome!!)

Mmh, I can't recall at the moment, but we should have a cleaner API for
this...

> So 'magic' function is one of these function that need to be redirected,
to
> my pager.
> Fernando, do you think it could be possible to add a hook to ipython to
> avoid this bad coding method??

Certaily we should :)

> For the syntax highlighting problem can you send a private mail @gmail? +
> tell me witch linux distrib you use and the version of wx too. Will take a
> look at it and add Experimental warning inside it :).

I'm on ubuntu gutsy, up to date.  I'll send you the screenshot off-list.

>
> By the way, can you give me the commands that didn't work as expected??
> (should only be ipython one...)

ls
cd

They worked, but produced junk.  You'll see it in the screenshot.

Cheers,
f



From fperez.net at gmail.com  Sun Jan 13 17:06:23 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 15:06:23 -0700
Subject: [IPython-dev] IPython development news and prospects
Message-ID: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>

Hi all,

I wanted to get a bit of feedback from others on dev plans, as well as
update everyone on a few things.  This is necessarily a long email,
sorry about that.

IPython continues to grow, thanks to the hard work of many (myself
notably excluded), but it's obvious to all of us that the current
codebase has problems and is hard to extend cleanly.  I think we
finally have both the manpower and the pieces in place to do something
about it.  First of all, there's enough of us working on the project
that I think we can tackle more changes, and second I'll have more
time soon to devote to some of this effort, as I'll be able to
directly use more of it in my research.

Next month I'm switching jobs, and at my new job we'll be using enough
of ipython's capabilities (the trunk as well as the parallel work)
that I'll be able to spend some more time on it. I know Brian will
also have soon more time for it, which is fantastic, and we've also
picked up new developers.  All this means that I expect this year to
make a lot of progress.  Ville and the rest of the team have done a
fantastic job keeping the main project alive while Brian and Min did
the distributed computing work, but I think we're at the point where
we can join forces to push through a fully integrated, well structured
project.

For this to be successful, there are several areas that will change,
so that the project evolves in a more sustainable way.

- The first is source control.  All this work will likely require a
fair amount of messing around with multiple parts of the codebase, by
multiple people.  Hence I'm thinking this is the time to switch out to
a DVCS.  I know we've talked about it, and it's something that seems
to be "in the air".  By this I mean that right now, the same
discussion is going on in the Emacs-dev, python-dev, matplotlib-dev
and numpy/scipy-dev lists.  I honestly don't want to get dragged into
a git/hg/bzr debate here, *all* of the above lists have to some extent
done that already.

My proposal is very simple: we go with hg.  Other related projects to
us are already using it (Sage & sympy), I know some of you already use
it yourselves, some of the MPL devs personally use it, I've used it
and liked it, and Robert Kern already uses it.  To me, that's enough
to let me make a choice.  It really seems like *both* bzr and hg are
pretty good, even if they have small differences here and there.  I
think we'll benefit a lot more from having *A* dvcs than from having
yet another 70-email-long thread on the subject.

So the question is: does anyone here *fundamentally object* to
switching out to hg for the project?

If not, then the big remaining issue is Trac support and SVN
migration.  If we all agree on the above, I'll look into how to make
that part happen, but I'd be *extremely grateful* if any of you has
already done this and offers a bit of help.

- Next comes documentation.  I'll work on transferring the manual over
to a bunch of reST files, so we can have reST based manual(s).  After
we do this, we will hopefully start doing a very, very good job of
always documenting things correctly in the reST sources, rather than
having the hodgepodge of outdated manual + random wiki pages we have
today.

Again, anyone who has done a lyx/latex->reST conversion in the past
and is willing to pitch in with help, let me know.

- Testing: we'll just go with nose.  All the ipython1 code that needs
Twisted has tests (Brian and Min have done a stellar job here) that
use twisted's test support to properly manage the reactor, but
fortunately they work out of the box with Twisted.  Nose makes it
really simple to have little snippets get picked up as tests, so this
means that we can have easier test development, and continue to use
the twisted test support where reactor management is needed.

At some point, we may need to turn some of the fancier doctest support
that SnakeOil has into a nose plugin to support pure ipython
functionality.  We'll cross that bridge when we get to it.

- Configuration: for this, we've more or less decided to just going
with TConfig:

http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/sandbox/tconfig

The only downside tconfig has is a dependence on Enthought Traits,
which has C code.  The upsides are vast, including a *clean* way of
handling many of the configuration-related problems that pop up here
all the time.  We can have a separate discussion thread for this topic
later if you want, because I think it's the only potentially
contentious one and we don't want to ram it down anyone's throat.


Having the above in place, what's left is actual code and project
evolution.  In the ipython1 subproject we already have the beginnings
of a cleaner separation of modular components for the interactive
component.  Eric Jones, Gael and I have taken (short) stabs at this,
and as of March I expect to spend some serious time on getting this
running.  Laurent's recent work would much more naturally fit here.

If any of you (in addition to Gael and Laurent) is in or near Paris
March 22/23, please let us know.  Gael and I are tentatively planning
a Paris IPython sprint for those dates.  If it's just the three of us
the logistics are one thing, but if more developers can come we can
look into a different setup.


What I'd like to know is if Ville and the others who have a stake on
trunk are willing to basically join in together, starting mid to late
March, on this effort.  My idea is that we'll eventually have ALL the
functionality that ipython trunk has today (a lot of which Ville has
shepherded), though some of the APIs may necessarily change.

But there will be enough work to be done here, and it touches on what
all have contributed so much, that I want to find the best way to
manage a transition so that:

- nobody feels like their work is dropped/ignored

- we don't end up with *two* projects.

This is still just one project, that will have multiple components but
all of which work in concert to provide various levels of
functionality (a terminal-based shell, a gui-based shell, a
javascript-based one, a remote shell, a parallel computing system, an
ide-like system for scientific work, etc).

Python is growing a lot, and in the scientific arena it's really,
really becoming a major player fast.  I think if we round up our joint
effort together, with the proper ground infrastructure, we can provide
something that will be tremendously useful to lots of people.  I'd
like to hear your thoughts on the matter.

If there's broad agreement on the above, we can plan specifics in
subsequent emails (hg transition, a 'final release plan' to put trunk
in maintenance mode with a good view for when we'll get the new system
functioning, etc).

I'd like to close by thanking all of you who donate your time and
energy to ipython.  It started as a 'quick afternoon hack' back in
2001 when I wanted an excuse to not write my dissertation and was
tired of Perl. Years later we have something that's used in major
research labs around the world, included in all the major Linux
distros, and shipped in every laptop from the OLPC program to kids
everywhere.  It's really been a great run so far, one that couldn't
have happended without all of your help.  Now I think we can make the
future of the project even better!

Regards,

f


From gael.varoquaux at normalesup.org  Sun Jan 13 17:13:14 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 13 Jan 2008 23:13:14 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
Message-ID: <20080113221314.GJ24375@phare.normalesup.org>

On Sun, Jan 13, 2008 at 03:06:23PM -0700, Fernando Perez wrote:
> So the question is: does anyone here *fundamentally object* to
> switching out to hg for the project?

No problem. The question is: how does the centralized hosting happens?
How can I commit to a centralized codebase with hg and who hosts it? This
is not clear in my head.

Cheers,

Ga?l


From gael.varoquaux at normalesup.org  Sun Jan 13 17:33:59 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 13 Jan 2008 23:33:59 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <20080113221314.GJ24375@phare.normalesup.org>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<20080113221314.GJ24375@phare.normalesup.org>
Message-ID: <20080113223359.GK24375@phare.normalesup.org>

On Sun, Jan 13, 2008 at 03:17:38PM -0700, Fernando Perez wrote:
> On Jan 13, 2008 3:12 PM, Gael Varoquaux <gael.varoquaux at normalesup.org>
> wrote:
> > On Sun, Jan 13, 2008 at 03:06:23PM -0700, Fernando Perez wrote:
> > > So the question is: does anyone here *fundamentally object* to
> > > switching out to hg for the project?

> > No problem. The question is: how does the centralized hosting
> > happens?
> > How can I commit to a centralized codebase with hg and who hosts it?
> > This
> > is not clear in my head.

> There's an official HG repo that devs commit to, in addition to their
> private local ones.  See for example:

> http://hg.sympy.org/sympy/

Sure, but who hosts it ? Is this sorted out in the case of ipython. This
means sysadmin work.

Ga?l


From fperez.net at gmail.com  Sun Jan 13 17:36:50 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 15:36:50 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <20080113223359.GK24375@phare.normalesup.org>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<20080113221314.GJ24375@phare.normalesup.org>
	<20080113223359.GK24375@phare.normalesup.org>
Message-ID: <db6b5ecc0801131436t613afa7es70b7b2c3a001834b@mail.gmail.com>

On Jan 13, 2008 3:33 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> > There's an official HG repo that devs commit to, in addition to their
> > private local ones.  See for example:
>
> > http://hg.sympy.org/sympy/
>
> Sure, but who hosts it ? Is this sorted out in the case of ipython. This
> means sysadmin work.

Enthought already publicly hosts hg for Robert, so I would approach
them first.  If they can't, we can register at google and use them for
the hosting (keeping the rest where it is at Enthought).  If we go
with the decision, we'll find a way to make it happen.

Cheers,

f


From fperez.net at gmail.com  Sun Jan 13 17:47:06 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 13 Jan 2008 15:47:06 -0700
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <fme4ab$nm8$1@ger.gmane.org>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<20080113221314.GJ24375@phare.normalesup.org>
	<20080113223359.GK24375@phare.normalesup.org>
	<db6b5ecc0801131436t613afa7es70b7b2c3a001834b@mail.gmail.com>
	<fme4ab$nm8$1@ger.gmane.org>
Message-ID: <db6b5ecc0801131447p48273e1ev4c9c5243fb40d94d@mail.gmail.com>

On Jan 13, 2008 3:45 PM, Robert Kern <robert.kern at gmail.com> wrote:
> Fernando Perez wrote:
> > On Jan 13, 2008 3:33 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
> >
> >>> There's an official HG repo that devs commit to, in addition to their
> >>> private local ones.  See for example:
> >>> http://hg.sympy.org/sympy/
> >> Sure, but who hosts it ? Is this sorted out in the case of ipython. This
> >> means sysadmin work.
> >
> > Enthought already publicly hosts hg for Robert, so I would approach
> > them first.  If they can't, we can register at google and use them for
> > the hosting (keeping the rest where it is at Enthought).
>
> I'm pretty sure that Google doesn't host Mercurial repositories. hg.sympy.org is
> Ondrej's, I believe (it is an alias for dakol.fsik.cvut.cz).

OK.  Then I can look into having it hosted externally if need be.

Cheers,

f


From ondrej at certik.cz  Sun Jan 13 18:07:35 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 14 Jan 2008 00:07:35 +0100
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <db6b5ecc0801131447p48273e1ev4c9c5243fb40d94d@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<20080113221314.GJ24375@phare.normalesup.org>
	<20080113223359.GK24375@phare.normalesup.org>
	<db6b5ecc0801131436t613afa7es70b7b2c3a001834b@mail.gmail.com>
	<fme4ab$nm8$1@ger.gmane.org>
	<db6b5ecc0801131447p48273e1ev4c9c5243fb40d94d@mail.gmail.com>
Message-ID: <85b5c3130801131507s75b9976dr80741839bd50b34c@mail.gmail.com>

On Jan 13, 2008 11:47 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Jan 13, 2008 3:45 PM, Robert Kern <robert.kern at gmail.com> wrote:
> > Fernando Perez wrote:
> > > On Jan 13, 2008 3:33 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
> > >
> > >>> There's an official HG repo that devs commit to, in addition to their
> > >>> private local ones.  See for example:
> > >>> http://hg.sympy.org/sympy/
> > >> Sure, but who hosts it ? Is this sorted out in the case of ipython. This
> > >> means sysadmin work.
> > >
> > > Enthought already publicly hosts hg for Robert, so I would approach
> > > them first.  If they can't, we can register at google and use them for
> > > the hosting (keeping the rest where it is at Enthought).
> >
> > I'm pretty sure that Google doesn't host Mercurial repositories. hg.sympy.org is
> > Ondrej's, I believe (it is an alias for dakol.fsik.cvut.cz).
>
> OK.  Then I can look into having it hosted externally if need be.

Yep, hg.sympy.org is hosted at my server. It's extremely easy to setup and works
like a charm.

+1 for DVCS


I would like to come for the Paris sprint, but I am not promising
anything, since I am coming
to Sage Days 8 and I need to work on my thesis too.

BTW, I started sympy in the same way as ipython has started - I was
lazy to do what I was supposed to do and instead I posed myself a
question -
is it possible to do symbolic manipulation in python? :)

Ondrej


From vivainio at gmail.com  Sun Jan 13 18:11:58 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 14 Jan 2008 01:11:58 +0200
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
Message-ID: <46cb515a0801131511n32a74e90v4916f2f7f8463bd0@mail.gmail.com>

On Jan 14, 2008 12:06 AM, Fernando Perez <fperez.net at gmail.com> wrote:

> My proposal is very simple: we go with hg.  Other related projects to

+1

> - Next comes documentation.  I'll work on transferring the manual over
> to a bunch of reST files, so we can have reST based manual(s).  After
> we do this, we will hopefully start doing a very, very good job of
> always documenting things correctly in the reST sources, rather than
> having the hodgepodge of outdated manual + random wiki pages we have
> today.

Great, I'm quite willing to update the manual with more current
information when this happens.

> - Configuration: for this, we've more or less decided to just going
> with TConfig:
>
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/sandbox/tconfig
>
> The only downside tconfig has is a dependence on Enthought Traits,
> which has C code.  The upsides are vast, including a *clean* way of

As long as the C code is not shipped with IPython and the binaries are
readily available, no problem.

> What I'd like to know is if Ville and the others who have a stake on
> trunk are willing to basically join in together, starting mid to late
> March, on this effort.  My idea is that we'll eventually have ALL the

Obviously I can't make any kind of formal commitment, but I will
certainly be kicking the tires of the new codebase.

Porting over existing trunk code is something that I can see myself
doing, at least.

> But there will be enough work to be done here, and it touches on what
> all have contributed so much, that I want to find the best way to
> manage a transition so that:
>
> - nobody feels like their work is dropped/ignored

I don't see the current trunk going away any day soon, even if we some
day decide to call it "ready" :-)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From darren.dale at cornell.edu  Sun Jan 13 18:37:26 2008
From: darren.dale at cornell.edu (Darren Dale)
Date: Sun, 13 Jan 2008 18:37:26 -0500
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
Message-ID: <200801131837.26953.darren.dale@cornell.edu>

On Sunday 13 January 2008 5:06:23 pm Fernando Perez wrote:
> So the question is: does anyone here *fundamentally object* to
> switching out to hg for the project?

Not me, I've been looking for an opportunity to start using hg.

> - Configuration: for this, we've more or less decided to just going
> with TConfig:
>
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/sandbox/tc
>onfig
>
> The only downside tconfig has is a dependence on Enthought Traits,
> which has C code.  The upsides are vast, including a *clean* way of
> handling many of the configuration-related problems that pop up here
> all the time.  We can have a separate discussion thread for this topic
> later if you want, because I think it's the only potentially
> contentious one and we don't want to ram it down anyone's throat.

I'm interested to see how that discussion plays out.

> What I'd like to know is if Ville and the others who have a stake on
> trunk are willing to basically join in together, starting mid to late
> March, on this effort.

I'll be quite buy around that time, but I'm not a major contributor and 
probably 

> This is still just one project, that will have multiple components but
> all of which work in concert to provide various levels of
> functionality (a terminal-based shell, a gui-based shell, a
> javascript-based one, a remote shell, a parallel computing system, an
> ide-like system for scientific work, etc).

Personally, I'd like to see a qt4 version of the gui as well. I would be 
willing to work on it, but before I do we would need to discuss the license 
issue. Specifically, would it need to be distributed as a separate package, 
or a namespace package? Would ipython entertain such a possibility? I'm not 
an advocate for setuptools, but I do use PyQt4 heavily for my own work and 
would like to find a way to work with it while keeping all parties happy 
concerning license. I guess this discussion should occur in another thread.

Darren





From steve at shrogers.com  Sun Jan 13 22:39:44 2008
From: steve at shrogers.com (Steven H. Rogers)
Date: Sun, 13 Jan 2008 20:39:44 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
Message-ID: <478AD980.3040206@shrogers.com>

Sounds like a plan.  I like it.

# Steve


From vivainio at gmail.com  Mon Jan 14 03:21:41 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Mon, 14 Jan 2008 10:21:41 +0200
Subject: [IPython-dev] PyQt4 & GPL in IPython
Message-ID: <46cb515a0801140021w25541cd4v8d67935d9718b1d8@mail.gmail.com>

On Jan 14, 2008 1:37 AM, Darren Dale <darren.dale at cornell.edu> wrote:

> Personally, I'd like to see a qt4 version of the gui as well. I would be
> willing to work on it, but before I do we would need to discuss the license
> issue. Specifically, would it need to be distributed as a separate package,
> or a namespace package? Would ipython entertain such a possibility? I'm not
> an advocate for setuptools, but I do use PyQt4 heavily for my own work and
> would like to find a way to work with it while keeping all parties happy
> concerning license. I guess this discussion should occur in another thread.

I suppose separate modules is enough, as long as the GPL'd modules can
be removed without disrupting the rest of IPython.

That is, if someone wants a pure-BSD (or BSD-like) IPython, he just
removes the GPL'd modules and distributes that version.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From pete.forman at westerngeco.com  Mon Jan 14 12:04:51 2008
From: pete.forman at westerngeco.com (Pete Forman)
Date: Mon, 14 Jan 2008 17:04:51 +0000
Subject: [IPython-dev] IPython, xemacs,
	win32 - raw_input does not work at all
References: <46cb515a0801130355j7a912fccpb562a94789ad680b@mail.gmail.com>
	<db6b5ecc0801130837v6b3d2ddfgc9acad20e33d13ff@mail.gmail.com>
	<46cb515a0801131204y22d530e3k7141577b061de452@mail.gmail.com>
Message-ID: <k5mcgwy4.fsf@wgmail2.gatwick.eur.slb.com>

[retry]

"Ville M. Vainio" <vivainio at gmail.com> writes:

 > I'm willing to devote a bit of time on this, but would appreciate
 > some pointers, or input from other win32 users. After all,
 > raw_input works with py-shell & standard python-mode.

I do not see the problem.

In [2]: x = raw_input('enter something')
enter somethingabc

In [3]: x
Out[3]: 'abc'


IPython 0.7.3
Python 2.5.1
XEmacs 21.5.28 native
Windows XP SP2
-- 
Pete Forman                -./\.-  Disclaimer: This post is originated
WesternGeco                  -./\.-   by myself and does not represent
pete.forman at westerngeco.com    -./\.-   the opinion of Schlumberger or
http://petef.port5.com           -./\.-   WesternGeco.



From barrywark at gmail.com  Mon Jan 14 16:29:16 2008
From: barrywark at gmail.com (Barry Wark)
Date: Mon, 14 Jan 2008 13:29:16 -0800
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <200801131837.26953.darren.dale@cornell.edu>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<200801131837.26953.darren.dale@cornell.edu>
Message-ID: <cd7634ce0801141329ued262e4i1566d45b03204d82@mail.gmail.com>

On Jan 13, 2008 3:37 PM, Darren Dale <darren.dale at cornell.edu> wrote:
>
> > This is still just one project, that will have multiple components but
> > all of which work in concert to provide various levels of
> > functionality (a terminal-based shell, a gui-based shell, a
> > javascript-based one, a remote shell, a parallel computing system, an
> > ide-like system for scientific work, etc).
>
> Personally, I'd like to see a qt4 version of the gui as well. I would be
> willing to work on it, but before I do we would need to discuss the license
> issue. Specifically, would it need to be distributed as a separate package,
> or a namespace package? Would ipython entertain such a possibility? I'm not
> an advocate for setuptools, but I do use PyQt4 heavily for my own work and
> would like to find a way to work with it while keeping all parties happy
> concerning license. I guess this discussion should occur in another thread.


Since we're quite heavily invested in both OS X and IPython in my
group, I'm willing to put time into an OS X front-end. We're working
on better OS X integration with matplotlib, so there's a natural
synergy to both efforts. Since Twisted is already well integrated with
all of the GUI toolkits that I've heard tossed about (WX, Qt, OS X,
gtk), is there still a thought towards making the ipython1 engine a
true ipython instance (rather than a pure python interpreter)? If so,
GUI integration gets much easier...

barry


From bwaters at nrao.edu  Mon Jan 14 16:40:40 2008
From: bwaters at nrao.edu (Boyd Waters)
Date: Mon, 14 Jan 2008 14:40:40 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <cd7634ce0801141329ued262e4i1566d45b03204d82@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<200801131837.26953.darren.dale@cornell.edu>
	<cd7634ce0801141329ued262e4i1566d45b03204d82@mail.gmail.com>
Message-ID: <20ABD774-560C-46CB-B02B-383AE599BCDC@nrao.edu>


On Jan 14, 2008, at 2:29 PM, Barry Wark wrote:

> Since we're quite heavily invested in both OS X and IPython in my
> group, I'm willing to put time into an OS X front-end. We're working
> on better OS X integration with matplotlib, so there's a natural
> synergy to both efforts. Since Twisted is already well integrated with
> all of the GUI toolkits that I've heard tossed about (WX, Qt, OS X,
> gtk), is there still a thought towards making the ipython1 engine a
> true ipython instance (rather than a pure python interpreter)? If so,
> GUI integration gets much easier...


Ya-hoo!

I could help with that, too!

We are heavy into Matplotlib, Qt4, Mac OS X, ipython, and ipython1 for  
our data analysis app.

I'd love to push against a Twisted driver for Qt 4.4 and/or PyObjC2...

... or whatever else makes sense. I suspect that our remoting  
architecture will NOT be at the Python layer, but It Would Be Nice to  
be able to invoke remote events from within the Python shell.

We are considering a move away from matplotlib towards QWT, but I  
daresay that we won't do that without *serious* contemplation, and  
some reworking or combination of IPython-Matplotlib such that we have  
a remote-procedure-call story for Matplotlib processes may be  
sufficient.


Regards,

  - boyd


Boyd Waters
Scientific Programmer
National Radio Astronomy Observatory
Socorro, New Mexico
http://www.aoc.nrao.edu/~bwaters




From michael.j.mclay at gmail.com  Mon Jan 14 17:29:00 2008
From: michael.j.mclay at gmail.com (Michael McLay)
Date: Mon, 14 Jan 2008 17:29:00 -0500
Subject: [IPython-dev] using class instances with ipython1
Message-ID: <683f83340801141429r1f31b9f7s7cfef7e2d982992f@mail.gmail.com>

Is it possible to use class instances with ipython1?  I'm getting an
import error message whenever I try to pass a list of class instances
to engines. I've attached a test script and a module, testclass.py,
with a simple, pickleable class defined in it. Here's the traceback:

mmclay at mmclay:~/src/scratchpad$ python test_ipython.py
[<class C 0>, <class C 1>, <class C 2>, <class C 3>, <class C 4>,
<class C 5>, <class C 6>, <class C 7>, <class C 8>, <class C 9>]
Traceback (most recent call last):
  File "test_ipython.py", line 16, in <module>
    rc.scatterAll('cs', test)
  File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
line 808, in scatterAll
    return self.scatter('all', key, seq, style, flatten)
  File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
line 798, in scatter
    localBlock, targets, key, bseq, style, flatten)
  File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
line 390, in _executeRemoteMethod
    result = self._unpackageResult(rawResult)
  File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
line 399, in _unpackageResult
    return self._returnOrRaise(result)
  File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
line 403, in _returnOrRaise
    result.raiseException()
  File "/usr/lib/python2.5/site-packages/twisted/python/failure.py",
line 259, in raiseException
    raise self.type, self.value, self.tb
ImportError: No module named testclass
mmclay at mmclay:~/src/scratchpad$
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_ipython.py
Type: text/x-python
Size: 371 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080114/e86a7727/attachment.py>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: testclass.py
Type: text/x-python
Size: 118 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080114/e86a7727/attachment-0001.py>

From michael.j.mclay at gmail.com  Mon Jan 14 17:53:38 2008
From: michael.j.mclay at gmail.com (Michael McLay)
Date: Mon, 14 Jan 2008 17:53:38 -0500
Subject: [IPython-dev] using class instances with ipython1
In-Reply-To: <683f83340801141429r1f31b9f7s7cfef7e2d982992f@mail.gmail.com>
References: <683f83340801141429r1f31b9f7s7cfef7e2d982992f@mail.gmail.com>
Message-ID: <683f83340801141453r250ce36cob9be6f60907411c0@mail.gmail.com>

I just determined the problem.  The testclass module was not in the
path  of ipcontroller.

On Jan 14, 2008 5:29 PM, Michael McLay <michael.j.mclay at gmail.com> wrote:
> Is it possible to use class instances with ipython1?  I'm getting an
> import error message whenever I try to pass a list of class instances
> to engines. I've attached a test script and a module, testclass.py,
> with a simple, pickleable class defined in it. Here's the traceback:
>
> mmclay at mmclay:~/src/scratchpad$ python test_ipython.py
> [<class C 0>, <class C 1>, <class C 2>, <class C 3>, <class C 4>,
> <class C 5>, <class C 6>, <class C 7>, <class C 8>, <class C 9>]
> Traceback (most recent call last):
>   File "test_ipython.py", line 16, in <module>
>     rc.scatterAll('cs', test)
>   File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
> line 808, in scatterAll
>     return self.scatter('all', key, seq, style, flatten)
>   File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
> line 798, in scatter
>     localBlock, targets, key, bseq, style, flatten)
>   File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
> line 390, in _executeRemoteMethod
>     result = self._unpackageResult(rawResult)
>   File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
> line 399, in _unpackageResult
>     return self._returnOrRaise(result)
>   File "/usr/lib/python2.5/site-packages/ipython1-0.9alpha2-py2.5.egg/ipython1/kernel/multienginexmlrpc.py",
> line 403, in _returnOrRaise
>     result.raiseException()
>   File "/usr/lib/python2.5/site-packages/twisted/python/failure.py",
> line 259, in raiseException
>     raise self.type, self.value, self.tb
> ImportError: No module named testclass
> mmclay at mmclay:~/src/scratchpad$
>


From m.vanland at gmail.com  Mon Jan 14 19:27:17 2008
From: m.vanland at gmail.com (Michael VanLandingham)
Date: Mon, 14 Jan 2008 16:27:17 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac osx)
In-Reply-To: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
Message-ID: <b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>

Forwarding as a re-send because the cut and paste missed 'i' at the
beginning of 'ipython'.

---------- Forwarded message ----------
From: Michael VanLandingham <m.vanland at gmail.com>
Date: Jan 14, 2008 4:01 PM
Subject: Multi-line history problems in ipython (mac osx)
To: pythonmac-sig at python.org, python-dev at scipy.org


Hi Group,
Perhaps this has been covered elsewhere, but I'm seeing problems in iPython
(0.8.2 , 0.8.3 r2919) on OS X 10.5.1 when using the up arrow to do history
completion after multi-line input (a function def, for example).

Like this:

#define a silly function or any multi-line statement:

In [1]: def foo(s=None):
   ...:   if s:
   ...:     print 'foo got "%s"' % s
   ...:     return
   ...:   print "foo got nuthin'"
   ...:

#play with it:
In [2]: foo('bar')
foo got "bar"

In [3]: foo()
foo got nuthin'

#When I press the arrow-up key a couple times to edit the function def:

  if s: def foo(s=None):
                                                  print 'foo got "%s"' % s

                    return
                                                                print "foo
got nuthin'"


Notice how the 'def foo(s=None)' is on the same line as the 'if s:'?  And
once it does this, _all_ arrow-ups are out in the middle of the console and
well, generally messed up.

You can still use this if you're careful, but it's sketchy.

Is this a known issue?  Someone else recently mentioned to me that they were
seeing this as well.

Thanks,
Michael
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080114/d5a69348/attachment.html>

From vivainio at gmail.com  Tue Jan 15 02:01:07 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Tue, 15 Jan 2008 09:01:07 +0200
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
Message-ID: <46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>

On Jan 15, 2008 2:27 AM, Michael VanLandingham <m.vanland at gmail.com> wrote:

> Notice how the 'def foo(s=None)' is on the same line as the 'if s:'?  And
> once it does this, _all_ arrow-ups are out in the middle of the console and
> well, generally messed up.
>
> You can still use this if you're careful, but it's sketchy.
>
> Is this a known issue?  Someone else recently mentioned to me that they were
> seeing this as well.

Seems like a readline issue to me.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From jorgen.stenarson at bostream.nu  Tue Jan 15 13:28:17 2008
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 15 Jan 2008 19:28:17 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
Message-ID: <478CFB41.2010908@bostream.nu>

Fernando Perez skrev:
> - The first is source control.  All this work will likely require a
> fair amount of messing around with multiple parts of the codebase, by
> multiple people.  Hence I'm thinking this is the time to switch out to
> a DVCS.  I know we've talked about it, and it's something that seems
> to be "in the air".  By this I mean that right now, the same
> discussion is going on in the Emacs-dev, python-dev, matplotlib-dev
> and numpy/scipy-dev lists.  I honestly don't want to get dragged into
> a git/hg/bzr debate here, *all* of the above lists have to some extent
> done that already.
> 
> My proposal is very simple: we go with hg.  Other related projects to
> us are already using it (Sage & sympy), I know some of you already use
> it yourselves, some of the MPL devs personally use it, I've used it
> and liked it, and Robert Kern already uses it.  To me, that's enough
> to let me make a choice.  It really seems like *both* bzr and hg are
> pretty good, even if they have small differences here and there.  I
> think we'll benefit a lot more from having *A* dvcs than from having
> yet another 70-email-long thread on the subject.
> 
> So the question is: does anyone here *fundamentally object* to
> switching out to hg for the project?
Personally I really like the nice integration of tortoise-svn into 
windows. That is something I'd hate to lose. That said I can see that 
the benefits of using a DVCS compensate the pain I will feel with a much 
smoother workflow for the project as a whole. I think it is a good idea 
to just go with what the others use, hopefully the other projects will 
follow our lead. So I say go for it.

...
> The only downside tconfig has is a dependence on Enthought Traits,
> which has C code.  The upsides are vast, including a *clean* way of
> handling many of the configuration-related problems that pop up here
> all the time.  We can have a separate discussion thread for this topic
> later if you want, because I think it's the only potentially
> contentious one and we don't want to ram it down anyone's throat.
> 
I think having C code in the project will be a major drawback in getting 
new developers on windows. Getting a compiler working can be a lot of 
work. But the positive aspects of using something trait-like seems it 
could definitely be worth it.

/J?rgen













From gael.varoquaux at normalesup.org  Tue Jan 15 13:33:33 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 15 Jan 2008 19:33:33 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <478CFB41.2010908@bostream.nu>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
Message-ID: <20080115183333.GD7918@phare.normalesup.org>

On Tue, Jan 15, 2008 at 07:28:17PM +0100, J?rgen Stenarson wrote:
> I think having C code in the project will be a major drawback in
> getting
> new developers on windows. Getting a compiler working can be a lot of
> work. But the positive aspects of using something trait-like seems it
> could definitely be worth it.

I agree. I think we (by we I mean he was does the work, probably not me)
should make it easy to use a standard Traits instal, if available. That
way someone with no compiler can simply download the binary Traits egg.

Cheers,

Ga?l


From fperez.net at gmail.com  Tue Jan 15 14:11:15 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jan 2008 12:11:15 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <20080115183333.GD7918@phare.normalesup.org>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
Message-ID: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>

Hi all,

first, thanks everyone for the feedback.  I think the enthusiastic
response is a sign that getting this ball rolling was indeed the right
thing to do.  A few things then:

- DVCS (HG). There seems to be broad agreement here, so I'll start
looking into how to actually make it happen.  I'll report back once I
have news of what the final setup will be.

- GUI front ends: we have interest on OSX, Qt4, WX (both the existing
prototype and Laurent's new work).  This is excellent.  A really cool
project once this is in will be a GTK vesion that runs on the OLPC, so
that the real ipython can be the shell of the Pippy programming
activity on the XO.

We'll look into the licensing issues regarding Qt4, and whether we
have to distribute that code as a standalone subproject to avoid GPL
contamination.  So far matplotlib has been shipping its Qt4 support
inside the project without any problems, but if that becomes an issue,
we'll break up the Qt code into a standalone component.  But we *will*
make sure that a Qt4 version can happen as a first-class citizen,
that's promised.

- Traits.  As expected, this was likely to be the most problematic
issue, and I appreciate the feedback from all.  It's also something
that worries *me* a lot, since being pure python, pure stdlib is a
huge advantage in terms of ease of installation, deployment and use
for us.  I can't count anymore how many times we've gone back and
forth on this issue, wanting the convenience and elegance of Traits'
declarative model for data validation and delegation, while worrying
about the cost of the dependency.

Brian and I had yesterday what I think was a very useful conversation
about it, and here's our current proposal on how to address the issue.
 Feedback on this point very much welcome.

The idea is to write a very lightweight, Traits-like module that does
only a fraction of what Traits does, but does so with 100% api
compatibility within that subset.  Basically we'd only implement

- Validation of types (probably only for a very small subset of types
like strings, numbers and lists)
- Simple registration of callbacks (listeners), so that changes to
configuration can trigger the appropriate callbacks to update other
things.

Since in ipython we're not trying to build something like say Mayavi2,
but just to organize our config issues cleanly, we'd have to limit
ourselves to these capabilities, which I don't think is the end of the
world.

However, we'd put in a global option to enable ipython to use the
*real* traits if a user wants.  This could be used for example by some
of the higher-level components, for example a Wx-based shell or an
Envisage (Enthought) plugin could decide to make use of the full power
of Traits.  Obviously for such a project, Traits would then become a
non-optional dependency, but it's a decision they can each make on a
case by case basis, rather than being imposed on everyone from the
very bottom of the project.

This would mean that there would always be a pure-python ipython that
operates in a terminal, along with a few optional gui shells that only
require a given gui toolkit to run.  And we may also have traits-using
tools as well, but those would be optional and "in the leaves" of the
logical dependency tree, not at the very core.

How does that sound to everyone?  I'm particularly interested in the
opinion of people like Gael and Robert, who have extensive experience
working with Traits.

Cheers,

f


From ellisonbg.net at gmail.com  Tue Jan 15 14:18:03 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 15 Jan 2008 12:18:03 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
Message-ID: <6ce0ac130801151118w6c1552d1gdfca9eb97caa5986@mail.gmail.com>

> - Traits.  As expected, this was likely to be the most problematic
> issue, and I appreciate the feedback from all.  It's also something
> that worries *me* a lot, since being pure python, pure stdlib is a
> huge advantage in terms of ease of installation, deployment and use
> for us.  I can't count anymore how many times we've gone back and
> forth on this issue, wanting the convenience and elegance of Traits'
> declarative model for data validation and delegation, while worrying
> about the cost of the dependency.
>
> Brian and I had yesterday what I think was a very useful conversation
> about it, and here's our current proposal on how to address the issue.
>  Feedback on this point very much welcome.
>
> The idea is to write a very lightweight, Traits-like module that does
> only a fraction of what Traits does, but does so with 100% api
> compatibility within that subset.  Basically we'd only implement
>
> - Validation of types (probably only for a very small subset of types
> like strings, numbers and lists)
> - Simple registration of callbacks (listeners), so that changes to
> configuration can trigger the appropriate callbacks to update other
> things.
>
> Since in ipython we're not trying to build something like say Mayavi2,
> but just to organize our config issues cleanly, we'd have to limit
> ourselves to these capabilities, which I don't think is the end of the
> world.
>
> However, we'd put in a global option to enable ipython to use the
> *real* traits if a user wants.  This could be used for example by some
> of the higher-level components, for example a Wx-based shell or an
> Envisage (Enthought) plugin could decide to make use of the full power
> of Traits.  Obviously for such a project, Traits would then become a
> non-optional dependency, but it's a decision they can each make on a
> case by case basis, rather than being imposed on everyone from the
> very bottom of the project.
>
> This would mean that there would always be a pure-python ipython that
> operates in a terminal, along with a few optional gui shells that only
> require a given gui toolkit to run.  And we may also have traits-using
> tools as well, but those would be optional and "in the leaves" of the
> logical dependency tree, not at the very core.
>
> How does that sound to everyone?  I'm particularly interested in the
> opinion of people like Gael and Robert, who have extensive experience
> working with Traits.

It is also worth mentioning that another factor that led us to this
conclusion is Jython and IronPython.  A C dependency would kill any
efforts to get IPython running in those settings.


> Cheers,
>
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From gael.varoquaux at normalesup.org  Tue Jan 15 14:21:04 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 15 Jan 2008 20:21:04 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
Message-ID: <20080115192104.GH7918@phare.normalesup.org>

On Tue, Jan 15, 2008 at 12:11:15PM -0700, Fernando Perez wrote:
> The idea is to write a very lightweight, Traits-like module that does
> only a fraction of what Traits does, but does so with 100% api
> compatibility within that subset.  Basically we'd only implement

> - Validation of types (probably only for a very small subset of types
> like strings, numbers and lists)
> - Simple registration of callbacks (listeners), so that changes to
> configuration can trigger the appropriate callbacks to update other
> things.

Very interesting. I think there is no way you could use Traits as
extensively as we use it in Traits-based project, doing reactive
programming using Traits callback. However, for a small use, this seems
like a good idea.

I wonder whether you shouldn't simply try to patch Traits, as there is
only one C file. So you could provide a replacement to it. I don't know
what Dave Morrill (the author of Traits, for those who do not know) would
think about this, but I think it would be a good idea.

My 2 cents,

Ga?l


From m.vanland at gmail.com  Tue Jan 15 15:20:48 2008
From: m.vanland at gmail.com (Michael VanLandingham)
Date: Tue, 15 Jan 2008 12:20:48 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
Message-ID: <b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>

Yeah it probably is a readline issue...  or editline, in this case.  But
what to do about it? :-)   I remember there were some fixes to get
tab-completion working back when Leopard was released.
I'm wondering, are most people (on this group) who are using iPython on OS X
running gnu readline (as opposed to editline)?

Thanks,
Michael


On Jan 14, 2008 11:01 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> On Jan 15, 2008 2:27 AM, Michael VanLandingham <m.vanland at gmail.com>
> wrote:
>
> > Notice how the 'def foo(s=None)' is on the same line as the 'if s:'?
>  And
> > once it does this, _all_ arrow-ups are out in the middle of the console
> and
> > well, generally messed up.
> >
> > You can still use this if you're careful, but it's sketchy.
> >
> > Is this a known issue?  Someone else recently mentioned to me that they
> were
> > seeing this as well.
>
> Seems like a readline issue to me.
>
> --
> Ville M. Vainio - vivainio.googlepages.com
> blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080115/687c7516/attachment.html>

From walter at livinglogic.de  Tue Jan 15 16:12:00 2008
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue, 15 Jan 2008 22:12:00 +0100
Subject: [IPython-dev] [IPython-user] IPython development news
	and	prospects
In-Reply-To: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	<478CFB41.2010908@bostream.nu>	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
Message-ID: <478D21A0.9070004@livinglogic.de>

Fernando Perez wrote:

> Hi all,
> 
> first, thanks everyone for the feedback.  I think the enthusiastic
> response is a sign that getting this ball rolling was indeed the right
> thing to do.  A few things then:
> 
> - DVCS (HG). There seems to be broad agreement here, so I'll start
> looking into how to actually make it happen.  I'll report back once I
> have news of what the final setup will be.

+1

Since we've switched from CVS to hg at work many problems are simply gone.

BTW, how will this work? Will we be able to push to the master 
repository, or is someone going to pull from our published repos?

> - GUI front ends: we have interest on OSX, Qt4, WX (both the existing
> prototype and Laurent's new work).  This is excellent.  A really cool
> project once this is in will be a GTK vesion that runs on the OLPC, so
> that the real ipython can be the shell of the Pippy programming
> activity on the XO.

I've haven't looked at the new wx shell, but combining it with igrid 
might give interesting results.

> [...]

Servus,
    Walter



From barrywark at gmail.com  Tue Jan 15 18:22:05 2008
From: barrywark at gmail.com (Barry Wark)
Date: Tue, 15 Jan 2008 15:22:05 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
Message-ID: <cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>

Yes, I'm using python.org's framework python (which uses readline),
though I've compiled a version that works with PyObjc2 (see
http://ipython.scipy.org/moin/InstallationOSXLeopard). There seem to
be constant headaches using Leopard's system python for
numpy/scipy/IPython. If you're developing Cocoa apps in Python, things
are different. Then the system python's a dream.

barry

On Jan 15, 2008 12:20 PM, Michael VanLandingham <m.vanland at gmail.com> wrote:
> Yeah it probably is a readline issue...  or editline, in this case.  But
> what to do about it? :-)   I remember there were some fixes to get
> tab-completion working back when Leopard was released.
>
>
> I'm wondering, are most people (on this group) who are using iPython on OS X
> running gnu readline (as opposed to editline)?
>
> Thanks,
> Michael
>
>
>
> On Jan 14, 2008 11:01 PM, Ville M. Vainio <vivainio at gmail.com> wrote:
> >
> > On Jan 15, 2008 2:27 AM, Michael VanLandingham <m.vanland at gmail.com>
> wrote:
> >
> > > Notice how the 'def foo(s=None)' is on the same line as the 'if s:'?
> And
> > > once it does this, _all_ arrow-ups are out in the middle of the console
> and
> > > well, generally messed up.
> > >
> > > You can still use this if you're careful, but it's sketchy.
> > >
> > > Is this a known issue?  Someone else recently mentioned to me that they
> were
> > > seeing this as well.
> >
> > Seems like a readline issue to me.
> >
> > --
> > Ville M. Vainio - vivainio.googlepages.com
> > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
> >
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>
>


From laurent.dufrechou at gmail.com  Tue Jan 15 18:33:22 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Wed, 16 Jan 2008 00:33:22 +0100
Subject: [IPython-dev] [IPython-user] IPython development
	news	and	prospects
In-Reply-To: <478D21A0.9070004@livinglogic.de>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	<478CFB41.2010908@bostream.nu>	<20080115183333.GD7918@phare.normalesup.org>	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<478D21A0.9070004@livinglogic.de>
Message-ID: <478d42d5.05ae660a.572d.4a30@mx.google.com>

Hi ipythoners,

Warning long mail... which main subject is GUI with ipython

***************** GUI integration ************************
Just some thought on Ipython GUI integration:

I've worked a little on one of the implementation of Ipython GUI integration
inside WX.
I've just discovered a big problem of my current implementation: I'm
creating an Ipython instance inside a thread and send it some line to
interpret.
The good thing is that the thread class is GUI toolkit independent --> good
The bad point is that prabhu asked me if it was possible to let say call
ipython with -wthread or -qthread.
(which is a good idea for example to develop a QTapp with wxIpython ^_^)
It seems that it is not possible or hard to do.(not tried a lot but from
I've seen...).

So my question is: 

Many people seems to be interested in GUI frontend for ipython, don't you
think it could be interesting to create a process whis a ipython instance
inside (the same as the wxIpython thread class in fact) and The GUI
wx,QT,everything else discuss with it via a network protocol like xml-rpc.
(I'm not software programmer so sure you will have better idea for
communication protocol:), I said xml-rpc because it's a base module of
python)

So the Gui part will only manage user interaction and will make ipython
process core execute line, get history and so on.
>From what I've seen from now the interace only needs a few function to
support ipython features (history,magic,tab_completion...)
This also could be reused with ipython1 later as the GUI interface will not
change only how we manage ipython1 inside the process
(well perhaps it's more tricky :) )

********************* QT4 **********************************************
For license question check pyqwt also, they've got a license section that
try to explain the problem.
http://pyqwt.sourceforge.net/license.html

What is interesting is that to have code that is not 'contamined' by gpl it
seems that we have to use exec to be OK with the license.
If anybody can confirm...
So it raise the interest of having an ipython interpreter in a separate
process... :)

By the way QT4 is a good idea, from what I've seen of WX there is some
multiplatform problem, and widget doesn't always react the same way. If we
could also use scintilla it could be good too because it is ported to
wx,QT,GTK and work on a lot of platform...
(it's only a suggestion), So perhaps it will be possible to share code...?

********** OLPC *****************
Hey for GTK question, I've recently tried PIDA under linux (too hard to
install on windows).
They've got a command shell in a process inside a gtk window, and if you
type in: ipython -wthread, well you've an ipython shell inside a gui that
support wx!(well no interaction with pida possible but well it's a good
start!).
That why I thought about the process thing...
So, perhaps if you speak with pida guys you can have an ipython gui + python
dev env for the olpc for nothing ;)
 
************** IGRID *****************
Can you tell me more about igrid?


Any comments, idea welcome!

Cheers,
Laurent



From fperez.net at gmail.com  Wed Jan 16 00:07:50 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jan 2008 22:07:50 -0700
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
Message-ID: <db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>

On Jan 15, 2008 4:22 PM, Barry Wark <barrywark at gmail.com> wrote:
> Yes, I'm using python.org's framework python (which uses readline),
> though I've compiled a version that works with PyObjc2 (see
> http://ipython.scipy.org/moin/InstallationOSXLeopard). There seem to
> be constant headaches using Leopard's system python for
> numpy/scipy/IPython. If you're developing Cocoa apps in Python, things
> are different. Then the system python's a dream.

Do you know if bug reports have been filed with Apple on their
readline-like thing?  It seems to me that the bug is on their end,
from what I've heard so far from everyone, though I could obviously be
wrong..

Cheers,

f


From prabhu at aero.iitb.ac.in  Wed Jan 16 00:20:02 2008
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Wed, 16 Jan 2008 10:50:02 +0530
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	<478CFB41.2010908@bostream.nu>	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
Message-ID: <478D9402.3030900@aero.iitb.ac.in>

Fernando Perez wrote:
[...]
> The idea is to write a very lightweight, Traits-like module that does
> only a fraction of what Traits does, but does so with 100% api
> compatibility within that subset.  Basically we'd only implement
[...]

IMNSHO, I think this is a step in the wrong direction!  Do you really 
have that much time on your hands?  How are you going to maintain b/w 
compat?  What about new versions/new features, tests, bugs?  An option 
would be to request Dave Morrill to see if this could be done in traits. 
  Kind of like elementtree/celementtree.

cheers,
prabhu


From fperez.net at gmail.com  Wed Jan 16 01:22:10 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jan 2008 23:22:10 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <478D9402.3030900@aero.iitb.ac.in>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<478D9402.3030900@aero.iitb.ac.in>
Message-ID: <db6b5ecc0801152222j3fb9bc80r650ec90879b8d50@mail.gmail.com>

On Jan 15, 2008 10:20 PM, Prabhu Ramachandran <prabhu at aero.iitb.ac.in> wrote:
> Fernando Perez wrote:
> [...]
> > The idea is to write a very lightweight, Traits-like module that does
> > only a fraction of what Traits does, but does so with 100% api
> > compatibility within that subset.  Basically we'd only implement
> [...]
>
> IMNSHO, I think this is a step in the wrong direction!  Do you really
> have that much time on your hands?  How are you going to maintain b/w
> compat?  What about new versions/new features, tests, bugs?  An option
> would be to request Dave Morrill to see if this could be done in traits.
>   Kind of like elementtree/celementtree.

I'm not sure it would take that much time: keep in mind I'm not
talking about implementing all of traits, but only:

1. Validation on assignment for a handful of basic, builtin types
(strings, numbers, lists).
2. Callback registration.

That's it.  It doesn't matter what new features the real Traits gets,
since we're not implementing those at all.

I'm a huge admirer of Traits, and would love to have it be *in the
language* so we could use all of its features for all  code (and so
that it was considered high-priority enough that the Jython/IronPython
people would implement its API on top of their own backends).  But the
reality today is that it's not in the core, and so we have to weigh
its cost against its benefits in the core.

I'm unfortunately in the middle of a packing/job change/cross country
move so I doubt I'll be able to prototype this quickly, but I honestly
don't think it's that hard at all.  Much of the really difficult work
in Traits has to do with a very rich API, GUI code, and a tight C
implementation for performance.  But just having validation and
callbacks is something that Python properties already almost do, and I
honestly don't see it being all  that hard.

Cheers,

f


From prabhu at aero.iitb.ac.in  Wed Jan 16 01:29:49 2008
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Wed, 16 Jan 2008 11:59:49 +0530
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801152222j3fb9bc80r650ec90879b8d50@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	
	<478CFB41.2010908@bostream.nu>	
	<20080115183333.GD7918@phare.normalesup.org>	
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>	
	<478D9402.3030900@aero.iitb.ac.in>
	<db6b5ecc0801152222j3fb9bc80r650ec90879b8d50@mail.gmail.com>
Message-ID: <478DA45D.2040004@aero.iitb.ac.in>

Fernando Perez wrote:
> On Jan 15, 2008 10:20 PM, Prabhu Ramachandran <prabhu at aero.iitb.ac.in> wrote:
> I'm unfortunately in the middle of a packing/job change/cross country
> move so I doubt I'll be able to prototype this quickly, but I honestly
> don't think it's that hard at all.  Much of the really difficult work
> in Traits has to do with a very rich API, GUI code, and a tight C
> implementation for performance.  But just having validation and
> callbacks is something that Python properties already almost do, and I
> honestly don't see it being all  that hard.

I'm not saying this can't be done.  I'm saying that rather than fork 
another implementation, why not discuss it with the creator and see if 
something can be worked out.  I think you should ask Dave Morrill about 
it.  He will have a better idea about it than anyone else.

cheers,
prabhu




From fperez.net at gmail.com  Wed Jan 16 01:30:31 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 15 Jan 2008 23:30:31 -0700
Subject: [IPython-dev] Brand new refactored version for linux users!
In-Reply-To: <478a8ac6.0407560a.61e1.7883@mx.google.com>
References: <47869441.0437560a.77de.3616@mx.google.com>
	<db6b5ecc0801130907x6b4dc946tcb3744d89c66652e@mail.gmail.com>
	<db6b5ecc0801131002j3c9cf77dg626592bad6b5efd5@mail.gmail.com>
	<478a7c2e.0b38560a.5c39.ffff80cd@mx.google.com>
	<db6b5ecc0801131312x316e4e87nbfc440493f111389@mail.gmail.com>
	<478a8ac6.0407560a.61e1.7883@mx.google.com>
Message-ID: <db6b5ecc0801152230x2ec5dc1ctc82838f24a86ed6f@mail.gmail.com>

On Jan 13, 2008 3:03 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:
> Ok got the screenshot.
> For syntax highlighting, it is for history only (look the little text
> written in blue ;) )
> Sadly I couldn't activate it for Ipython shell widget because it messed up
> the colors of In[,Out[ and ipython ouput (which still colored as normal
> ipython shell indeed).
> Parhaps will add it to my todo list, but I was so happy to have a similar
> working behavior on liux and windows that I let the things like this :)
> For the ESC bell etc...
> Ouch this is a big bug!
> ls,cd works well here under liux and windows.
>
> Please can you tell wich version of wx you use?
> Sudo synaptic, search wx: I've got python-wxgtk-2.8.4.0-0ubunutu3

Same here.

> I suspect a locales error.
> Can you do this?:
>
> Around line 424 in ipython_view.py:
> Replace line
> result =
> stdout.read()#.decode('cp437').encode(locale.getpreferredencoding())
> by
> result = stdout.read()
> #we use print command because the shell command is called inside IPython
> instance and thus is
> #redirected to thread cout
> #"\x01\x1b[1;36m\x02" <-- add colour to the text...
> print "\x01\x1b[1;36m\x02"+result
>
> I suspect that your terminal is not cp437 encoded.
> If it doesn't works try also to remove
> print "\x01\x1b[1;36m\x02"+result
> by:
> print result

Tried both, neither made any difference.  Same result as before.

>
> by the way I'm upgrading my ubuntu gutsy or gusty perhaps I will have same
> bug as yours :)

OK, let us know how that goes.

Cheers,

f


From fperez.net at gmail.com  Wed Jan 16 02:15:59 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Jan 2008 00:15:59 -0700
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <478D21A0.9070004@livinglogic.de>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<478D21A0.9070004@livinglogic.de>
Message-ID: <db6b5ecc0801152315u6aa55f32ya1cdc40d4cdfd07c@mail.gmail.com>

On Jan 15, 2008 2:12 PM, Walter D?rwald <walter at livinglogic.de> wrote:

> Since we've switched from CVS to hg at work many problems are simply gone.

Good to know.  I think this decision is made then, all the feedback so
far is positive.

>
> BTW, how will this work? Will we be able to push to the master
> repository, or is someone going to pull from our published repos?

I'm not sure yet, but my preferred mode would be to have a master hg
repo where all current devs have push rights.  This one would act much
like SVN today.  We'll need to look into the details later.

Cheers,

f


From fperez.net at gmail.com  Wed Jan 16 02:24:39 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Jan 2008 00:24:39 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <20080115192037.GG7918@phare.normalesup.org>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<20080115192037.GG7918@phare.normalesup.org>
Message-ID: <db6b5ecc0801152324h272939bamf9a47223aaed35ad@mail.gmail.com>

On Jan 15, 2008 12:20 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> Very interesting. I think there is no way you could use Traits as
> extensively as we use it in Traits-based project, doing reactive
> programming using Traits callback. However, for a small use, this seems
> like a good idea.

yes, we certainly couldn't use any of the real 'traits way', since we
would only implement the minimal stuff necessary for something like
tconfig to exist (except with support only for a few basic types being
validated).

>
> I wonder whether you shouldn't simply try to patch Traits, as there is
> only one C file. So you could provide a replacement to it. I don't know
> what Dave Morrill (the author of Traits, for those who do not know) would
> think about this, but I think it would be a good idea.

Before I bug David M, I'd like to have a little test of my own, just
to get a better feel for whether the whole idea even makes any sense
at all in practice.  This is one of those cases where working code
will end all arguments: if it's really easy just to write the little
piece we want, then that's OK, and we move on.  If it becomes rapidly
clear that there's a slippery slope of having to do half of the real
traits, I'll just punt and rethink the whole question.

Cheers,

f


From fperez.net at gmail.com  Wed Jan 16 02:52:15 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 16 Jan 2008 00:52:15 -0700
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <478d42d5.05ae660a.572d.4a30@mx.google.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<478D21A0.9070004@livinglogic.de>
	<478d42d5.05ae660a.572d.4a30@mx.google.com>
Message-ID: <db6b5ecc0801152352g56f810dbwd59596952319d6a@mail.gmail.com>

On Jan 15, 2008 4:33 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:

> ***************** GUI integration ************************
> Just some thought on Ipython GUI integration:
>
> I've worked a little on one of the implementation of Ipython GUI integration
> inside WX.
> I've just discovered a big problem of my current implementation: I'm
> creating an Ipython instance inside a thread and send it some line to
> interpret.
> The good thing is that the thread class is GUI toolkit independent --> good
> The bad point is that prabhu asked me if it was possible to let say call
> ipython with -wthread or -qthread.
> (which is a good idea for example to develop a QTapp with wxIpython ^_^)
> It seems that it is not possible or hard to do.(not tried a lot but from
> I've seen...).
>
> So my question is:
>
> Many people seems to be interested in GUI frontend for ipython, don't you
> think it could be interesting to create a process whis a ipython instance
> inside (the same as the wxIpython thread class in fact) and The GUI
> wx,QT,everything else discuss with it via a network protocol like xml-rpc.
> (I'm not software programmer so sure you will have better idea for
> communication protocol:), I said xml-rpc because it's a base module of
> python)
>
> So the Gui part will only manage user interaction and will make ipython
> process core execute line, get history and so on.
> From what I've seen from now the interace only needs a few function to
> support ipython features (history,magic,tab_completion...)
> This also could be reused with ipython1 later as the GUI interface will not
> change only how we manage ipython1 inside the process
> (well perhaps it's more tricky :) )

What you're asking for/describing is more or less the model for the
code in the new ipython1 core.  The idea is to have an object that
implements all the ipython basic functionality (magics, system access,
object introspection and completion, fancy exceptions, etc) but does
so simply with a clean API.  This object can then be used in-process
by an interactor that is based on readline and assumes it's running in
a terminal, but it can also be used by a WX or Qt interactor,  and it
can also be exposed over the network for out-of-process use.  You can
then have an out of process interface (say in javascript in a browser,
or even a Wx or Qt one that wants to be out of process).

And by having this, the distributed/parallel computing functionality
we have today in ipython1 (the stuff that the recent MPI questions
were about, for example) can then make full use of all the ipython
functionality in the engines.  This opens great possibilities for
distributed interactive debugging, attaching control consoles to
remote engines, etc.

As you can see, the possibilities opened by a cleaner design are many,
while keeping the fact that a simple, single-process, terminal based
ipython will continue to work much like today's, as that's a very
common and important use case.

I hope this is clear and answers your questions, let me know if it
isn't the case.

Cheers,

f

> ********** OLPC *****************
> Hey for GTK question, I've recently tried PIDA under linux (too hard to
> install on windows).
> They've got a command shell in a process inside a gtk window, and if you
> type in: ipython -wthread, well you've an ipython shell inside a gui that
> support wx!(well no interaction with pida possible but well it's a good
> start!).
> That why I thought about the process thing...
> So, perhaps if you speak with pida guys you can have an ipython gui + python
> dev env for the olpc for nothing ;)

Thanks for the pointer.  The OLPC thing is probably more future work,
but if anyone wants to jump on it now, great!

Cheers,

f


From vivainio at gmail.com  Wed Jan 16 03:23:48 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 16 Jan 2008 10:23:48 +0200
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
Message-ID: <46cb515a0801160023n327bc192u53ac74ecc664a341@mail.gmail.com>

On Jan 15, 2008 9:11 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> The idea is to write a very lightweight, Traits-like module that does
> only a fraction of what Traits does, but does so with 100% api
> compatibility within that subset.  Basically we'd only implement
>
> - Validation of types (probably only for a very small subset of types
> like strings, numbers and lists)
> - Simple registration of callbacks (listeners), so that changes to
> configuration can trigger the appropriate callbacks to update other
> things.

Isn't this rather easy to hack into current "Struct" class? See
ipstruct.py / Struct.__setitem__

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From pete.forman at westerngeco.com  Wed Jan 16 04:55:48 2008
From: pete.forman at westerngeco.com (Pete Forman)
Date: Wed, 16 Jan 2008 09:55:48 +0000
Subject: [IPython-dev] IPython development news and prospects
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
Message-ID: <fxwygkm3.fsf@wgmail2.gatwick.eur.slb.com>

J?rgen Stenarson <jorgen.stenarson at bostream.nu> writes:

 > Personally I really like the nice integration of tortoise-svn into
 > windows.  That is something I'd hate to lose.

TortoiseHg should fill that gap.

    http://tortoisehg.sourceforge.net/

-- 
Pete Forman                -./\.-  Disclaimer: This post is originated
WesternGeco                  -./\.-   by myself and does not represent
pete.forman at westerngeco.com    -./\.-   the opinion of Schlumberger or
http://petef.port5.com           -./\.-   WesternGeco.



From vivainio at gmail.com  Wed Jan 16 05:40:44 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 16 Jan 2008 12:40:44 +0200
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <fxwygkm3.fsf@wgmail2.gatwick.eur.slb.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<fxwygkm3.fsf@wgmail2.gatwick.eur.slb.com>
Message-ID: <46cb515a0801160240s6a7434b8m80b4332104022f7a@mail.gmail.com>

On Jan 16, 2008 11:55 AM, Pete Forman <pete.forman at westerngeco.com> wrote:

> J?rgen Stenarson <jorgen.stenarson at bostream.nu> writes:
>
>  > Personally I really like the nice integration of tortoise-svn into
>  > windows.  That is something I'd hate to lose.
>
> TortoiseHg should fill that gap.
>
>     http://tortoisehg.sourceforge.net/

And even w/o tortoisehg, the other GUI tools shipped w/ the rather new
"batteries included" hg package for windows cover all the bases that
need covering:

hg qct (commit tool, like tortoisesvn commit)
hg view & hg glog (history browsing)

http://qct.sourceforge.net/Mercurial-BI.html

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From hans_meine at gmx.net  Wed Jan 16 11:49:45 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 16 Jan 2008 17:49:45 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <200801131837.26953.darren.dale@cornell.edu>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<200801131837.26953.darren.dale@cornell.edu>
Message-ID: <200801161749.45795.hans_meine@gmx.net>

Am Montag, 14. Januar 2008 00:37:26 schrieb Darren Dale:
> > This is still just one project, that will have multiple components but
> > all of which work in concert to provide various levels of
> > functionality (a terminal-based shell, a gui-based shell, a
> > javascript-based one, a remote shell, a parallel computing system, an
> > ide-like system for scientific work, etc).
>
> Personally, I'd like to see a qt4 version of the gui as well.

Same here.

> I would be willing to work on it,

..and I would be happy to help.  (Gimme some weeks, I'll hopefully have time 
for this starting with Feb/Mar, being very busy/focused ATM.)

It is not fully clear to me what the state of IPyhon1, trunk, the new WX code 
in the latter, and the past SOC projects (based on the former?) is.  But 
that'll probably get sorted out soon.

Ciao, /  /
     /--/
    /  / ANS


From ellisonbg.net at gmail.com  Wed Jan 16 11:57:14 2008
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Wed, 16 Jan 2008 09:57:14 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <200801161749.45795.hans_meine@gmx.net>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<200801131837.26953.darren.dale@cornell.edu>
	<200801161749.45795.hans_meine@gmx.net>
Message-ID: <6ce0ac130801160857m2cad7dbav1d92eac385c01d99@mail.gmail.com>

> It is not fully clear to me what the state of IPyhon1, trunk, the new WX code
> in the latter, and the past SOC projects (based on the former?) is.  But
> that'll probably get sorted out soon.

Yep.  One of the first things we will need to do is begin to organize
the old+new codebases into subprojects that reflect where we are
headed.  Soon, we will probably begin a new thread to discuss these
things.

Brian

> Ciao, /  /
>      /--/
>     /  / ANS
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>


From hans_meine at gmx.net  Wed Jan 16 12:00:51 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 16 Jan 2008 18:00:51 +0100
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <478CFB41.2010908@bostream.nu>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
Message-ID: <200801161800.52077.hans_meine@gmx.net>

Am Dienstag, 15. Januar 2008 19:28:17 schrieb J?rgen Stenarson:
> > So the question is: does anyone here *fundamentally object* to
> > switching out to hg for the project?
>
> Personally I really like the nice integration of tortoise-svn into
> windows. That is something I'd hate to lose.

I heard that before, this is an important point.  (Disclaimer: I don't use 
Windows.)  I heard there's a Tortoise-bzr that is already usable, and a quick 
search suggests that there's an *experimental* hg port.

Given the popularity of hg and Tortoise-Foo, this will probably be worked 
on "soon".

Ciao, /  /
     /--/
    /  / ANS


From hans_meine at gmx.net  Wed Jan 16 12:16:02 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Wed, 16 Jan 2008 18:16:02 +0100
Subject: [IPython-dev]
	=?iso-8859-1?q?=5BIPython-user=5D_IPython_developme?=
	=?iso-8859-1?q?nt_news=09and=09prospects?=
In-Reply-To: <478d42d5.05ae660a.572d.4a30@mx.google.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478D21A0.9070004@livinglogic.de>
	<478d42d5.05ae660a.572d.4a30@mx.google.com>
Message-ID: <200801161816.02797.hans_meine@gmx.net>

Am Mittwoch, 16. Januar 2008 00:33:22 schrieb Laurent Dufr?chou:
> Many people seems to be interested in GUI frontend for ipython, don't you
> think it could be interesting to create a process whis a ipython instance
> inside (the same as the wxIpython thread class in fact) and The GUI
> wx,QT,everything else discuss with it via a network protocol like xml-rpc.

This is what we (mainly Ullrich K?the) has done for our VIGRA library python 
bindings (alas, not officially released yet).  There's a "pyterm" terminal 
that works this way.

The major feature is that the shell's GUI is always responsive, even with 
python code running.

Major obstacles AFAICS were Ctrl-C support (1), pluggable menu items (2), and 
completion, but all of them could be solved.

1: See the "interrupt" module I once posted to this list.  OTOH, Fernando 
(IIRC) lately introduced an ingenious solution in IPython that made this work 
(at least with threads, which is even more difficult AFAICS) with pure Python 
code.

2: For instance, if you type "import vigra" in pyterm, you'll get image 
analysis-submenus that open dialogs for composing python commands.  If you 
fill in variable names and parameters and press "OK", appropriate code is 
pasted into the terminal.  Similarly, I have implemented a menu that 
automatically contains all parameter-free functions from a specific file for 
quick execution (for a teaching project).

All features require communication between the frontend and backend.  I have 
also thought about porting our home-brewn communication code to xml-rpc, but 
have not much experience with the latter (only from the python 
challenge ;-) ).

Ciao, /  /
     /--/
    /  / ANS


From barrywark at gmail.com  Wed Jan 16 13:13:51 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 16 Jan 2008 10:13:51 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
Message-ID: <cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>

Fernando,

I'd check with Boyd. I know he filed some bug reports as he was
working on the existing editline support for Ipython. Unfortunately,
I'm a complete hack and don't really understand either readline or
editline, so I'm not a great source of useful reports for Apple. I
suspect that many issues we're having are not "bugs" per se, just
differences between the configuration of readline and editline.
editline is supposed to have a realine compatibility mode, so any
differences we can identify could be submitted to Apple as a bug
report. Like I said, I'm not really the best person for that job,
though I'm happy to compile information from others if they want to
send it to me.

Barry

On Jan 15, 2008 9:07 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Jan 15, 2008 4:22 PM, Barry Wark <barrywark at gmail.com> wrote:
> > Yes, I'm using python.org's framework python (which uses readline),
> > though I've compiled a version that works with PyObjc2 (see
> > http://ipython.scipy.org/moin/InstallationOSXLeopard). There seem to
> > be constant headaches using Leopard's system python for
> > numpy/scipy/IPython. If you're developing Cocoa apps in Python, things
> > are different. Then the system python's a dream.
>
> Do you know if bug reports have been filed with Apple on their
> readline-like thing?  It seems to me that the bug is on their end,
> from what I've heard so far from everyone, though I could obviously be
> wrong..
>
> Cheers,
>
> f
>


From barrywark at gmail.com  Wed Jan 16 13:19:10 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 16 Jan 2008 10:19:10 -0800
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <20ABD774-560C-46CB-B02B-383AE599BCDC@nrao.edu>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<200801131837.26953.darren.dale@cornell.edu>
	<cd7634ce0801141329ued262e4i1566d45b03204d82@mail.gmail.com>
	<20ABD774-560C-46CB-B02B-383AE599BCDC@nrao.edu>
Message-ID: <cd7634ce0801161019i5efaf8ebv53217f045698e64a@mail.gmail.com>

On Jan 14, 2008 1:40 PM, Boyd Waters <bwaters at nrao.edu> wrote:
>
> On Jan 14, 2008, at 2:29 PM, Barry Wark wrote:
>
> > Since we're quite heavily invested in both OS X and IPython in my
> > group, I'm willing to put time into an OS X front-end. We're working
> > on better OS X integration with matplotlib, so there's a natural
> > synergy to both efforts. Since Twisted is already well integrated with
> > all of the GUI toolkits that I've heard tossed about (WX, Qt, OS X,
> > gtk), is there still a thought towards making the ipython1 engine a
> > true ipython instance (rather than a pure python interpreter)? If so,
> > GUI integration gets much easier...
>
>
> Ya-hoo!
>
> I could help with that, too!
>
> We are heavy into Matplotlib, Qt4, Mac OS X, ipython, and ipython1 for
> our data analysis app.
>
> I'd love to push against a Twisted driver for Qt 4.4 and/or PyObjC2...

My understanding is that Twisted's NSRunLoop integration works fine
with PyObjC2. I've done a bit of work with PyObjC apps that use
Twisted and I'd push for this approach as well.

On the plotting end, we've started working on a quartz backend for
matplotlib and have been investigating enthought.chaco2 (and kiva)
quartz backends as well (rkern has already done much of the heavy
lifting on that end).  Getting one of these integrated into
Cocoa/Twisted would be a really huge win for us. Let me know if you'd
like to collaborate a bit. Qwt looks great, but until Qt is ported to
Cocoa, it's a non-starter for us. We use Cocoa for much of our UI and
Apple officially frowns on incorporating Carbon (like Qt/Qwt) views
into Cocoa apps :(

Barry


From barrywark at gmail.com  Wed Jan 16 13:21:08 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 16 Jan 2008 10:21:08 -0800
Subject: [IPython-dev] [IPython-user] IPython development news and
	prospects
In-Reply-To: <478d42d5.05ae660a.572d.4a30@mx.google.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<478D21A0.9070004@livinglogic.de>
	<478d42d5.05ae660a.572d.4a30@mx.google.com>
Message-ID: <cd7634ce0801161021v2e66c092oaec5b1ce1352c60e@mail.gmail.com>

On Jan 15, 2008 3:33 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:
> Hi ipythoners,
>
> Warning long mail... which main subject is GUI with ipython
>
> ***************** GUI integration ************************
> Just some thought on Ipython GUI integration:
>
> I've worked a little on one of the implementation of Ipython GUI integration
> inside WX.
> I've just discovered a big problem of my current implementation: I'm
> creating an Ipython instance inside a thread and send it some line to
> interpret.
> The good thing is that the thread class is GUI toolkit independent --> good
> The bad point is that prabhu asked me if it was possible to let say call
> ipython with -wthread or -qthread.
> (which is a good idea for example to develop a QTapp with wxIpython ^_^)
> It seems that it is not possible or hard to do.(not tried a lot but from
> I've seen...).
>
> So my question is:
>
> Many people seems to be interested in GUI frontend for ipython, don't you
> think it could be interesting to create a process whis a ipython instance
> inside (the same as the wxIpython thread class in fact) and The GUI
> wx,QT,everything else discuss with it via a network protocol like xml-rpc.
> (I'm not software programmer so sure you will have better idea for
> communication protocol:), I said xml-rpc because it's a base module of
> python)
>
> So the Gui part will only manage user interaction and will make ipython
> process core execute line, get history and so on.
> >From what I've seen from now the interace only needs a few function to
> support ipython features (history,magic,tab_completion...)
> This also could be reused with ipython1 later as the GUI interface will not
> change only how we manage ipython1 inside the process
> (well perhaps it's more tricky :) )

I'd put in a big vote for using Twisted for this. There's already
Twisted integration for most UI toolkits (including GTK, Wx, Qt, and
Cocoa). Since ipython1 is already invested in Twisted, it seems to
make sense to leverage that experience.

barry


From m.vanland at gmail.com  Wed Jan 16 16:12:15 2008
From: m.vanland at gmail.com (Michael VanLandingham)
Date: Wed, 16 Jan 2008 13:12:15 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
	<cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>
Message-ID: <AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>

I recall that there was a bug filed in October regarding tab- 
completion / rlcompleter (Boyd filed it).   The thing is, the multi- 
line history problem doesn't occur in the stock python 2.5.1on  
Leopard.  It works fine.   But it doesn't work in iPython.  I also  
don't fully understand readline or editline, so I don't know (and  
haven't found the time to discover) where the problem originates.

I can/will log a bug, but I'd like to figure out the specifics -- I  
don't think that 'multi-line history in iPython borked' would get the  
right attention.  If anyone has anymore insight as to _why_ this  
occurs, please let me know.

Meanwhile I think I'll try to compile readline for my system -- it's  
better software (than libedit) anyway.

Thanks,
-mvl


On Jan 16, 2008, at 10:13 AM, Barry Wark wrote:

> Fernando,
>
> I'd check with Boyd. I know he filed some bug reports as he was
> working on the existing editline support for Ipython. Unfortunately,
> I'm a complete hack and don't really understand either readline or
> editline, so I'm not a great source of useful reports for Apple. I
> suspect that many issues we're having are not "bugs" per se, just
> differences between the configuration of readline and editline.
> editline is supposed to have a realine compatibility mode, so any
> differences we can identify could be submitted to Apple as a bug
> report. Like I said, I'm not really the best person for that job,
> though I'm happy to compile information from others if they want to
> send it to me.
>
> Barry
>
> On Jan 15, 2008 9:07 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>> On Jan 15, 2008 4:22 PM, Barry Wark <barrywark at gmail.com> wrote:
>>> Yes, I'm using python.org's framework python (which uses readline),
>>> though I've compiled a version that works with PyObjc2 (see
>>> http://ipython.scipy.org/moin/InstallationOSXLeopard). There seem to
>>> be constant headaches using Leopard's system python for
>>> numpy/scipy/IPython. If you're developing Cocoa apps in Python,  
>>> things
>>> are different. Then the system python's a dream.
>>
>> Do you know if bug reports have been filed with Apple on their
>> readline-like thing?  It seems to me that the bug is on their end,
>> from what I've heard so far from everyone, though I could obviously  
>> be
>> wrong..
>>
>> Cheers,
>>
>> f
>>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev



From vivainio at gmail.com  Wed Jan 16 16:15:02 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 16 Jan 2008 23:15:02 +0200
Subject: [IPython-dev] Splitting interact() to non-blocking parts
Message-ID: <46cb515a0801161315h497090ddv1adb0df0ca9284e2@mail.gmail.com>

As previously seen, there is need for non-blocking interact(), for GUI use

I figured I'll add few extra functions to InteractiveShell to provide for this

interact_prompt - displays prompt
interact_handle_input(s) - handles input

of course normal interact can be implemented using these & raw_input.

It should then be possible to just instantiate interactiveshell and
start calling these in succession from GUI.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From bwaters at nrao.edu  Wed Jan 16 17:38:01 2008
From: bwaters at nrao.edu (Boyd Waters)
Date: Wed, 16 Jan 2008 15:38:01 -0700
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
	<cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>
	<AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>
Message-ID: <EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>


On Jan 16, 2008, at 2:12 PM, Michael VanLandingham wrote:

> I recall that there was a bug filed in October regarding tab-
> completion / rlcompleter (Boyd filed it).   The thing is, the multi-
> line history problem doesn't occur in the stock python 2.5.1on
> Leopard.  It works fine.

I think Apple tweaked something with the update to 10.5.1

Can you point me to the bug that I filed? I'll take another look.



>   But it doesn't work in iPython.  I also
> don't fully understand readline or editline, so I don't know (and
> haven't found the time to discover) where the problem originates.

I suspect that something isn't getting bound correctly in .editrc -- I  
can probably deal with this in an ipython profile but not sure. Again,  
if I can understand what "multi-line history" means maybe I can be  
more intelligent. Looking at that bug report may remind me.



> I can/will log a bug, but I'd like to figure out the specifics -- I
> don't think that 'multi-line history in iPython borked' would get the
> right attention.  If anyone has anymore insight as to _why_ this
> occurs, please let me know.
>


> Meanwhile I think I'll try to compile readline for my system -- it's
> better software (than libedit) anyway.


Readline is easy to compile on Leopard; I use MacPorts to build a  
Python 2.5.1 with GNU readline. There are a number of dependencies,  
mostly internationalization stuff, if I recall correctly; I have all  
that solved via MacPorts. I can provide these port files, or a binary  
installer for a Python 2.5.1 with readline, if you like.

I see a mention of dtrace support in Apple's Python. Hmm. I don't know  
that the binary needs to do anything special to provide dtrace -- I  
don't think you need to compile in any "hooks", that's the whole point  
of dtrace. In any event I'll take a look at Apple's source, if any,  
and see if we can't get a Leopard Python/readline set up.

But I have a strong preference for using the system Python in Leopard,  
so I'd like to solve the libedit readline issues if possible.


   - boyd


Boyd Waters
Scientific Programmer
National Radio Astronomy Observatory
Socorro, New Mexico



From robert.kern at gmail.com  Wed Jan 16 17:51:11 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 16 Jan 2008 16:51:11 -0600
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>	<cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>	<AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>
	<EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>
Message-ID: <fmm1ov$pgl$1@ger.gmane.org>

Boyd Waters wrote:

> I see a mention of dtrace support in Apple's Python. Hmm. I don't know  
> that the binary needs to do anything special to provide dtrace -- I  
> don't think you need to compile in any "hooks", that's the whole point  
> of dtrace. In any event I'll take a look at Apple's source, if any,  
> and see if we can't get a Leopard Python/readline set up.

You do need to patch Python to get dtrace support for Python-specific events.

http://www.opensource.apple.com/darwinsource/10.5/python-30/fix/ceval.c.ed

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From m.vanland at gmail.com  Wed Jan 16 18:37:36 2008
From: m.vanland at gmail.com (Michael VanLandingham)
Date: Wed, 16 Jan 2008 15:37:36 -0800
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
	<cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>
	<AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>
	<EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>
Message-ID: <3E42E015-5CB9-4832-A07E-AD6C28609D3F@gmail.com>


On Jan 16, 2008, at 2:38 PM, Boyd Waters wrote:

> On Jan 16, 2008, at 2:12 PM, Michael VanLandingham wrote:
>
>> I recall that there was a bug filed in October regarding tab-
>> completion / rlcompleter (Boyd filed it).   The thing is, the multi-
>> line history problem doesn't occur in the stock python 2.5.1on
>> Leopard.  It works fine.
>
> I think Apple tweaked something with the update to 10.5.1
>
> Can you point me to the bug that I filed? I'll take another look.


<rdar://problem/5563035> Python readline - rlcompleter does not work
(on https://bugreport.apple.com/, do 'Find Problem' at the top. etc.)


>>  But it doesn't work in iPython.  I also
>> don't fully understand readline or editline, so I don't know (and
>> haven't found the time to discover) where the problem originates.
>
> I suspect that something isn't getting bound correctly in .editrc --  
> I can probably deal with this in an ipython profile but not sure.  
> Again, if I can understand what "multi-line history" means maybe I  
> can be more intelligent. Looking at that bug report may remind me.

So what I mean is that, in iPython, if I enter in anything that spans  
more than one line, such as a function definition.  Say it's this:

def dog():
     print 'bark'

and then I want to edit that (e.g. because what I really want is for  
dog() to print 'woof'). So I hit the up-arrow key to insert the  
previous history entry into the current line.  But when I do the  
terminal formatting becomes corrupted - for example, the cursor is out  
in the middle of the row, and so on.  It makes editing almost  
impossible.

I think there was a better example in my original post.  It's easier  
to show than to explain, but you'd know it if you saw it.  It happens  
with stock Leopard & current iPython.

I too prefer to use the stock python framework on Leopard, since I  
tend to use a lot of PyObjC and bridged frameworks, not to mention I  
don't want to have to customize every system I might use.

Thanks,
-mvl


>> I can/will log a bug, but I'd like to figure out the specifics -- I
>> don't think that 'multi-line history in iPython borked' would get the
>> right attention.  If anyone has anymore insight as to _why_ this
>> occurs, please let me know.
>>
>
>
>> Meanwhile I think I'll try to compile readline for my system -- it's
>> better software (than libedit) anyway.
>
>
> Readline is easy to compile on Leopard; I use MacPorts to build a  
> Python 2.5.1 with GNU readline. There are a number of dependencies,  
> mostly internationalization stuff, if I recall correctly; I have all  
> that solved via MacPorts. I can provide these port files, or a  
> binary installer for a Python 2.5.1 with readline, if you like.
>
> I see a mention of dtrace support in Apple's Python. Hmm. I don't  
> know that the binary needs to do anything special to provide dtrace  
> -- I don't think you need to compile in any "hooks", that's the  
> whole point of dtrace. In any event I'll take a look at Apple's  
> source, if any, and see if we can't get a Leopard Python/readline  
> set up.
>
> But I have a strong preference for using the system Python in  
> Leopard, so I'd like to solve the libedit readline issues if possible.
>
>
>  - boyd
>
>
> Boyd Waters
> Scientific Programmer
> National Radio Astronomy Observatory
> Socorro, New Mexico
>



From bwaters at nrao.edu  Wed Jan 16 19:58:59 2008
From: bwaters at nrao.edu (Boyd Waters)
Date: Wed, 16 Jan 2008 17:58:59 -0700
Subject: [IPython-dev] Fwd: Multi-line history problems in ipython (mac
	osx)
In-Reply-To: <3E42E015-5CB9-4832-A07E-AD6C28609D3F@gmail.com>
References: <b364be0801141601w28b6316dxa08dd8390875dcd8@mail.gmail.com>
	<b364be0801141627t4e3c4004sc6b4de1d01c27eac@mail.gmail.com>
	<46cb515a0801142301t5d8acd5cm66b583959c10193a@mail.gmail.com>
	<b364be0801151220n639821cu53ea2c440ae3aca0@mail.gmail.com>
	<cd7634ce0801151522p595f0331s999bd477dffabb02@mail.gmail.com>
	<db6b5ecc0801152107l71466178xccd97ffd3b818b40@mail.gmail.com>
	<cd7634ce0801161013yde0aae7ifba1f4198b241d4@mail.gmail.com>
	<AC090690-6D28-4323-93C0-8A442144B94F@gmail.com>
	<EBB22BC5-526C-4CC8-A9D5-F246707D717D@nrao.edu>
	<3E42E015-5CB9-4832-A07E-AD6C28609D3F@gmail.com>
Message-ID: <00F05F5B-D055-4E56-86C5-69E267092578@nrao.edu>


On Jan 16, 2008, at 4:37 PM, Michael VanLandingham wrote:

> So what I mean is that, in iPython, if I enter in anything that  
> spans more than one line, such as a function definition.  Say it's  
> this:
>
> def dog():
>    print 'bark'
>
> and then I want to edit that (e.g. because what I really want is for  
> dog() to print 'woof'). So I hit the up-arrow key to insert the  
> previous history entry into the current line.  But when I do the  
> terminal formatting becomes corrupted - for example, the cursor is  
> out in the middle of the row, and so on.  It makes editing almost  
> impossible.
>
> I think there was a better example in my original post.  It's easier  
> to show than to explain, but you'd know it if you saw it.  It  
> happens with stock Leopard & current iPython.


Oh. right.

that's a termcap issue.

comments at this blog entry
http://www.osxautomation.com/2007/10/26/ipython-tab-completion-fix-on-leopard/

report the same problem.


I ran across another place recently where they were having that  
trouble.... hmm... I can't find it now. I'll keep looking.


Regarding <rdar://problem/5563035>, I suppose I should update that -  
I'm no longer getting duplicate completions.

I haven't tried to reproduce the ipython crasher (binding TAB key to  
unknown function causes segfault in IPython but not "stock" Python).



   - boyd

Boyd Waters
http://www.aoc.nrao.edu/~bwaters







From strawman at astraw.com  Thu Jan 17 04:07:11 2008
From: strawman at astraw.com (Andrew Straw)
Date: Thu, 17 Jan 2008 01:07:11 -0800
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <db6b5ecc0801152324h272939bamf9a47223aaed35ad@mail.gmail.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	<478CFB41.2010908@bostream.nu>	<20080115183333.GD7918@phare.normalesup.org>	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>	<20080115192037.GG7918@phare.normalesup.org>
	<db6b5ecc0801152324h272939bamf9a47223aaed35ad@mail.gmail.com>
Message-ID: <478F1ABF.6080706@astraw.com>

IIRC, way back in the day (2003?) traits (1.x?) was pure-Python. I think 
there's a good chance you could grab it and run it as is, simply 
updating the API.

Fernando Perez wrote:
> On Jan 15, 2008 12:20 PM, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
>
>   
>> Very interesting. I think there is no way you could use Traits as
>> extensively as we use it in Traits-based project, doing reactive
>> programming using Traits callback. However, for a small use, this seems
>> like a good idea.
>>     
>
> yes, we certainly couldn't use any of the real 'traits way', since we
> would only implement the minimal stuff necessary for something like
> tconfig to exist (except with support only for a few basic types being
> validated).
>
>   
>> I wonder whether you shouldn't simply try to patch Traits, as there is
>> only one C file. So you could provide a replacement to it. I don't know
>> what Dave Morrill (the author of Traits, for those who do not know) would
>> think about this, but I think it would be a good idea.
>>     
>
> Before I bug David M, I'd like to have a little test of my own, just
> to get a better feel for whether the whole idea even makes any sense
> at all in practice.  This is one of those cases where working code
> will end all arguments: if it's really easy just to write the little
> piece we want, then that's OK, and we move on.  If it becomes rapidly
> clear that there's a slippery slope of having to do half of the real
> traits, I'll just punt and rethink the whole question.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev
>   



From fperez.net at gmail.com  Thu Jan 17 13:14:09 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 17 Jan 2008 11:14:09 -0700
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <478F1ABF.6080706@astraw.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>
	<478CFB41.2010908@bostream.nu>
	<20080115183333.GD7918@phare.normalesup.org>
	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>
	<20080115192037.GG7918@phare.normalesup.org>
	<db6b5ecc0801152324h272939bamf9a47223aaed35ad@mail.gmail.com>
	<478F1ABF.6080706@astraw.com>
Message-ID: <db6b5ecc0801171014i79223565mc75219a593dde91e@mail.gmail.com>

On Jan 17, 2008 2:07 AM, Andrew Straw <strawman at astraw.com> wrote:
> IIRC, way back in the day (2003?) traits (1.x?) was pure-Python. I think
> there's a good chance you could grab it and run it as is, simply
> updating the API.

Yup, here it is:

https://svn.enthought.com/enthought/browser/trunk/traits/traits.py?rev=4015

I'd thought about looking at this, but hadn't spent the time tracking
down the revision history.

I'll play a bit more with this as I find some time and will get in
touch with Enthought-dev to get their feedback (in particular David's)
regarding the best approach, esp. concerning the ability to have
jython/ironpython support.

Cheers,

f


From robert.kern at gmail.com  Thu Jan 17 13:38:07 2008
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 17 Jan 2008 12:38:07 -0600
Subject: [IPython-dev] IPython development news and prospects
In-Reply-To: <478F1ABF.6080706@astraw.com>
References: <db6b5ecc0801131406j3634b8e2t1b585fadfee144ca@mail.gmail.com>	<478CFB41.2010908@bostream.nu>	<20080115183333.GD7918@phare.normalesup.org>	<db6b5ecc0801151111t1840964bpdf4901963065f162@mail.gmail.com>	<20080115192037.GG7918@phare.normalesup.org>	<db6b5ecc0801152324h272939bamf9a47223aaed35ad@mail.gmail.com>
	<478F1ABF.6080706@astraw.com>
Message-ID: <fmo7af$k5r$1@ger.gmane.org>

Andrew Straw wrote:
> IIRC, way back in the day (2003?) traits (1.x?) was pure-Python. I think 
> there's a good chance you could grab it and run it as is, simply 
> updating the API.

Traits 1 is *extremely* different from Traits 2 or 3. Updating its API is 
tantamount to rewriting from scratch.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From hans_meine at gmx.net  Fri Jan 18 11:46:16 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 18 Jan 2008 17:46:16 +0100
Subject: [IPython-dev] Clearing exception info stored for %debug?
Message-ID: <200801181746.26728.hans_meine@gmx.net>

Hi!

I just suffered from a complicated situation; I had a file being clobbered on 
IPython quit by a __del__ method from one of the objects in the stack within 
an exception that occurred earlier.  After the exception, I only ran one 
corrected command which stored the right file after a long computation, but 
when I quit IPython the data was overwritten by the partial results from the 
earlier, broken run.

This situation is probably a rare one, but I thought that one might want to 
introduce "%clear exception" or sth. along that lines.

Ciao, /  /                                                    .o.
     /--/                                                     ..o
    /  / ANS                                                  ooo
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part.
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080118/b3f45c16/attachment.sig>

From edreamleo at yahoo.com  Fri Jan 18 12:29:44 2008
From: edreamleo at yahoo.com (Edward Ream)
Date: Fri, 18 Jan 2008 09:29:44 -0800 (PST)
Subject: [IPython-dev] IPython and Leo
Message-ID: <450916.81767.qm@web57508.mail.re1.yahoo.com>


Hello All,





I have been following the IPython
project with interest for several years.  It's an admirable project
for at least the following reasons:





- It provides a simple, yet powerful
user interface.





- It handles complex implementation
difficulties smoothly behind the scenes.





- It has many interesting and important
users.





- It interfaces well with several major
tools.





- It has an excellent installation
model.





Several years ago, while discussing the
IPython summer of code proposal, I made a wildly inaccurate
statement, namely that Leo might already be an IPython notebook. 
This statement caused consternation;  I apologize again for the upset
I caused.





Recently I discovered, at long last,
why Leo is *not* a prototype for an IPython notebook.  Unless I am
again mistaken, I would characterize the IPython notebook project as
an attempt (roughly speaking) to do an Open Source version of
Mathematica.  There would be an outline component, a math engine, and
a renderer capable of drawing equations and the results of those
equations.





Is this characterization of the IPython
notebook project even vaguely correct?  If so, it seems to me that
Leo and the IPython notebook projects are on roughly parallel tracks.
 Yes, there are substantial differences between the two projects, but
there may well be ways that the two projects could benefit each
other.





Enough for now.  I hope we can begin a
friendly and mutually beneficial exchange between our two projects.





Edward

--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo at yahoo.com
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------




      ____________________________________________________________________________________
Looking for last minute shopping deals?  
Find them fast with Yahoo! Search.  http://tools.search.yahoo.com/newsearch/category.php?category=shopping
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080118/a150bc6e/attachment.html>

From laurent.dufrechou at gmail.com  Fri Jan 18 17:55:03 2008
From: laurent.dufrechou at gmail.com (=?us-ascii?Q?Laurent_Dufrechou?=)
Date: Fri, 18 Jan 2008 23:55:03 +0100
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
Message-ID: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>

Hi fernando,ipython-devs,

 

Fernando,I see the same bug here under ubunutu. ('ls' command line not
working under wxIPython)

In fact it is an ipython update problem J

 

Here is a part of code I use to redirect shell command:

 

#we replace the ipython default shell command caller by our shell handler

#FIXME: any better solution welcome

 IPython.genutils.shell_ori = self._shell                #needed by windows

self._IP.system = self._shell                                     #needed
for linux

   

So it seems that with last commits (I was working with ipython 0.8.1)
_IP.system is no more called under linux.

Anybody as an idea of what have recently changed in the code that could
explain that?

I've looked at commit and traced the code, but it's a little bit hard  for
me to find which function I need to bypass.

 

I'll continue to look at it but any help is welcome!

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080118/c148cf0b/attachment.html>

From vivainio at gmail.com  Sat Jan 19 06:05:19 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 19 Jan 2008 13:05:19 +0200
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
Message-ID: <46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>

On Jan 19, 2008 12:55 AM, Laurent Dufrechou <laurent.dufrechou at gmail.com> wrote:

> #FIXME: any better solution welcome
>
>  IPython.genutils.shell_ori = self._shell                #needed by windows
>
> self._IP.system = self._shell                                     #needed
> for linux

Better solution is indeed welcome :-P

I will create a system command hook and you can use that. That's what
hooks are for & I've been planning a 'subprocess' (module) based
'system' command a while now (to provide better process handling on
newer python versions).

Hold on, it will be ready today.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Jan 19 06:33:24 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 19 Jan 2008 13:33:24 +0200
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
Message-ID: <46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>

On Jan 19, 2008 1:05 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> I will create a system command hook and you can use that. That's what
> hooks are for & I've been planning a 'subprocess' (module) based
> 'system' command a while now (to provide better process handling on
> newer python versions).

Done in svn, r2951. I will add a process killing extension next to 'demo' this.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Sat Jan 19 09:01:02 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sat, 19 Jan 2008 15:01:02 +0100
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>	
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
	<46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>
Message-ID: <479202a6.0305560a.110a.20da@mx.google.com>

Thanks ville :)
Will check this!
I've tried to find a workaround, but it isn't successful for now.
By the way don't know if it is hard to do, but can you add in your todo list
callback for:

Ipython.genutils.page

-->
#we replace the ipython default pager by our pager
#FIXME: add a pager callback to IPython
IPython.OInspect.page = self._pager
IPython.genutils.page = self._pager
IPython.iplib.page = self._pager
IPython.Magic.page = self._pager

'cause it's one of these function hard to shortcut, a hook should be better.
In fact here I'm not sure to cover all page call so a hook is really really
needed to :)



-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: samedi 19 janvier 2008 12:33
??: Laurent Dufrechou
Cc?: IPython-dev at scipy.org; Fernando Perez
Objet?: Re: [IPython-dev] wxIPython, where as gone _IP.system() ?

On Jan 19, 2008 1:05 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> I will create a system command hook and you can use that. That's what
> hooks are for & I've been planning a 'subprocess' (module) based
> 'system' command a while now (to provide better process handling on
> newer python versions).

Done in svn, r2951. I will add a process killing extension next to 'demo'
this.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From vivainio at gmail.com  Sat Jan 19 09:49:25 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 19 Jan 2008 16:49:25 +0200
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <479202a6.0305560a.110a.20da@mx.google.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
	<46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>
	<479202a6.0305560a.110a.20da@mx.google.com>
Message-ID: <46cb515a0801190649p6cd42dddy6ba0a78636f09f18@mail.gmail.com>

On Jan 19, 2008 4:01 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:

> Thanks ville :)
> Will check this!
> I've tried to find a workaround, but it isn't successful for now.
> By the way don't know if it is hard to do, but can you add in your todo list
> callback for:
>
> Ipython.genutils.page

I will add a 'page_object' to generics.py, so you can hook different
pagers for different object types (as opposed to just strings).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Jan 19 10:04:41 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 19 Jan 2008 17:04:41 +0200
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <46cb515a0801190649p6cd42dddy6ba0a78636f09f18@mail.gmail.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
	<46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>
	<479202a6.0305560a.110a.20da@mx.google.com>
	<46cb515a0801190649p6cd42dddy6ba0a78636f09f18@mail.gmail.com>
Message-ID: <46cb515a0801190704s40b90034i1a55d8accce182d2@mail.gmail.com>

On Jan 19, 2008 4:49 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> I will add a 'page_object' to generics.py, so you can hook different
> pagers for different object types (as opposed to just strings).

... or not. I realized a good ol' hook is better for this application
(where most of the time paging a string is all that is needed), so
I'll just add a hook.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sat Jan 19 10:24:39 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sat, 19 Jan 2008 17:24:39 +0200
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <479202a6.0305560a.110a.20da@mx.google.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>
	<46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>
	<479202a6.0305560a.110a.20da@mx.google.com>
Message-ID: <46cb515a0801190724o57e2c8f3wee54335b3d966d85@mail.gmail.com>

On Jan 19, 2008 4:01 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com> wrote:

> By the way don't know if it is hard to do, but can you add in your todo list
> callback for:
>
> Ipython.genutils.page

Done (r2955).

The hook name is 'show_in_pager', and it gets a string to page.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Sat Jan 19 13:32:12 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Sat, 19 Jan 2008 19:32:12 +0100
Subject: [IPython-dev] wxIPython, where as gone _IP.system() ?
In-Reply-To: <46cb515a0801190724o57e2c8f3wee54335b3d966d85@mail.gmail.com>
References: <47912e4c.1438560a.7490.ffff8e6b@mx.google.com>	
	<46cb515a0801190305y14966043i8f015a17f5db5145@mail.gmail.com>	
	<46cb515a0801190333w2685817dv2e9e0ed7f63768cf@mail.gmail.com>	
	<479202a6.0305560a.110a.20da@mx.google.com>
	<46cb515a0801190724o57e2c8f3wee54335b3d966d85@mail.gmail.com>
Message-ID: <47924236.0405560a.6773.5738@mx.google.com>

Thanks ville!
Both hooks work very well.

Fernando, the bug you had has disappeared now :)

-----Message d'origine-----
De?: Ville M. Vainio [mailto:vivainio at gmail.com] 
Envoy??: samedi 19 janvier 2008 16:25
??: Laurent Dufr?chou
Cc?: IPython-dev at scipy.org; Fernando Perez
Objet?: Re: [IPython-dev] wxIPython, where as gone _IP.system() ?

On Jan 19, 2008 4:01 PM, Laurent Dufr?chou <laurent.dufrechou at gmail.com>
wrote:

> By the way don't know if it is hard to do, but can you add in your todo
list
> callback for:
>
> Ipython.genutils.page

Done (r2955).

The hook name is 'show_in_pager', and it gets a string to page.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'



From vivainio at gmail.com  Sun Jan 20 12:48:00 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 20 Jan 2008 19:48:00 +0200
Subject: [IPython-dev] IPython and Leo
In-Reply-To: <450916.81767.qm@web57508.mail.re1.yahoo.com>
References: <450916.81767.qm@web57508.mail.re1.yahoo.com>
Message-ID: <46cb515a0801200948g632f6cd2x927334e2bbeda434@mail.gmail.com>

On Jan 18, 2008 7:29 PM, Edward Ream <edreamleo at yahoo.com> wrote:

> are on roughly parallel tracks. Yes, there are substantial differences
> between the two projects, but there may well be ways that the two projects
> could benefit each other.

Yeah, it might be fun to play with the IPython integration with Leo a
bit; I somehow missed/forgot the original discussion. Perhaps at some
point we might want to push out ipy_leo.py IPython extension to
facilitate some interoperability tasks. I haven't used Leo in a while,
so perhaps it's time to have another look...

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From vivainio at gmail.com  Sun Jan 20 13:06:36 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Sun, 20 Jan 2008 20:06:36 +0200
Subject: [IPython-dev] IPython and Leo
In-Reply-To: <46cb515a0801200948g632f6cd2x927334e2bbeda434@mail.gmail.com>
References: <450916.81767.qm@web57508.mail.re1.yahoo.com>
	<46cb515a0801200948g632f6cd2x927334e2bbeda434@mail.gmail.com>
Message-ID: <46cb515a0801201006r2d7c09b7y705f6817f30185f1@mail.gmail.com>

On Jan 20, 2008 7:48 PM, Ville M. Vainio <vivainio at gmail.com> wrote:

> Yeah, it might be fun to play with the IPython integration with Leo a
> bit; I somehow missed/forgot the original discussion. Perhaps at some
> point we might want to push out ipy_leo.py IPython extension to
> facilitate some interoperability tasks. I haven't used Leo in a while,
> so perhaps it's time to have another look...

Note - I suppose Leo can also be extended to integrate well with IPython?

What would really make my day is the ability to do this:

[~]|29> s = "some sring"
[~]|30> %leo s

and then you would have the 's' as node in leo, and editing the string
(or perhaps some other variable type, e.g. pretty printed array) would
update the IPython variable s as well!

Too bad I was unable to install Leo now that I tried - the .exe
installer complained that I did not have python installed (which is of
course untrue, the standard distutils .exe installer worked ok...)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From edreamleo at yahoo.com  Thu Jan 24 10:54:01 2008
From: edreamleo at yahoo.com (Edward Ream)
Date: Thu, 24 Jan 2008 07:54:01 -0800 (PST)
Subject: [IPython-dev]  IPython and Leo
Message-ID: <654480.45388.qm@web57508.mail.re1.yahoo.com>

[quote]
What would really make my day is the ability to do this:

[~]|29> s = "some sring"
[~]|30> %leo s

and then you would have the 's' as node in leo, and editing the string
(or perhaps some other variable type, e.g. pretty printed array) would
update the IPython variable s as well!
[end quote]

A most interesting idea.  There are similar kinds of interactions possible
 now between Leo and Vim or Emacs, so such a scheme might be possible
with not too much work.  See the vim.py and xemacs.py plugins.

This scheme happily keeps the separation between Leo and IPython.  That is,
there would be no need to emulate IPython's powerful features in Leo.

As far as "%leo s" goes, s could be stored in a temp file, and then
Leo could be opened with a new command line option,
say --s <tempfilename>. Such a command-line option could easily
be added. 

> Too bad I was unable to install Leo now that I tried - the .exe
installer complained that I did not have python installed (which is of
course untrue, the standard distutils .exe installer worked ok...)

The .exe installer is a minor convenience, or inconvenience in your case.
To install Leo, just download and unpack leo-4.4.6-b2.zip from
http://sourceforge.net/project/showfiles.php?group_id=3458&package_id=29106&release_id=568796
You can put the Leo folder anywhere convenient.

Then add the folder to your sys.path.  HTH.

Edward


--------------------------------------------------------------------
Edward K. Ream   email:  edreamleo at yahoo.com
Leo: http://webpages.charter.net/edreamleo/front.html
--------------------------------------------------------------------




      ____________________________________________________________________________________
Be a better friend, newshound, and 
know-it-all with Yahoo! Mobile.  Try it now.  http://mobile.yahoo.com/;_ylt=Ahu06i62sR8HDtDypao8Wcj9tAcJ 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080124/db0d285b/attachment.html>

From hans_meine at gmx.net  Thu Jan 24 10:59:47 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 24 Jan 2008 16:59:47 +0100
Subject: [IPython-dev] IPython and Leo
In-Reply-To: <654480.45388.qm@web57508.mail.re1.yahoo.com>
References: <654480.45388.qm@web57508.mail.re1.yahoo.com>
Message-ID: <200801241659.48155.hans_meine@gmx.net>

Am Donnerstag, 24. Januar 2008 16:54:01 schrieb Edward Ream:
> [quote]
> What would really make my day is the ability to do this:
>
> [~]|29> s = "some sring"
> [~]|30> %leo s
>
> and then you would have the 's' as node in leo, and editing the string
> (or perhaps some other variable type, e.g. pretty printed array) would
> update the IPython variable s as well!
> [end quote]

That reminds me of traits, as used e.g. in envisage.  Just a heads-up; I 
always read about the enthought tool suite but only recently started to take 
a serious look at the stuff.  Traits are rather self-contained and 
independent of the higher-level stuff (e.g. envisage, chaco, mayavi,...), so 
maybe that's something you could use in Leo.

BTW: the GUI -> python direction is not the difficult one, but traits help you 
with the other direction, i.e. automatic GUI updating

Ciao, /  /
     /--/
    /  / ANS


From gael.varoquaux at normalesup.org  Thu Jan 24 11:05:25 2008
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 24 Jan 2008 17:05:25 +0100
Subject: [IPython-dev] IPython and Leo
In-Reply-To: <200801241659.48155.hans_meine@gmx.net>
References: <654480.45388.qm@web57508.mail.re1.yahoo.com>
	<200801241659.48155.hans_meine@gmx.net>
Message-ID: <20080124160525.GA19283@phare.normalesup.org>

On Thu, Jan 24, 2008 at 04:59:47PM +0100, Hans Meine wrote:
> That reminds me of traits, as used e.g. in envisage.  Just a heads-up; I 
> always read about the enthought tool suite but only recently started to take 
> a serious look at the stuff.  Traits are rather self-contained and 
> independent of the higher-level stuff (e.g. envisage, chaco, mayavi,...), so 
> maybe that's something you could use in Leo.

Good news on this side. Traits (and mayavi, and envisage...) are now in
Debian testing, and I have just sent a sync request to get them in
Ubuntu. With some luck they'll land in Hardy in a few days, and be
shipped when Hardy comes out.

-- 
    Ga?l
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080124/8ee7d94c/attachment.sig>

From vivainio at gmail.com  Thu Jan 24 15:17:12 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 24 Jan 2008 22:17:12 +0200
Subject: [IPython-dev] exe installer + vista
Message-ID: <46cb515a0801241217pebb2c9axe20422550a6d1955@mail.gmail.com>

Has anyone succeeded in running the exe installer in Vista recently? I
just tried it, and get a series of error dialogs, and no start menu
shortcuts are created.

I did not have time to seriously look into the issue, but if need be
we should make some workaround, or just suggest the vista users to
choose the source distribution.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From laurent.dufrechou at gmail.com  Thu Jan 24 18:07:13 2008
From: laurent.dufrechou at gmail.com (=?iso-8859-1?Q?Laurent_Dufr=E9chou?=)
Date: Fri, 25 Jan 2008 00:07:13 +0100
Subject: [IPython-dev] exe installer + vista
In-Reply-To: <46cb515a0801241217pebb2c9axe20422550a6d1955@mail.gmail.com>
References: <46cb515a0801241217pebb2c9axe20422550a6d1955@mail.gmail.com>
Message-ID: <47991a33.0c58560a.33c2.ffffe1c8@mx.google.com>

I'm on vista here and didn't had such problem.  (well with 0.8.1)
Since I got lot of problem with software installation, what I did is to
deactivate UAC (user access control).
It's most of the time from where problems comes under vista.
There is an option in vista if you don't deactivate it, something like
execute as administrator.(with a right click on the .exe)
Give it a try.
Another option is to right click and select properties and compatibility
mode windows XP SP2 for the installer .exe.
This last one solves the issues not solved by execute as administrator...

-----Message d'origine-----
De?: ipython-dev-bounces at scipy.org [mailto:ipython-dev-bounces at scipy.org] De
la part de Ville M. Vainio
Envoy??: jeudi 24 janvier 2008 21:17
??: ipython-dev Mailing list
Objet?: [IPython-dev] exe installer + vista

Has anyone succeeded in running the exe installer in Vista recently? I
just tried it, and get a series of error dialogs, and no start menu
shortcuts are created.

I did not have time to seriously look into the issue, but if need be
we should make some workaround, or just suggest the vista users to
choose the source distribution.

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'
_______________________________________________
IPython-dev mailing list
IPython-dev at scipy.org
http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev



From vivainio at gmail.com  Fri Jan 25 03:00:49 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 25 Jan 2008 10:00:49 +0200
Subject: [IPython-dev] exe installer + vista
In-Reply-To: <6586e22e0801241522r8e0bc84oc014a2711b12b6e7@mail.gmail.com>
References: <46cb515a0801241217pebb2c9axe20422550a6d1955@mail.gmail.com>
	<6586e22e0801241522r8e0bc84oc014a2711b12b6e7@mail.gmail.com>
Message-ID: <46cb515a0801250000j55b7701ey6fb6d0bb6b3ad16@mail.gmail.com>

On Jan 25, 2008 1:22 AM, Suraj Barkale <suraj+ipython at barkale.com> wrote:

> It seems Vista's elevation for installers just checks the file names
> so renaming ipython-0.8.2.win32.exe to ipython-0.8.2.win32-setup.exe
> will solve the problem :)

If that works, we should rename the exe installers we distribute!
Though of course this is something that should be fixed all the way up
in distutils...

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Fri Jan 25 03:09:13 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 25 Jan 2008 01:09:13 -0700
Subject: [IPython-dev] exe installer + vista
In-Reply-To: <46cb515a0801250000j55b7701ey6fb6d0bb6b3ad16@mail.gmail.com>
References: <46cb515a0801241217pebb2c9axe20422550a6d1955@mail.gmail.com>
	<6586e22e0801241522r8e0bc84oc014a2711b12b6e7@mail.gmail.com>
	<46cb515a0801250000j55b7701ey6fb6d0bb6b3ad16@mail.gmail.com>
Message-ID: <db6b5ecc0801250009l15055b98h924dd9cad3a61244@mail.gmail.com>

On Jan 25, 2008 1:00 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Jan 25, 2008 1:22 AM, Suraj Barkale <suraj+ipython at barkale.com> wrote:
>
> > It seems Vista's elevation for installers just checks the file names
> > so renaming ipython-0.8.2.win32.exe to ipython-0.8.2.win32-setup.exe
> > will solve the problem :)
>
> If that works, we should rename the exe installers we distribute!
> Though of course this is something that should be fixed all the way up
> in distutils...

Can you verify this?  If that's the case, I'll quickly rename our
installer (I'm sure XP doesn't care what the name is) and will update
the release script to make the 'fixed' name by default.

Cheers,

f


From vivainio at gmail.com  Fri Jan 25 11:49:36 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Fri, 25 Jan 2008 18:49:36 +0200
Subject: [IPython-dev] IPython and Leo
In-Reply-To: <654480.45388.qm@web57508.mail.re1.yahoo.com>
References: <654480.45388.qm@web57508.mail.re1.yahoo.com>
Message-ID: <46cb515a0801250849g268efa33v275c882c77b0948d@mail.gmail.com>

What would I do to accomplish this:

- Write this in node contents:

def foo():
 print "hello"

- Press some key (ctrl+b does not work, that's what I tried), and the
code gets run inside the embedded IPython instance (i.e. executed in
the IPython's user namespace)

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From kschutte at csail.mit.edu  Tue Jan 29 17:10:11 2008
From: kschutte at csail.mit.edu (Ken Schutte)
Date: Tue, 29 Jan 2008 17:10:11 -0500
Subject: [IPython-dev] automatic cpaste
Message-ID: <479FA443.8010406@csail.mit.edu>

Hello,

I've been trying to figure out a way to make cpaste more "automatic".

The closest I've come is the attached cpaste modification.  Using 
"cpaste -a" automatically executes the contents of the clipboard; 
however, it requires the 'xclip' command, which is run with os.popen3(). 
(btw, xclip is a nice command-line utility - I've found it's in most 
distro's repositories, but usually not installed by default).

I'm not sure if you'd want to add this patch because of this dependency, 
but it should be backward compatible as is. (or is there a better way to 
get the clipbard contents?)

This is maybe a slight improvement for me, but I'd love something where 
I can truly do a simple paste with the mouse and everything is done 
automatically.  I guess this is basically not possible within ipython?

I usually use ipython within a Konsole window, so the only other thing I 
can think of is to figure out if you can easily do something there to 
make this happen, e.g. intercept a mouse paste, and instead paste in the 
text "cpaste\n"+$clipboard+"\n--\n".

If anyone has any suggestions, please let me know.

Ken
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython.diff
Type: text/x-patch
Size: 2512 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080129/79856be5/attachment.bin>

From michael.j.mclay at gmail.com  Tue Jan 29 17:16:07 2008
From: michael.j.mclay at gmail.com (Michael McLay)
Date: Tue, 29 Jan 2008 17:16:07 -0500
Subject: [IPython-dev] TaskController security
Message-ID: <683f83340801291416h1723fd7chbef9c51ff74d2625@mail.gmail.com>

Some experimentation with the TaskController in the ipython1 trunk has
been very encouraging. It looks like a great tool for managing a queue
of tasks that need to be executed on a cluster, or even distributed to
a bunch of workstations in an office. TaskController seems to do a
decent job of scheduling the execution of tasks without any tweeking
required. Thanks for making the software available. It was very easy
to get the software to work with our software.  Almost too easy. I am
puzzled that there is almost nothing written on using TaskController.
Also, the development seems to have stopped. Is development completed
and documentation simply not necessary?

What are the security recommendations for using ipython1? How can an
application protect the RemoteController and a TaskController used to
parcel out tasks on a local network? It looks like a rogue application
could simply connection to the RemoteController and corrupt data on
any engine connected to the controller. The following email suggests
work was planned to add security to ipython1:

    http://article.gmane.org/gmane.comp.python.ipython.devel/1467

The mailing list archive was cold on the subject after that thread and
the word security only exists in the twisted portion of the source
tree. What was planned and what happened to those plans?

Any leads on adding security would be appreciated? Where you going to
use Twisted's security to manage access to controllers?


From fperez.net at gmail.com  Wed Jan 30 02:57:10 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 30 Jan 2008 00:57:10 -0700
Subject: [IPython-dev] Development plans update
Message-ID: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>

Hi all,

I'd like to drop a quick update on the current plans, as well as
propose a few specific steps to take from here on.  It was great to
hear so much enthusiasm coming from different corners, so it's obvious
that we must now make good use of such 'capital', lest we lose it to
other projects :)

- First, an update on the HG situation: Ryan Earl, from Enthought, has
kindly agreed to help up set up hg on their servers, with a likely
timeline of mid- to late-February.  In the meantime we'll continue
operating on our existing SVN, but I'm sure it will be very handy to
have hg once we start making deeper changes and keeping more parallel
lines of development running.

The plan is to initially only set up hg without any Trac coupling, so
we'll continue to use Trac for tickets but will simply point to the hg
changeset URLs instead of having the convenience of integrated
source/ticket management.    This is a small, and hopefully temporary
loss that I'm sure we can live with.  Full Trac integration remains in
the plans, but for later as time permits.  We may simply modify the
Trac config to stop showing the 'browse source' part of the tree to
avoid confusing new visitors.

I spent a few hours today familiarizing myself with hg in more depth,
and I think we can start with a work model where we serve multiple hg
repos via hgwebdir, and set the server permissions so that current
devs can commit to them.  We have a small and well behaved enough team
that I don't foresee having to implement more granular controls, since
each of us can just touch only what we're supposed to at any point in
time.  These repos will serve as the 'official' ones for the public to
download, contribute patches, etc.  Obviously with hg being fully
distributed, anyone can simply set up a publicly visible repo to offer
material for experimental development, and that type of work is
absolutely welcome.  But I think that having an official repo (or
group thereof) is a good idea so that newcomers can know what to use
from the start.

Some references on the matter for the curious:

http://www.selenic.com/mercurial/wiki/index.cgi/PublishingRepositories
http://www.selenic.com/mercurial/wiki/index.cgi/HgWebDirStepByStep

Comments on hg-based workflows are welcome.


- Next, what to do: we've seen many new ideas coming around and offers
for collaboration, so I think our key priority is to organize things a
bit for that to happen smoothly.  Up until now we've had basically two
lines of work proceeding in parallel: the trunk, dutifully maintained
by Ville and the ipython1 work, mostly done by Brian and Min.  I'm not
sure what exactly I've been doing :)

Our idea has always been that the work in ipython1 would eventually
absorb the trunk, since there's no question whatsoever about ever
dropping the command-line ipython we use today as a tool we offer.
But with finite resources, and the trunk continuing to evolve, I think
we might make more definitive progress by following a slightly
different strategy.  What I'd like to see happen, if all the devs are
interested, would be to all focus over the next few months on actually
pushing out a 'new ipython' that would be developed off the current
ipython1/ codebase, but trying to incorporate, as fast as possible,
all the existing functionality.

Specifically, this is my proposal:

1. We push out an 0.8.3 release with a fair amount of care.  We
actually go through Trac and find anything we can reasonably expect to
fix and try to close it.  This would likely be the 'end of life' of
the series.  I'm willing to participate on this work directly, since I
know the codebase pretty well, but obviously it's not a decision I'll
push without the agreement of Ville, Jorgen, Walter and the rest of
the team.

2. Once this release is out, we really pretty much stop working on the
trunk altogether.  *All of us*. We put it in pure maintenance mode,
and only fix critical bugs.  New feature requests get tickets made for
them, but their fulfillment will actually happen in the new series.

3. We all start working on getting the code in ipython1/ in a usable
form.  I expect by the time we get here, the hg repos will be up, so
we can partition the work in logical components that various teams can
work on.   I'll outline roughly what those pieces are below, but this
message is getting long already, so I won't go into too much detail
yet.  If this plan is agreeable to all, I'll probably start putting
these ideas as reST docs in the codebase itself, so we can discuss
them as design documents.

The basic system is a core capable of doing what ipython does today in
a terminal, but with a cleanly defined API for all of its public
functionality, and subcomponents well separated.  The raw draft of
this is already in place:

http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core
http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/frontend

The first order of business would be, using this, to get working:

- a terminal-based, single-process ipython, much like today.
- in-terminal GUI support like our -Xthread options of today.
- a gui embedded one, likely Laurent's WX tool.

If the above goals are met, likely it will be because the right
abstractions will have been made (things like separating prompt and
readline handling from stdin/out, etc).

At this point, we'd have specific projects that are either already
functioning but without a 'true ipython backend' or that don't exist
yet:

- All of the distributed computing facilities that Brian and I are
very interested in, and for which he's already working on here, in
addition to all the existing ipython1 machinery:
http://projects.scipy.org/ipython/ipython/browser/ipythondistarray

- A JavaScript notebook, for which we already have also prototype code:
http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/notebook

- The Qt/Cocoa frontends and other ideas that some of you have
expressed interest in.

- The Leo integration work that has been recently discussed.

- End-user interactive functionality like a better help system (Steve
Roger's project),

- Integration with Enthought's Envisage

- Etc.

In order to achieve all this, the basic guidelines we have in mind are:

- We'll try to keep compatibility with existing APIs (like ipapi)
where feasible, but there are no promises.  We'll break backwards
compatibility wherever we need to in order to do the right thing.

- No untested code.  Between nose, doctests and Twisted's trial (for
the network parts), we have more than enough for properly testing new
code.

- Good docstrings using epydoc/reST for everything.

- A reST-based set of manuals from which we can produce at any time
HTML and PDF documentation.  I'll do the work of exporting the current
lyx user manual out to reST, but that will be just a start.


I'd like to make a release 1.0 of ipython once the 'first order of
business' items above are reasonably met, even if we have a few
functionality regressions on the current terminal-based ipython.
After that, as the various components and subprojects improve, we'll
continue releasing.  This release would also include at least all of
the existing ipython1 distributed computing code, since that already
works quite well for many uses.

Do these sound like reasonable steps to all of you, especially to the
other devs?  If so, we can get going on Trac ticket triage for the
0.8.3 release as best we all can.  I should add that my
moving/changing jobs will make me more useless than usual for the next
few weeks, which is why at least I wanted to get this message out.
I'll do my best for this period, though I'm sure that won't be much.
But once I'm settled in I intend to commit more time to this effort,
since the distributed computing aspects of it will be immediately
needed in my research.

Ok, I better stop.  Thanks to anyone still reading :)  That was long...

Cheers,

f


From fperez.net at gmail.com  Wed Jan 30 03:13:56 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 30 Jan 2008 01:13:56 -0700
Subject: [IPython-dev] TaskController security
In-Reply-To: <683f83340801291416h1723fd7chbef9c51ff74d2625@mail.gmail.com>
References: <683f83340801291416h1723fd7chbef9c51ff74d2625@mail.gmail.com>
Message-ID: <db6b5ecc0801300013j24747444v640b2ea8da3660cb@mail.gmail.com>

Hi Michael,

On Jan 29, 2008 3:16 PM, Michael McLay <michael.j.mclay at gmail.com> wrote:
> Some experimentation with the TaskController in the ipython1 trunk has
> been very encouraging. It looks like a great tool for managing a queue
> of tasks that need to be executed on a cluster, or even distributed to
> a bunch of workstations in an office. TaskController seems to do a
> decent job of scheduling the execution of tasks without any tweeking
> required. Thanks for making the software available. It was very easy
> to get the software to work with our software.  Almost too easy. I am
> puzzled that there is almost nothing written on using TaskController.
> Also, the development seems to have stopped. Is development completed
> and documentation simply not necessary?

I'm glad to hear it worked for you.  I won't go into much detail now,
since I just sent that XXXL message about future development a moment
ago.  But hopefully from that message it should be clear that we DO
intend, very much, to continue developing this, and that better docs
are a key objective.  All hands welcome :)

> What are the security recommendations for using ipython1? How can an
> application protect the RemoteController and a TaskController used to
> parcel out tasks on a local network? It looks like a rogue application
> could simply connection to the RemoteController and corrupt data on
> any engine connected to the controller. The following email suggests
> work was planned to add security to ipython1:
>
>     http://article.gmane.org/gmane.comp.python.ipython.devel/1467
>
> The mailing list archive was cold on the subject after that thread and
> the word security only exists in the twisted portion of the source
> tree. What was planned and what happened to those plans?
>
> Any leads on adding security would be appreciated? Where you going to
> use Twisted's security to manage access to controllers?

Security so far simply hasn't been done.  Most of us have been using
this on trusted subnets, but it would obviously be madness to put
these engines on publicly visible ports as they are right now.  I know
next to nothing about network security, and in particular about how to
set up a system where we can balance convenience, performance (I don't
want mandatory encryption for every controller/engine communication,
which would obliterate performance) and security.  This is precisely
one area where contributions from others who know the problem domain
better would be very welcome.

The simplest thing to do would be to at least put some form of
authentication in front of the controller, so that an open controller
doesn't simply accept random client connections.  That would be
already a big help, and from there more refined schemes can be
implemented.  Since this is by definition network-based, reusing all
that Twisted has to offer here certainly seems like a reasonable
starting point, but I can't really say anything in detail, since I
just don't know.

Cheers,

f


From vivainio at gmail.com  Wed Jan 30 04:34:18 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 30 Jan 2008 11:34:18 +0200
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
Message-ID: <46cb515a0801300134h4a419b3n5f30d088560d60ee@mail.gmail.com>

On Jan 30, 2008 9:57 AM, Fernando Perez <fperez.net at gmail.com> wrote:

> 1. We push out an 0.8.3 release with a fair amount of care.  We
> actually go through Trac and find anything we can reasonably expect to
> fix and try to close it.  This would likely be the 'end of life' of
> the series.  I'm willing to participate on this work directly, since I
> know the codebase pretty well, but obviously it's not a decision I'll
> push without the agreement of Ville, Jorgen, Walter and the rest of
> the team.

I feel I have to chip in on this -

I don't see the need to declare "end of life", quite on the contrary.
We should call it "complete". New Extensions can still be created
(they are pretty independent of the core), and I don't see a problem
creating some new extension points if need be (they can be trivially
added, and can be easily ported to next-gen IPython as well).

However, I'm more than happy to reject as "WONTFIX" everything that is
disruptive to the core, or would require lots of work (bye bye,
unicode defects) :-P

Also, we should still keep accepting patches that are obviously correct.

The bottom line is, if something won't require much time, I don't see
a problem submitting it to trunk. Most importantly, as far as Leo
integration goes, I would like to do that with IPython trunk. on
IPython side, this will happen mostly in Extensions/, and i don't
foresee much (if any) changes to IPython core.

One important point regarding IPython1 is dogfooding. We need to get
it to "usable" state quickly, in order to give it some testing (I
currently depend heavily on trunk for my daily work).

Does the current ipapi work with IPython1 already?

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From ondrej at certik.cz  Wed Jan 30 04:41:11 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 30 Jan 2008 10:41:11 +0100
Subject: [IPython-dev] Development plans update
In-Reply-To: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
Message-ID: <85b5c3130801300141y73cfec6bg8c64afbadb27e30c@mail.gmail.com>

> download, contribute patches, etc.  Obviously with hg being fully
> distributed, anyone can simply set up a publicly visible repo to offer
> material for experimental development, and that type of work is
> absolutely welcome.  But I think that having an official repo (or
> group thereof) is a good idea so that newcomers can know what to use
> from the start.
>
> Some references on the matter for the curious:
>
> http://www.selenic.com/mercurial/wiki/index.cgi/PublishingRepositories
> http://www.selenic.com/mercurial/wiki/index.cgi/HgWebDirStepByStep
>
> Comments on hg-based workflows are welcome.

When we switched to hg in sympy, we also started to review every
single patch. So my workflow is:

"hg email -ao" this will send my patches into sympy-patches
mailinglist. People will review it there, point out weaknesses, etc. I
rewrite it, fix it. When it's polished, only then I'll push it in.

With svn, this would be pain, because the trunk (=our official hg
repo) evolves all the time, and so my patches will soon become old.
With hg, I simply create a new clone of the repo for every new feature
I work on (right now I have around 10 clones for example) and I leave
my patches in there until they are approved. And I merge with the
official branch (hg makes this quite easy) in the meantime.

Comments on this are welcome.

Ondrej


From walter at livinglogic.de  Wed Jan 30 09:25:19 2008
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 30 Jan 2008 15:25:19 +0100
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
Message-ID: <47A088CF.6040800@livinglogic.de>

Fernando Perez wrote:

> Hi all,
> 
> I'd like to drop a quick update on the current plans, as well as
> propose a few specific steps to take from here on.  It was great to
> hear so much enthusiasm coming from different corners, so it's obvious
> that we must now make good use of such 'capital', lest we lose it to
> other projects :)
> 
> - First, an update on the HG situation: Ryan Earl, from Enthought, has
> kindly agreed to help up set up hg on their servers, with a likely
> timeline of mid- to late-February.  In the meantime we'll continue
> operating on our existing SVN, but I'm sure it will be very handy to
> have hg once we start making deeper changes and keeping more parallel
> lines of development running.
 >
> [...]
> 
> Comments on hg-based workflows are welcome.

So will the hg repository start with a fresh history or will we do a hg 
convert?

> [...]

> [...] 
> The first order of business would be, using this, to get working:
> 
> - a terminal-based, single-process ipython, much like today.
> - in-terminal GUI support like our -Xthread options of today.
> - a gui embedded one, likely Laurent's WX tool.
> 
> If the above goals are met, likely it will be because the right
> abstractions will have been made (things like separating prompt and
> readline handling from stdin/out, etc).

Porting ipipe/ibrowse to ipython1 would be simple, once we have a 
display hook extension mechanism. Except for the hook that invokes the 
display object, there are no other dependencies.

I especially liked the generic function generics.result_display(), but 
it seems ipython1 uses a different tool for that:
http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core/display_trap.py

Is there any chance of porting generics.result_display() to ipython1?

Servus,
    Walter



From vivainio at gmail.com  Wed Jan 30 09:46:15 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Wed, 30 Jan 2008 16:46:15 +0200
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <47A088CF.6040800@livinglogic.de>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<47A088CF.6040800@livinglogic.de>
Message-ID: <46cb515a0801300646j3708b456j4b1b4be9f6ed605f@mail.gmail.com>

On Jan 30, 2008 4:25 PM, Walter D?rwald <walter at livinglogic.de> wrote:

> I especially liked the generic function generics.result_display(), but
> it seems ipython1 uses a different tool for that:
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core/display_trap.py
>
> Is there any chance of porting generics.result_display() to ipython1?

I don't see why not. We can call both, because it is a priority to
retain API compatibility with the trunk and the next-gen IPython
(generics, hooks and ipapi are all part of the API).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From fperez.net at gmail.com  Wed Jan 30 12:12:00 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 30 Jan 2008 10:12:00 -0700
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <46cb515a0801300134h4a419b3n5f30d088560d60ee@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<46cb515a0801300134h4a419b3n5f30d088560d60ee@mail.gmail.com>
Message-ID: <db6b5ecc0801300912y60637244od7faad90a9808145@mail.gmail.com>

On Jan 30, 2008 2:34 AM, Ville M. Vainio <vivainio at gmail.com> wrote:
> On Jan 30, 2008 9:57 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> > 1. We push out an 0.8.3 release with a fair amount of care.  We
> > actually go through Trac and find anything we can reasonably expect to
> > fix and try to close it.  This would likely be the 'end of life' of
> > the series.  I'm willing to participate on this work directly, since I
> > know the codebase pretty well, but obviously it's not a decision I'll
> > push without the agreement of Ville, Jorgen, Walter and the rest of
> > the team.
>
> I feel I have to chip in on this -
>
> I don't see the need to declare "end of life", quite on the contrary.
> We should call it "complete". New Extensions can still be created
> (they are pretty independent of the core), and I don't see a problem
> creating some new extension points if need be (they can be trivially
> added, and can be easily ported to next-gen IPython as well).

Well, the problem I see is twofold:

1. Finite developer resources: in building the new system so that it
is truly as functional in a terminal as today's ipython is, we'll need
your eyes, Walter's, etc.  There's just too much code in there beyond
what I originally wrote for me not to make possibly serious mistakes.
I'm not sure if you'll have the time to both do work on the trunk and
to also help us ensure that the new code really covers all the bases
of the trunk.  There will also be design decisions, API discussions,
etc, where your input would be very important, so we don't
accidentally implement something in a way that breaks parts of the
trunk you've worked on that I don't know well.  But if you think that
you can keep things moving on both fronts well, obviously I'll be the
last to complain :)

Furthermore, eventually the point is that the new code should be much
cleaner to develop for, so that nobody should need to use today's
trunk.  We want to retain 100% feature coverage, except with a cleaner
and more modular codebase that is easier to reuse in various ways.  So
I'd hope that any work you are interested in doing (be it Leo or
anything else) would also make more sense there at some point.

In my view, the sooner we reach feature parity, the better off we'll
all be, since maintaining two parallel lines of development causes
confusion, and that will increase once the two begin to seriously
overlap in functionality.

2. Name collisions: we called the new code with that silly '1' at the
end just to avoid namespace clashes in case insensitive filesystems,
where

import ipython

would clash with today's package that is called IPython.  And
eventually, we'll have a script that will be called just
'/usr/bin/ipython', that will clash with today's.  So at some point it
will become hard (without path tricks) to run both codebases at the
same time, and we'll have to seriously deprecate the existing trunk.
Think of it like major linux kernel revisions: while it's easy to run
say 2.6.21 and 2.6.22 in parallel, it's impossible to run 2.0.10 and
2.6.22 on the same system, because too many low-level structures are
incompatible.  At some point you need to make the jump and transition
over, and we'll reach that point with this code.

> However, I'm more than happy to reject as "WONTFIX" everything that is
> disruptive to the core, or would require lots of work (bye bye,
> unicode defects) :-P

Here's one for you :)

http://ipython.scipy.org/ipython/ipython/ticket/156

> Also, we should still keep accepting patches that are obviously correct.
>
> The bottom line is, if something won't require much time, I don't see
> a problem submitting it to trunk. Most importantly, as far as Leo
> integration goes, I would like to do that with IPython trunk. on
> IPython side, this will happen mostly in Extensions/, and i don't
> foresee much (if any) changes to IPython core.
>
> One important point regarding IPython1 is dogfooding. We need to get
> it to "usable" state quickly, in order to give it some testing (I
> currently depend heavily on trunk for my daily work).

Yes, fully agreed.  That's why I'd like to make the transition to that
state be as short as possible, so we begin the dogfooding cycle
ourselves.

> Does the current ipapi work with IPython1 already?

No, not yet.

Cheers,

f


From fperez.net at gmail.com  Wed Jan 30 12:16:52 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 30 Jan 2008 10:16:52 -0700
Subject: [IPython-dev] Development plans update
In-Reply-To: <85b5c3130801300141y73cfec6bg8c64afbadb27e30c@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<85b5c3130801300141y73cfec6bg8c64afbadb27e30c@mail.gmail.com>
Message-ID: <db6b5ecc0801300916u150a334eu6b00f6e6801901f2@mail.gmail.com>

On Jan 30, 2008 2:41 AM, Ondrej Certik <ondrej at certik.cz> wrote:

> > Comments on hg-based workflows are welcome.
>
> When we switched to hg in sympy, we also started to review every
> single patch. So my workflow is:
>
> "hg email -ao" this will send my patches into sympy-patches
> mailinglist. People will review it there, point out weaknesses, etc. I
> rewrite it, fix it. When it's polished, only then I'll push it in.
>
> With svn, this would be pain, because the trunk (=our official hg
> repo) evolves all the time, and so my patches will soon become old.
> With hg, I simply create a new clone of the repo for every new feature
> I work on (right now I have around 10 clones for example) and I leave
> my patches in there until they are approved. And I merge with the
> official branch (hg makes this quite easy) in the meantime.
>
> Comments on this are welcome.

Thanks for this feedback: I certainly would love to see us adopting a
workflow with regular reviews.  I'm glad to hear that you find hg
makes this natural and painless, since I think that regular patch
reviewing is something that will greatly increase the quality of the
code.

I do have some questions:

- with major restructuring and writing wholesale new code, it seems
that waiting for reviews on everything might become a burden.  Does
sympy have enough active developers that this is never an issue for
you?  Do you always find someone to review a patch quickly?

- how do you deal with running from say 10 parallel repos day to day?
Do you keep changing symlinks to point to the active one, or do you
use setuptools 'develop' magic?

Cheers,

f


From fperez.net at gmail.com  Wed Jan 30 12:18:33 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 30 Jan 2008 10:18:33 -0700
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <47A088CF.6040800@livinglogic.de>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<47A088CF.6040800@livinglogic.de>
Message-ID: <db6b5ecc0801300918u2189e2c4n5cc2e5bbb62bb6e6@mail.gmail.com>

On Jan 30, 2008 7:25 AM, Walter D?rwald <walter at livinglogic.de> wrote:
> Fernando Perez wrote:
>
> > Hi all,
> >
> > I'd like to drop a quick update on the current plans, as well as
> > propose a few specific steps to take from here on.  It was great to
> > hear so much enthusiasm coming from different corners, so it's obvious
> > that we must now make good use of such 'capital', lest we lose it to
> > other projects :)
> >
> > - First, an update on the HG situation: Ryan Earl, from Enthought, has
> > kindly agreed to help up set up hg on their servers, with a likely
> > timeline of mid- to late-February.  In the meantime we'll continue
> > operating on our existing SVN, but I'm sure it will be very handy to
> > have hg once we start making deeper changes and keeping more parallel
> > lines of development running.
>  >
> > [...]
> >
> > Comments on hg-based workflows are welcome.
>
> So will the hg repository start with a fresh history or will we do a hg
> convert?

Definitely convert.  I don't want to lose the history (we have it
since CVS from the early days, since that was imported into SVN with
full history).

> Porting ipipe/ibrowse to ipython1 would be simple, once we have a
> display hook extension mechanism. Except for the hook that invokes the
> display object, there are no other dependencies.

Good.

>
> I especially liked the generic function generics.result_display(), but
> it seems ipython1 uses a different tool for that:
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core/display_trap.py
>
> Is there any chance of porting generics.result_display() to ipython1?

Certainly.  This is precisely why I want to have everyone who knows
the trunk codebase looking over this, so we don't miss anything
important/good from the trunk in the transition.

Cheers,

f


From ondrej at certik.cz  Wed Jan 30 13:02:25 2008
From: ondrej at certik.cz (Ondrej Certik)
Date: Wed, 30 Jan 2008 19:02:25 +0100
Subject: [IPython-dev] Development plans update
In-Reply-To: <db6b5ecc0801300916u150a334eu6b00f6e6801901f2@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<85b5c3130801300141y73cfec6bg8c64afbadb27e30c@mail.gmail.com>
	<db6b5ecc0801300916u150a334eu6b00f6e6801901f2@mail.gmail.com>
Message-ID: <85b5c3130801301002y613f08c0t7d421e1b112fc919@mail.gmail.com>

On Jan 30, 2008 6:16 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Jan 30, 2008 2:41 AM, Ondrej Certik <ondrej at certik.cz> wrote:
>
> > > Comments on hg-based workflows are welcome.
> >
> > When we switched to hg in sympy, we also started to review every
> > single patch. So my workflow is:
> >
> > "hg email -ao" this will send my patches into sympy-patches
> > mailinglist. People will review it there, point out weaknesses, etc. I
> > rewrite it, fix it. When it's polished, only then I'll push it in.
> >
> > With svn, this would be pain, because the trunk (=our official hg
> > repo) evolves all the time, and so my patches will soon become old.
> > With hg, I simply create a new clone of the repo for every new feature
> > I work on (right now I have around 10 clones for example) and I leave
> > my patches in there until they are approved. And I merge with the
> > official branch (hg makes this quite easy) in the meantime.
> >
> > Comments on this are welcome.
>
> Thanks for this feedback: I certainly would love to see us adopting a
> workflow with regular reviews.  I'm glad to hear that you find hg
> makes this natural and painless, since I think that regular patch
> reviewing is something that will greatly increase the quality of the
> code.

It indeed improves the quality of the code a lot. Also I am learning
new things both being reviewed and by doing the review.
CCing Kirill, as it was him who brought reviewing patches to sympy.

>
> I do have some questions:
>
> - with major restructuring and writing wholesale new code, it seems
> that waiting for reviews on everything might become a burden.  Does

If we are restructuring or doing some other big changes, we always
review, as this is important to make things right.
But if we do some small change, like adding another name to credits,
or if we fix some really important bug, that the patch should be
in as soon as possible, I usually just post the patch for review, but
reply to it myself that this needs to be in soon and push it in.

> sympy have enough active developers that this is never an issue for
> you?  Do you always find someone to review a patch quickly?

Yes, you can see it here:

http://groups.google.com/group/sympy-patches/

At least me and Kirill always review and sometimes others too.

>
> - how do you deal with running from say 10 parallel repos day to day?
> Do you keep changing symlinks to point to the active one, or do you
> use setuptools 'develop' magic?

I try to avoid setuptools if I can. :) I just have one repository that
is equivalent to our official repo and from which I push the final
patches in.

Then I clone this repository locally (e.g. 10 times now). Then I
implement some feature, prepare a reviewed polished patch, do "hg
push"
from some of those clonned repos, this will go to my first repo and
from there I do the final "push". If I do "hg pull" from any of those
clonned repos,
if will bring in the latest updates from my first repo (which I
regularly sync with our official one).

Actually, I use versioned Mercurial Queues for development, as it
helps me keep distinct features in separate patches.

Or do you mean how do I install things? I don't install sympy when I
develop it, as we have just pure python code, so I just download the
hg (or tarball), go to the directory and "import sympy" and that's it.
We have setup.py install, as this is the standard, but I don't use it
for development.

We are still learning how to do things more efficiently. I am playing
with a review board for patches:

http://reviews.sympy.org/

So that reviewing is more fun and more people join in. Reviewing is
just talking and no coding, so I think people should like to do it -
certainly I do. :)

Ondrej


From barrywark at gmail.com  Wed Jan 30 14:46:36 2008
From: barrywark at gmail.com (Barry Wark)
Date: Wed, 30 Jan 2008 11:46:36 -0800
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
Message-ID: <cd7634ce0801301146j5d625c57mf425168f6d124807@mail.gmail.com>

I'm very interested in helping with a Cocoa frontend. It looks like
ipython/frontend/IPReadline is soon to become my friend. A quick
related question: ipython1.frontend.PendingResult looks like it could
be a twisted.internet.Deferred. Is there a reason not to use Twisted
for this? There's already Twisted integration with all the GUI
toolkits that folks have mentioned (Qt, Wx, GTK, Cocoa). Perhaps a
switch to the IPReadline constructor that sets things up to use
synchronous (for e.g. terminal use) or Twisted InterpreterResults.

barry

On Jan 29, 2008 11:57 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi all,
>
> I'd like to drop a quick update on the current plans, as well as
> propose a few specific steps to take from here on.  It was great to
> hear so much enthusiasm coming from different corners, so it's obvious
> that we must now make good use of such 'capital', lest we lose it to
> other projects :)
>
> - First, an update on the HG situation: Ryan Earl, from Enthought, has
> kindly agreed to help up set up hg on their servers, with a likely
> timeline of mid- to late-February.  In the meantime we'll continue
> operating on our existing SVN, but I'm sure it will be very handy to
> have hg once we start making deeper changes and keeping more parallel
> lines of development running.
>
> The plan is to initially only set up hg without any Trac coupling, so
> we'll continue to use Trac for tickets but will simply point to the hg
> changeset URLs instead of having the convenience of integrated
> source/ticket management.    This is a small, and hopefully temporary
> loss that I'm sure we can live with.  Full Trac integration remains in
> the plans, but for later as time permits.  We may simply modify the
> Trac config to stop showing the 'browse source' part of the tree to
> avoid confusing new visitors.
>
> I spent a few hours today familiarizing myself with hg in more depth,
> and I think we can start with a work model where we serve multiple hg
> repos via hgwebdir, and set the server permissions so that current
> devs can commit to them.  We have a small and well behaved enough team
> that I don't foresee having to implement more granular controls, since
> each of us can just touch only what we're supposed to at any point in
> time.  These repos will serve as the 'official' ones for the public to
> download, contribute patches, etc.  Obviously with hg being fully
> distributed, anyone can simply set up a publicly visible repo to offer
> material for experimental development, and that type of work is
> absolutely welcome.  But I think that having an official repo (or
> group thereof) is a good idea so that newcomers can know what to use
> from the start.
>
> Some references on the matter for the curious:
>
> http://www.selenic.com/mercurial/wiki/index.cgi/PublishingRepositories
> http://www.selenic.com/mercurial/wiki/index.cgi/HgWebDirStepByStep
>
> Comments on hg-based workflows are welcome.
>
>
> - Next, what to do: we've seen many new ideas coming around and offers
> for collaboration, so I think our key priority is to organize things a
> bit for that to happen smoothly.  Up until now we've had basically two
> lines of work proceeding in parallel: the trunk, dutifully maintained
> by Ville and the ipython1 work, mostly done by Brian and Min.  I'm not
> sure what exactly I've been doing :)
>
> Our idea has always been that the work in ipython1 would eventually
> absorb the trunk, since there's no question whatsoever about ever
> dropping the command-line ipython we use today as a tool we offer.
> But with finite resources, and the trunk continuing to evolve, I think
> we might make more definitive progress by following a slightly
> different strategy.  What I'd like to see happen, if all the devs are
> interested, would be to all focus over the next few months on actually
> pushing out a 'new ipython' that would be developed off the current
> ipython1/ codebase, but trying to incorporate, as fast as possible,
> all the existing functionality.
>
> Specifically, this is my proposal:
>
> 1. We push out an 0.8.3 release with a fair amount of care.  We
> actually go through Trac and find anything we can reasonably expect to
> fix and try to close it.  This would likely be the 'end of life' of
> the series.  I'm willing to participate on this work directly, since I
> know the codebase pretty well, but obviously it's not a decision I'll
> push without the agreement of Ville, Jorgen, Walter and the rest of
> the team.
>
> 2. Once this release is out, we really pretty much stop working on the
> trunk altogether.  *All of us*. We put it in pure maintenance mode,
> and only fix critical bugs.  New feature requests get tickets made for
> them, but their fulfillment will actually happen in the new series.
>
> 3. We all start working on getting the code in ipython1/ in a usable
> form.  I expect by the time we get here, the hg repos will be up, so
> we can partition the work in logical components that various teams can
> work on.   I'll outline roughly what those pieces are below, but this
> message is getting long already, so I won't go into too much detail
> yet.  If this plan is agreeable to all, I'll probably start putting
> these ideas as reST docs in the codebase itself, so we can discuss
> them as design documents.
>
> The basic system is a core capable of doing what ipython does today in
> a terminal, but with a cleanly defined API for all of its public
> functionality, and subcomponents well separated.  The raw draft of
> this is already in place:
>
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/frontend
>
> The first order of business would be, using this, to get working:
>
> - a terminal-based, single-process ipython, much like today.
> - in-terminal GUI support like our -Xthread options of today.
> - a gui embedded one, likely Laurent's WX tool.
>
> If the above goals are met, likely it will be because the right
> abstractions will have been made (things like separating prompt and
> readline handling from stdin/out, etc).
>
> At this point, we'd have specific projects that are either already
> functioning but without a 'true ipython backend' or that don't exist
> yet:
>
> - All of the distributed computing facilities that Brian and I are
> very interested in, and for which he's already working on here, in
> addition to all the existing ipython1 machinery:
> http://projects.scipy.org/ipython/ipython/browser/ipythondistarray
>
> - A JavaScript notebook, for which we already have also prototype code:
> http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/notebook
>
> - The Qt/Cocoa frontends and other ideas that some of you have
> expressed interest in.
>
> - The Leo integration work that has been recently discussed.
>
> - End-user interactive functionality like a better help system (Steve
> Roger's project),
>
> - Integration with Enthought's Envisage
>
> - Etc.
>
> In order to achieve all this, the basic guidelines we have in mind are:
>
> - We'll try to keep compatibility with existing APIs (like ipapi)
> where feasible, but there are no promises.  We'll break backwards
> compatibility wherever we need to in order to do the right thing.
>
> - No untested code.  Between nose, doctests and Twisted's trial (for
> the network parts), we have more than enough for properly testing new
> code.
>
> - Good docstrings using epydoc/reST for everything.
>
> - A reST-based set of manuals from which we can produce at any time
> HTML and PDF documentation.  I'll do the work of exporting the current
> lyx user manual out to reST, but that will be just a start.
>
>
> I'd like to make a release 1.0 of ipython once the 'first order of
> business' items above are reasonably met, even if we have a few
> functionality regressions on the current terminal-based ipython.
> After that, as the various components and subprojects improve, we'll
> continue releasing.  This release would also include at least all of
> the existing ipython1 distributed computing code, since that already
> works quite well for many uses.
>
> Do these sound like reasonable steps to all of you, especially to the
> other devs?  If so, we can get going on Trac ticket triage for the
> 0.8.3 release as best we all can.  I should add that my
> moving/changing jobs will make me more useless than usual for the next
> few weeks, which is why at least I wanted to get this message out.
> I'll do my best for this period, though I'm sure that won't be much.
> But once I'm settled in I intend to commit more time to this effort,
> since the distributed computing aspects of it will be immediately
> needed in my research.
>
> Ok, I better stop.  Thanks to anyone still reading :)  That was long...
>
> Cheers,
>
> f
> _______________________________________________
> IPython-user mailing list
> IPython-user at scipy.org
> http://lists.ipython.scipy.org/mailman/listinfo/ipython-user
>


From walters at verbum.org  Wed Jan 30 22:40:11 2008
From: walters at verbum.org (Colin Walters)
Date: Wed, 30 Jan 2008 22:40:11 -0500
Subject: [IPython-dev] iPipe and Hotwire Shell
In-Reply-To: <faa16b610801301927v1f969001k6b2187367b381281@mail.gmail.com>
References: <faa16b610801301927v1f969001k6b2187367b381281@mail.gmail.com>
Message-ID: <faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>

[originally sent with wrong From:]

Hi,

Recently I came across IPipe
(http://ipython.scipy.org/ipython/ipython/wiki/UsingIPipe), and
noticed a lot of similarity with the HotwirePipe language:
http://code.google.com/p/hotwire-shell/wiki/HotwireArchitecture

Looking at the ipipe.py source, we actually have a number of things we
could share.  First, for Hotwire we had to invent our own extended
types such as File
(http://hotwire-shell.googlecode.com/svn/trunk/hotwire/sysdep/fs.py),
because Python sadly doesn't include one.  For example, on Windows you
want to know whether or not the file is hidden, on modern Linux
systems you want to use GnomeVFS to get extended file metadata like
MIME type and file thumbnail, etc.  Basically we want something a lot
closer to Java's File object.  It looks like ipipe defines a lot of
its own classes like this.

Secondly, iPipe has some operators like "iwalk" which are close to
Hotwire builtins
(http://hotwire-shell.googlecode.com/svn/trunk/hotwire/builtins/walk.py).
 But, sharing these things leads to a the first big distinction
between Hotwire and IPipe:

Hotwire is closer to a "system" shell built on Python, and IPipe due
to its IPython base is more a Python toplevel where you can call out
to the system if you want.  Practically speaking, because I'm more of
an OS developer, I find myself wanting to call out to system binaries
more often than not.  I could use IPython and type "!make", "!sudo rpm
...", etc., but it's much nicer to just type "make" and "sudo rpm
...".  I do prefer Hotwire's approach of just taking over the system
namespace with improved versions, rather than having two worlds
separated by "!".

So, the main difference that is reflected here is that IPython's
operators are really *just* Python generators, while Hotwire builtins
are structured to take Unix-style options and string arguments.  But
the goal is that the Hotwire builtins are just a thin convenience
wrapper around a real API, so we could have Hotwire builtins just call
out to a shared implementation.

There's also some code in Hotwire like the SysBuiltin for executing
subprocesses that would be good to share.  I see IPipe just does
os.popen4(self.cmd).  I remember when I thought it was that
easy...before I discovered the horrors of Unix pseudoterminals just so
you can force subprocesses to be line buffered, and the pitfalls of
processing through /bin/sh.

In the longer term, I was recently thinking about trying to rebase
Hotwire around some of the IPython infrastructure; so the HotwirePipe
input language would be compiled to something like IPipe, which is
itself compiled to Python.  Then if you switch the Hotwire input
language to Python, we'd use IPython to evaluate rather than our own
custom stuff.

Your thoughts appreciated!  Personally I enjoy PyGTK ninjitsu a lot
more than I like maintaining File APIs and Python evaluation hacks, so
I'm hoping we can share some of that =)


From fperez.net at gmail.com  Thu Jan 31 02:24:46 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 31 Jan 2008 00:24:46 -0700
Subject: [IPython-dev] Development plans update
In-Reply-To: <85b5c3130801301002y613f08c0t7d421e1b112fc919@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<85b5c3130801300141y73cfec6bg8c64afbadb27e30c@mail.gmail.com>
	<db6b5ecc0801300916u150a334eu6b00f6e6801901f2@mail.gmail.com>
	<85b5c3130801301002y613f08c0t7d421e1b112fc919@mail.gmail.com>
Message-ID: <db6b5ecc0801302324g1fc5b23akdd0a016bd397427e@mail.gmail.com>

On Jan 30, 2008 11:02 AM, Ondrej Certik <ondrej at certik.cz> wrote:

[...]

> Actually, I use versioned Mercurial Queues for development, as it
> helps me keep distinct features in separate patches.

Thanks a lot for your comments on hg workflow, very useful.

> Or do you mean how do I install things? I don't install sympy when I
> develop it, as we have just pure python code, so I just download the
> hg (or tarball), go to the directory and "import sympy" and that's it.
> We have setup.py install, as this is the standard, but I don't use it
> for development.
>
> We are still learning how to do things more efficiently. I am playing
> with a review board for patches:
>
> http://reviews.sympy.org/
>
> So that reviewing is more fun and more people join in. Reviewing is
> just talking and no coding, so I think people should like to do it -
> certainly I do. :)

I try to use as few tools as possible so that the process doesn't get
in the way of doing things, but this type of thing might actually come
in handy if we develop a good patch review culture.  Thanks for the
link, I'll keep it in mind...

Cheers,

f


From fperez.net at gmail.com  Thu Jan 31 02:27:33 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 31 Jan 2008 00:27:33 -0700
Subject: [IPython-dev] [IPython-user] Development plans update
In-Reply-To: <cd7634ce0801301146j5d625c57mf425168f6d124807@mail.gmail.com>
References: <db6b5ecc0801292357j41e3ce40sd24ceb287f7a66c2@mail.gmail.com>
	<cd7634ce0801301146j5d625c57mf425168f6d124807@mail.gmail.com>
Message-ID: <db6b5ecc0801302327u70bdd8f5yac1e2c6a7bae362a@mail.gmail.com>

On Jan 30, 2008 12:46 PM, Barry Wark <barrywark at gmail.com> wrote:
> I'm very interested in helping with a Cocoa frontend. It looks like

Great!

> ipython/frontend/IPReadline is soon to become my friend. A quick
> related question: ipython1.frontend.PendingResult looks like it could
> be a twisted.internet.Deferred. Is there a reason not to use Twisted
> for this? There's already Twisted integration with all the GUI
> toolkits that folks have mentioned (Qt, Wx, GTK, Cocoa). Perhaps a
> switch to the IPReadline constructor that sets things up to use
> synchronous (for e.g. terminal use) or Twisted InterpreterResults.

This is something we'll want to think about: we certainly want to have
a terminal-only core implementation that does NOT depend on twisted at
all.  Twisted isn't small, and it has C code in it.  Just as much as
we hesitate to depend on Traits for the core, this would be an issue.
For true network functionality, obviously we use it, but I'd like to
keep a small lightweight core that can be used without twisted.  So
far we've been able to provide pretty decent GUI support without it
(via manual threading tricks), so we might be able to continue doing
so.

But certainly such implementation specifics are the things we'll have
to work on over the next few weeks.

Cheers,

f


From fperez.net at gmail.com  Thu Jan 31 02:32:17 2008
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 31 Jan 2008 00:32:17 -0700
Subject: [IPython-dev] iPipe and Hotwire Shell
In-Reply-To: <faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
References: <faa16b610801301927v1f969001k6b2187367b381281@mail.gmail.com>
	<faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
Message-ID: <db6b5ecc0801302332n4f59fba1ifcea00de220117c0@mail.gmail.com>

On Jan 30, 2008 8:40 PM, Colin Walters <walters at verbum.org> wrote:
> [originally sent with wrong From:]
>
> Hi,
>
> Recently I came across IPipe
> (http://ipython.scipy.org/ipython/ipython/wiki/UsingIPipe), and
> noticed a lot of similarity with the HotwirePipe language:
> http://code.google.com/p/hotwire-shell/wiki/HotwireArchitecture

Neat!  As you may have noticed (I'm not sure when you subscribed)
we're talking about making our codebase much more modular so that it's
easier to get reuse of the core ipython functionality in different
contexts.

I won't say much about ipipe, since Walter is our resident expert on
the matter, I just wanted to welcome you and your enthusiasm to the
list.  It would be great to see good collaborations between the two
projects.

One comment: your code is GPL while IPython is BSD, so we'd be unable
to use any of hotwire's functionality in ipython at all.  I don't know
if you'd be willing to consider relicensing code, at least the parts
meant for mutual use.

Cheers,

f


From vivainio at gmail.com  Thu Jan 31 03:15:51 2008
From: vivainio at gmail.com (Ville M. Vainio)
Date: Thu, 31 Jan 2008 10:15:51 +0200
Subject: [IPython-dev] iPipe and Hotwire Shell
In-Reply-To: <faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
References: <faa16b610801301927v1f969001k6b2187367b381281@mail.gmail.com>
	<faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
Message-ID: <46cb515a0801310015i516973fse78ee063feb0eb89@mail.gmail.com>

On Jan 31, 2008 5:40 AM, Colin Walters <walters at verbum.org> wrote:

> Hotwire is closer to a "system" shell built on Python, and IPipe due
> to its IPython base is more a Python toplevel where you can call out
> to the system if you want.  Practically speaking, because I'm more of
> an OS developer, I find myself wanting to call out to system binaries
> more often than not.  I could use IPython and type "!make", "!sudo rpm
> ...", etc., but it's much nicer to just type "make" and "sudo rpm
> ...".  I do prefer Hotwire's approach of just taking over the system
> namespace with improved versions, rather than having two worlds
> separated by "!".

Try using "ipython -p sh", you might be in for a treat. I use IPython
primarily as OS shell, and such functionality is definitely a
priority.

> There's also some code in Hotwire like the SysBuiltin for executing
> subprocesses that would be good to share.  I see IPipe just does
> os.popen4(self.cmd).  I remember when I thought it was that
> easy...before I discovered the horrors of Unix pseudoterminals just so
> you can force subprocesses to be line buffered, and the pitfalls of
> processing through /bin/sh.

Thanks for the heads up, this is something I'd like to explore (and
have pondered quite a bit in the past).

-- 
Ville M. Vainio - vivainio.googlepages.com
blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio'


From walter at livinglogic.de  Thu Jan 31 06:58:04 2008
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Thu, 31 Jan 2008 12:58:04 +0100
Subject: [IPython-dev] iPipe and Hotwire Shell
In-Reply-To: <faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
References: <faa16b610801301927v1f969001k6b2187367b381281@mail.gmail.com>
	<faa16b610801301940i6b27d81dn8970c76b2bc5b813@mail.gmail.com>
Message-ID: <47A1B7CC.2060906@livinglogic.de>

Colin Walters wrote:

> [originally sent with wrong From:]
> 
> Hi,
> 
> Recently I came across IPipe
> (http://ipython.scipy.org/ipython/ipython/wiki/UsingIPipe), and
> noticed a lot of similarity with the HotwirePipe language:
> http://code.google.com/p/hotwire-shell/wiki/HotwireArchitecture
> 
> Looking at the ipipe.py source, we actually have a number of things we
> could share.  First, for Hotwire we had to invent our own extended
> types such as File
> (http://hotwire-shell.googlecode.com/svn/trunk/hotwire/sysdep/fs.py),
> because Python sadly doesn't include one.  For example, on Windows you
> want to know whether or not the file is hidden, on modern Linux
> systems you want to use GnomeVFS to get extended file metadata like
> MIME type and file thumbnail, etc.  Basically we want something a lot
> closer to Java's File object.  It looks like ipipe defines a lot of
> its own classes like this.

Currently ipipe's ifile class is a thin wrapper around Jason Orendorff's
path class (actually it's a subclass). The only thing ifile does is add 
the properties that will we used when displaying the output and 
implementing the method __xattrs__() which tells ipipe which attributes 
to display. The basic file functionality (getting file metadata, listing 
directories etc.) is inherited directly from path.path.

> 
> Secondly, iPipe has some operators like "iwalk" which are close to
> Hotwire builtins
> (http://hotwire-shell.googlecode.com/svn/trunk/hotwire/builtins/walk.py).
>  But, sharing these things leads to a the first big distinction
> between Hotwire and IPipe:
> 
> Hotwire is closer to a "system" shell built on Python, and IPipe due
> to its IPython base is more a Python toplevel where you can call out
> to the system if you want.  Practically speaking, because I'm more of
> an OS developer, I find myself wanting to call out to system binaries
> more often than not.  I could use IPython and type "!make", "!sudo rpm
> ...", etc., but it's much nicer to just type "make" and "sudo rpm
> ...".  I do prefer Hotwire's approach of just taking over the system
> namespace with improved versions, rather than having two worlds
> separated by "!".

ipipe was implemented as pure Python expressions, because at the time I 
was more familiar with Python than with IPython's commandline 
preprocessing capabilities (and still am).

> So, the main difference that is reflected here is that IPython's
> operators are really *just* Python generators, while Hotwire builtins
> are structured to take Unix-style options and string arguments.  But
> the goal is that the Hotwire builtins are just a thin convenience
> wrapper around a real API, so we could have Hotwire builtins just call
> out to a shared implementation.

This sounds indeed interesting. I've looked around the hotwire webpages 
and a few other differences I see are the following (correct me, if I'm 
wrong):

According to 
http://code.google.com/p/hotwire-shell/wiki/HotwireArchitecture hotwire 
uses threading, ipipe uses simple iterators/generator.

hotwire propably does its own command line parsing, ipipe tables just 
implement the  __or__/__ror__ magic methods. But thats a superficial 
difference, ipipe could switch to command line parsing, or hotwire might 
add the required magic.

hotwire has one output method, the rendering infrastructure where each 
class can register a specific renderer for instances of the class. ipipe 
has three "render infrastructures": idump (basically this does just a 
print), ibrowse (curses based) and igrid (wx based), but each class can 
only specify which attributes (or methods) should be output, then for 
each attribute value (or method return value) an extended version of 
repr() (that supports colored output) kicks in.

> There's also some code in Hotwire like the SysBuiltin for executing
> subprocesses that would be good to share.  I see IPipe just does
> os.popen4(self.cmd).  I remember when I thought it was that
> easy...before I discovered the horrors of Unix pseudoterminals just so
> you can force subprocesses to be line buffered, and the pitfalls of
> processing through /bin/sh.

Actually I was trying to get an asynchronous system command that outputs 
the result as a generator yielding lines, but I could never get this to 
work.

> In the longer term, I was recently thinking about trying to rebase
> Hotwire around some of the IPython infrastructure; so the HotwirePipe
> input language would be compiled to something like IPipe, which is
> itself compiled to Python.

IPipe isn't "compiled to Python", it just *is* Python, so you don't have 
to generate an ipipe expression, you could simply instantiate the ipipe 
objects directly.

> Then if you switch the Hotwire input
> language to Python, we'd use IPython to evaluate rather than our own
> custom stuff.
> 
> Your thoughts appreciated!  Personally I enjoy PyGTK ninjitsu a lot
> more than I like maintaining File APIs and Python evaluation hacks, so
> I'm hoping we can share some of that =)

Unfortunately I'm on OS X, so I can't really test hotwire, so I'm 
limited to reading source and looking at screenshots.

As Fernando stated, we can't use any of the GPL licensed parts of 
hotwire, but if I've read COPYING/COPYING.API/COPYING.UI correctly, that 
only affects the GUI parts.

Looking forward to your comments!

Servus,
    Walter



From Nathan.Johnson at RheinLand-Versicherungen.de  Thu Jan 31 07:18:15 2008
From: Nathan.Johnson at RheinLand-Versicherungen.de (Nathan Johnson)
Date: Thu, 31 Jan 2008 13:18:15 +0100
Subject: [IPython-dev] Wiki Dead Links - 0
Message-ID: <OFC0BEA029.20695D65-ONC12573E1.003BB435-C12573E1.004396B9@RheinLand-Versicherungen.de>

http://ipython.scipy.org/moin/PyReadline/Intro
Link "
Download installer from here. 
" is broken.

Link "
Questions and bug reports can be directed to the ipython-user mail list. 
" too.
RheinLand Versicherungs AG | 41456 Neuss | Amtsgericht Neuss HRB 1477 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 914 118 5
RheinLand Lebensversicherung AG | Postfach 10 12 61, 41412 Neuss | Amtsgericht Neuss HRB 9279 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 909 084 600
.
Vorsitzender des Aufsichtsrates: Dr. Wolf-R. Bringewald | Vorstand: Christoph Buchbender, Udo Klanten, Jutta St?cker | Ust-IdNr. DE 120683573
Hausanschrift der Gesellschaften: RheinLandplatz, 41460 Neuss | Telefon (0 21 31) 2 90-0 | Telefax (0 21 31)  2 90-300 | www.rheinland-versicherungen.de
.
Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
.
This e-mail may contain confidential and/or privileged information. If you are  not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

RheinLand Versicherungs AG | 41456 Neuss | Amtsgericht Neuss HRB 1477 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 914 118 5
RheinLand Lebensversicherung AG | Postfach 10 12 61, 41412 Neuss | Amtsgericht Neuss HRB 9279 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 909 084 600
.
Vorsitzender des Aufsichtsrates: Dr. Wolf-R. Bringewald | Vorstand: Christoph Buchbender, Udo Klanten, Jutta St?cker | Ust-IdNr. DE 120683573
Hausanschrift der Gesellschaften: RheinLandplatz, 41460 Neuss | Telefon (0 21 31) 2 90-0 | Telefax (0 21 31)  2 90-300 | www.rheinland-versicherungen.de
.
Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
.
This e-mail may contain confidential and/or privileged information. If you are  not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.

RheinLand Versicherungs AG | 41456 Neuss | Amtsgericht Neuss HRB 1477 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 914 118 5
RheinLand Lebensversicherung AG | Postfach 10 12 61, 41412 Neuss | Amtsgericht Neuss HRB 9279 | Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 909 084 600
.
Vorsitzender des Aufsichtsrates: Dr. Wolf-R. Bringewald | Vorstand: Christoph Buchbender, Udo Klanten, Jutta St?cker | Ust-IdNr. DE 120683573
Hausanschrift der Gesellschaften: RheinLandplatz, 41460 Neuss | Telefon (0 21 31) 2 90-0 | Telefax (0 21 31)  2 90-300 | www.rheinland-versicherungen.de
.
Diese E-Mail enth?lt vertrauliche und/oder rechtlich gesch?tzte Informationen. Wenn Sie nicht der richtige Adressat sind oder diese E-Mail irrt?mlich erhalten haben, informieren Sie bitte sofort den Absender und vernichten Sie diese Mail. Das unerlaubte Kopieren sowie die unbefugte Weitergabe dieser E-Mail ist nicht gestattet.
.
This e-mail may contain confidential and/or privileged information. If you are  not the intended recipient (or have received this e-mail in error) please notify the sender immediately and destroy this e-mail. Any unauthorised copying, disclosure or distribution of the material in this e-mail is strictly forbidden.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20080131/43b2369c/attachment.html>

From hans_meine at gmx.net  Thu Jan 31 09:11:34 2008
From: hans_meine at gmx.net (Hans Meine)
Date: Thu, 31 Jan 2008 15:11:34 +0100
Subject: [IPython-dev] Wiki Dead Links - 0
In-Reply-To: <OFC0BEA029.20695D65-ONC12573E1.003BB435-C12573E1.004396B9@RheinLand-Versicherungen.de>
References: <OFC0BEA029.20695D65-ONC12573E1.003BB435-C12573E1.004396B9@RheinLand-Versicherungen.de>
Message-ID: <200801311511.35335.hans_meine@gmx.net>

Am Donnerstag, 31. Januar 2008 13:18:15 schrieb Nathan Johnson:
> RheinLand Versicherungs AG | 41456 Neuss | Amtsgericht Neuss HRB 1477 |
> Deutsche Bank AG, Filiale Neuss (BLZ 300 700 10) Konto 914 118 5 RheinLand
> [... snip roughly 2kb ..]
> unauthorised copying, disclosure or distribution of the material in this
> e-mail is strictly forbidden.

Argh! This reminds me of http://www.goldmark.org/jeff/stupid-disclaimers/ :-)

Ciao, /  /
     /--/
    /  / ANS

-- 
Notice: Unless you are named "Arnold P. Fasnock", you may read only the "odd 
numbered words" (every other word beginning with the first) of the message 
above. If you have violated that, then you hereby owe the sender 10 Euro for 
each even numbered word you have read.