From ellisonbg at gmail.com  Wed Sep  1 16:13:18 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Wed, 1 Sep 2010 13:13:18 -0700
Subject: [IPython-dev] [matplotlib-devel] Uniform GUI support across
	matplotlib, ets and ipython
In-Reply-To: <277011.10259.qm@web62401.mail.re1.yahoo.com>
References: <AANLkTimUOVXSzX8+MJ5EN7Y5ONg2pRxHqQ9OORpW91L2@mail.gmail.com>
	<277011.10259.qm@web62401.mail.re1.yahoo.com>
Message-ID: <AANLkTim-XO2FqH_GWwzsY7H7G64p+NXskUF8mp+D8h60@mail.gmail.com>

On Tue, Aug 31, 2010 at 7:02 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
>> 1. Our networking event loop that is based on zeromq/pyzmq
>> 2. A single GUI event loop from wx, qt4, etc.
>>
>> We do this by triggering an iteration of our networking
>> event loop on a periodic GUI timer.
>
> So right now you're in a loop in which you let qt4 (or wx) watch the file descriptors qt4 needs, then zeromq the file descriptors that zeromq needs, and so on?

ZMQ sockets are not really sockets in that they do not have a file
descriptor interface.  That is something that is being worked on in
the zeromq development, but it is not there yet.  Also, I don't think
the API will be fully compatible, so I am not sure how it will play
with what Qt is expecting.

> Just use the qt4 event loop to watch both the file descriptors zeromq wants to watch, in addition to whatever events qt4 needs. Qt4 already has the API that allows you to do this (QSocketNotifier et al.). I am not familiar with zeromq, but if there is a way to determine which file descriptors it wants to watch then you're almost done. If not, you could discuss this with the zeromq developers. Then you won't need to poll, you'll get better response times, and the code will be scalable too.
>
> Best,
> --Michiel.
>
>
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From almar.klein at gmail.com  Thu Sep  2 09:16:57 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Thu, 2 Sep 2010 15:16:57 +0200
Subject: [IPython-dev] Fwd:  Kernel-client communication
In-Reply-To: <AANLkTikwOCjjapM9-5DzpbiX7gN=e-3g_UokfbgySR+9@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikwOCjjapM9-5DzpbiX7gN=e-3g_UokfbgySR+9@mail.gmail.com>
Message-ID: <AANLkTinfi=_eRzT2b8q64bndN4CtTf-0s2HYeNYkqJzK@mail.gmail.com>

<snip>

Getting a robust and efficient message transport layer written is not
> easy work.  It takes expertise and detailed knowledge, coupled with
> extensive real-world experience, to do it right.  We simply decided to
> piggy back on some of the best that was out there, rather than trying
> to rewrite our own.  The features we gain from zmq (it's not just the
> low-level performance, it's also the simple but powerful semantics of
> their various socket types, which we've baked into the very core of
> our design) are well worth the price of a C dependency in this case.
>

I totally agree with you. And the more I read about zmq, the more I like it.
However, the C dependency is a larger hurdle for me than it is for you. You
see, IEP is an IDE that's independent of any Python installation. One can
just install IEP, and use it for any python version installed on the system.
I'd like to keep it that way, and that means I can only use pure Python my
kernel code (and the code should work on Python 2 and Python 3).



> > Further, am I right that the heartbeat is not necessary when
> communicating
> > between processes on the same box using 'localhost' (since some network
> > layers are bypassed)? That would give a short term solution for IEP.
>
> Yes, on local host you can detect the process via other mechanisms.
> The question is whether the system recovers gracefully from dropped
> messages or incomplete connections.  You do need to engineer that into
> the code itself, so that you don't lock up your client when the kernel
> becomes unresponsive, for example.
>

But is it at all possible to lose a connection when you connect two
processes using 'localhost'? Since it skips some of the lower layers of the
networking (http://docs.python.org/howto/sockets.html), I'd expect much less
can go wrong.


Concluding. I think I'll stick to my implementation for the time being. For
now, IEP will communicate 1to1 with a kernel, so I think it's pretty save as
long as long as the kernel runs on the same box and I use 'localhost'.
Later, when I implement remote processing and other more advanced stuff, I
might use zmq (and require users to install it in order to use
remote/parallel processing).

These discussion with you, and the stuff I read about zmq have got me
thinking, and I'll probably improve a few things here and there. I
definitely want to do some more testing (what happens if a connection is
lost? Maybe I can try to recover it...).


I should probably also explain that I do not use a request/reply pattern on
the socket level. I just connect the socket pair as the kernel starts up,
and then it keeps sending data in both directions. So what happens if the
kernel is busy running extension code, is that it will not be able to
socket.send() or socket.recv(). This means:

  * For the messages this should not be a problem, it will send the queued
messages as the program returns from the extension code.
  * There will be a couple of missed heartbeats though (but on the same box,
should not be a problem, right?).
  * I'm not sure what happens when the client tries to send large amounts of
data. Will network buffers overflow, or will this be correctly handled by
TCP? (I use "select" to check whether I can send/recv over the socket.)
  * If the connection is lost, I'll get a socket error. So maybe I don't
even need a heartbeat signal. However, I won't be able to distinguish a lost
connection from the process being killed.




 Thanks a lot for sharing your ideas, it's always super useful to look
> at these questions from multiple perspectives.
>

And thank you you for discussing this stuff with me. I appreciate it a lot!

  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100902/6d0c2448/attachment.html>

From satra at mit.edu  Thu Sep  2 10:09:53 2010
From: satra at mit.edu (Satrajit Ghosh)
Date: Thu, 2 Sep 2010 10:09:53 -0400
Subject: [IPython-dev] debugging python c modules
Message-ID: <AANLkTi=LDuB0jigSbKNz1V6nCwmcOET_=vVF+W+AcqGS@mail.gmail.com>

hi all,

is there a python debugger that can step me through c code if it was being
used? or do i need to launch python using xcode/gdb/msvc and put breakpoints
in the code. if the latter, can i launch ipython instead of system python?

cheers,

satra
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100902/28a84404/attachment.html>

From fperez.net at gmail.com  Thu Sep  2 14:12:01 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 11:12:01 -0700
Subject: [IPython-dev] !commands in the zmq model
In-Reply-To: <i5c8ud$3kt$1@dough.gmane.org>
References: <AANLkTi=3Agm4T2c7kcc9XhfkUbs2g2k7zneVoywWQKZx@mail.gmail.com>
	<i5c8ud$3kt$1@dough.gmane.org>
Message-ID: <AANLkTi=M8W3SQdiD6FnVJecwHBWZtG6D_3BfW4Gy_=KO@mail.gmail.com>

On Sat, Aug 28, 2010 at 5:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
>
> I believe the current development is here:
>
> ? http://sage.math.washington.edu/home/goreckc/sage/wexpect/
>
> You may want to check with the author, Chris K. Gorecki
> <firstname.middleinitial.lastname at gmail (fill in the blanks)> for information.

Robert, we owe you one more :)  I just got a full confirmation from
Chris Gorecki that his version remains MIT licensed, and he says that
as far as he knows, it's working and fully functional.  So at least we
have a good starting point.  He also was interested in any
fixes/improvements we might make, so the code isn't really dead per
se.

If it works well for us, later it might be worth cleaning it up to
have a common base plus the windows/posix layers in separate files,
rather than as an inline 'if platform' set of statements like it is
now.

Thanks again for pointing that out to me!  We'll report on how it goes
once we can do some windows testing, it would be fabulous to have on
win32 the kind of very robust subprocess support we have right now on
posix.

Regards,

f


From gael.varoquaux at normalesup.org  Thu Sep  2 16:43:32 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 2 Sep 2010 22:43:32 +0200
Subject: [IPython-dev] !commands in the zmq model
In-Reply-To: <AANLkTikV5qTtm7bTz_+gBRiea=Ov3WV=HCXBnii+=f_C@mail.gmail.com>
References: <AANLkTi=3Agm4T2c7kcc9XhfkUbs2g2k7zneVoywWQKZx@mail.gmail.com>
	<20100828125156.GA582@phare.normalesup.org>
	<AANLkTikV5qTtm7bTz_+gBRiea=Ov3WV=HCXBnii+=f_C@mail.gmail.com>
Message-ID: <20100902204332.GB2472@phare.normalesup.org>

On Sun, Aug 29, 2010 at 10:42:54PM -0700, Fernando Perez wrote:
> MMh, I've been testing your ipythonx implementation, and it definitely
> does *not* show subprocess output asynchronously as it happens.  At
> least not with the example above run in a subprocess, and it won't let
> me interrupt the subprocess either (I see the KeyboardInterrupt happen
> only at the end, and coming from the ipythonx code:

Puzzling! I tried that old piece of crappy code that I wrote, and was
very surprised to find that it still runs. I can run '!pdflatex' and get
immediate interaction with the spawned program (I remember that achieving
this was hard), however I can also confirm that when I spawn a Python
process, things don't go well (try simply '!python'): synchronous display
doesn't work.

I have no clue why this is the case, and it's certainly not worth wasting
time investigating this if you have a better option (and it looks like
you do).

Thanks for your (collective 'your') efforts on IPython. It is really
starting to look good in terms of providing the basis for a scientific
IDE.

Ga?l


From fperez.net at gmail.com  Thu Sep  2 16:58:07 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 13:58:07 -0700
Subject: [IPython-dev] !commands in the zmq model
In-Reply-To: <20100902204332.GB2472@phare.normalesup.org>
References: <AANLkTi=3Agm4T2c7kcc9XhfkUbs2g2k7zneVoywWQKZx@mail.gmail.com>
	<20100828125156.GA582@phare.normalesup.org>
	<AANLkTikV5qTtm7bTz_+gBRiea=Ov3WV=HCXBnii+=f_C@mail.gmail.com>
	<20100902204332.GB2472@phare.normalesup.org>
Message-ID: <AANLkTikRbe+wE36a+=QFxZDwOfy3zyedW01Fnjf-UEF3@mail.gmail.com>

On Thu, Sep 2, 2010 at 1:43 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> Puzzling! I tried that old piece of crappy code that I wrote, and was
> very surprised to find that it still runs. I can run '!pdflatex' and get
> immediate interaction with the spawned program (I remember that achieving
> this was hard), however I can also confirm that when I spawn a Python
> process, things don't go well (try simply '!python'): synchronous display
> doesn't work.
>
> I have no clue why this is the case, and it's certainly not worth wasting
> time investigating this if you have a better option (and it looks like
> you do).

It's worth pointing out that we will lose one thing from a
terminal-based setup: the ability to *interact* with subprocesses.
The moment you get the network involved in all of this, transporting
reliably both the stdin of the kernel *and* of subprocesses with
different file descriptors is more or less impossible.

But that's OK: I'd rather have reliable async communication and subp
killing than interaction; after all, we're not making a raw shell.
People can always open a separate terminal for 100% pure shell work.

And if later, someone comes along and improves things to also get
stdin to work, that would be fabulous, but I'm not holding my breath
given what I know about how pexpect works.  Basically pexpect now
holds control of the subprocess, and we can send whatever we want, but
I think only in response to text patterns or a timeout event.  But I
don't know how to make pexpect fire a callback if read() is called on
the stdin file descriptor of the subprocess, which is what we'd need.

> Thanks for your (collective 'your') efforts on IPython. It is really
> starting to look good in terms of providing the basis for a scientific
> IDE.

It's been a lot of fun, and I think we're going to have something very
cool, and for potential for pretty amazing ideas to be implemented,
quite soon.

Regards,

f


From benjaminrk at gmail.com  Thu Sep  2 18:49:57 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 2 Sep 2010 15:49:57 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
Message-ID: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>

I can delete the In [n]: prompts in the ipythonqt window in either of the
following two ways:
1) cursor at empty line before In... and forward-delete (ctrl-D)
2) cursor at opening bracket, as in: In [|1]:..., and backspace

These delete content, and break the input for future commands, due to
raising 'QTextCursor::setPosition: Position 'X' out of range' errors.

Works on both Ubuntu and OSX.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100902/5122762d/attachment.html>

From fperez.net at gmail.com  Thu Sep  2 18:54:34 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 15:54:34 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
Message-ID: <AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>

On Thu, Sep 2, 2010 at 3:49 PM, MinRK <benjaminrk at gmail.com> wrote:
> I can delete the In [n]: prompts in the ipythonqt window in either of the
> following two ways:
> 1) cursor at empty line before In... and forward-delete (ctrl-D)
> 2) cursor at opening bracket, as in: In [|1]:..., and backspace
> These delete content, and break the input for future commands, due to
> raising 'QTextCursor::setPosition: Position 'X' out of range' errors.
> Works on both Ubuntu and OSX.

Evan, if you scroll up on your IRC log you can find the conversation
where Min showed us this problem.  It's pretty serious because it
completely wedges the console, you have to kill it and restart it.  So
a user's inadvertent click in the wrong place could easily cost them
the whole session.

I don't know how easy it would be to forbid the cursor from being
placed anywhere other than past the last prompt, while retaining the
ability to highlight prior text for copy/paste purposes.

Preventing that would be great in that it would block a class of
problems I've had occur quite often: pasting in that window thining it
behaves like a terminal, via middle-click, but finding that the pasted
code goes 'inside' the old text, and makes  a giant mess.

More for your usability plate :)

Cheers,

f


From ellisonbg at gmail.com  Thu Sep  2 19:50:09 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 2 Sep 2010 16:50:09 -0700
Subject: [IPython-dev] debugging python c modules
In-Reply-To: <AANLkTi=LDuB0jigSbKNz1V6nCwmcOET_=vVF+W+AcqGS@mail.gmail.com>
References: <AANLkTi=LDuB0jigSbKNz1V6nCwmcOET_=vVF+W+AcqGS@mail.gmail.com>
Message-ID: <AANLkTik_0MH8nGXawMTKKG+YsmY2MTRx_pNh1YZyuunF@mail.gmail.com>

On Thu, Sep 2, 2010 at 7:09 AM, Satrajit Ghosh <satra at mit.edu> wrote:
> hi all,
>
> is there a python debugger that can step me through c code if it was being
> used? or do i need to launch python using xcode/gdb/msvc and put breakpoints
> in the code. if the latter, can i launch ipython instead of system python?

No, the python debugger knows nothing about C.  You will need to use
traditional C debugging tools for that part of your code.

Brian

> cheers,
>
> satra
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From epatters at enthought.com  Thu Sep  2 20:02:15 2010
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 2 Sep 2010 19:02:15 -0500
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
Message-ID: <AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>

Min: I will fix these issues tomorrow. At least one of these is a
recent regression.

Fernando: I thought I had hijacked all the possible ways to paste
code, but I forgot about middle click (I never use that one). I will
also fix this tomorrow.

In general, there is a large amount of code in place to ensure that
the user does not delete or modify text that should be static, but I'm
not surprised that there are still ways for the determined user to
break things. Eventually, all the holes will be plugged, but this is
the price you pay for using a generic text widget to implement a
terminal interface. (Of course, there are also many advantages to not
using a true terminal widget.)

Evan

On Thu, Sep 2, 2010 at 5:54 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 3:49 PM, MinRK <benjaminrk at gmail.com> wrote:
>> I can delete the In [n]: prompts in the ipythonqt window in either of the
>> following two ways:
>> 1) cursor at empty line before In... and forward-delete (ctrl-D)
>> 2) cursor at opening bracket, as in: In [|1]:..., and backspace
>> These delete content, and break the input for future commands, due to
>> raising 'QTextCursor::setPosition: Position 'X' out of range' errors.
>> Works on both Ubuntu and OSX.
>
> Evan, if you scroll up on your IRC log you can find the conversation
> where Min showed us this problem. ?It's pretty serious because it
> completely wedges the console, you have to kill it and restart it. ?So
> a user's inadvertent click in the wrong place could easily cost them
> the whole session.
>
> I don't know how easy it would be to forbid the cursor from being
> placed anywhere other than past the last prompt, while retaining the
> ability to highlight prior text for copy/paste purposes.
>
> Preventing that would be great in that it would block a class of
> problems I've had occur quite often: pasting in that window thining it
> behaves like a terminal, via middle-click, but finding that the pasted
> code goes 'inside' the old text, and makes ?a giant mess.
>
> More for your usability plate :)
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From fperez.net at gmail.com  Thu Sep  2 20:05:46 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 17:05:46 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
Message-ID: <AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>

On Thu, Sep 2, 2010 at 5:02 PM, Evan Patterson <epatters at enthought.com> wrote:
>
> Fernando: I thought I had hijacked all the possible ways to paste
> code, but I forgot about middle click (I never use that one). I will
> also fix this tomorrow.
>

Thanks, Evan.

BTW, I'm trying to implement all the messaging spec, but it's really
tricky to trace in the Qt code if something isn't quite right, because
it just sits silent and there's no output anywhere.  Any hints on how
to get it to be more verbose about what it's doing?

Cheers,

f


From ellisonbg at gmail.com  Thu Sep  2 20:10:24 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 2 Sep 2010 17:10:24 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
Message-ID: <AANLkTi=VpBTA+-nyAy8MjtZ-rqiKvXwQ-Q=O9fixNAOe@mail.gmail.com>

Fernando, for printing debug messages I use:

self._append_plain_text('Hi there')

That way you can at least print debug messages in a nice manner.

Brian

On Thu, Sep 2, 2010 at 5:05 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 5:02 PM, Evan Patterson <epatters at enthought.com> wrote:
>>
>> Fernando: I thought I had hijacked all the possible ways to paste
>> code, but I forgot about middle click (I never use that one). I will
>> also fix this tomorrow.
>>
>
> Thanks, Evan.
>
> BTW, I'm trying to implement all the messaging spec, but it's really
> tricky to trace in the Qt code if something isn't quite right, because
> it just sits silent and there's no output anywhere. ?Any hints on how
> to get it to be more verbose about what it's doing?
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From epatters at enthought.com  Thu Sep  2 20:10:49 2010
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 2 Sep 2010 19:10:49 -0500
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
Message-ID: <AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>

On Thu, Sep 2, 2010 at 7:05 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 5:02 PM, Evan Patterson <epatters at enthought.com> wrote:
>>
>> Fernando: I thought I had hijacked all the possible ways to paste
>> code, but I forgot about middle click (I never use that one). I will
>> also fix this tomorrow.
>>
>
> Thanks, Evan.
>
> BTW, I'm trying to implement all the messaging spec, but it's really
> tricky to trace in the Qt code if something isn't quite right, because
> it just sits silent and there's no output anywhere. ?Any hints on how
> to get it to be more verbose about what it's doing?
>

The fundamental issue is that the frontend has no way to tell whether
a reply that has not come back from the kernel is a result of the
kernel being broken or merely busy--the frontend has no choice but to
be silent. Although I've never seen the frontend outright freeze, you
can verify that this has not happened by pressing Ctrl-Period.

The only way to verify that the kernel is sending the messages is to
look at the debug output.

Evan


From fperez.net at gmail.com  Thu Sep  2 20:12:03 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 17:12:03 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTi=VpBTA+-nyAy8MjtZ-rqiKvXwQ-Q=O9fixNAOe@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
	<AANLkTi=VpBTA+-nyAy8MjtZ-rqiKvXwQ-Q=O9fixNAOe@mail.gmail.com>
Message-ID: <AANLkTi=qBAztb-odYyEUFLznn=_W97Ft5aYRTwJ3KvUR@mail.gmail.com>

On Thu, Sep 2, 2010 at 5:10 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> self._append_plain_text('Hi there')
>
> That way you can at least print debug messages in a nice manner.
>

OK, thanks.


From fperez.net at gmail.com  Thu Sep  2 20:13:31 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 17:13:31 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
	<AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>
Message-ID: <AANLkTinhpjmP7EXxNBcw845nnD7Bubo7XPfF3p6Dq2n3@mail.gmail.com>

On Thu, Sep 2, 2010 at 5:10 PM, Evan Patterson <epatters at enthought.com> wrote:
> The fundamental issue is that the frontend has no way to tell whether
> a reply that has not come back from the kernel is a result of the
> kernel being broken or merely busy--the frontend has no choice but to
> be silent. Although I've never seen the frontend outright freeze, you
> can verify that this has not happened by pressing Ctrl-Period.
>
> The only way to verify that the kernel is sending the messages is to
> look at the debug output.

In this case I had an error at the very beginning, and the frontend
was just frozen, without any messages having been emmitted by the
kernel yet.  Kind of staring at a blank wall...

But I figured it out by the sheer power of my raw intellect (aka,  I
stared at the same line of code for 5 minutes until I realized how
dumb I was).  All good now :)

Cheers,

f


From ellisonbg at gmail.com  Thu Sep  2 20:17:43 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 2 Sep 2010 17:17:43 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTinhpjmP7EXxNBcw845nnD7Bubo7XPfF3p6Dq2n3@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
	<AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>
	<AANLkTinhpjmP7EXxNBcw845nnD7Bubo7XPfF3p6Dq2n3@mail.gmail.com>
Message-ID: <AANLkTi=OTTp99e6Dghrr0JbT_vz+vTG7cn+2n0fQKqvv@mail.gmail.com>

On Thu, Sep 2, 2010 at 5:13 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 5:10 PM, Evan Patterson <epatters at enthought.com> wrote:
>> The fundamental issue is that the frontend has no way to tell whether
>> a reply that has not come back from the kernel is a result of the
>> kernel being broken or merely busy--the frontend has no choice but to
>> be silent. Although I've never seen the frontend outright freeze, you
>> can verify that this has not happened by pressing Ctrl-Period.
>>
>> The only way to verify that the kernel is sending the messages is to
>> look at the debug output.
>
> In this case I had an error at the very beginning, and the frontend
> was just frozen, without any messages having been emmitted by the
> kernel yet. ?Kind of staring at a blank wall...
>
> But I figured it out by the sheer power of my raw intellect (aka, ?I
> stared at the same line of code for 5 minutes until I realized how
> dumb I was). ?All good now :)

Debugging like that is a super nice skill to have - not easy though...

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Thu Sep  2 23:21:44 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 20:21:44 -0700
Subject: [IPython-dev] user_variables/expressions ready in kernel
Message-ID: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>

Evan:

I've added user_variables/expressions to _execute() in ipython_widget,
but the caller in parent_class.execute() doesn't have these
parameters, so for now they are dead.  I wasn't quite sure given the
ConsoleWidget/IPythonWidget hierarchy how you wanted to handle this
part of the API, which is somewhat ipython-specific.  I'd suggest we
promote this idea even to the normal non-iptyhon widget, since the
idea of fetching names/expressions from a running interpreter is not
ipython-specific.  But I wasn't sure in how many places through the Qt
code I'd have to chage things, so I didn't touch it.  But the kernel
is now correctly processing and returning values for these fields.

This should be pushed shortly (I'm still cleaning up some), but I
wanted to record this for you while it's fresh in my head.

We can go over the details tomorrow, but I did test it manually and
now those fields behave as per the messaging spec.

I'll now implement the object_info details, and the get/setattr calls.

Cheers,

f


From fperez.net at gmail.com  Thu Sep  2 23:24:38 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 20:24:38 -0700
Subject: [IPython-dev] Broken ~/<tab> completion in newkernel
Message-ID: <AANLkTinDubysFKZ5V6-opStraZ7Q2WJFvkWH8jmwF+79@mail.gmail.com>

Hey Evan,

I think this is a recent regression:

In [3]: run ~/tes<TAB HERE>

becomes

In [3]: run ~/ome/fperez/test/


The completion message is correctly giving the info, so something is
mis-computing how to rewrite the input line:

{'content': {'matches': [u'/home/fperez/test/'], 'status': 'ok',
'matched_text': u'~/tes'},
 'header': {'username': u'kernel', 'msg_id': 16, 'session':
'ffeb049d-575a-4f50-8933-032977908e7f'},
 'msg_type': 'complete_reply',
 'parent_header': {u'username': u'fperez', u'msg_id': 11, u'session':
u'b45c7701-0c0b-43da-b605-cfd2d4781b5c'}}

Since I know you just did some work on the completion code, you may
have inadvertedly offset a cursor position somewhere.

Cheers,

f


From ellisonbg at gmail.com  Thu Sep  2 23:35:31 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 2 Sep 2010 20:35:31 -0700
Subject: [IPython-dev] user_variables/expressions ready in kernel
In-Reply-To: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
References: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
Message-ID: <AANLkTiksCaFnDp01yX2J88=tY0B3-i7g7YPc2ZN2p_Xk@mail.gmail.com>

Fernando,

This is great, we can do a review tomorrow.  I just pushed some fixes
to the heartbeat polling.  I had a few bugs, and I think there is a
minor bug in zeromq with this.  But i added logic to deal with it, so
it should behave well now.

Cheers,

Brian

On Thu, Sep 2, 2010 at 8:21 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Evan:
>
> I've added user_variables/expressions to _execute() in ipython_widget,
> but the caller in parent_class.execute() doesn't have these
> parameters, so for now they are dead. ?I wasn't quite sure given the
> ConsoleWidget/IPythonWidget hierarchy how you wanted to handle this
> part of the API, which is somewhat ipython-specific. ?I'd suggest we
> promote this idea even to the normal non-iptyhon widget, since the
> idea of fetching names/expressions from a running interpreter is not
> ipython-specific. ?But I wasn't sure in how many places through the Qt
> code I'd have to chage things, so I didn't touch it. ?But the kernel
> is now correctly processing and returning values for these fields.
>
> This should be pushed shortly (I'm still cleaning up some), but I
> wanted to record this for you while it's fresh in my head.
>
> We can go over the details tomorrow, but I did test it manually and
> now those fields behave as per the messaging spec.
>
> I'll now implement the object_info details, and the get/setattr calls.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Thu Sep  2 23:42:28 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 20:42:28 -0700
Subject: [IPython-dev] user_variables/expressions ready in kernel
In-Reply-To: <AANLkTiksCaFnDp01yX2J88=tY0B3-i7g7YPc2ZN2p_Xk@mail.gmail.com>
References: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
	<AANLkTiksCaFnDp01yX2J88=tY0B3-i7g7YPc2ZN2p_Xk@mail.gmail.com>
Message-ID: <AANLkTi=LyU943o707=Qs+6kXq8iYnPtw=YB_J26B7TFy@mail.gmail.com>

On Thu, Sep 2, 2010 at 8:35 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> This is great, we can do a review tomorrow. ?I just pushed some fixes
> to the heartbeat polling. ?I had a few bugs, and I think there is a
> minor bug in zeromq with this. ?But i added logic to deal with it, so
> it should behave well now.

Thanks, I'll ping as soon as I push so you guys can have a look in
advance, because it will *definitely* need a good look by many eyes.
I've had to go through many critical paths in the code, so we really
want a careful look by others at this.

Cheers,

f


From ellisonbg at gmail.com  Thu Sep  2 23:45:48 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Thu, 2 Sep 2010 20:45:48 -0700
Subject: [IPython-dev] user_variables/expressions ready in kernel
In-Reply-To: <AANLkTi=LyU943o707=Qs+6kXq8iYnPtw=YB_J26B7TFy@mail.gmail.com>
References: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
	<AANLkTiksCaFnDp01yX2J88=tY0B3-i7g7YPc2ZN2p_Xk@mail.gmail.com>
	<AANLkTi=LyU943o707=Qs+6kXq8iYnPtw=YB_J26B7TFy@mail.gmail.com>
Message-ID: <AANLkTinSp_MfSXKKRVH_8eu+7wDazz1+nfcj7kHU0MV6@mail.gmail.com>

Sounds good.

On Thu, Sep 2, 2010 at 8:42 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 8:35 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> This is great, we can do a review tomorrow. ?I just pushed some fixes
>> to the heartbeat polling. ?I had a few bugs, and I think there is a
>> minor bug in zeromq with this. ?But i added logic to deal with it, so
>> it should behave well now.
>
> Thanks, I'll ping as soon as I push so you guys can have a look in
> advance, because it will *definitely* need a good look by many eyes.
> I've had to go through many critical paths in the code, so we really
> want a careful look by others at this.
>
> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Thu Sep  2 23:48:02 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 20:48:02 -0700
Subject: [IPython-dev] user_variables/expressions ready in kernel
In-Reply-To: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
References: <AANLkTikFotWxGHp_+6xO+XuBkprh7kttAPTMqy4zfBB0@mail.gmail.com>
Message-ID: <AANLkTinG-icoJv_0-St2S_MLcJESk5gcXsR61D6K3b0k@mail.gmail.com>

On Thu, Sep 2, 2010 at 8:21 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Evan:
>
> I've added user_variables/expressions to _execute() in ipython_widget,
>

Sorry, in frontend_widget, L134, not ipython_widget.

Cheers,

f


From fperez.net at gmail.com  Fri Sep  3 02:37:17 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 2 Sep 2010 23:37:17 -0700
Subject: [IPython-dev] completed messaging spec update for exec requests
Message-ID: <AANLkTi=__7TW9jV-J3Z1YdnV=ukEUvoGR9Yj+yRisP+u@mail.gmail.com>

Hey Evan,

the exec requests now have full silent and extra field support, and
all prompt request traces are gone from the kernel.  I may do the
object_info before crashing, but I'm a little tired.  If not, at least
this should get us going with a cleaner spec.

Regards,

f


From hans_meine at gmx.net  Fri Sep  3 04:08:32 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 3 Sep 2010 10:08:32 +0200
Subject: [IPython-dev] debugging python c modules
In-Reply-To: <AANLkTik_0MH8nGXawMTKKG+YsmY2MTRx_pNh1YZyuunF@mail.gmail.com>
References: <AANLkTi=LDuB0jigSbKNz1V6nCwmcOET_=vVF+W+AcqGS@mail.gmail.com>
	<AANLkTik_0MH8nGXawMTKKG+YsmY2MTRx_pNh1YZyuunF@mail.gmail.com>
Message-ID: <201009031008.33095.hans_meine@gmx.net>

Am Freitag 03 September 2010, 01:50:09 schrieb Brian Granger:
> On Thu, Sep 2, 2010 at 7:09 AM, Satrajit Ghosh <satra at mit.edu> wrote:
> > hi all,
> > 
> > is there a python debugger that can step me through c code if it was
> > being used? or do i need to launch python using xcode/gdb/msvc and put
> > breakpoints in the code. if the latter, can i launch ipython instead of
> > system python?
> 
> No, the python debugger knows nothing about C.  You will need to use
> traditional C debugging tools for that part of your code.

Interesting idea.  Wouldn't it be possible to support attaching gdb to the 
process from python?  Then, it could even be recognized when the call stack 
goes back into python, in order to issue 'continue' and return control to the 
PDB.

Wouldn't that be terrific?  The thought deserves some really good reasons to 
be abandoned at least.. ;-)

Have a nice day,
  Hans


From almar.klein at gmail.com  Fri Sep  3 04:30:29 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Fri, 3 Sep 2010 10:30:29 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
Message-ID: <AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>

Hi,

I've been thinking about your protocol last night. (arg! This whole thing is
getting to my head too much!)

I see some advantages of loose messages such as your using in favor of my
channels approach. Specifically, the order of stdout and stderr messages is
not preserved, as they are send over a different channel (same socket
though). On a 1to1 system, this is not much of a problem, but for multiple
users it definitely will be, specifically if you also want to show the other
users input (pyin).

Anyway, I took another good look at your approach and I have a few
remarks/ideas. Might be useful, might be total BS.

* You say the prompt is a thing the client decides what it looks like. I
disagree. The kernel gives a prompt to indicate that the user can give
input, and maybe also what kind of input. A debugger might produce a
different prompt to indicate that the user is in debug mode. See also the
doc on sys.ps1 and sys.ps2 (http://docs.python.org/library/sys.html): the
user (or interpreter) can put an object on it that evaluates to something
nice when str()'nged.

* You plan on keeping history in the kernel. In this case I think this is
the task of the client. Otherwise you'd get your own history mixed with that
of someone else using that kernel? History is, I think, a feature of the
client to help the programmer. I see no use for storing it at the kernel.

* I really think you can do with less sockets. I believe that the (black)
req/rep pair is not needed. You only seem to use it for when raw_input is
used. But why? When raw_input is used, you can just block and wait for some
stdin (I think that'll be the execute_request message). This should not be
too hard by replacing sys.stdin with an object that has a readline method
that does this. If two users are present, and one calls raw_input, they can
both provide input (whoever's first). To indicate this to the *other* user,
however, his prompt should be replaced with an empty string, so his cursor
is positioned right after the <text> in raw_input('<text>').

* I think you can do with even less sockets :)  But this is more of a wild
idea. Say that John set up an experiment at work and wants to check the
results in the bar on his Android (sorry I stole your example here,
Fernando). Now his experiment crashed, producing a traceback in the client
at his work PC. But now he cannot see the traceback as he just logged in!
-----  So what about storing all stdout, stderr and pyin (basically all
"terminal-output") at the kernel? And rather than pub/sub, use the existing
req/rep to obtain this stuff. Maybe you can even pass new terminal-output
along with other replies. The client should indicate in the request a
sequence number to indicate to the kernel what messages were already
received. This does mean, however, that the client would have to
periodically query the kernel. But maybe this can also be done automatically
by writing a thin layer on top of the zmq interface. Oh, and you'd need to
encapsulate multiple terminal-messages in a single reply.

Just my two cents,
  Almar


On 31 August 2010 07:28, Fernando Perez <fperez.net at gmail.com> wrote:

> On Mon, Aug 30, 2010 at 1:51 AM, Almar Klein <almar.klein at gmail.com>
> wrote:
> > Ah right. Although I'm not sure how often one would use such this in
> > practice, it's certainly a nice feature, and seems to open op a range of
> > possibilities. I can imagine this requirement makes things considerably
> > harder to implement, but since you're designing a whole new protocol from
> > scratch, it's probably a good choice to include it now.
>
> And the whole thing fits naturally in our design for tools that enable
> both interactive/collaborative computing and distributed/parallel work
> within one single framework.  After all, it's just manipulating
> namespaces :)
>
> >> In our case obviously the kernel itself remains unresponsive, but the
> >> important part is that the networking doesn't suffer.  So we have
> >> enough information to take action even in the face of an unresponsive
> >> kernel.
> >
> > I'm quite a new to networking, so sorry for if this sounds stupid: Other
> > than the heartbeat stuff not working, would it also have other effects? I
> > mean, data can not be send or received, so would maybe network buffers
> > overflow or anything?
>
> Depending on how you implemented your networking layer, you're likely
> to lose data.  And you'll need to ensure that your api recovers
> gracefully from half-sent messages, unreplied messages, etc.
>
> Getting a robust and efficient message transport layer written is not
> easy work.  It takes expertise and detailed knowledge, coupled with
> extensive real-world experience, to do it right.  We simply decided to
> piggy back on some of the best that was out there, rather than trying
> to rewrite our own.  The features we gain from zmq (it's not just the
> low-level performance, it's also the simple but powerful semantics of
> their various socket types, which we've baked into the very core of
> our design) are well worth the price of a C dependency in this case.
>
> > Further, am I right that the heartbeat is not necessary when
> communicating
> > between processes on the same box using 'localhost' (since some network
> > layers are bypassed)? That would give a short term solution for IEP.
>
> Yes, on local host you can detect the process via other mechanisms.
> The question is whether the system recovers gracefully from dropped
> messages or incomplete connections.  You do need to engineer that into
> the code itself, so that you don't lock up your client when the kernel
> becomes unresponsive, for example.
>
> I'm sure we still have corner cases in our code where we can lock up,
> it's not easy to prevent all such occurrences.
>
> > No, that's the great thing! All channels are multiplexed over the same
> > socket pair. When writing a message to a channel, it is put in a queue,
> > adding a small header to indicate the channel id. There is a single
> thread
> > that sends and receives messages over the socket. It just pops the
> messages
> > from the queue and sends them to the other side. At the receiver side,
> the
> > messages are distributed to the queue corresponding to the right channel.
> So
> > there's one 'global' queue on the sending side and one queue per channel
> on
> > the receiver side.
>
> Ah, excellent!  It seems your channels are similar to our message
> types, we simply dispatch on the message type (a string) with the
> appropriate handler.  The twist in ipython is that we have used as an
> integral part of the design the various types of zmq sockets: req/rep
> for stdin control, xrep/xreq for execution requests multiplexed across
> clients, and pub/sub for side effects (things that don't fit in a
> functional paradigm).
>
> We thus have a very strong marriage between the abstractions that zmq
> exposes and our design.  Honestly, I sometimes feel as if zmq had been
> designed for us, because it makes certain things we'd wanted for a
> very long time almost embarrassingly easy.
>
> Thanks a lot for sharing your ideas, it's always super useful to look
> at these questions from multiple perspectives.
>
> Regards,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100903/134987f9/attachment.html>

From hans_meine at gmx.net  Fri Sep  3 05:18:38 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 3 Sep 2010 11:18:38 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
Message-ID: <201009031118.38731.hans_meine@gmx.net>

Am Freitag 03 September 2010, 10:30:29 schrieb Almar Klein:
> * You plan on keeping history in the kernel. In this case I think this is
> the task of the client. Otherwise you'd get your own history mixed with
> that of someone else using that kernel? History is, I think, a feature of
> the client to help the programmer. I see no use for storing it at the
> kernel.

That's an interesting issue.  Some more random thoughts:

- If you decide to connect to an existing session (from home or the bar via 
your smartphone) to check something and/or issue a command, it is very useful 
to reuse your existing history -> should be in the kernel

- If you are using only one frontend/client, it does not matter anyhow, only..

- If the kernel is busy, it could still be useful to browse through the 
history to compose the next command(s) -> should be in the client?

- In the multiple GUI/observer scenario, I think it would not hurt to share 
the history with the observer either -> could be in the kernel

Is there any other reason to have the history in the client, Almar?

Otherwise, I's say of course it's a feature of the client, but obviously there 
are good reasons (in particular when detaching/attaching) to have the history 
be stored in the kernel.  Maybe it needs to be duplicated/synchronized to 
support situations where the kernel cannot respond, though.

HTH,
  Hans


From almar.klein at gmail.com  Fri Sep  3 06:12:51 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Fri, 3 Sep 2010 12:12:51 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <201009031118.38731.hans_meine@gmx.net>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<201009031118.38731.hans_meine@gmx.net>
Message-ID: <AANLkTinvK3Tpw6DDF8H1M28+MBF4dhmNNSnEdyTy=UDc@mail.gmail.com>

Hi Hans,

On 3 September 2010 11:18, Hans Meine <hans_meine at gmx.net> wrote:

> Am Freitag 03 September 2010, 10:30:29 schrieb Almar Klein:
> > * You plan on keeping history in the kernel. In this case I think this is
> > the task of the client. Otherwise you'd get your own history mixed with
> > that of someone else using that kernel? History is, I think, a feature of
> > the client to help the programmer. I see no use for storing it at the
> > kernel.
>
> That's an interesting issue.  Some more random thoughts:
>
> - If you decide to connect to an existing session (from home or the bar via
> your smartphone) to check something and/or issue a command, it is very
> useful
> to reuse your existing history -> should be in the kernel
>

You're right, that makes sense too.



> - If you are using only one frontend/client, it does not matter anyhow,
> only..
>
> - If the kernel is busy, it could still be useful to browse through the
> history to compose the next command(s) -> should be in the client?
>

Well,  the kernel could be implemented in such a way that messages are
received in a separate thread, which may also handle the introspection
requests. Introspection (+history) will then still work if the process is
busy (unless it's running extension code).


- In the multiple GUI/observer scenario, I think it would not hurt to share
> the history with the observer either -> could be in the kernel
>


> Is there any other reason to have the history in the client, Almar?
>
> Otherwise, I's say of course it's a feature of the client, but obviously
> there
> are good reasons (in particular when detaching/attaching) to have the
> history
> be stored in the kernel.  Maybe it needs to be duplicated/synchronized to
> support situations where the kernel cannot respond, though.
>

It seemed intuitively more correct (it still does, to me), but in practice
it may be better to store it in the kernel. Thanks for enlightening me!

  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100903/f8dcc635/attachment.html>

From hans_meine at gmx.net  Fri Sep  3 07:51:31 2010
From: hans_meine at gmx.net (Hans Meine)
Date: Fri, 3 Sep 2010 13:51:31 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTinvK3Tpw6DDF8H1M28+MBF4dhmNNSnEdyTy=UDc@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<201009031118.38731.hans_meine@gmx.net>
	<AANLkTinvK3Tpw6DDF8H1M28+MBF4dhmNNSnEdyTy=UDc@mail.gmail.com>
Message-ID: <201009031351.31511.hans_meine@gmx.net>

Am Freitag 03 September 2010, 12:12:51 schrieb Almar Klein:
> It seemed intuitively more correct (it still does, to me), but in practice
> it may be better to store it in the kernel. Thanks for enlightening me!

I was also confused because of the term "kernel" - recall that this is still 
the *IPython* kernel, so this is not only about code execution, but also about 
many of ipython's nifty facilities.

Indeed, a wild guess would be that in the multiprocessing case, with many 
execution kernels, there would still be only one ipython kernel (since we 
share history etc.).  That might be wrong though. ;-)

Have a nice day,
  Hans


From epatters at enthought.com  Fri Sep  3 12:17:53 2010
From: epatters at enthought.com (Evan Patterson)
Date: Fri, 3 Sep 2010 11:17:53 -0500
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTi=OTTp99e6Dghrr0JbT_vz+vTG7cn+2n0fQKqvv@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
	<AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>
	<AANLkTinhpjmP7EXxNBcw845nnD7Bubo7XPfF3p6Dq2n3@mail.gmail.com>
	<AANLkTi=OTTp99e6Dghrr0JbT_vz+vTG7cn+2n0fQKqvv@mail.gmail.com>
Message-ID: <AANLkTikKRTtv8E6cqSHpMKr4Zskyq+O0fyDp55XnyDd9@mail.gmail.com>

All the bugs reported in this thread should now be fixed. Please let
me know if you discover new and inventive ways to break the input
buffer.

Evan

On Thu, Sep 2, 2010 at 7:17 PM, Brian Granger <ellisonbg at gmail.com> wrote:
> On Thu, Sep 2, 2010 at 5:13 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>> On Thu, Sep 2, 2010 at 5:10 PM, Evan Patterson <epatters at enthought.com> wrote:
>>> The fundamental issue is that the frontend has no way to tell whether
>>> a reply that has not come back from the kernel is a result of the
>>> kernel being broken or merely busy--the frontend has no choice but to
>>> be silent. Although I've never seen the frontend outright freeze, you
>>> can verify that this has not happened by pressing Ctrl-Period.
>>>
>>> The only way to verify that the kernel is sending the messages is to
>>> look at the debug output.
>>
>> In this case I had an error at the very beginning, and the frontend
>> was just frozen, without any messages having been emmitted by the
>> kernel yet. ?Kind of staring at a blank wall...
>>
>> But I figured it out by the sheer power of my raw intellect (aka, ?I
>> stared at the same line of code for 5 minutes until I realized how
>> dumb I was). ?All good now :)
>
> Debugging like that is a super nice skill to have - not easy though...
>
> Cheers,
>
> Brian
>
>> Cheers,
>>
>> f
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
>
>
> --
> Brian E. Granger, Ph.D.
> Assistant Professor of Physics
> Cal Poly State University, San Luis Obispo
> bgranger at calpoly.edu
> ellisonbg at gmail.com
>


From seongjoo.lee at gmail.com  Fri Sep  3 12:52:12 2010
From: seongjoo.lee at gmail.com (Lee Seongjoo)
Date: Sat, 4 Sep 2010 01:52:12 +0900
Subject: [IPython-dev] Wrong line number after exception occurs
Message-ID: <AANLkTi=p-nr6qHwQ+p7fHNqu3maMAB_rOdLX+8eOGO_V@mail.gmail.com>

I am not quite sure how to reproduce but wrong line is returned when %rep
command issued after series of exception occurs. I am pretty positive that
this is a known issue since I have experienced this unstable line numbering
quite often. Still, I want to know why and how it happens if possible along
with its status if anyone could tell me. Thanks.

-- 
Lee Seongjoo
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100904/7333e2e3/attachment.html>

From ellisonbg at gmail.com  Fri Sep  3 14:28:11 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Fri, 3 Sep 2010 11:28:11 -0700
Subject: [IPython-dev] [matplotlib-devel] Uniform GUI support across
	matplotlib, ets and ipython
In-Reply-To: <540233.16065.qm@web62407.mail.re1.yahoo.com>
References: <AANLkTim-XO2FqH_GWwzsY7H7G64p+NXskUF8mp+D8h60@mail.gmail.com>
	<540233.16065.qm@web62407.mail.re1.yahoo.com>
Message-ID: <AANLkTi=JiJYeESRSQYNcq3m5g1CcJkAo2urc8=fxHiz0@mail.gmail.com>

Michiel,

On Fri, Sep 3, 2010 at 9:22 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> --- On Wed, 9/1/10, Brian Granger <ellisonbg at gmail.com> wrote:
>> > So right now you're in a loop in which you let qt4 (or
>> wx) watch the file descriptors qt4 needs, then zeromq the
>> file descriptors that zeromq needs, and so on?
>>
>> ZMQ sockets are not really sockets in that they do not have
>> a file descriptor interface.
>
> That may be, but at the bottom of the ZMQ event loop is a call to select(), which expects a pair of ordinary file descriptors. At the same time, at the bottom of the qt4 event loop is also a call to select(), also using a set of file descriptors.

What you are saying is mostly right.  ZMQ uses the best polling
mechanism for the platform (select/poll/epoll/kqueue/etc).  In some
cases, there are actual file descriptors in the internals of zeromq.
As I understand it:

* Zeromq runs on other transports other than TCP and not all of those
transports have file descriptors.
* Even if zeromq is using TCP for transport, a single zeromq socket
may be implemented with many low level sockets during its lifetime.
These sockets are not in any way exposed in the public zeromq API.
* zeromq sockets are connectionless.  If you were able to get those
low level file descriptors, the model would become leaky because you
would need to start handling the connection related events.

Thus, it is currently not an option to get those low level file
descriptors and do what you are proposing.

> Currently your event loop works approximately as follows:
>
> 1) Run the qt4 event loop, which calls select() inside qt4, watching the file descriptors that qt4 knows about.
> 2) Interrupt the qt4 event loop, and enter the ZMQ event loop, which calls select() inside ZMQ to watch the file descriptors ZMQ knows about.
> 3) If none if the ZMQ file descriptors have been signaled, go to 1).
>
> Ideally, you would want to call select() only once, and have it watch both the file descriptors qt4 knows about, as well as the file descriptors ZMQ knows about.
>
> So how to implement this?
>
> Think about just the ZMQ event loop for now. Just before entering the ZMQ event loop, ZMQ knows which file descriptors it should watch when it calls select(). Which means that ZMQ maintains in some form or another a list of file descriptors to watch. Now I don't know the details of ZMQ, but I suppose there is some function inside ZMQ that effectively adds one file descriptor to the ZMQ list of file descriptors to watch. Now, instead of this function, call an ipython-specific function that effectively adds this file descriptor to the list of file descriptors that qt should watch for (probably using QSocketNotifier). Now you don't need to enter the ZMQ event loop at all: When you enter the qt event loop, and there is any activity on one of the ZMQ file descriptors, the qt event loop will notice and take the appropriate action.
>
> Does this make sense?

Definitely, it makes total sense, it is just not possible today with
how zeromq is implemented.  The zeromq devs are working on a file
descriptor interface, but it will not be fully compatible with the
usual one in that you won't be able to distinguish between read, write
and error events with only the file descriptor.  I am not sure the
file descriptor interface that zeromq will eventually have will
actually work with what qt is expecting.

The other issue is one of maintainability.  Currently the GUI code and
zeromq event loop code is completely separate and orthogonal.  We can
easily change one without affecting the other.  If we were to combine
the event loops, we would loose that nice separation that makes our
code more robust, stable and maintainable.  Also, I should mention
that the current approach functions perfectly well.  Both the GUI and
IPython remain extremely responsive at all times.

I should also mention that the help we need with matplotlib right now
is *completely* independent of these issues of how we integrate the
two event loops.

Thanks for the ideas though.

Cheers,

Brian

> --Michiel.
>
>
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Sat Sep  4 01:26:49 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 3 Sep 2010 22:26:49 -0700
Subject: [IPython-dev] Wrong line number after exception occurs
In-Reply-To: <AANLkTi=p-nr6qHwQ+p7fHNqu3maMAB_rOdLX+8eOGO_V@mail.gmail.com>
References: <AANLkTi=p-nr6qHwQ+p7fHNqu3maMAB_rOdLX+8eOGO_V@mail.gmail.com>
Message-ID: <AANLkTikEASro7v6jXUGV-A8PjKLe+EKV8=_0HgwMg0XN@mail.gmail.com>

Hi Lee,

On Fri, Sep 3, 2010 at 9:52 AM, Lee Seongjoo <seongjoo.lee at gmail.com> wrote:
> I am not quite sure how to reproduce but wrong line is returned when %rep
> command issued after series of exception occurs. I am pretty positive that
> this is a known issue since I have experienced this unstable line numbering
> quite often. Still, I want to know why and how it happens if possible along
> with its status if anyone could tell me. Thanks.

unfortunately without a reproducible test case that can be automated,
catching something like that will be very, very hard.  I know there's
broken logic with the history synchronization because that state is
managed all over the place, rather than in a single, atomic location.
So when something like an exception breaks the normal execution flow,
it's possible to leave 'dangling' stuff and have the internal state
become inconsistent.

In the work we're currently doing, we're paying a lot of attention to
trying to move various pieces out into independent components, which
should help at least with making problems like these easier to track.
And indeed, we're already seeing the benefits of that from having
kernel and client codes live in independent processes.  But we still
have work to do.

Sorry not to have a more precise answer,  but basically the problem
you have is buried in a rat's nest that I doubt we'll disentangle.
Instead, let's hope our rework gives us a cleaner base where these
problems simply go away.

regards

f


From fperez.net at gmail.com  Sat Sep  4 02:57:13 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 3 Sep 2010 23:57:13 -0700
Subject: [IPython-dev] [matplotlib-devel] Uniform GUI support across
	matplotlib, ets and ipython
In-Reply-To: <925447.15482.qm@web62405.mail.re1.yahoo.com>
References: <AANLkTi=JiJYeESRSQYNcq3m5g1CcJkAo2urc8=fxHiz0@mail.gmail.com>
	<925447.15482.qm@web62405.mail.re1.yahoo.com>
Message-ID: <AANLkTinawGMtB88BAgJuqEvbHJnw2EBsGQUPfqJnDrtf@mail.gmail.com>

On Fri, Sep 3, 2010 at 11:16 PM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> Then I prefer to wait until ZMQ is fixed.

Note that it's not a matter of 'fixing': part of the design of zmq, as
best as I understand it, precisely requires that the low-level fds be
kept completely *private* to zmq and *never* exposed to the library
users.  This allows zmq to expose networking semantics that are
richer, more flexible and easier to use than classic socket ones.  But
the price you pay is that manual manipulation of certain internal
details becomes forbidden.

Cases like this are very common in software: you hide something to
expose a different high-level interface, and users can *not* decide to
manipulate the private bits of your object and come out unscathed.
It's like having a C extension that sets a pointer to a numpy array's
buffer and assumes it can work with it forever, forgetting reference
counting: at some point python may delete that high-level object, and
the code that "went behind numpy's back" will break.

So zmq is not 'broken', it's *different* from a purely fd-based
network api. And as Brian indicated, even if they do offer something
that looks like a fd, it may not be quite the one that other event
loops expect, so integration by 'melding' event loops may never be
possible.

Keep in mind that these design choices in zmq make it a *phenomenal*
library in many ways, whose semantics are clear and easy to work with,
so this is a price that we are happy to pay.

And the point about orthogonality Brian made is also important to keep
in mind: by blending event loops, we'd need N implementations of
gui+zmq for N GUI toolkits.  That's a maintenance *nightmare*,
considering how currently our networking logic can stay completely
uncoupled from GUI specifics.

In summary, the approach we've used with Qt/Wx of simply calling the
zmq loop on an idle timer of the GUI event loop works fine, keeps the
coupling between networking and GUI code to a *tiny* mininum, and has
only a very small performance penalty.

In software development, optmizing for a little performance at an
astronomical complexity and maintainability cost isn't a good idea.
The new architecture we have is very nice but not exactly trivial;
weaving 4 gui toolkits into its very core would likely gum it up with
impossible to debug problems.

I hope you see why we're looking for a more decoupled approach. It's
not a hack, it's a solution based on a set of well defined reasons
based on the problem at hand.

Thanks for your feedback.

Regards,

f


From fperez.net at gmail.com  Sat Sep  4 04:42:24 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 4 Sep 2010 01:42:24 -0700
Subject: [IPython-dev] kernel crash fixed, little utilities written
Message-ID: <AANLkTim=x5qDhVMGNbKkdsEaFrL2KDMOm=PN77HqtvEr@mail.gmail.com>

Hi,

the 'figure(' crash was an easy fix:

http://github.com/ipython/ipython/commit/f0e4ac07db31f9b21e84dfb1eae8380e9f4e9c4b

once I had finished a little utility I wanted you to know about:

http://github.com/ipython/ipython/commit/5c6d229ce5f249ef1f122da2ae813b70e14a3c40

Make sure you use this whenever about to send something that you're
not 100% is json encode-safe.

I just realized that it may be worth adding a noerror flag that
suppresses the ValueError exceptions that dicts can generate... There
may be cases where it's better to send a dict that drops a value
rather than crashing a kernel.  I'll add that now.

But otherwise we're in good shape.

Evan, the usability of the system is really great.  I've added little
things to the todo list, but I'm using it as my main ipython now, and
loving it!

The kernel heartbeat is also awesome: since I was checking a kernel
crasher, I could keep my session open and just keep crashing it,
editing the kernel files, and letting it restart, without losing any
of the state in my terminal :)

I did notice one thing: when we restart a kernel, we might want to
keep the client-side history alive, since precisely with a crashed
kernel you may want to re-execute prior inputs (possibly changing
something to prevent the crash again, obviously).

Anyhow, moving ahead...

Cheers,

f


From fperez.net at gmail.com  Sat Sep  4 19:29:03 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 4 Sep 2010 16:29:03 -0700
Subject: [IPython-dev] [matplotlib-devel] Uniform GUI support across
	matplotlib, ets and ipython
In-Reply-To: <687147.80651.qm@web62406.mail.re1.yahoo.com>
References: <AANLkTinawGMtB88BAgJuqEvbHJnw2EBsGQUPfqJnDrtf@mail.gmail.com>
	<687147.80651.qm@web62406.mail.re1.yahoo.com>
Message-ID: <AANLkTinR1+DS_jckWQu4BxrOiL5zSc+frMyBEPD4MNBL@mail.gmail.com>

Hi Michiel,

On Sat, Sep 4, 2010 at 1:11 AM, Michiel de Hoon <mjldehoon at yahoo.com> wrote:
> OK, then let's say ZMQ events are handled by polling. I assume that ZMQ does not use X11, otherwise this may break. If not, you should be OK.
>
> I assume that you plan to do this via QTimer in qt, or something similar for the other backends. So you need to find out if the Qt event loop is actually running. I know that in matplotlib's non-interactive mode the event loop may not be running. But since the drawing code was changed near the end of 2008 / early 2009, I don't know if there is still a good reason for matplotlib's non-interactive mode. If matplotlib were always in interactive mode, and you can be sure that there is an event loop running, your problem is solved, right?


Actually, many thanks for your help, coming from a surprising place!
This discussion you had with Ville:

https://bugs.launchpad.net/ipython/+bug/270856

especially around comment #6, put me in the right track.  I've now
committed the GTK support:

http://github.com/ipython/ipython/commit/13751b1c86126f974f13041a10f8da3920e3335c


I stumbled on that old thread on launchpad by accident, from my new
'Gmail Priority Inbox' showing it from my inbox because I had it
starred.  It was buried deep down in my normal inbox, but the new
priority inbox in gmail popped it to my attention, and when I read it
I realized that might actually do the trick for us.  Combining the
example in that thread with logic similar to what Brian had done for
Qt/Wx, along with some reading of the GTK apis, seems to have done the
trick.

I've been testing it a lot, and so far it seems to be working.

Regards,

f


From fperez.net at gmail.com  Sat Sep  4 19:30:54 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 4 Sep 2010 16:30:54 -0700
Subject: [IPython-dev] [matplotlib-devel] Uniform GUI support across
 matplotlib, ets and ipython
In-Reply-To: <F13F4057-D798-4ADD-8D04-3364A5741F01@informatik.uni-hamburg.de>
References: <AANLkTi=JiJYeESRSQYNcq3m5g1CcJkAo2urc8=fxHiz0@mail.gmail.com>
	<925447.15482.qm@web62405.mail.re1.yahoo.com>
	<AANLkTinawGMtB88BAgJuqEvbHJnw2EBsGQUPfqJnDrtf@mail.gmail.com>
	<F13F4057-D798-4ADD-8D04-3364A5741F01@informatik.uni-hamburg.de>
Message-ID: <AANLkTikCSAAwHc687SM5ZE1pMpA9cd0ZGX9RaZLAWHYt@mail.gmail.com>

On Sat, Sep 4, 2010 at 3:30 AM, Hans Meine
<meine at informatik.uni-hamburg.de> wrote:
> AFAICS, "fixing" would not necessarily mean to make any of the
> (rightfully) internal stuff public, but simply to support a
> non-polling (/battery-draining) way of integrating into an eventloop.
>
> That's certainly a sound requirement, which ZMQ might fail to meet
> today, but maybe not because it's inherently impossible given the
> design goals.

Yes, very good point, that's the right way to frame the question.  ZMQ
is a young project and it seems to be driven by a very receptive team
of developers, so I hope they can tackle this in future releases, and
that there's no fundamental reason that prevents it from working.

Regards

f


From fperez.net at gmail.com  Sat Sep  4 19:33:41 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 4 Sep 2010 16:33:41 -0700
Subject: [IPython-dev] can delete prompts in ipythonqt
In-Reply-To: <AANLkTikKRTtv8E6cqSHpMKr4Zskyq+O0fyDp55XnyDd9@mail.gmail.com>
References: <AANLkTimXMyp0dwS5oArWnh-eXsbvEVYHs3W9FuAPKGyt@mail.gmail.com>
	<AANLkTino=CtJN_NPxPC90OQ=zmStw4M9jBaFqmyy3eHi@mail.gmail.com>
	<AANLkTi=J1yXV9_jaoV3pGqaC3gm+RoBO6c9dfW7mKUSz@mail.gmail.com>
	<AANLkTikYbbxG2kGnboZnNnh0ZwuOEq8k6L5R_DgMdRPk@mail.gmail.com>
	<AANLkTikjfVqFz4TX0sr4zWz8FE6uxeiErzXh5JfWiq8Z@mail.gmail.com>
	<AANLkTinhpjmP7EXxNBcw845nnD7Bubo7XPfF3p6Dq2n3@mail.gmail.com>
	<AANLkTi=OTTp99e6Dghrr0JbT_vz+vTG7cn+2n0fQKqvv@mail.gmail.com>
	<AANLkTikKRTtv8E6cqSHpMKr4Zskyq+O0fyDp55XnyDd9@mail.gmail.com>
Message-ID: <AANLkTinFJjLo_0WRo-dVwTjGD8dmCD=JypuAgdvWoDxm@mail.gmail.com>

On Fri, Sep 3, 2010 at 9:17 AM, Evan Patterson <epatters at enthought.com> wrote:
> All the bugs reported in this thread should now be fixed. Please let
> me know if you discover new and inventive ways to break the input
> buffer.

Just to let you know that I'm now using it pretty much exclusively,
and it's working *great*.  Right now my only real bothers are (already
noted in your todo list):

- the up-arrow-only-walk-what-matches: this saves a TON of typing and
up-up-up-up...
- sliding the popups down a tiny bit so they don't obscure the line
being typed on.
- making the popups dismissable without killing the input buffer.

Honestly with those three out of the way, the rest is becoming more
wish-list than must-have.  You've done a tremendous job, many thanks.

Cheers,

f


From fperez.net at gmail.com  Sat Sep  4 19:42:00 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 4 Sep 2010 16:42:00 -0700
Subject: [IPython-dev] Nice and short ZeroMQ intro article
Message-ID: <AANLkTinT5qQr=VdkaKYq8753CWjJdnNP=8iiqSZKZ11N@mail.gmail.com>

since we're doing so much with zmq, and not all of it  is completly
trivial (e.g. the connection diagrams in
http://ipython.scipy.org/doc/nightly/html/development/parallel_connections.html
for a taste of what I'm talking about), I figured this could be useful
to anyone interested in understanding the basic concepts involved in
zeromq and how it compares/differs from raw BSD sockets:

http://www.igvita.com/2010/09/03/zeromq-modern-fast-networking-stack&root7/

Cheers,

f


From takowl at gmail.com  Mon Sep  6 09:07:25 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Mon, 6 Sep 2010 14:07:25 +0100
Subject: [IPython-dev] IPython on Python 3
Message-ID: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>

Hi all,

I know running IPython on Python 3 has been discussed before, and that doing
a proper port might take a long time. However, I was messing about with
Python 3, and found IPython was the thing I missed most, so I thought I'd
investigate how difficult it was to get it running.

Using 2to3, and tweaking the result a bit (mostly for the new Unicode
strings), I have a version that works, at least for basic use on my
computers:
http://github.com/takowl/ipython

Basic use includes starting it up, running commands, docstrings ("object?"),
tab completion (see below) and history. I don't claim that the codebase is
neat: I've done a largely automatic conversion, without attempting to
understand how it all fits together.

One specific issue I'd like some advice on: tab completion currently doesn't
work in the main namespace, before any dots. So mystring.isdig<tab> works,
but prin<tab> doesn't do anything. Filenames within strings don't
tab-complete either. I've tried to investigate, but I guess the code
silences any errors, and I don't really know where to start looking. I
imagine it's the Unicode question again.

Naturally, anyone is welcome to use it or improve it.

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100906/920b10c0/attachment.html>

From ellisonbg at gmail.com  Mon Sep  6 13:02:16 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 6 Sep 2010 10:02:16 -0700
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
Message-ID: <AANLkTin-iGOBUMJ710eS=FWPPnvoT9qUAO-79T5TsO9r@mail.gmail.com>

On Fri, Sep 3, 2010 at 1:30 AM, Almar Klein <almar.klein at gmail.com> wrote:
> Hi,
>
> I've been thinking about your protocol last night. (arg! This whole thing is
> getting to my head too much!)
>
> I see some advantages of loose messages such as your using in favor of my
> channels approach. Specifically, the order of stdout and stderr messages is
> not preserved, as they are send over a different channel (same socket
> though). On a 1to1 system, this is not much of a problem, but for multiple
> users it definitely will be, specifically if you also want to show the other
> users input (pyin).
>
> Anyway, I took another good look at your approach and I have a few
> remarks/ideas. Might be useful, might be total BS.
>
> * You say the prompt is a thing the client decides what it looks like. I
> disagree. The kernel gives a prompt to indicate that the user can give
> input, and maybe also what kind of input. A debugger might produce a
> different prompt to indicate that the user is in debug mode. See also the
> doc on sys.ps1 and sys.ps2 (http://docs.python.org/library/sys.html): the
> user (or interpreter) can put an object on it that evaluates to something
> nice when str()'nged.
>
> * You plan on keeping history in the kernel. In this case I think this is
> the task of the client. Otherwise you'd get your own history mixed with that
> of someone else using that kernel? History is, I think, a feature of the
> client to help the programmer. I see no use for storing it at the kernel.


We keep the history is both places, which allows either party to
reconstruct things if the other goes down.

> * I really think you can do with less sockets. I believe that the (black)
> req/rep pair is not needed. You only seem to use it for when raw_input is
> used. But why? When raw_input is used, you can just block and wait for some
> stdin (I think that'll be the execute_request message). This should not be
> too hard by replacing sys.stdin with an object that has a readline method
> that does this. If two users are present, and one calls raw_input, they can
> both provide input (whoever's first). To indicate this to the *other* user,
> however, his prompt should be replaced with an empty string, so his cursor
> is positioned right after the <text> in raw_input('<text>').

We could do with fewer pairs, but having different socket pairs for
different purposes buys us a lot of isolation, encapsulation and
robustness in our code base.

> * I think you can do with even less sockets :)? But this is more of a wild
> idea. Say that John set up an experiment at work and wants to check the
> results in the bar on his Android (sorry I stole your example here,
> Fernando). Now his experiment crashed, producing a traceback in the client
> at his work PC. But now he cannot see the traceback as he just logged in!
> -----? So what about storing all stdout, stderr and pyin (basically all
> "terminal-output") at the kernel? And rather than pub/sub, use the existing
> req/rep to obtain this stuff. Maybe you can even pass new terminal-output
> along with other replies. The client should indicate in the request a
> sequence number to indicate to the kernel what messages were already
> received. This does mean, however, that the client would have to
> periodically query the kernel. But maybe this can also be done automatically
> by writing a thin layer on top of the zmq interface. Oh, and you'd need to
> encapsulate multiple terminal-messages in a single reply.
>
> Just my two cents,
> ? Almar
>
>
> On 31 August 2010 07:28, Fernando Perez <fperez.net at gmail.com> wrote:
>>
>> On Mon, Aug 30, 2010 at 1:51 AM, Almar Klein <almar.klein at gmail.com>
>> wrote:
>> > Ah right. Although I'm not sure how often one would use such this in
>> > practice, it's certainly a nice feature, and seems to open op a range of
>> > possibilities. I can imagine this requirement makes things considerably
>> > harder to implement, but since you're designing a whole new protocol
>> > from
>> > scratch, it's probably a good choice to include it now.
>>
>> And the whole thing fits naturally in our design for tools that enable
>> both interactive/collaborative computing and distributed/parallel work
>> within one single framework. ?After all, it's just manipulating
>> namespaces :)
>>
>> >> In our case obviously the kernel itself remains unresponsive, but the
>> >> important part is that the networking doesn't suffer. ?So we have
>> >> enough information to take action even in the face of an unresponsive
>> >> kernel.
>> >
>> > I'm quite a new to networking, so sorry for if this sounds stupid: Other
>> > than the heartbeat stuff not working, would it also have other effects?
>> > I
>> > mean, data can not be send or received, so would maybe network buffers
>> > overflow or anything?
>>
>> Depending on how you implemented your networking layer, you're likely
>> to lose data. ?And you'll need to ensure that your api recovers
>> gracefully from half-sent messages, unreplied messages, etc.
>>
>> Getting a robust and efficient message transport layer written is not
>> easy work. ?It takes expertise and detailed knowledge, coupled with
>> extensive real-world experience, to do it right. ?We simply decided to
>> piggy back on some of the best that was out there, rather than trying
>> to rewrite our own. ?The features we gain from zmq (it's not just the
>> low-level performance, it's also the simple but powerful semantics of
>> their various socket types, which we've baked into the very core of
>> our design) are well worth the price of a C dependency in this case.
>>
>> > Further, am I right that the heartbeat is not necessary when
>> > communicating
>> > between processes on the same box using 'localhost' (since some network
>> > layers are bypassed)? That would give a short term solution for IEP.
>>
>> Yes, on local host you can detect the process via other mechanisms.
>> The question is whether the system recovers gracefully from dropped
>> messages or incomplete connections. ?You do need to engineer that into
>> the code itself, so that you don't lock up your client when the kernel
>> becomes unresponsive, for example.
>>
>> I'm sure we still have corner cases in our code where we can lock up,
>> it's not easy to prevent all such occurrences.
>>
>> > No, that's the great thing! All channels are multiplexed over the same
>> > socket pair. When writing a message to a channel, it is put in a queue,
>> > adding a small header to indicate the channel id. There is a single
>> > thread
>> > that sends and receives messages over the socket. It just pops the
>> > messages
>> > from the queue and sends them to the other side. At the receiver side,
>> > the
>> > messages are distributed to the queue corresponding to the right
>> > channel. So
>> > there's one 'global' queue on the sending side and one queue per channel
>> > on
>> > the receiver side.
>>
>> Ah, excellent! ?It seems your channels are similar to our message
>> types, we simply dispatch on the message type (a string) with the
>> appropriate handler. ?The twist in ipython is that we have used as an
>> integral part of the design the various types of zmq sockets: req/rep
>> for stdin control, xrep/xreq for execution requests multiplexed across
>> clients, and pub/sub for side effects (things that don't fit in a
>> functional paradigm).
>>
>> We thus have a very strong marriage between the abstractions that zmq
>> exposes and our design. ?Honestly, I sometimes feel as if zmq had been
>> designed for us, because it makes certain things we'd wanted for a
>> very long time almost embarrassingly easy.
>>
>> Thanks a lot for sharing your ideas, it's always super useful to look
>> at these questions from multiple perspectives.
>>
>> Regards,
>>
>> f
>
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From ellisonbg at gmail.com  Mon Sep  6 13:04:13 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 6 Sep 2010 10:04:13 -0700
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
Message-ID: <AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>

Almar,

Here is the new guisupport module that we are using for these things.
We are encouraging projects to simply copy it into their code base and
use it:

http://github.com/ipython/ipython/blob/newkernel/IPython/lib/guisupport.py

Cheers,

Brian

On Mon, Aug 30, 2010 at 10:29 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Mon, Aug 30, 2010 at 2:09 AM, Almar Klein <almar.klein at gmail.com> wrote:
>> Still, may I suggest the following: IPython or IEP, or any environment,
>> could inject a function 'start_event_loop' in the module namespace of the
>> GUI toolkit it integrates. My main argument for this, is that it would be
>> independent of IPython or any specific library or IDE. The user can then
>> simply call:
>>
>> import wx
>> if hasattr(wx, 'start_event_loop'):
>> ??? wx.start_event_loop()
>> else:
>> ??? # Start the "native" way
>> ??? app = wx.PySimpleApp(*args, **kwargs)
>> ??? app.MainLoop()
>>
>
> I mentioned it to Brian on IRC and he saw a catch with this idea, but
> I'm not sure of the details. ?Over the next couple of days we can hash
> it over here, thanks a lot for the feedback. ?We certainly want a
> solution that covers all the bases for all projects, so we can all
> reuse a common approach.
>
> Regards,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From almar.klein at gmail.com  Mon Sep  6 15:53:37 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Mon, 6 Sep 2010 21:53:37 +0200
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
Message-ID: <AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>

On 6 September 2010 19:04, Brian Granger <ellisonbg at gmail.com> wrote:

> Almar,
>
> Here is the new guisupport module that we are using for these things.
> We are encouraging projects to simply copy it into their code base and
> use it:
>
> http://github.com/ipython/ipython/blob/newkernel/IPython/lib/guisupport.py
>

I saw this module earlier, only then did not realize it was stand-alone (or
maybe it wasn't then?)

So, to be clear, the user can now just do:
import guisupport
guisupport.start_event_loop_wx() # or _qt4, or _gtk, ...

That sounds great! I'll take a look at filling in the blanks for tk, fltk
and gtk if you like.

Cheers,
  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100906/0300fe85/attachment.html>

From fperez.net at gmail.com  Mon Sep  6 16:04:28 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Sep 2010 13:04:28 -0700
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
Message-ID: <AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>

Hi Almar,

On Mon, Sep 6, 2010 at 12:53 PM, Almar Klein <almar.klein at gmail.com> wrote:
>
> That sounds great! I'll take a look at filling in the blanks for tk, fltk
> and gtk if you like.

that would be great. Note that as far as we've seen, in light testing
it seems that for gtk nothing is needed: with this code on our side:

http://github.com/ipython/ipython/blob/newkernel/IPython/zmq/gui/gtkembed.py

every single gtk example I've tried so far has worked (with the only
caveat being that you have to close them with the window manager
buttons rather than the app's own close button/menu).  So we may be
home free for gtk, though that's still up for confirmation with more
testing and someone more knowledgeable in the matter than me.

It also looks like something similar may be OK for tk, where with a
bit of hijacking from our side, we can let unmodified Tk code run
happily within our own environment.

For Qt/Wx it definitely seems that the cooperation via guisupport will
be needed.  And for fltk, I have no idea.

Thanks for your help and interest!

Regards,

f


From almar.klein at gmail.com  Mon Sep  6 16:32:43 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Mon, 6 Sep 2010 22:32:43 +0200
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
Message-ID: <AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>

On 6 September 2010 22:04, Fernando Perez <fperez.net at gmail.com> wrote:

> Hi Almar,
>
> On Mon, Sep 6, 2010 at 12:53 PM, Almar Klein <almar.klein at gmail.com>
> wrote:
> >
> > That sounds great! I'll take a look at filling in the blanks for tk, fltk
> > and gtk if you like.
>
> that would be great. Note that as far as we've seen, in light testing
> it seems that for gtk nothing is needed: with this code on our side:
>
>
> http://github.com/ipython/ipython/blob/newkernel/IPython/zmq/gui/gtkembed.py
>
> every single gtk example I've tried so far has worked (with the only
> caveat being that you have to close them with the window manager
> buttons rather than the app's own close button/menu).  So we may be
> home free for gtk, though that's still up for confirmation with more
> testing and someone more knowledgeable in the matter than me.
>
> It also looks like something similar may be OK for tk, where with a
> bit of hijacking from our side, we can let unmodified Tk code run
> happily within our own environment.
>
> For Qt/Wx it definitely seems that the cooperation via guisupport will
> be needed.  And for fltk, I have no idea.
>

I don't think there's much to do for fltk either, I'll look into it tomorrow
I hope.

May I suggest to put in a start_event_loop_* for tk and gtk even if they'd
just do "pass"? It would look more uniform to the user. Now it looks a bit
like gtk and tk are not yet supported.

  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100906/eb0b5eef/attachment.html>

From fperez.net at gmail.com  Mon Sep  6 16:45:48 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Sep 2010 13:45:48 -0700
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
	<AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
Message-ID: <AANLkTikvvv0a8qEJt9Pta-r5mnkOfjTe0v2Ct7sJb0P3@mail.gmail.com>

On Mon, Sep 6, 2010 at 1:32 PM, Almar Klein <almar.klein at gmail.com> wrote:
>
> May I suggest to put in a start_event_loop_* for tk and gtk even if they'd
> just do "pass"? It would look more uniform to the user. Now it looks a bit
> like gtk and tk are not yet supported.

Even better, we just put in there

# GTK

# Go home happy, you don't need to do anything for GTK!  Your code
works without any extra modifications needed :)

I'd rather just explain that to people than make them do busy calls
that serve no purpose.  Most people (with a few exceptions, obviously)
only write code for the toolkit they use, so the gtk crowd (and
possibly tk/fltk) will just register "I don't have to worry about it"
and move on, rather than sprinkling all their codes with extra no-ops.

Cheers,

f


From fperez.net at gmail.com  Mon Sep  6 19:57:15 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Sep 2010 16:57:15 -0700
Subject: [IPython-dev] IPython on Python 3
In-Reply-To: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>
References: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>
Message-ID: <AANLkTimQzh9_iQ=8Vb98R2szaafxWsjbwgdACVF7Ko34@mail.gmail.com>

Hi Thomas,

On Mon, Sep 6, 2010 at 6:07 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> Hi all,
>
> I know running IPython on Python 3 has been discussed before, and that doing
> a proper port might take a long time. However, I was messing about with
> Python 3, and found IPython was the thing I missed most, so I thought I'd
> investigate how difficult it was to get it running.
>
> Using 2to3, and tweaking the result a bit (mostly for the new Unicode
> strings), I have a version that works, at least for basic use on my
> computers:
> http://github.com/takowl/ipython
>
> Basic use includes starting it up, running commands, docstrings ("object?"),
> tab completion (see below) and history. I don't claim that the codebase is
> neat: I've done a largely automatic conversion, without attempting to
> understand how it all fits together.
>
> One specific issue I'd like some advice on: tab completion currently doesn't
> work in the main namespace, before any dots. So mystring.isdig<tab> works,
> but prin<tab> doesn't do anything. Filenames within strings don't
> tab-complete either. I've tried to investigate, but I guess the code
> silences any errors, and I don't really know where to start looking. I
> imagine it's the Unicode question again.
>
> Naturally, anyone is welcome to use it or improve it.

First, many, many thanks for this!

Now that with zeromq we have a way forward for our entire codebase on
py3k, it's really time to start thinking about it.  It may be a while
until we completely shed the bulk of the py2-only code, but we should
be able to move with partial functionality forward.

I see you worked off trunk, which is great.  We have a fair amount of
new code in a separate branch that we're waiting to stabilize before
we merge it in, it's here:

http://github.com/ipython/ipython/tree/newkernel

It should be no more than a couple of weeks before we merge it in,
I'll leave it up to you to decide if you want to track that or just
wait for us.

The tab completion issues you mention are indeed tricky to debug, but
I'd suggest if you want to go there, that you do work off the
newkernel branch, since we've cleaned things up quite a bit in the
completion code.  I just don't want you having to redo any bug fixing
you may do now, manually again later after we merge.

For debugging the completion mechanism, see this comment/flag (which I
added just yesterday :) :

http://github.com/ipython/ipython/blob/newkernel/IPython/core/completer.py#L819

Once newkernel is merged, we can do a branch comparison between it and
your code, and start looking at integrating as much as possible
straight in.

Having an easier path to 3.x is one of the reasons why we went for 2.6
as a minimum requirement for trunk, and in all new code we're using
print() as a function as well as trying to keep 3.x in mind all the
time.

And if you want/can play with pyzmq to produce 3.x bindings and run
the new PyQt code on 3.x, I won't stop you :)

Regards,

f


From fperez.net at gmail.com  Tue Sep  7 02:06:10 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 6 Sep 2010 23:06:10 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
Message-ID: <AANLkTimq7dZ6CzoAArUPg+eidYU5J64_1vYsriepmqiH@mail.gmail.com>

Hey Evan,

I just fixed the paste-trailing-newline annoyance:

http://github.com/ipython/ipython/commit/92971904bc9fd2b988d8c16e9502edc39a70ff25

I think that approach is good, because it gives the user a chance to
edit the code before actually executing, but otherwise just needs  a
simple return to execute.

I do have one question though: why disallow unicode paste?  People are
quite likely to have non-ascii in their examples, and it seems odd to
block them from pasting it in.  Consider for example that I can't
paste this:

name = "Fernando P?rez"

I consider the fact that I can't type my own name into ipython a bug :)

I think the solution is to set the GUI encoding by default to UTF-8,
with an option for the user to change that according to their
preferences later.  I had a quick go at it, but it was getting too
complicated so I didn't commit anything anywhere.  Here's the diff in
case you find it useful as a starting point (I just reverted locally):

####
(newkernel)amirbar[qt]> git diff
diff --git a/IPython/frontend/qt/console/console_widget.py
b/IPython/frontend/qt/console/console_widget.py
index d78cd63..f6ae9fd 100644
--- a/IPython/frontend/qt/console/console_widget.py
+++ b/IPython/frontend/qt/console/console_widget.py
@@ -10,7 +10,7 @@ from PyQt4 import QtCore, QtGui
 # Local imports
 from IPython.config.configurable import Configurable
 from IPython.frontend.qt.util import MetaQObjectHasTraits
-from IPython.utils.traitlets import Bool, Enum, Int
+from IPython.utils.traitlets import Bool, Enum, Int, Str
 from ansi_code_processor import QtAnsiCodeProcessor
 from completion_widget import CompletionWidget

@@ -37,6 +37,9 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
     # non-positive number disables text truncation (not recommended).
     buffer_size = Int(500, config=True)

+    # The default encoding used by the GUI.
+    encoding = Str('utf-8')
+
     # Whether to use a list widget or plain text output for tab completion.
     gui_completion = Bool(False, config=True)

@@ -233,7 +236,7 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
             text = QtGui.QApplication.clipboard().text()
             if not text.isEmpty():
                 try:
-                    str(text)
+                    text.encode(self.encoding)
                     return True
                 except UnicodeEncodeError:
                     pass
@@ -421,7 +424,8 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
             try:
                 # Remove any trailing newline, which confuses the GUI and
                 # forces the user to backspace.
-                text = str(QtGui.QApplication.clipboard().text(mode)).rstrip()
+                raw = QtGui.QApplication.clipboard().text(mode).rstrip()
+                text = raw.encode(self.encoding)
             except UnicodeEncodeError:
                 pass
             else:
@@ -1034,7 +1038,7 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
         cursor.movePosition(QtGui.QTextCursor.StartOfBlock)
         cursor.movePosition(QtGui.QTextCursor.EndOfBlock,
                             QtGui.QTextCursor.KeepAnchor)
-        return str(cursor.selection().toPlainText())
+        return unicode(cursor.selection().toPlainText()).encode(self.encoding)

     def _get_cursor(self):
         """ Convenience method that returns a cursor for the current position.
####

By the way, this isn't an odd corner case: in other countries, people
are likely to have files and directories with unicode in them *all the
time*, so this problem will hit us immediately once the code is out,
I'm afraid.

I saw multiple calls of the form str(some.Qt.Code()) that were
throwing exceptions and decided to stop before I get myself too deep
into Qt code I don't know well.  But the right approach is probably to
encapsulate all those into a single common call that manages the
encoding.

The tricky part, I suspect, will be to do the cursor positioning logic
with unicode in play: you need to correctly compute the lengths in
terms of characters on the unicode string (more precisely, the number
of glyphs that the code points map to), not bytes on the raw one.

Welcome to the wonderful world of unicode!

Cheers,

f

ps - and on py3k it's *only* unicode everywhere, so we might as well
get this code right from the get go.  Now that we have people starting
to help towards py3, the last thing we should do is write a ton of new
code that is unicode-unsafe for a py3 transition.  We're not writing
py3 code yet, but we should write *with an eye towards py3*.


From fperez.net at gmail.com  Tue Sep  7 03:19:22 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 00:19:22 -0700
Subject: [IPython-dev] subprocess vs popen
In-Reply-To: <20100517004941.31359.40712.launchpad@wampee.canonical.com>
References: <20100517004941.31359.40712.launchpad@wampee.canonical.com>
Message-ID: <AANLkTik3LXfAXmfH9MJcOdRO949GOyFe5dHbEzN3Qysc@mail.gmail.com>

[ Ccing the dev list where this discussion fits best]

On Sun, May 16, 2010 at 5:49 PM, Kevin Beckford <lcc at lazyweb.ca> wrote:
>There was a bug on lp a while ago about why you were not going to fix:
>
> !ls
>
> Deprecation Warning .....
>
> in ipython.
>
> I agree , subprocess is a horrid interface, that's the word I want,
> horrid. Thus i went looking for and found this:
>
> http://pypi.python.org/pypi/iterpipes/0.4
>
> But ipython does _not_ use external libs right?

Cool, thanks for the link!  I hadn't seen it.

IPython *does* use external libs if they are either:

- small enough to carry our private copy in IPython.external

- large but central to some (optional) functionality, like Wx for gui
support or Twisted for parallel computing.

This would fit case #1.

I'd suggest that if you like it and use it for a while, you propose it
on the ipython-dev list for inclusion.  We now tend to be more
conservative in pulling in new libraries than before (it's all code to
maintain, after all), but I could see an 'ipython system shell'
project growing in this direction, if enough people (aka not me) pitch
in to do the
work.

Regards

f


From almar.klein at gmail.com  Tue Sep  7 03:36:19 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Tue, 7 Sep 2010 09:36:19 +0200
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTikvvv0a8qEJt9Pta-r5mnkOfjTe0v2Ct7sJb0P3@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
	<AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
	<AANLkTikvvv0a8qEJt9Pta-r5mnkOfjTe0v2Ct7sJb0P3@mail.gmail.com>
Message-ID: <AANLkTingNQC1KVan1vAH64Nvd2FRjrpsnRoWSn8_qUXy@mail.gmail.com>

On 6 September 2010 22:45, Fernando Perez <fperez.net at gmail.com> wrote:

> On Mon, Sep 6, 2010 at 1:32 PM, Almar Klein <almar.klein at gmail.com> wrote:
> >
> > May I suggest to put in a start_event_loop_* for tk and gtk even if
> they'd
> > just do "pass"? It would look more uniform to the user. Now it looks a
> bit
> > like gtk and tk are not yet supported.
>
> Even better, we just put in there
>
> # GTK
>
> # Go home happy, you don't need to do anything for GTK!  Your code
> works without any extra modifications needed :)
>
> I'd rather just explain that to people than make them do busy calls
> that serve no purpose.  Most people (with a few exceptions, obviously)
> only write code for the toolkit they use, so the gtk crowd (and
> possibly tk/fltk) will just register "I don't have to worry about it"
> and move on, rather than sprinkling all their codes with extra no-ops.
>

In general, this "busy call" is only done once per application right? So not
much harm done IMO.

But well, if properly documented, I can agree with your suggestion.

Cheers,
  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100907/cd89b095/attachment.html>

From takowl at gmail.com  Tue Sep  7 13:27:29 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Tue, 7 Sep 2010 18:27:29 +0100
Subject: [IPython-dev] IPython on Python 3
In-Reply-To: <AANLkTimQzh9_iQ=8Vb98R2szaafxWsjbwgdACVF7Ko34@mail.gmail.com>
References: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>
	<AANLkTimQzh9_iQ=8Vb98R2szaafxWsjbwgdACVF7Ko34@mail.gmail.com>
Message-ID: <AANLkTikWLRpcJ0gFgNGe_LQicrh3-AOjW5GMnw6J8YcJ@mail.gmail.com>

On 7 September 2010 00:57, Fernando Perez <fperez.net at gmail.com> wrote:

> I see you worked off trunk, which is great.  We have a fair amount of
> new code in a separate branch that we're waiting to stabilize before
> we merge it in, it's here:
>

Great, I'll see what I can make of it.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100907/5df3fff3/attachment.html>

From takowl at gmail.com  Tue Sep  7 19:08:20 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 8 Sep 2010 00:08:20 +0100
Subject: [IPython-dev] IPython on Python 3
In-Reply-To: <AANLkTikWLRpcJ0gFgNGe_LQicrh3-AOjW5GMnw6J8YcJ@mail.gmail.com>
References: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>
	<AANLkTimQzh9_iQ=8Vb98R2szaafxWsjbwgdACVF7Ko34@mail.gmail.com>
	<AANLkTikWLRpcJ0gFgNGe_LQicrh3-AOjW5GMnw6J8YcJ@mail.gmail.com>
Message-ID: <AANLkTikYjkVJS=p8xxwEsmnd77ymmHaw=LVi=n2YkUZf@mail.gmail.com>

OK, the branch based on the new kernel is at:
http://github.com/takowl/ipython/tree/ipy3-newkernel

It already seems to be working better. Thanks for the completer debug
switch, Fernando; it was a trivial fix once I could see the error.

I've not looked at the the PyQt side at all, I've just used it in the
terminal.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100908/c5b35ca3/attachment.html>

From fperez.net at gmail.com  Tue Sep  7 19:17:14 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 16:17:14 -0700
Subject: [IPython-dev] IPython on Python 3
In-Reply-To: <AANLkTikYjkVJS=p8xxwEsmnd77ymmHaw=LVi=n2YkUZf@mail.gmail.com>
References: <AANLkTi=dA0KMhwX2TavKtT5Kwz1HiB3LqdrxjD+LEYPu@mail.gmail.com>
	<AANLkTimQzh9_iQ=8Vb98R2szaafxWsjbwgdACVF7Ko34@mail.gmail.com>
	<AANLkTikWLRpcJ0gFgNGe_LQicrh3-AOjW5GMnw6J8YcJ@mail.gmail.com>
	<AANLkTikYjkVJS=p8xxwEsmnd77ymmHaw=LVi=n2YkUZf@mail.gmail.com>
Message-ID: <AANLkTinXD7jdLhajSQXFnSq++PoiKaEFrt21ge51UP22@mail.gmail.com>

On Tue, Sep 7, 2010 at 4:08 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> OK, the branch based on the new kernel is at:
> http://github.com/takowl/ipython/tree/ipy3-newkernel
>
> It already seems to be working better. Thanks for the completer debug
> switch, Fernando; it was a trivial fix once I could see the error.

Glad to hear that!

> I've not looked at the the PyQt side at all, I've just used it in the
> terminal.

No problem.  This is excellent, many thanks.

I won't be able to look at it in detail for a few weeks, as the
trunk/newkernel work is taking all the time I can devote to IPython.
But if you start seeing little things that we could change upstream
that would make the conversion easier/cleaner, please do submit them
on their own branch.  Basically you could have a branch that is 2.x
only, but with fixes for the 3k one:

ipython/(newkernel now, trunk soon) -> takowl/ipy3-preparation -> takowl/ipy3

The changes in the -preparation branch could then be submitted back to
upstream and we could pull from you in the usual fashion.  Eventually
we'd like -prep to disappear, and ipy3 to be as similar as possible to
trunk itself to minimize duplicate maintenance.

This will let you work in parallel with us even if we're busy on
trunk, because all you need to do is give us changes that make the 3k
port easier but that remain 2.6-valid, and we incorporate those
upstream.

Regards,

f


From fperez.net at gmail.com  Tue Sep  7 19:46:19 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 16:46:19 -0700
Subject: [IPython-dev] Smart copy rocks, let's make it even better :)
Message-ID: <AANLkTi=JDXExmqoFqR2HKXnuhtHbe3Ds8C5OMHW9fxNt@mail.gmail.com>

Hey Evan,

I'm loving smart copy, but I was thinking it would be even better to
strip not just prompts, but all output, so that one can just highlight
a bunch of prior input, C-C, C-V and then edit it as a new multiline
cell without having to clean out output.

We could then leave C-Shift-C or a menu entry for 'full copy' that
would not do any stripping, for copy/pasting full sessions into an
external document, for example.

With that improvement, the workflow begins to feel really fantastic,
since it's possible to do things like:

In [29]: f = qt.QFont('Consolas')

In [30]: fi = qt.QFontInfo(f)

In [31]: fi.family()
Out[31]: PyQt4.QtCore.QString(u'Consolas')

< Copy-paste here of the previous three lines, now I have a single
block I can easily modify and re-execute>

In [32]: f = qt.QFont('Consolas')
    ...: fi = qt.QFontInfo(f)
    ...: fi.family()
Out[32]: PyQt4.QtCore.QString(u'Consolas')

<re-execute changing one parameter>

In [33]: f = qt.QFont('Consolata')
    ...: fi = qt.QFontInfo(f)
    ...: fi.family()
Out[33]: PyQt4.QtCore.QString(u'DejaVu Sans')


Since I made the run_cell semantics analyze the input and run the
*last* statement in single mode, you get the benefits of a cell-based
system with the fluidity of a console :)

Cheers,

ps - that and the up-arrow-filter-matches, and we'll be in heaven :)


From epatters at enthought.com  Tue Sep  7 20:16:51 2010
From: epatters at enthought.com (Evan Patterson)
Date: Tue, 7 Sep 2010 17:16:51 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTimq7dZ6CzoAArUPg+eidYU5J64_1vYsriepmqiH@mail.gmail.com>
References: <AANLkTimq7dZ6CzoAArUPg+eidYU5J64_1vYsriepmqiH@mail.gmail.com>
Message-ID: <AANLkTinUjP==6YYtswREuOEeMhhZN7M=EkKiKLfr7Scv@mail.gmail.com>

I appreciate the desire for Unicode support (although I firmly believe
that source code itself should *always* be in ASCII). Unfortunately,
you're correct that there may be fair amount of effort involved in
supporting Unicode robustly.

In short, I may not have time to get this done in the next week and a
half. We should discuss this further off-line, though.

Evan

On Mon, Sep 6, 2010 at 11:06 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey Evan,
>
> I just fixed the paste-trailing-newline annoyance:
>
> http://github.com/ipython/ipython/commit/92971904bc9fd2b988d8c16e9502edc39a70ff25
>
> I think that approach is good, because it gives the user a chance to
> edit the code before actually executing, but otherwise just needs ?a
> simple return to execute.
>
> I do have one question though: why disallow unicode paste? ?People are
> quite likely to have non-ascii in their examples, and it seems odd to
> block them from pasting it in. ?Consider for example that I can't
> paste this:
>
> name = "Fernando P?rez"
>
> I consider the fact that I can't type my own name into ipython a bug :)
>
> I think the solution is to set the GUI encoding by default to UTF-8,
> with an option for the user to change that according to their
> preferences later. ?I had a quick go at it, but it was getting too
> complicated so I didn't commit anything anywhere. ?Here's the diff in
> case you find it useful as a starting point (I just reverted locally):
>
> ####
> (newkernel)amirbar[qt]> git diff
> diff --git a/IPython/frontend/qt/console/console_widget.py
> b/IPython/frontend/qt/console/console_widget.py
> index d78cd63..f6ae9fd 100644
> --- a/IPython/frontend/qt/console/console_widget.py
> +++ b/IPython/frontend/qt/console/console_widget.py
> @@ -10,7 +10,7 @@ from PyQt4 import QtCore, QtGui
> ?# Local imports
> ?from IPython.config.configurable import Configurable
> ?from IPython.frontend.qt.util import MetaQObjectHasTraits
> -from IPython.utils.traitlets import Bool, Enum, Int
> +from IPython.utils.traitlets import Bool, Enum, Int, Str
> ?from ansi_code_processor import QtAnsiCodeProcessor
> ?from completion_widget import CompletionWidget
>
> @@ -37,6 +37,9 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
> ? ? # non-positive number disables text truncation (not recommended).
> ? ? buffer_size = Int(500, config=True)
>
> + ? ?# The default encoding used by the GUI.
> + ? ?encoding = Str('utf-8')
> +
> ? ? # Whether to use a list widget or plain text output for tab completion.
> ? ? gui_completion = Bool(False, config=True)
>
> @@ -233,7 +236,7 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
> ? ? ? ? ? ? text = QtGui.QApplication.clipboard().text()
> ? ? ? ? ? ? if not text.isEmpty():
> ? ? ? ? ? ? ? ? try:
> - ? ? ? ? ? ? ? ? ? ?str(text)
> + ? ? ? ? ? ? ? ? ? ?text.encode(self.encoding)
> ? ? ? ? ? ? ? ? ? ? return True
> ? ? ? ? ? ? ? ? except UnicodeEncodeError:
> ? ? ? ? ? ? ? ? ? ? pass
> @@ -421,7 +424,8 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
> ? ? ? ? ? ? try:
> ? ? ? ? ? ? ? ? # Remove any trailing newline, which confuses the GUI and
> ? ? ? ? ? ? ? ? # forces the user to backspace.
> - ? ? ? ? ? ? ? ?text = str(QtGui.QApplication.clipboard().text(mode)).rstrip()
> + ? ? ? ? ? ? ? ?raw = QtGui.QApplication.clipboard().text(mode).rstrip()
> + ? ? ? ? ? ? ? ?text = raw.encode(self.encoding)
> ? ? ? ? ? ? except UnicodeEncodeError:
> ? ? ? ? ? ? ? ? pass
> ? ? ? ? ? ? else:
> @@ -1034,7 +1038,7 @@ class ConsoleWidget(Configurable, QtGui.QWidget):
> ? ? ? ? cursor.movePosition(QtGui.QTextCursor.StartOfBlock)
> ? ? ? ? cursor.movePosition(QtGui.QTextCursor.EndOfBlock,
> ? ? ? ? ? ? ? ? ? ? ? ? ? ? QtGui.QTextCursor.KeepAnchor)
> - ? ? ? ?return str(cursor.selection().toPlainText())
> + ? ? ? ?return unicode(cursor.selection().toPlainText()).encode(self.encoding)
>
> ? ? def _get_cursor(self):
> ? ? ? ? """ Convenience method that returns a cursor for the current position.
> ####
>
> By the way, this isn't an odd corner case: in other countries, people
> are likely to have files and directories with unicode in them *all the
> time*, so this problem will hit us immediately once the code is out,
> I'm afraid.
>
> I saw multiple calls of the form str(some.Qt.Code()) that were
> throwing exceptions and decided to stop before I get myself too deep
> into Qt code I don't know well. ?But the right approach is probably to
> encapsulate all those into a single common call that manages the
> encoding.
>
> The tricky part, I suspect, will be to do the cursor positioning logic
> with unicode in play: you need to correctly compute the lengths in
> terms of characters on the unicode string (more precisely, the number
> of glyphs that the code points map to), not bytes on the raw one.
>
> Welcome to the wonderful world of unicode!
>
> Cheers,
>
> f
>
> ps - and on py3k it's *only* unicode everywhere, so we might as well
> get this code right from the get go. ?Now that we have people starting
> to help towards py3, the last thing we should do is write a ton of new
> code that is unicode-unsafe for a py3 transition. ?We're not writing
> py3 code yet, but we should write *with an eye towards py3*.
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From epatters at enthought.com  Tue Sep  7 20:19:54 2010
From: epatters at enthought.com (Evan Patterson)
Date: Tue, 7 Sep 2010 17:19:54 -0700
Subject: [IPython-dev] Smart copy rocks, let's make it even better :)
In-Reply-To: <AANLkTi=JDXExmqoFqR2HKXnuhtHbe3Ds8C5OMHW9fxNt@mail.gmail.com>
References: <AANLkTi=JDXExmqoFqR2HKXnuhtHbe3Ds8C5OMHW9fxNt@mail.gmail.com>
Message-ID: <AANLkTinjZhF2mFf9jfRku+sJCQKAPGWAHP1AP=cdNOBK@mail.gmail.com>

On Tue, Sep 7, 2010 at 4:46 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey Evan,
>
> I'm loving smart copy, but I was thinking it would be even better to
> strip not just prompts, but all output, so that one can just highlight
> a bunch of prior input, C-C, C-V and then edit it as a new multiline
> cell without having to clean out output.
>
> We could then leave C-Shift-C or a menu entry for 'full copy' that
> would not do any stripping, for copy/pasting full sessions into an
> external document, for example.

This sounds reasonable, and it would not take much effort to implement
this. I'll put it on the list.

>
> With that improvement, the workflow begins to feel really fantastic,
> since it's possible to do things like:
>
> In [29]: f = qt.QFont('Consolas')
>
> In [30]: fi = qt.QFontInfo(f)
>
> In [31]: fi.family()
> Out[31]: PyQt4.QtCore.QString(u'Consolas')
>
> < Copy-paste here of the previous three lines, now I have a single
> block I can easily modify and re-execute>
>
> In [32]: f = qt.QFont('Consolas')
> ? ?...: fi = qt.QFontInfo(f)
> ? ?...: fi.family()
> Out[32]: PyQt4.QtCore.QString(u'Consolas')
>
> <re-execute changing one parameter>
>
> In [33]: f = qt.QFont('Consolata')
> ? ?...: fi = qt.QFontInfo(f)
> ? ?...: fi.family()
> Out[33]: PyQt4.QtCore.QString(u'DejaVu Sans')
>
>
Since you're playing with Consolas, I should note parenthetically that
I have made this the default font on Windows (when it is available).

> Since I made the run_cell semantics analyze the input and run the
> *last* statement in single mode, you get the benefits of a cell-based
> system with the fluidity of a console :)
>
> Cheers,
>
> ps - that and the up-arrow-filter-matches, and we'll be in heaven :)

I'm actually working on this feature right now. It will probably be
checked in sometime tomorrow morning.

Evan


From fperez.net at gmail.com  Tue Sep  7 20:29:17 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 17:29:17 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTinUjP==6YYtswREuOEeMhhZN7M=EkKiKLfr7Scv@mail.gmail.com>
References: <AANLkTimq7dZ6CzoAArUPg+eidYU5J64_1vYsriepmqiH@mail.gmail.com>
	<AANLkTinUjP==6YYtswREuOEeMhhZN7M=EkKiKLfr7Scv@mail.gmail.com>
Message-ID: <AANLkTikGdHB2Xj7bPCoPD4_6CNh_AUhHf0yp88i38Zvo@mail.gmail.com>

On Tue, Sep 7, 2010 at 5:16 PM, Evan Patterson <epatters at enthought.com> wrote:
> I appreciate the desire for Unicode support (although I firmly believe
> that source code itself should *always* be in ASCII). Unfortunately,
> you're correct that there may be fair amount of effort involved in
> supporting Unicode robustly.

I also like my source code in ascii, though we've been already
overridden on that one:

http://www.python.org/dev/peps/pep-3131/

In python3, unicode *source* is valid, so eventually we'll have to
deal with it, like it or not.  But the issue here is about unicode in
strings that we handle as input (paths, filenames, constants, etc),
not source code.  Though I realize that in the terminal, there's
always code intermixed with other input.  Fun...

By the way, this isn't really a new problem: there are bugs in the Qt
code re. unicode, but in the rest of ipython unicode is probably our
worst bug.  I even made a label for that:

http://github.com/ipython/ipython/issues/labels/unicode

Some of those bugs are very old, and we haven't been able to fix them...

> In short, I may not have time to get this done in the next week and a
> half. We should discuss this further off-line, though.

OK.  At the very least we should audit and mark the places in the code
with text manipulations that are unicode-unsafe.  Otherwise it will
never get fixed...

Cheers,

f


From fperez.net at gmail.com  Tue Sep  7 20:30:52 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 17:30:52 -0700
Subject: [IPython-dev] Smart copy rocks, let's make it even better :)
In-Reply-To: <AANLkTinjZhF2mFf9jfRku+sJCQKAPGWAHP1AP=cdNOBK@mail.gmail.com>
References: <AANLkTi=JDXExmqoFqR2HKXnuhtHbe3Ds8C5OMHW9fxNt@mail.gmail.com>
	<AANLkTinjZhF2mFf9jfRku+sJCQKAPGWAHP1AP=cdNOBK@mail.gmail.com>
Message-ID: <AANLkTikGz3b-ZQh=fqi_AAZz06wPFqh9V5C7g6ffQnx2@mail.gmail.com>

On Tue, Sep 7, 2010 at 5:19 PM, Evan Patterson <epatters at enthought.com> wrote:
> Since you're playing with Consolas, I should note parenthetically that
> I have made this the default font on Windows (when it is available).

I did.  My input was a not-so-subtle hit of a patch I'm making to
improve our font selection, I'll ping you on irc in a minute :)

>> Since I made the run_cell semantics analyze the input and run the
>> *last* statement in single mode, you get the benefits of a cell-based
>> system with the fluidity of a console :)
>>
>> Cheers,
>>
>> ps - that and the up-arrow-filter-matches, and we'll be in heaven :)
>
> I'm actually working on this feature right now. It will probably be
> checked in sometime tomorrow morning.

Fabulous, many thanks!

f


From fperez.net at gmail.com  Wed Sep  8 02:40:38 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 7 Sep 2010 23:40:38 -0700
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTingNQC1KVan1vAH64Nvd2FRjrpsnRoWSn8_qUXy@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
	<AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
	<AANLkTikvvv0a8qEJt9Pta-r5mnkOfjTe0v2Ct7sJb0P3@mail.gmail.com>
	<AANLkTingNQC1KVan1vAH64Nvd2FRjrpsnRoWSn8_qUXy@mail.gmail.com>
Message-ID: <AANLkTi=29+8BbSwkPpR-=8Haty1uyp_uV9BvX9sOZNkj@mail.gmail.com>

On Tue, Sep 7, 2010 at 12:36 AM, Almar Klein <almar.klein at gmail.com> wrote:
> In general, this "busy call" is only done once per application right? So not
> much harm done IMO.

Oh, my concern isn't performance, but the creation of extra code for
no necessary reason.  More code is always more code to maintain :)

> But well, if properly documented, I can agree with your suggestion.

Cheers,

f


From fperez.net at gmail.com  Wed Sep  8 03:02:30 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Sep 2010 00:02:30 -0700
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
Message-ID: <AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>

Hi Almar,

sorry for the delayed reply, I hunkered down on the code and dropped
everything else for the weekend :)

On Fri, Sep 3, 2010 at 1:30 AM, Almar Klein <almar.klein at gmail.com> wrote:
> I've been thinking about your protocol last night. (arg! This whole thing is
> getting to my head too much!)

I think I know the feeling :)

> I see some advantages of loose messages such as your using in favor of my
> channels approach. Specifically, the order of stdout and stderr messages is
> not preserved, as they are send over a different channel (same socket
> though). On a 1to1 system, this is not much of a problem, but for multiple
> users it definitely will be, specifically if you also want to show the other
> users input (pyin).

Yes, and those notions of collaborative, distributed work are very
much central to our ideas.

> Anyway, I took another good look at your approach and I have a few
> remarks/ideas. Might be useful, might be total BS.

Hans already made some very good comments pretty much 100% in line
with what I would have said, so I won't repeat his.

> * You say the prompt is a thing the client decides what it looks like. I
> disagree. The kernel gives a prompt to indicate that the user can give
> input, and maybe also what kind of input. A debugger might produce a
> different prompt to indicate that the user is in debug mode. See also the
> doc on sys.ps1 and sys.ps2 (http://docs.python.org/library/sys.html): the
> user (or interpreter) can put an object on it that evaluates to something
> nice when str()'nged.

Following up on Hans' comments, yes: a history synchronization
mechanism is available (or at least the basic info is there).  The
client keeps its history local for fast things like 'up arrow', but
when you type '%history', that's executed in the kernel.  A client who
joins a kernel can always request a full history to bring its local
copy up to date.

> * You plan on keeping history in the kernel. In this case I think this is
> the task of the client. Otherwise you'd get your own history mixed with that
> of someone else using that kernel? History is, I think, a feature of the
> client to help the programmer. I see no use for storing it at the kernel.
>
> * I really think you can do with less sockets. I believe that the (black)
> req/rep pair is not needed. You only seem to use it for when raw_input is
> used. But why? When raw_input is used, you can just block and wait for some
> stdin (I think that'll be the execute_request message). This should not be
> too hard by replacing sys.stdin with an object that has a readline method
> that does this. If two users are present, and one calls raw_input, they can
> both provide input (whoever's first). To indicate this to the *other* user,
> however, his prompt should be replaced with an empty string, so his cursor
> is positioned right after the <text> in raw_input('<text>').

Keep in mind that the direction of those sockets (the normal xreq/xrep
pair for client input and the req/rep for kernel stdin) is opposite,
and that's because they represent fundamentally different operations.

I'm not worried about 'too many sockets', I would worry about having
more sockets, or *less* sockets, than we have separate, independent
concepts and streams of information.  It seems that now, we do have
one socket pair for each type of information flow, and this way we
only multiplex when the data may vary but the communication semantics
are constant (such as having multiple streams in the pub/sub).  I
think this actually buys us a lot of simplicity, because each
connection has precisely the socket and semantics of the type of
information flow it needs to transfer.

> * I think you can do with even less sockets :)? But this is more of a wild
> idea. Say that John set up an experiment at work and wants to check the
> results in the bar on his Android (sorry I stole your example here,
> Fernando). Now his experiment crashed, producing a traceback in the client
> at his work PC. But now he cannot see the traceback as he just logged in!
> -----? So what about storing all stdout, stderr and pyin (basically all
> "terminal-output") at the kernel? And rather than pub/sub, use the existing
> req/rep to obtain this stuff. Maybe you can even pass new terminal-output
> along with other replies. The client should indicate in the request a
> sequence number to indicate to the kernel what messages were already
> received. This does mean, however, that the client would have to
> periodically query the kernel. But maybe this can also be done automatically
> by writing a thin layer on top of the zmq interface. Oh, and you'd need to
> encapsulate multiple terminal-messages in a single reply.

Rather than forcing the kernel to store all that info and play back
multiple data streams, I'd separate this (valid) idea into its own
entity.  Someone could easily write a client whose job is simply to
listen to a kernel and store all streams of information, nicely
organized for later replay as needed.  Because this client would only
be storing the string data of the messages, there's no danger of
making the kernel leak memory by asking it to hold on to every object
it has ever produced (a very dangerous proposition).  And such a
logger could then save that info, replay it for you, make it available
over the web, selectively SMS you if a message on a certain stream
matches something, etc.

For example, with this design it becomes *trivial* to write a little
program that subscribes only to the pyerr channel, monitors
exceptions, and sends you an SMS if an exception of a particular type
you care about arrives.  Something like that could probably be written
in 2 hours.  And that, thanks to the clear separation of information
flow semantics across the various channels, that make it very easy to
grab/focus only on the data you need.

It's to some extent trying to carry the ideas from Unix software:
rather than making one single object (a super socket) that does 20
different things, create a few simple ones that do each one job well,
with relative decoupling and isolation from each other, and then use
the parts you need to build what you want.

So far I have to say it's working fantastically well, though we could
always be proven wrong by some horrible bottleneck we haven't yet
foreseen :)

But thanks for your feedback and ideas: only if we can explain and
clarify our thoughts sufficiently to justify them, can we be sure that
we actually understand what we're doing.

Regards,

f


From fperez.net at gmail.com  Wed Sep  8 03:04:54 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Sep 2010 00:04:54 -0700
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTinvK3Tpw6DDF8H1M28+MBF4dhmNNSnEdyTy=UDc@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<201009031118.38731.hans_meine@gmx.net>
	<AANLkTinvK3Tpw6DDF8H1M28+MBF4dhmNNSnEdyTy=UDc@mail.gmail.com>
Message-ID: <AANLkTikuOhm0r3fxiaDy6ADPpHN1V5smyNdmUQxz4gr=@mail.gmail.com>

On Fri, Sep 3, 2010 at 3:12 AM, Almar Klein <almar.klein at gmail.com> wrote:
> Well,? the kernel could be implemented in such a way that messages are
> received in a separate thread, which may also handle the introspection
> requests. Introspection (+history) will then still work if the process is
> busy (unless it's running extension code).

Except you want introspection to act on valid information, so you want
to either wait to have it evaluated on the real status of the kernel
(so f? doesn't tell you f is an integer when now it has become  a
function), or have it timeout and not give you anything.

Cheers,

f


From takowl at gmail.com  Wed Sep  8 05:24:04 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 8 Sep 2010 10:24:04 +0100
Subject: [IPython-dev] Pasting fix, unicode woes
Message-ID: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>

On 8 September 2010 07:41, <ipython-dev-request at scipy.org> wrote:

> In python3, unicode *source* is valid, so eventually we'll have to
> deal with it, like it or not.
>

For information, in making it run on python3, I stripped out one piece of
code that checks if identifiers are pure ascii:
http://github.com/takowl/ipython/commit/0636636f5da692f198f3d4cbf3f04fc4e2b131bf#diff-2

This works, at least in my simple tests. Only script characters (not symbols
and punctuation) can be used as identifiers, but I imagine that this is by
design.

I've just tested, and the python3 version can handle unicode filenames with
zero fuss (including tab completion), though I think this is down to Python
3 itself rather than anything I can take credit for.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100908/dbe7e1f0/attachment.html>

From fperez.net at gmail.com  Wed Sep  8 15:24:42 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Sep 2010 12:24:42 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
Message-ID: <AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>

Hi Thomas,

On Wed, Sep 8, 2010 at 2:24 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> For information, in making it run on python3, I stripped out one piece of
> code that checks if identifiers are pure ascii:
> http://github.com/takowl/ipython/commit/0636636f5da692f198f3d4cbf3f04fc4e2b131bf#diff-2

yes, though perhaps that one shouldn't be deleted but instead only
turned off in an 'if py3' conditional.  Because in py2x, we do still
have the ascii-only restriction for identifiers.  So it would be good
to leave the code be 2x and 3x correct, even if it costs us a few ugly
'if py2' and 'if py3' statements in a few places (I suspect that will
be inevitable, we'll just have to work on making them as cleanly
organized and isolated as possible).

> This works, at least in my simple tests. Only script characters (not symbols
> and punctuation) can be used as identifiers, but I imagine that this is by
> design.
>
> I've just tested, and the python3 version can handle unicode filenames with
> zero fuss (including tab completion), though I think this is down to Python
> 3 itself rather than anything I can take credit for.

Great news to hear!

BTW, Evan and I just pushed some changes to newkernel, so you can stay
up to date.

Best regards, and many thanks again for taking the lead on the py3
problem for us!

f


From fperez.net at gmail.com  Wed Sep  8 22:40:14 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 8 Sep 2010 19:40:14 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
	<AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>
	<AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
Message-ID: <AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>

Hi Thomas,

On Wed, Sep 8, 2010 at 3:26 PM, Thomas Kluyver <takowl at gmail.com> wrote:
> Is using conditional code a path we want to take? I'd intended the code to
> form a separate 'py3 only' branch, which could be kept up to date with
> changes in the main branch. It would take quite a lot of conditional code to
> support 2.x and 3.x on a single version.

That's a good question, to which I honestly don't have an answer yet
for lack of experience.  I haven't done any more than very cursory
playing with py3, so my intuition here fails me.

The issue is balancing:

- the nastiness of a codebase chock full of conditionals

- the load of maintaining two codebases without them diverging.

I really don't know which way the balance will tilt between these
competing forces; I'm more than happy to hear from you or others on
this matter.

For now you're doing the bulk of the work so I'm happy to follow your
lead, as your hands-on experience will inform your judgment much
better than any thought experiment I can cook up.

If you think that syncing a py3 branch with semi-automatic conversion
is feasible, then that's fine with me.  It would let us develop on a
single, clean py2-compatible codebase at least for a while longer.
We'd certainly be mindful of evolving that branch as closely as
possible in the py3 direction over time.

> I've started a 2.x branch updating bits of the code to what is supported in
> 2.6, as you suggested:
> http://github.com/takowl/ipython/tree/ipy3-preparation
> I'll update this from newkernel, and try to synchronise ipy3-newkernel.

Great.  Keep us posted, and when you think you are ready to start
considering changes from -prep back upstream, we'll look at them.

I hope we'll be merging newkernel into ipython/trunk soon enough (next
week, hopefully) so that from now on you could be following just
trunk.  The mad sprint on newkernel is calming down and it looks like
we pulled it off without burning down the house, so we should be able
to move back to a model with a stable trunk and small, localized
feature branches that you can ignore until their content finds its way
into trunk.

Regards,

f


From takowl at gmail.com  Thu Sep  9 05:40:17 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 9 Sep 2010 10:40:17 +0100
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
	<AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>
	<AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
	<AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>
Message-ID: <AANLkTi=qALBjUt49fCYdsHMC93R2EEfcdnX5aYrTE2aD@mail.gmail.com>

On 9 September 2010 03:40, Fernando Perez <fperez.net at gmail.com> wrote:

> The issue is balancing:
>
> - the nastiness of a codebase chock full of conditionals
>
> - the load of maintaining two codebases without them diverging.
>
> I really don't know which way the balance will tilt between these
> competing forces; I'm more than happy to hear from you or others on
> this matter.
>

I've just looked at the changes made by 2to3, and it's actually closer than
I had imagined, although I still don't think it's quite close enough to try
to have a single codebase. We could support several of the changes with
little effort, e.g. using print_function from __future__. Unicode strings
would be the big problem: there'd either be conditional code scattered
throughout, or confusing abstractions to hide the difference. We'd also need
to do something with the code that handles old-style classes (I removed
references to ClassType and InstanceType from the py3 branch).

To sum up: I think it's easiest to use the magic of DVCSs and 2to3 to keep
separate branches.


>  Great.  Keep us posted, and when you think you are ready to start
> considering changes from -prep back upstream, we'll look at them.
>

I've done all the changes I've got planned for now (I essentially looked
through the changes I'd had to apply after running 2to3, and reapplied those
that work in 2.6). Is there a standard set of tests I should run to confirm
I haven't broken anything?

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/6af30a3d/attachment.html>

From almar.klein at gmail.com  Thu Sep  9 06:42:00 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Thu, 9 Sep 2010 12:42:00 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
Message-ID: <AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>

Hi,

> But thanks for your feedback and ideas: only if we can explain and
> clarify our thoughts sufficiently to justify them, can we be sure that
> we actually understand what we're doing.

Hehe, I can imagine you (or others reading this thread) start to think I'm
stubborn. Well I'm a bit of a purist at times and I keep to my opinion
unless I'm convinced by good arguments :)  But hey, its your project, so
please let me know if you've had enough of my criticism.

So here's a little more ...

> * I really think you can do with less sockets. I believe that the (black)
> > req/rep pair is not needed. You only seem to use it for when raw_input is
> > used. But why? When raw_input is used, you can just block and wait for
> some
> > stdin (I think that'll be the execute_request message). This should not
> be
> > too hard by replacing sys.stdin with an object that has a readline method
> > that does this. If two users are present, and one calls raw_input, they
> can
> > both provide input (whoever's first). To indicate this to the *other*
> user,
> > however, his prompt should be replaced with an empty string, so his
> cursor
> > is positioned right after the <text> in raw_input('<text>').
>
> Keep in mind that the direction of those sockets (the normal xreq/xrep
> pair for client input and the req/rep for kernel stdin) is opposite,
> and that's because they represent fundamentally different operations.
>

I get that, but I'm not sure whether this is correct/necessary for the
raw_input. In the original Python interpreter, raw_input just reads from
stdin, the same stream that's used for executing commands. The interpreter
just waits for the next "command", which is then interpreted as text, rather
than executing it. In a shell, this idea works quite well.


I'm not worried about 'too many sockets', I would worry about having
> more sockets, or *less* sockets, than we have separate, independent
> concepts and streams of information.  It seems that now, we do have
> one socket pair for each type of information flow, and this way we
> only multiplex when the data may vary but the communication semantics
> are constant (such as having multiple streams in the pub/sub).  I
> think this actually buys us a lot of simplicity, because each
> connection has precisely the socket and semantics of the type of
> information flow it needs to transfer.
>

So in your words, I believe that raw_input should actually be the same data
stream as the commands. Sure, because you now have a GUI at the other end,
you could create a pop-up dialog for the user to input the requested text.
But is that really better/nicer? It can also be a bit intrusive.

However, my main argument against this approach is what I already mentioned
in the previous e-mail: whatever client executes raw_input, the data should
be input at the "main client" (the client having the particular socket
connection). What if that happens to be the PC at work, while you're in the
bar?

Concluding: I see no reason why commands can be executed by all clients,
while a response to raw_input can only be given from one.



<snip stupid idea to get stdout stderr etc via a req/rep pattern>
>


> Rather than forcing the kernel to store all that info and play back
> multiple data streams, I'd separate this (valid) idea into its own
> entity.  Someone could easily write a client whose job is simply to
> listen to a kernel and store all streams of information, nicely
> organized for later replay as needed.  Because this client would only
> be storing the string data of the messages, there's no danger of
> making the kernel leak memory by asking it to hold on to every object
> it has ever produced (a very dangerous proposition).  And such a
> logger could then save that info, replay it for you, make it available
> over the web, selectively SMS you if a message on a certain stream
> matches something, etc.
>
> For example, with this design it becomes *trivial* to write a little
> program that subscribes only to the pyerr channel, monitors
> exceptions, and sends you an SMS if an exception of a particular type
> you care about arrives.  Something like that could probably be written
> in 2 hours.  And that, thanks to the clear separation of information
> flow semantics across the various channels, that make it very easy to
> grab/focus only on the data you need.
>

You're right. That seems very natural. However, I'm still a little concerned
about when a user connects to a kernel that's running extension code and is
therefore unresponsive. This would immediately become clear to him/her if he
saw the preceding stdout/stderr.

What about storing these messages (to be send from the PUB socket) at the
kernel? I mean store the string messages, not the objects used to create
them (i.e. no holding onto large objects). You could then send this history
when a client connects. Something like "Hi there, this is what we've been
doing so far.".

Cheers,
  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/54c1576a/attachment.html>

From robert.kern at gmail.com  Thu Sep  9 10:58:55 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Sep 2010 09:58:55 -0500
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
Message-ID: <i6asng$2oh$1@dough.gmane.org>

On 9/9/10 5:42 AM, Almar Klein wrote:
> Hi,
>
>  > But thanks for your feedback and ideas: only if we can explain and
>  > clarify our thoughts sufficiently to justify them, can we be sure that
>  > we actually understand what we're doing.
>
> Hehe, I can imagine you (or others reading this thread) start to think I'm
> stubborn. Well I'm a bit of a purist at times and I keep to my opinion unless
> I'm convinced by good arguments :)  But hey, its your project, so please let me
> know if you've had enough of my criticism.
>
> So here's a little more ...
>
>      > * I really think you can do with less sockets. I believe that the (black)
>      > req/rep pair is not needed. You only seem to use it for when raw_input is
>      > used. But why? When raw_input is used, you can just block and wait for some
>      > stdin (I think that'll be the execute_request message). This should not be
>      > too hard by replacing sys.stdin with an object that has a readline method
>      > that does this. If two users are present, and one calls raw_input, they can
>      > both provide input (whoever's first). To indicate this to the *other* user,
>      > however, his prompt should be replaced with an empty string, so his cursor
>      > is positioned right after the <text> in raw_input('<text>').
>
>     Keep in mind that the direction of those sockets (the normal xreq/xrep
>     pair for client input and the req/rep for kernel stdin) is opposite,
>     and that's because they represent fundamentally different operations.
>
>
> I get that, but I'm not sure whether this is correct/necessary for the
> raw_input. In the original Python interpreter, raw_input just reads from stdin,
> the same stream that's used for executing commands. The interpreter just waits
> for the next "command", which is then interpreted as text, rather than executing
> it. In a shell, this idea works quite well.

Sending a command to the interpreter and sending input to raw_input() are 
conceptually two different things. By keeping the interfaces for them separate, 
we allow a greater flexibility to present different things in different ways. 
Just because the original interpreter implementation conflated them out of 
necessity due to the limitations of a terminal environment doesn't mean that is 
the best thing to do. That would lock us down to the limitations of the original 
implementation.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ben.root at ou.edu  Thu Sep  9 13:38:12 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Thu, 9 Sep 2010 12:38:12 -0500
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
Message-ID: <AANLkTikMdFiTTTvtS4S+-KFnoX6W-R9rYrCdYydT6u7S@mail.gmail.com>

On Thu, Sep 9, 2010 at 5:42 AM, Almar Klein <almar.klein at gmail.com> wrote:
<snip>

> You're right. That seems very natural. However, I'm still a little
> concerned about when a user connects to a kernel that's running extension
> code and is therefore unresponsive. This would immediately become clear to
> him/her if he saw the preceding stdout/stderr.
>
> What about storing these messages (to be send from the PUB socket) at the
> kernel? I mean store the string messages, not the objects used to create
> them (i.e. no holding onto large objects). You could then send this history
> when a client connects. Something like "Hi there, this is what we've been
> doing so far.".
>
> Cheers,
>   Almar
>
>
Just piping up on this thought... I picture something similar to IRC in this
respect.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/1599b0ce/attachment.html>

From almar.klein at gmail.com  Thu Sep  9 15:48:59 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Thu, 9 Sep 2010 21:48:59 +0200
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <i6asng$2oh$1@dough.gmane.org>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
	<i6asng$2oh$1@dough.gmane.org>
Message-ID: <AANLkTim28mg5dL84idr9oJx9hdz0yYv=CJOAN878sdpC@mail.gmail.com>

On 9 September 2010 16:58, Robert Kern <robert.kern at gmail.com> wrote:

> On 9/9/10 5:42 AM, Almar Klein wrote:
> > Hi,
> >
> >  > But thanks for your feedback and ideas: only if we can explain and
> >  > clarify our thoughts sufficiently to justify them, can we be sure that
> >  > we actually understand what we're doing.
> >
> > Hehe, I can imagine you (or others reading this thread) start to think
> I'm
> > stubborn. Well I'm a bit of a purist at times and I keep to my opinion
> unless
> > I'm convinced by good arguments :)  But hey, its your project, so please
> let me
> > know if you've had enough of my criticism.
> >
> > So here's a little more ...
> >
> >      > * I really think you can do with less sockets. I believe that the
> (black)
> >      > req/rep pair is not needed. You only seem to use it for when
> raw_input is
> >      > used. But why? When raw_input is used, you can just block and wait
> for some
> >      > stdin (I think that'll be the execute_request message). This
> should not be
> >      > too hard by replacing sys.stdin with an object that has a readline
> method
> >      > that does this. If two users are present, and one calls raw_input,
> they can
> >      > both provide input (whoever's first). To indicate this to the
> *other* user,
> >      > however, his prompt should be replaced with an empty string, so
> his cursor
> >      > is positioned right after the <text> in raw_input('<text>').
> >
> >     Keep in mind that the direction of those sockets (the normal
> xreq/xrep
> >     pair for client input and the req/rep for kernel stdin) is opposite,
> >     and that's because they represent fundamentally different operations.
> >
> >
> > I get that, but I'm not sure whether this is correct/necessary for the
> > raw_input. In the original Python interpreter, raw_input just reads from
> stdin,
> > the same stream that's used for executing commands. The interpreter just
> waits
> > for the next "command", which is then interpreted as text, rather than
> executing
> > it. In a shell, this idea works quite well.
>
> Sending a command to the interpreter and sending input to raw_input() are
> conceptually two different things. By keeping the interfaces for them
> separate,
> we allow a greater flexibility to present different things in different
> ways.
> Just because the original interpreter implementation conflated them out of
> necessity due to the limitations of a terminal environment doesn't mean
> that is
> the best thing to do. That would lock us down to the limitations of the
> original
> implementation.
>

Fair enough. But what about my second argument: why can commands be executed
by all clients, but a response to raw_input only from one?

  Almar
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/a744e504/attachment.html>

From fperez.net at gmail.com  Thu Sep  9 16:12:55 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Sep 2010 13:12:55 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTi=qALBjUt49fCYdsHMC93R2EEfcdnX5aYrTE2aD@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
	<AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>
	<AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
	<AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>
	<AANLkTi=qALBjUt49fCYdsHMC93R2EEfcdnX5aYrTE2aD@mail.gmail.com>
Message-ID: <AANLkTi=g3Z6UX6pB94Jp63mNwdhq+_uazmWi83wmoc7N@mail.gmail.com>

Hi Thomas,

On Thu, Sep 9, 2010 at 2:40 AM, Thomas Kluyver <takowl at gmail.com> wrote:
> I've just looked at the changes made by 2to3, and it's actually closer than
> I had imagined, although I still don't think it's quite close enough to try
> to have a single codebase. We could support several of the changes with
> little effort, e.g. using print_function from __future__. Unicode strings
> would be the big problem: there'd either be conditional code scattered
> throughout, or confusing abstractions to hide the difference. We'd also need
> to do something with the code that handles old-style classes (I removed
> references to ClassType and InstanceType from the py3 branch).
>
> To sum up: I think it's easiest to use the magic of DVCSs and 2to3 to keep
> separate branches.

Sounds good.   We've been doing print_function in all new code and
adding it to existing files as time allows (basically I do it if the
file isn't too huge as I work on something, I've left the really big
ones alone for now).  And hopefully we'll be able to get rid of
old-style classes altogether at some point.

>>
>> Great. ?Keep us posted, and when you think you are ready to start
>> considering changes from -prep back upstream, we'll look at them.
>
> I've done all the changes I've got planned for now (I essentially looked
> through the changes I'd had to apply after running 2to3, and reapplied those
> that work in 2.6). Is there a standard set of tests I should run to confirm
> I haven't broken anything?

Yes, once you install ipython,  a script called 'iptest' is installed
that runs the entire test suite and tries to be smart about not
running things for unmet dependencies.  Though it's quite likely that
the dependency checking may have gone a bit stale, and that it will
attempt to run tests it shouldn't, that would need to be cleaned up
first.

Cheers,

f


From robert.kern at gmail.com  Thu Sep  9 16:46:50 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Sep 2010 15:46:50 -0500
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTim28mg5dL84idr9oJx9hdz0yYv=CJOAN878sdpC@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>	<i6asng$2oh$1@dough.gmane.org>
	<AANLkTim28mg5dL84idr9oJx9hdz0yYv=CJOAN878sdpC@mail.gmail.com>
Message-ID: <i6bh3s$9d3$1@dough.gmane.org>

On 9/9/10 2:48 PM, Almar Klein wrote:
>
>
> On 9 September 2010 16:58, Robert Kern <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     On 9/9/10 5:42 AM, Almar Klein wrote:
>      > Hi,
>      >
>      > > But thanks for your feedback and ideas: only if we can explain and
>      > > clarify our thoughts sufficiently to justify them, can we be sure that
>      > > we actually understand what we're doing.
>      >
>      > Hehe, I can imagine you (or others reading this thread) start to think I'm
>      > stubborn. Well I'm a bit of a purist at times and I keep to my opinion unless
>      > I'm convinced by good arguments :)  But hey, its your project, so please
>     let me
>      > know if you've had enough of my criticism.
>      >
>      > So here's a little more ...
>      >
>      > > * I really think you can do with less sockets. I believe that the (black)
>      > > req/rep pair is not needed. You only seem to use it for when raw_input is
>      > > used. But why? When raw_input is used, you can just block and wait for some
>      > > stdin (I think that'll be the execute_request message). This should not be
>      > > too hard by replacing sys.stdin with an object that has a readline method
>      > > that does this. If two users are present, and one calls raw_input, they can
>      > > both provide input (whoever's first). To indicate this to the *other* user,
>      > > however, his prompt should be replaced with an empty string, so his cursor
>      > > is positioned right after the <text> in raw_input('<text>').
>      >
>      >     Keep in mind that the direction of those sockets (the normal xreq/xrep
>      >     pair for client input and the req/rep for kernel stdin) is opposite,
>      >     and that's because they represent fundamentally different operations.
>      >
>      >
>      > I get that, but I'm not sure whether this is correct/necessary for the
>      > raw_input. In the original Python interpreter, raw_input just reads from
>     stdin,
>      > the same stream that's used for executing commands. The interpreter just
>     waits
>      > for the next "command", which is then interpreted as text, rather than
>     executing
>      > it. In a shell, this idea works quite well.
>
>     Sending a command to the interpreter and sending input to raw_input() are
>     conceptually two different things. By keeping the interfaces for them separate,
>     we allow a greater flexibility to present different things in different ways.
>     Just because the original interpreter implementation conflated them out of
>     necessity due to the limitations of a terminal environment doesn't mean that is
>     the best thing to do. That would lock us down to the limitations of the original
>     implementation.
>
>
> Fair enough. But what about my second argument: why can commands be executed by
> all clients, but a response to raw_input only from one?

That may be a reason to turn that into an XREQ socket on the kernel and XREP 
socket on the frontends, submitting one XREQ message to each frontend, and 
letting the first response "win", but it doesn't let you reuse the 
kernel-XREP/frontend-XREQ pair of sockets for this purpose.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From benjaminrk at gmail.com  Thu Sep  9 18:47:43 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 9 Sep 2010 15:47:43 -0700
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <i6bh3s$9d3$1@dough.gmane.org>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>
	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>
	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>
	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>
	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>
	<i6asng$2oh$1@dough.gmane.org>
	<AANLkTim28mg5dL84idr9oJx9hdz0yYv=CJOAN878sdpC@mail.gmail.com>
	<i6bh3s$9d3$1@dough.gmane.org>
Message-ID: <AANLkTimN79iP9U7ZyYObo8vmjeccogsDY2vGqj93iBfH@mail.gmail.com>

On Thu, Sep 9, 2010 at 13:46, Robert Kern <robert.kern at gmail.com> wrote:

> On 9/9/10 2:48 PM, Almar Klein wrote:
> >
> >
> > On 9 September 2010 16:58, Robert Kern <robert.kern at gmail.com
> > <mailto:robert.kern at gmail.com>> wrote:
> >
> >     On 9/9/10 5:42 AM, Almar Klein wrote:
> >      > Hi,
> >      >
> >      > > But thanks for your feedback and ideas: only if we can explain
> and
> >      > > clarify our thoughts sufficiently to justify them, can we be
> sure that
> >      > > we actually understand what we're doing.
> >      >
> >      > Hehe, I can imagine you (or others reading this thread) start to
> think I'm
> >      > stubborn. Well I'm a bit of a purist at times and I keep to my
> opinion unless
> >      > I'm convinced by good arguments :)  But hey, its your project, so
> please
> >     let me
> >      > know if you've had enough of my criticism.
> >      >
> >      > So here's a little more ...
> >      >
> >      > > * I really think you can do with less sockets. I believe that
> the (black)
> >      > > req/rep pair is not needed. You only seem to use it for when
> raw_input is
> >      > > used. But why? When raw_input is used, you can just block and
> wait for some
> >      > > stdin (I think that'll be the execute_request message). This
> should not be
> >      > > too hard by replacing sys.stdin with an object that has a
> readline method
> >      > > that does this. If two users are present, and one calls
> raw_input, they can
> >      > > both provide input (whoever's first). To indicate this to the
> *other* user,
> >      > > however, his prompt should be replaced with an empty string, so
> his cursor
> >      > > is positioned right after the <text> in raw_input('<text>').
> >      >
> >      >     Keep in mind that the direction of those sockets (the normal
> xreq/xrep
> >      >     pair for client input and the req/rep for kernel stdin) is
> opposite,
> >      >     and that's because they represent fundamentally different
> operations.
> >      >
> >      >
> >      > I get that, but I'm not sure whether this is correct/necessary for
> the
> >      > raw_input. In the original Python interpreter, raw_input just
> reads from
> >     stdin,
> >      > the same stream that's used for executing commands. The
> interpreter just
> >     waits
> >      > for the next "command", which is then interpreted as text, rather
> than
> >     executing
> >      > it. In a shell, this idea works quite well.
> >
> >     Sending a command to the interpreter and sending input to raw_input()
> are
> >     conceptually two different things. By keeping the interfaces for them
> separate,
> >     we allow a greater flexibility to present different things in
> different ways.
> >     Just because the original interpreter implementation conflated them
> out of
> >     necessity due to the limitations of a terminal environment doesn't
> mean that is
> >     the best thing to do. That would lock us down to the limitations of
> the original
> >     implementation.
> >
> >
> > Fair enough. But what about my second argument: why can commands be
> executed by
> > all clients, but a response to raw_input only from one?
>
> That may be a reason to turn that into an XREQ socket on the kernel and
> XREP
> socket on the frontends, submitting one XREQ message to each frontend, and
> letting the first response "win", but it doesn't let you reuse the
> kernel-XREP/frontend-XREQ pair of sockets for this purpose.
>

It probably shouldn't be XREQ on the Kernel, because then raw_input requests
would be load balanced across clients, which doesn't really make sense. If
it's XREP on the Kernel, then the kernel could explicitly request raw_input
from the client who caused the prompt.

In fact, the broken load-balanced case is the current behavior (even on
REQ/REP sockets) when multiple frontends are connected, so this should be
listed as a bug, since as it stands, raw_input is broken for multiple
clients.

Using an XREP socket as a keyed requester rather than replier does require a
certain amount of zmq.IDENTITY bookkeeping, as I well know from using them
in exactly this way in the Parallel IPython code, but I think it's the right
approach here, unless you really want to solicit raw_input from everybody
(note that most clients will have no information about what is causing the
raw_input call they just received until *after* entering the input), which
would more appropriately require a two-socket PUB/SUB model.

-MinRK


>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma
>  that is made terrible by our own mad attempt to interpret it as though it
> had
>  an underlying truth."
>   -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/1e275ad9/attachment.html>

From benjaminrk at gmail.com  Thu Sep  9 18:58:25 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 9 Sep 2010 15:58:25 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should not
 require specifying 4 ports
Message-ID: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>

Hello,

In order to connect a second ipythonqt frontend to an existing kernel, I
must specify by hand all 4 ports at the command-line.  This really shouldn't
be the case, especially since the default behavior is to have the ports
ordered sequentially.

I think it should at least be able to try using consecutive ports when a
single port is given, or use a two-stage connection model that doesn't
require clients to ever know more than one port, as is done in the parallel
code.

Having to type 'ipythonqt -e --xreq 65273 --sub 65274 --rep 65275 --hb
65276', as I just had to, just doesn't make sense.

-MinRK
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/fc137d0e/attachment.html>

From robert.kern at gmail.com  Thu Sep  9 19:06:46 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Sep 2010 18:06:46 -0500
Subject: [IPython-dev] Kernel-client communication
In-Reply-To: <AANLkTimN79iP9U7ZyYObo8vmjeccogsDY2vGqj93iBfH@mail.gmail.com>
References: <AANLkTi=ohFUEp+xcrRXa564h7Z7w=i2qDQro554UeeOs@mail.gmail.com>	<AANLkTi=+3i7_P7eW1n911EpcYcXoYHL+s5wXHR5CuPor@mail.gmail.com>	<AANLkTikWz=Zda-J1mPAfJiih=eF1LhBaJGin_WCWzdDV@mail.gmail.com>	<AANLkTimqoVoC54vVra3ghfTkWuQh2-GDGYt3oXHT8xfR@mail.gmail.com>	<AANLkTi=-D5iD9f-mNN014AbxkCWHz-b05s+OH+ZsYva-@mail.gmail.com>	<i6asng$2oh$1@dough.gmane.org>	<AANLkTim28mg5dL84idr9oJx9hdz0yYv=CJOAN878sdpC@mail.gmail.com>	<i6bh3s$9d3$1@dough.gmane.org>
	<AANLkTimN79iP9U7ZyYObo8vmjeccogsDY2vGqj93iBfH@mail.gmail.com>
Message-ID: <i6bpa6$dah$1@dough.gmane.org>

On 9/9/10 5:47 PM, MinRK wrote:
>
>
> On Thu, Sep 9, 2010 at 13:46, Robert Kern <robert.kern at gmail.com
> <mailto:robert.kern at gmail.com>> wrote:
>
>     On 9/9/10 2:48 PM, Almar Klein wrote:
>      > Fair enough. But what about my second argument: why can commands be
>     executed by
>      > all clients, but a response to raw_input only from one?
>
>     That may be a reason to turn that into an XREQ socket on the kernel and XREP
>     socket on the frontends, submitting one XREQ message to each frontend, and
>     letting the first response "win", but it doesn't let you reuse the
>     kernel-XREP/frontend-XREQ pair of sockets for this purpose.
>
>
> It probably shouldn't be XREQ on the Kernel, because then raw_input requests
> would be load balanced across clients, which doesn't really make sense. If it's
> XREP on the Kernel, then the kernel could explicitly request raw_input from the
> client who caused the prompt.

Ah, right. I was assuming you could send XREQs with a targeted IDENTITY, but I 
guess you do have to reverse the XREQ/XREP relationship to do that.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From takowl at gmail.com  Thu Sep  9 19:12:54 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 10 Sep 2010 00:12:54 +0100
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTi=g3Z6UX6pB94Jp63mNwdhq+_uazmWi83wmoc7N@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
	<AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>
	<AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
	<AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>
	<AANLkTi=qALBjUt49fCYdsHMC93R2EEfcdnX5aYrTE2aD@mail.gmail.com>
	<AANLkTi=g3Z6UX6pB94Jp63mNwdhq+_uazmWi83wmoc7N@mail.gmail.com>
Message-ID: <AANLkTimLnwcuO5Ot+LRVDyTJFqp1KGfh+WOua7SdN7TD@mail.gmail.com>

On 9 September 2010 21:12, Fernando Perez <fperez.net at gmail.com> wrote:

> And hopefully we'll be able to get rid of
> old-style classes altogether at some point.
>

Can't stop the user typing them in, unfortunately ;-).

 Yes, once you install ipython,  a script called 'iptest' is installed
> that runs the entire test suite and tries to be smart about not
> running things for unmet dependencies.
>

Can this be run without installing? I'm working with ipython in a standard
directory, so as not to confuse versions. I did find you could get it via
"from IPython.testing import iptest", but then iptest.run_iptest() hangs in
the middle of testing (unless this is just a very slow test).

Thanks,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100910/e8920632/attachment.html>

From robert.kern at gmail.com  Thu Sep  9 19:10:21 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 09 Sep 2010 18:10:21 -0500
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
Message-ID: <i6bpgv$fdl$1@dough.gmane.org>

On 9/9/10 5:58 PM, MinRK wrote:
> Hello,
>
> In order to connect a second ipythonqt frontend to an existing kernel, I must
> specify by hand all 4 ports at the command-line.  This really shouldn't be the
> case, especially since the default behavior is to have the ports ordered
> sequentially.
>
> I think it should at least be able to try using consecutive ports when a single
> port is given, or use a two-stage connection model that doesn't require clients
> to ever know more than one port, as is done in the parallel code.
>
> Having to type 'ipythonqt -e --xreq 65273 --sub 65274 --rep 65275 --hb 65276',
> as I just had to, just doesn't make sense.

I like the two-stage connection model; you open and advertise one REQ/REP port 
for configuring clients with the other ports.

Hopefully, 0MQ will grow the ability to share multiple named sockets on a single 
TCP/IP port, but they've only briefly discussed it so far.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From fperez.net at gmail.com  Thu Sep  9 19:22:31 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Sep 2010 16:22:31 -0700
Subject: [IPython-dev] Pasting fix, unicode woes
In-Reply-To: <AANLkTimLnwcuO5Ot+LRVDyTJFqp1KGfh+WOua7SdN7TD@mail.gmail.com>
References: <AANLkTimtN=pKpsSARr2QTO7ZVK3gYLO0dSRO3sFATnkT@mail.gmail.com>
	<AANLkTi=cG0ohZLAhXQfALaFhNsg-CWW+ZhGou57Gwqxr@mail.gmail.com>
	<AANLkTinn0StPSEPNnVX8fU6LzgGaqnfsFPVEu2DNWr8Y@mail.gmail.com>
	<AANLkTim19u11VbTw6SxHuXYB=6aO717v7-18vrGrAsGk@mail.gmail.com>
	<AANLkTi=qALBjUt49fCYdsHMC93R2EEfcdnX5aYrTE2aD@mail.gmail.com>
	<AANLkTi=g3Z6UX6pB94Jp63mNwdhq+_uazmWi83wmoc7N@mail.gmail.com>
	<AANLkTimLnwcuO5Ot+LRVDyTJFqp1KGfh+WOua7SdN7TD@mail.gmail.com>
Message-ID: <AANLkTinrCKxBGaMc5qXU-younP1jPjMEXzfzaWxaxznD@mail.gmail.com>

On Thu, Sep 9, 2010 at 4:12 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>>
>
> Can this be run without installing? I'm working with ipython in a standard
> directory, so as not to confuse versions. I did find you could get it via
> "from IPython.testing import iptest", but then iptest.run_iptest() hangs in
> the middle of testing (unless this is just a very slow test).

Oh yes, either have a copy/symlink of iptest in your $PATH, or use
that trick to run it by hand.  But you do need to have that version of
ipython be the one that is imported, obviously.  For that you can
either:

1 symlink  the IPython/ directory to your something in your PYTHONPATH
2 add the parent directory of IPython to your PYTHONPATH
3 use setupegg.py develop in a virtualenv

I personally use #1, I think Brian uses #3...

Cheers,

f


From fperez.net at gmail.com  Thu Sep  9 19:26:17 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 9 Sep 2010 16:26:17 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
Message-ID: <AANLkTimGoi0+X2GXbHrWSegaG2rY1TOVTRbiV7=q0ABL@mail.gmail.com>

On Thu, Sep 9, 2010 at 3:58 PM, MinRK <benjaminrk at gmail.com> wrote:
> In order to connect a second ipythonqt frontend to an existing kernel, I
> must specify by hand all 4 ports at the command-line. ?This really shouldn't
> be the case, especially since the default behavior is to have the ports
> ordered sequentially.
> I think it should at least be able to try using consecutive ports when a
> single port is given, or use a two-stage connection model that doesn't
> require clients to ever know more than one port, as is done in the parallel
> code.
> Having to type 'ipythonqt -e --xreq 65273 --sub 65274 --rep 65275 --hb
> 65276', as I just had to, just doesn't make sense.

I agree that's annoying.  Perhaps as a start we could change the model
to 'find a random port and then use the next 3 above that'...  Then
you'd only need to specify the starting port.

The two-stage model is indeed the cleanest approach.  Right now we
have some nasty business with zmq on win32 (surprise, surprise) so I
doubt we'll be able to look at it.

But once we start integrating with your code, we could uniformize the
connection model to use your approach.

Cheers,

f


From epatters at enthought.com  Thu Sep  9 20:21:37 2010
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 9 Sep 2010 17:21:37 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
Message-ID: <AANLkTi=TodcOUMj68wbz9Z_SFiMq=B8wGbjFDUOt1Ui8@mail.gmail.com>

On Thu, Sep 9, 2010 at 3:58 PM, MinRK <benjaminrk at gmail.com> wrote:
> Hello,
> In order to connect a second ipythonqt frontend to an existing kernel, I
> must specify by hand all 4 ports at the command-line. ?This really shouldn't
> be the case, especially since the default behavior is to have the ports
> ordered sequentially.

That may be the default behavior of your OS, but that's not the
default behavior in general. Random port is selection is currently
left entirely up to the OS (as it should be), and on some systems this
means that you get ports that appear to be totally random.

Guaranteeing that the ports are in consecutive order requires ugly
code (a while loop that keeps binding a port until you find one that
has three consecutive ports that are also open). Frankly, I think that
if you care what the ports are, you should just pass them when
launching the kernel in the first place.

Evan


From benjaminrk at gmail.com  Thu Sep  9 20:32:05 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 9 Sep 2010 17:32:05 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTi=TodcOUMj68wbz9Z_SFiMq=B8wGbjFDUOt1Ui8@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
	<AANLkTi=TodcOUMj68wbz9Z_SFiMq=B8wGbjFDUOt1Ui8@mail.gmail.com>
Message-ID: <AANLkTin2cKRAvg4XDw6ZMX00=ZZ0t1hGw6CTy5CT+T0K@mail.gmail.com>

On Thu, Sep 9, 2010 at 17:21, Evan Patterson <epatters at enthought.com> wrote:

> On Thu, Sep 9, 2010 at 3:58 PM, MinRK <benjaminrk at gmail.com> wrote:
> > Hello,
> > In order to connect a second ipythonqt frontend to an existing kernel, I
> > must specify by hand all 4 ports at the command-line.  This really
> shouldn't
> > be the case, especially since the default behavior is to have the ports
> > ordered sequentially.
>
> That may be the default behavior of your OS, but that's not the
> default behavior in general. Random port is selection is currently
> left entirely up to the OS (as it should be), and on some systems this
> means that you get ports that appear to be totally random.
>

Good point, that makes a two-stage connect even more important, since you
can't expect the relationship between the port numbers to be well behaved.


>
> Guaranteeing that the ports are in consecutive order requires ugly
> code (a while loop that keeps binding a port until you find one that
> has three consecutive ports that are also open). Frankly, I think that
> if you care what the ports are, you should just pass them when
> launching the kernel in the first place.
>

The problem is not that I care what the ports are, quite the opposite. I
don't want to care what the ports are, but the current state requires me to
track a new set of 4 every time. This is quite unpleasant for launching
multiple clients on a kernel, in addition to being unnecessary.

It should be very easy to connect additional clients to a running kernel,
and specifying every port by hand does not qualify.

$> ipythonqt -e
should successfully connect to a kernel started with:
$> ipythonqt

At the _very worst_, a single port (or file) should have to be specified to
connect to a kernel launched with defaults.

This can be done, as it was in Twisted code, via a file in IPYTHON_DIR, or
even better with a two-stage connect.

-MinRK


> Evan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/2ab42db5/attachment.html>

From epatters at enthought.com  Thu Sep  9 21:11:29 2010
From: epatters at enthought.com (Evan Patterson)
Date: Thu, 9 Sep 2010 18:11:29 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTin2cKRAvg4XDw6ZMX00=ZZ0t1hGw6CTy5CT+T0K@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
	<AANLkTi=TodcOUMj68wbz9Z_SFiMq=B8wGbjFDUOt1Ui8@mail.gmail.com>
	<AANLkTin2cKRAvg4XDw6ZMX00=ZZ0t1hGw6CTy5CT+T0K@mail.gmail.com>
Message-ID: <AANLkTim-nOtOWKm6=LfRFCJk9X4DkN9FpakFgPTTAK5=@mail.gmail.com>

On Thu, Sep 9, 2010 at 5:32 PM, MinRK <benjaminrk at gmail.com> wrote:
>
>
> On Thu, Sep 9, 2010 at 17:21, Evan Patterson <epatters at enthought.com> wrote:
>>
>> On Thu, Sep 9, 2010 at 3:58 PM, MinRK <benjaminrk at gmail.com> wrote:
>> > Hello,
>> > In order to connect a second ipythonqt frontend to an existing kernel, I
>> > must specify by hand all 4 ports at the command-line. ?This really
>> > shouldn't
>> > be the case, especially since the default behavior is to have the ports
>> > ordered sequentially.
>>
>> That may be the default behavior of your OS, but that's not the
>> default behavior in general. Random port is selection is currently
>> left entirely up to the OS (as it should be), and on some systems this
>> means that you get ports that appear to be totally random.
>
> Good point, that makes a two-stage connect even more important, since you
> can't expect the relationship between the port numbers to be well behaved.
>
>>
>> Guaranteeing that the ports are in consecutive order requires ugly
>> code (a while loop that keeps binding a port until you find one that
>> has three consecutive ports that are also open). Frankly, I think that
>> if you care what the ports are, you should just pass them when
>> launching the kernel in the first place.
>
> The problem is not that I care what the ports are, quite the opposite. I
> don't want to care what the ports are, but the current state requires me to
> track a new set of 4 every time. This is quite unpleasant for launching
> multiple clients on a kernel, in addition to being unnecessary.
> It should be very easy to connect additional clients to a running kernel,
> and specifying every port by hand does not qualify.
> $> ipythonqt -e
> should successfully connect to a kernel started with:
> $> ipythonqt
> At the _very worst_, a single port (or file) should have to be specified to
> connect to a kernel launched with defaults.
> This can be done, as it was in Twisted code, via a file in IPYTHON_DIR, or
> even better with a two-stage connect.

I see your point; something does have to be done about this. That
being said, it's important to keep in mind that ipythonqt is currently
not so much an application as a tech demo, so there are definitely
some usability issues to be worked out. The Qt widget itself is
becoming fairly polished, but the application needs some work.

Evan


From benjaminrk at gmail.com  Thu Sep  9 21:25:28 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 9 Sep 2010 18:25:28 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <AANLkTim-nOtOWKm6=LfRFCJk9X4DkN9FpakFgPTTAK5=@mail.gmail.com>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
	<AANLkTi=TodcOUMj68wbz9Z_SFiMq=B8wGbjFDUOt1Ui8@mail.gmail.com>
	<AANLkTin2cKRAvg4XDw6ZMX00=ZZ0t1hGw6CTy5CT+T0K@mail.gmail.com>
	<AANLkTim-nOtOWKm6=LfRFCJk9X4DkN9FpakFgPTTAK5=@mail.gmail.com>
Message-ID: <AANLkTikG2n7voec5XCdKPZ84BH_TXgoPPiYAsY46EZB0@mail.gmail.com>

On Thu, Sep 9, 2010 at 18:11, Evan Patterson <epatters at enthought.com> wrote:

> On Thu, Sep 9, 2010 at 5:32 PM, MinRK <benjaminrk at gmail.com> wrote:
> >
> >
> > On Thu, Sep 9, 2010 at 17:21, Evan Patterson <epatters at enthought.com>
> wrote:
> >>
> >> On Thu, Sep 9, 2010 at 3:58 PM, MinRK <benjaminrk at gmail.com> wrote:
> >> > Hello,
> >> > In order to connect a second ipythonqt frontend to an existing kernel,
> I
> >> > must specify by hand all 4 ports at the command-line.  This really
> >> > shouldn't
> >> > be the case, especially since the default behavior is to have the
> ports
> >> > ordered sequentially.
> >>
> >> That may be the default behavior of your OS, but that's not the
> >> default behavior in general. Random port is selection is currently
> >> left entirely up to the OS (as it should be), and on some systems this
> >> means that you get ports that appear to be totally random.
> >
> > Good point, that makes a two-stage connect even more important, since you
> > can't expect the relationship between the port numbers to be well
> behaved.
> >
> >>
> >> Guaranteeing that the ports are in consecutive order requires ugly
> >> code (a while loop that keeps binding a port until you find one that
> >> has three consecutive ports that are also open). Frankly, I think that
> >> if you care what the ports are, you should just pass them when
> >> launching the kernel in the first place.
> >
> > The problem is not that I care what the ports are, quite the opposite. I
> > don't want to care what the ports are, but the current state requires me
> to
> > track a new set of 4 every time. This is quite unpleasant for launching
> > multiple clients on a kernel, in addition to being unnecessary.
> > It should be very easy to connect additional clients to a running kernel,
> > and specifying every port by hand does not qualify.
> > $> ipythonqt -e
> > should successfully connect to a kernel started with:
> > $> ipythonqt
> > At the _very worst_, a single port (or file) should have to be specified
> to
> > connect to a kernel launched with defaults.
> > This can be done, as it was in Twisted code, via a file in IPYTHON_DIR,
> or
> > even better with a two-stage connect.
>
> I see your point; something does have to be done about this. That
> being said, it's important to keep in mind that ipythonqt is currently
> not so much an application as a tech demo, so there are definitely
> some usability issues to be worked out. The Qt widget itself is
> becoming fairly polished, but the application needs some work.
>

Certainly, this sort of thing falls in priority well behind polishing the
already super cool qtwidget functionality (well done on that, by the way). I
just noticed that it requires more information than it should when I tried
testing using raw_input with multiple clients, and I had to tyep it out
several times, since the current raw_input is sufficiently broken that I had
to start many fresh frontends, rather than just killing the kernel.

-MinRK


> Evan
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100909/e85fad1c/attachment.html>

From ellisonbg at gmail.com  Mon Sep 13 14:27:39 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 13 Sep 2010 11:27:39 -0700
Subject: [IPython-dev] connecting ipythonqt to an existing kernel should
 not require specifying 4 ports
In-Reply-To: <i6bpgv$fdl$1@dough.gmane.org>
References: <AANLkTikzM5NiR8QR=OvNYLfAsVf-HMFUJ1QWPrKGCJUQ@mail.gmail.com>
	<i6bpgv$fdl$1@dough.gmane.org>
Message-ID: <AANLkTin=V4Ag5EuNekgNvvDYjL3BEZe4TsOiLZMAyZxG@mail.gmail.com>

On Thu, Sep 9, 2010 at 4:10 PM, Robert Kern <robert.kern at gmail.com> wrote:
> On 9/9/10 5:58 PM, MinRK wrote:
>> Hello,
>>
>> In order to connect a second ipythonqt frontend to an existing kernel, I must
>> specify by hand all 4 ports at the command-line. ?This really shouldn't be the
>> case, especially since the default behavior is to have the ports ordered
>> sequentially.
>>
>> I think it should at least be able to try using consecutive ports when a single
>> port is given, or use a two-stage connection model that doesn't require clients
>> to ever know more than one port, as is done in the parallel code.
>>
>> Having to type 'ipythonqt -e --xreq 65273 --sub 65274 --rep 65275 --hb 65276',
>> as I just had to, just doesn't make sense.
>
> I like the two-stage connection model; you open and advertise one REQ/REP port
> for configuring clients with the other ports.

Yep, we are going to implement this.

Cheers,

Brian

> Hopefully, 0MQ will grow the ability to share multiple named sockets on a single
> TCP/IP port, but they've only briefly discussed it so far.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
> ?that is made terrible by our own mad attempt to interpret it as though it had
> ?an underlying truth."
> ? -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Mon Sep 13 16:58:13 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 13 Sep 2010 13:58:13 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
Message-ID: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>

Hi folks,

[ sorry for the cross-post, but devs on both lists will care about this]

I just went through the exercise of pasting 100 randomly chosen
examples from the gallery into the new ipython console with inline
graphics.  Report:

- 98 worked perfectly: the figures I got were identical to those on the website.

- 1 had minor visual differences:
http://matplotlib.sourceforge.net/examples/pylab_examples/quadmesh_demo.html:
in the SVG render, the masked region
appears black instead of transparent.

- One produced an error:
http://matplotlib.sourceforge.net/examples/axes_grid/simple_axisline4.html

...
   ...: plt.draw()
   ...: plt.show()
   ...:
Received invalid plot data.

But when I save the file and try to load it  into firefox, it seems to
indeed be bad SVG:

XML Parsing Error: mismatched tag. Expected: </g>.
Location: file:///home/fperez/ipython/ipython/bad.svg
Line Number 287, Column 3:</svg>
--^

In summary: we can run pretty much any MPL example by straight
copy/paste, and the only two glitches I see are in the SVG data
itself.  Once the other two buglets I reported earlier get fixed up,
this will be a very nice way to interact with MPL.

One small request: is it possible/easy to add to the MPL examples a
little 'copy to clipboard' button or link?  Now that one can
copy/paste wholesale examples into an interactive session to explore
them, it feels annoying to have to highlight the whole text box and
then do Ctrl-C or menu->copy.  It would be really nice to have a
one-click 'copy to clipboard'...  But I have no idea if that's easy or
hard in HTML...

Anyway, I think we're starting to be in pretty good shape!

Cheers,

f


From ellisonbg at gmail.com  Mon Sep 13 17:10:23 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 13 Sep 2010 14:10:23 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
Message-ID: <AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>

Fernando,



On Mon, Sep 13, 2010 at 1:58 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> [ sorry for the cross-post, but devs on both lists will care about this]
>
> I just went through the exercise of pasting 100 randomly chosen
> examples from the gallery into the new ipython console with inline
> graphics. ?Report:
>
> - 98 worked perfectly: the figures I got were identical to those on the website.

That is a pretty significant test of the new console....100 is a lot
of copying and pasting.

> - 1 had minor visual differences:
> http://matplotlib.sourceforge.net/examples/pylab_examples/quadmesh_demo.html:
> in the SVG render, the masked region
> appears black instead of transparent.
>
> - One produced an error:
> http://matplotlib.sourceforge.net/examples/axes_grid/simple_axisline4.html
>
> ...
> ? ...: plt.draw()
> ? ...: plt.show()
> ? ...:
> Received invalid plot data.
>
> But when I save the file and try to load it ?into firefox, it seems to
> indeed be bad SVG:
>
> XML Parsing Error: mismatched tag. Expected: </g>.
> Location: file:///home/fperez/ipython/ipython/bad.svg
> Line Number 287, Column 3:</svg>
> --^
>
> In summary: we can run pretty much any MPL example by straight
> copy/paste, and the only two glitches I see are in the SVG data
> itself. ?Once the other two buglets I reported earlier get fixed up,
> this will be a very nice way to interact with MPL.
>
> One small request: is it possible/easy to add to the MPL examples a
> little 'copy to clipboard' button or link? ?Now that one can
> copy/paste wholesale examples into an interactive session to explore
> them, it feels annoying to have to highlight the whole text box and
> then do Ctrl-C or menu->copy. ?It would be really nice to have a
> one-click 'copy to clipboard'... ?But I have no idea if that's easy or
> hard in HTML...

+1 to this!

Cheers,

Brian

> Anyway, I think we're starting to be in pretty good shape!
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From gokhansever at gmail.com  Mon Sep 13 17:22:16 2010
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Mon, 13 Sep 2010 16:22:16 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
Message-ID: <AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>

Hi Fernando,

On Mon, Sep 13, 2010 at 3:58 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi folks,
>
> One small request: is it possible/easy to add to the MPL examples a
> little 'copy to clipboard' button or link?  Now that one can
> copy/paste wholesale examples into an interactive session to explore
> them, it feels annoying to have to highlight the whole text box and
> then do Ctrl-C or menu->copy.  It would be really nice to have a
> one-click 'copy to clipboard'...  But I have no idea if that's easy or
> hard in HTML...
>

Either in Firefox or Chrome you could use extensions [Auto Copy] to copy
text selections into clipboard.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100913/27b728a8/attachment.html>

From fperez.net at gmail.com  Mon Sep 13 19:44:00 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 13 Sep 2010 16:44:00 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
Message-ID: <AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>

On Mon, Sep 13, 2010 at 2:22 PM, G?khan Sever <gokhansever at gmail.com> wrote:
>
> Either in Firefox or Chrome you could use extensions [Auto Copy] to copy
> text selections into clipboard.

Thanks, that's good to know.  But I'm mostly thinking of teaching
situations, so it would be nice to have this in the source: it's not
for my use but for the benefit of students who may be in a lab where
they can't install extensions.  But I don't know if that can even be
done in html in the first place.

Cheers,

f


From ben.root at ou.edu  Mon Sep 13 21:02:55 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Mon, 13 Sep 2010 20:02:55 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
Message-ID: <AANLkTiniZyyZOEzB+4383Yct0ERrSZteSOAMOn9_aoKV@mail.gmail.com>

On Mon, Sep 13, 2010 at 6:44 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Mon, Sep 13, 2010 at 2:22 PM, G?khan Sever <gokhansever at gmail.com>
> wrote:
> >
> > Either in Firefox or Chrome you could use extensions [Auto Copy] to copy
> > text selections into clipboard.
>
> Thanks, that's good to know.  But I'm mostly thinking of teaching
> situations, so it would be nice to have this in the source: it's not
> for my use but for the benefit of students who may be in a lab where
> they can't install extensions.  But I don't know if that can even be
> done in html in the first place.
>
> Cheers,
>
> f
>

In github, there is something like this for copying the address of someone's
git repo, but it might be done using Flash, I am not sure.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100913/cf3a70a9/attachment.html>

From almar.klein at gmail.com  Tue Sep 14 05:20:22 2010
From: almar.klein at gmail.com (Almar Klein)
Date: Tue, 14 Sep 2010 11:20:22 +0200
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
	<AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
Message-ID: <AANLkTin_QvfAzZutxsiN06SOp+CxNRxNjX5hVe8GCbxU@mail.gmail.com>

On 6 September 2010 22:32, Almar Klein <almar.klein at gmail.com> wrote:

>
>
> On 6 September 2010 22:04, Fernando Perez <fperez.net at gmail.com> wrote:
>
>> Hi Almar,
>>
>> On Mon, Sep 6, 2010 at 12:53 PM, Almar Klein <almar.klein at gmail.com>
>> wrote:
>> >
>> > That sounds great! I'll take a look at filling in the blanks for tk,
>> fltk
>> > and gtk if you like.
>>
>> that would be great. Note that as far as we've seen, in light testing
>> it seems that for gtk nothing is needed: with this code on our side:
>>
>>
>> http://github.com/ipython/ipython/blob/newkernel/IPython/zmq/gui/gtkembed.py
>>
>> every single gtk example I've tried so far has worked (with the only
>> caveat being that you have to close them with the window manager
>> buttons rather than the app's own close button/menu).  So we may be
>> home free for gtk, though that's still up for confirmation with more
>> testing and someone more knowledgeable in the matter than me.
>>
>
The code that I use is quite similar to the code behind the link. (Except
that IEP uses its own mainloop in which it periodically calls
gtk.main_iteration().)  In addition to gtk.main() and gtk.main_quit(), I
also replace gtk.mainloop() and gtk.mainquit() with a dummy. I suspect I did
that to also support older versions.



> It also looks like something similar may be OK for tk, where with a
>> bit of hijacking from our side, we can let unmodified Tk code run
>> happily within our own environment.
>>
>> For Qt/Wx it definitely seems that the cooperation via guisupport will
>> be needed.  And for fltk, I have no idea.
>>
>
> I don't think there's much to do for fltk either, I'll look into it
> tomorrow I hope.
>

A bit later than I hoped, but here it goes:

Gtk, Tk, and fltk can all be supported without cooperation via guisupport. I
guess its all about how they define application objects, and in these
toolkits, they're all global. On the kernel side, if integrating the event
loop for these toolkits, you'll need to disable the mainloops. I put some
example code below. Probably nothing new here for you guys, except maybe for
fltk.

Maybe a good idea to put examples such as these in the docs of
guisupport.py?

Cheers,
  Almar


=== Example code for hiding mainloops for TK, FLTK and GTK ===

# Dummy function to replace mainloops with
def dummy(*args,**kwargs):
    pass

# Hijack TK
import Tkinter
Tkinter.Misc.mainloop = dummy
Tkinter.mainloop = dummy

# Create (hidden) root object, to create a Tcl interpreter
r = Tkinter.Tk()
r.withdraw()

# TK events can be processed using:
r.update()


# Hijack FLTK
import fltk
from types import MethodType
fltk.Fl.run = MethodType(dummy, fltk.Fl)

# FLTK events can be processed using:
fltk.Fl.wait(0)


# Hijkack GTK
gtk.mainloop = dummy
gtk.main = dummy
gtk.main_quit = dummy
gtk.mainquit = dummy

# Make sure main_iteration exists even on older versions
if not hasattr(gtk, 'main_iteration'):
    gtk.main_iteration = gtk.mainiteration

# GTK events can be processed using:
while gtk.events_pending():
    gtk.main_iteration(False)

=== End of code ===
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100914/c41881c3/attachment.html>

From gokhansever at gmail.com  Tue Sep 14 11:08:11 2010
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Tue, 14 Sep 2010 10:08:11 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
Message-ID: <AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>

On Mon, Sep 13, 2010 at 6:44 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Thanks, that's good to know.  But I'm mostly thinking of teaching
> situations, so it would be nice to have this in the source: it's not
> for my use but for the benefit of students who may be in a lab where
> they can't install extensions.  But I don't know if that can even be
> done in html in the first place.
>

I think there might be a couple different approaches that might be useful
for educational purposes of IPython + matplotlib usage. For instance:

1-) When one downloads a script from the matplotlib gallery via an external
script (name it load_into_ipython or open_with_ipython) the contents of that
gallery script (or any python script) can be executed locally inside an
ipython session.

2-) Matplotlib gallery might turn to an interactive environment where you
can execute the script from right within your browser and change parameters
in the same browser window. As far as I know mpl figures can now be drawn on
html canvas. This might for sure boost the number of matplotlib audience.


-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100914/c87f78cb/attachment.html>

From jdh2358 at gmail.com  Tue Sep 14 11:21:20 2010
From: jdh2358 at gmail.com (John Hunter)
Date: Tue, 14 Sep 2010 10:21:20 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
Message-ID: <AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>

On Mon, Sep 13, 2010 at 4:10 PM, Brian Granger <ellisonbg at gmail.com> wrote:

>> One small request: is it possible/easy to add to the MPL examples a
>> little 'copy to clipboard' button or link? ?Now that one can
>> copy/paste wholesale examples into an interactive session to explore
>> them, it feels annoying to have to highlight the whole text box and
>> then do Ctrl-C or menu->copy. ?It would be really nice to have a
>> one-click 'copy to clipboard'... ?But I have no idea if that's easy or
>> hard in HTML...
>
> +1 to this!

On a quick googling, there are some IE only Javascript examples to do
this.  Apparently you can enable them in firefox but it requires a
significant amount of about:config hackery
(http://www.febooti.com/support/website-help/website-javascript-copy-clipboard.html).

How about this as an alternative: on my box, I can drag the "source
code" link from the browser into my terminal, which by default pastes
the URL of the referenced *.py into the terminal.  If "run" supported
a -w (web) option, or automatically detected that the URL starts with
http, it could do a web run of the file.  Of course, you may want the
source code pasted in for illustrative purposes... To support this,
you could add a -u (url) option to "paste" which assumes the input is
a url, fetches it, and pastes the contents into ipython.  So you could
type "paste -u" and then drag the link into the terminal, and it would
fetch it and paste the code into an input block.

JDH


From robert.kern at gmail.com  Tue Sep 14 11:29:08 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 14 Sep 2010 10:29:08 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
Message-ID: <i6o4c5$v56$1@dough.gmane.org>

On 9/13/10 3:58 PM, Fernando Perez wrote:

> One small request: is it possible/easy to add to the MPL examples a
> little 'copy to clipboard' button or link?  Now that one can
> copy/paste wholesale examples into an interactive session to explore
> them, it feels annoying to have to highlight the whole text box and
> then do Ctrl-C or menu->copy.  It would be really nice to have a
> one-click 'copy to clipboard'...  But I have no idea if that's easy or
> hard in HTML...

You have to use Flash. I'm sure there are easy-to-integrate snippets floating 
around for free on the web, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From fperez.net at gmail.com  Tue Sep 14 14:14:29 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 11:14:29 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <B057648D-4E6C-4359-B276-CD09F0ED42FA@iro.umontreal.ca>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<B057648D-4E6C-4359-B276-CD09F0ED42FA@iro.umontreal.ca>
Message-ID: <AANLkTininK0Cd=j4XM6+4N7yHXDL01Qa9cA5O_dKzwXD@mail.gmail.com>

On Mon, Sep 13, 2010 at 7:08 PM, David Warde-Farley
<wardefar at iro.umontreal.ca> wrote:
>
> Nice work IPython people! I haven't been following too closely but this looks exciting.

Thanks, David.  Hopefully by next week we'll complete our
stabilization so that we're willing to foist this on the ipython-dev
and mpl-dev denizens.  Anyone is welcome to try it out now, but we're
moving things around enough that at any given point it may or may not
work.  We'll move it into trunk once the churn stops.

Regards,

f


From fperez.net at gmail.com  Tue Sep 14 14:20:52 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 11:20:52 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
Message-ID: <AANLkTimRGyC=rcFLax2t-PA7mDSfxpf=YtqxDE2TikE8@mail.gmail.com>

Hey,

On Tue, Sep 14, 2010 at 8:21 AM, John Hunter <jdh2358 at gmail.com> wrote:
>
> How about this as an alternative: on my box, I can drag the "source
> code" link from the browser into my terminal, which by default pastes
> the URL of the referenced *.py into the terminal. ?If "run" supported
> a -w (web) option, or automatically detected that the URL starts with
> http, it could do a web run of the file. ?Of course, you may want the
> source code pasted in for illustrative purposes... To support this,
> you could add a -u (url) option to "paste" which assumes the input is
> a url, fetches it, and pastes the contents into ipython. ?So you could
> type "paste -u" and then drag the link into the terminal, and it would
> fetch it and paste the code into an input block.

I'll play with those ideas, good thoughts.  %paste may not be the best
location because paste is now a terminal-only magic, since gui clients
have 'real' paste.  What's a little trickier now is that we really
need to think all the time in terms of completely separate kernel and
client codes, that only communicate via messages.  So a magic can't
muck with code in the client, because the magic executes inside the
kernel and the client is in a separate process (and possibly in a
separate computer).  While this is a bit more constraining, it forces
us to have a clean separation between different kinds of
functionality.

We've toyed with the idea of enabling a special syntax for *purely
client side* commands, something like

:cmd

that would never be sent to the kernel, and would be something for the
client to process internally.  But we'll have to mull this a little
longer...

One very important point in all of this is that the client *may not be
written in Python*.  All we have is a messaging protocol, the client
could be a web browser or anything else.

But in any case, I'll play with ways to expose this that are as easy
as possible for the users thanks for the feedback.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 14 14:20:52 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 11:20:52 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
Message-ID: <AANLkTimRGyC=rcFLax2t-PA7mDSfxpf=YtqxDE2TikE8@mail.gmail.com>

Hey,

On Tue, Sep 14, 2010 at 8:21 AM, John Hunter <jdh2358 at gmail.com> wrote:
>
> How about this as an alternative: on my box, I can drag the "source
> code" link from the browser into my terminal, which by default pastes
> the URL of the referenced *.py into the terminal. ?If "run" supported
> a -w (web) option, or automatically detected that the URL starts with
> http, it could do a web run of the file. ?Of course, you may want the
> source code pasted in for illustrative purposes... To support this,
> you could add a -u (url) option to "paste" which assumes the input is
> a url, fetches it, and pastes the contents into ipython. ?So you could
> type "paste -u" and then drag the link into the terminal, and it would
> fetch it and paste the code into an input block.

I'll play with those ideas, good thoughts.  %paste may not be the best
location because paste is now a terminal-only magic, since gui clients
have 'real' paste.  What's a little trickier now is that we really
need to think all the time in terms of completely separate kernel and
client codes, that only communicate via messages.  So a magic can't
muck with code in the client, because the magic executes inside the
kernel and the client is in a separate process (and possibly in a
separate computer).  While this is a bit more constraining, it forces
us to have a clean separation between different kinds of
functionality.

We've toyed with the idea of enabling a special syntax for *purely
client side* commands, something like

:cmd

that would never be sent to the kernel, and would be something for the
client to process internally.  But we'll have to mull this a little
longer...

One very important point in all of this is that the client *may not be
written in Python*.  All we have is a messaging protocol, the client
could be a web browser or anything else.

But in any case, I'll play with ways to expose this that are as easy
as possible for the users thanks for the feedback.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 14 14:58:14 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 11:58:14 -0700
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
Message-ID: <AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>

On Tue, Sep 14, 2010 at 11:48 AM, Anne Archibald
<aarchiba at physics.mcgill.ca> wrote:
> On 14 September 2010 11:08, G?khan Sever <gokhansever at gmail.com> wrote:
>
>> 1-) When one downloads a script from the matplotlib gallery via an external
>> script (name it load_into_ipython or open_with_ipython) the contents of that
>> gallery script (or any python script) can be executed locally inside an
>> ipython session.
>
> Not to be difficult, but I should point out that allowing users to run
> code with one click, particularly if that code is from a wiki or other
> user-submitted gallery, is just asking for trouble. How long before
> someone submits "import os, shutil;
> shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> inoffensive script?

Very valid points.  I'm leaning more towards something like a
combination of (hopefully) a 'copy code' button on the MPL webpages
themselves, so users don't have to scroll/highlight a lot but would
still do paste, execute manually, and a special %mplexample magic.

This would only run examples from the mpl gallery (hardcoding the
path), would display the code to the  user first, and would ask for
confirmation before execution.  Since those html pages are built by
executing those same scripts, there's a layer of sanity already built
into it (the rmtree call would have already nuked the builder's home
directory in the build process if it had been there).  Showing the
code to the user and confirming execution before proceeding adds a
final chance for the person to check her parachute before  jumping off
the cliff.

Does that sound reasonable?

>> 2-) Matplotlib gallery might turn to an interactive environment where you
>> can execute the script from right within your browser and change parameters
>> in the same browser window. As far as I know mpl figures can now be drawn on
>> html canvas. This might for sure boost the number of matplotlib audience.
>
> Is there a sandboxed browser plugin? Or server plugin, depending on
> where you run the script?

This would have to be server-side, and code needs to be written.  Part
of our interest with this explicit separation of ipython kernel and
clients with a well-defined protocol is to make the above possible.
But we haven't written any of the code necessary to have a browser
client, and to serve code read from a sphinx-generated HTML page.
Gokhan, your patches will be welcome, the infrastructure is now ready
and waiting for you :)

Cheers,

f


From fperez.net at gmail.com  Tue Sep 14 15:03:49 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 12:03:49 -0700
Subject: [IPython-dev] Uniform way of integrating event loops among
 different IDE's
In-Reply-To: <AANLkTin_QvfAzZutxsiN06SOp+CxNRxNjX5hVe8GCbxU@mail.gmail.com>
References: <AANLkTimCPPTC=BsCvXtRfTKer2bqyr5S21xryue8M7Qv@mail.gmail.com>
	<AANLkTi=wgNEhea066mdX6LJKKMvDE2fAQVGdfdouRJ6g@mail.gmail.com>
	<AANLkTi=iqzAhRj0H-vAM1S0+AfaS0hzZR7YbcZsw-nAm@mail.gmail.com>
	<AANLkTindbCkUQs2gt1Q_gxo=f-eztZHX7P8NE8MzFrjB@mail.gmail.com>
	<AANLkTi=KOkOJcej3u6CMuFf-RaxZb6hDHBTjYJjCw7nw@mail.gmail.com>
	<AANLkTi=twfbO+drmsyh+gKe32f4oF4MTP0OJVggNXyw_@mail.gmail.com>
	<AANLkTikcScwitvZ9X1ZEowKRAJGvzcuHBXDdOey1Nfbb@mail.gmail.com>
	<AANLkTin766VHAwBWXjwqJ0r_ohpF7A6tPW_Eo0wPBPa5@mail.gmail.com>
	<AANLkTin_QvfAzZutxsiN06SOp+CxNRxNjX5hVe8GCbxU@mail.gmail.com>
Message-ID: <AANLkTikSO8-fzZVqbU=J5_vQXkzWpbecnoiC-QWWA_jW@mail.gmail.com>

On Tue, Sep 14, 2010 at 2:20 AM, Almar Klein <almar.klein at gmail.com> wrote:
> The code that I use is quite similar to the code behind the link. (Except
> that IEP uses its own mainloop in which it periodically calls
> gtk.main_iteration().)? In addition to gtk.main() and gtk.main_quit(), I
> also replace gtk.mainloop() and gtk.mainquit() with a dummy. I suspect I did
> that to also support older versions.

yes, that's for older versions, I'm not sure if a gtk that old is
still in significant use or not.

>>> It also looks like something similar may be OK for tk, where with a
>>> bit of hijacking from our side, we can let unmodified Tk code run
>>> happily within our own environment.
>>>
>>> For Qt/Wx it definitely seems that the cooperation via guisupport will
>>> be needed. ?And for fltk, I have no idea.
>>
>> I don't think there's much to do for fltk either, I'll look into it
>> tomorrow I hope.
>
> A bit later than I hoped, but here it goes:
>
> Gtk, Tk, and fltk can all be supported without cooperation via guisupport. I
> guess its all about how they define application objects, and in these
> toolkits, they're all global. On the kernel side, if integrating the event
> loop for these toolkits, you'll need to disable the mainloops. I put some
> example code below. Probably nothing new here for you guys, except maybe for
> fltk.
>
> Maybe a good idea to put examples such as these in the docs of
> guisupport.py?

Sure!  Brian is working hard on that code, so when the dust settles
we'll include this info for the others.

Thanks!

f


From gokhansever at gmail.com  Tue Sep 14 15:23:03 2010
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Tue, 14 Sep 2010 14:23:03 -0500
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
Message-ID: <AANLkTi=tqi3Mzqr4U6mYDLNZrw35E9GFTg8vaiD+SgRf@mail.gmail.com>

On Tue, Sep 14, 2010 at 1:48 PM, Anne Archibald
<aarchiba at physics.mcgill.ca>wrote:

> On 14 September 2010 11:08, G?khan Sever <gokhansever at gmail.com> wrote:
>
> > 1-) When one downloads a script from the matplotlib gallery via an
> external
> > script (name it load_into_ipython or open_with_ipython) the contents of
> that
> > gallery script (or any python script) can be executed locally inside an
> > ipython session.
>
> Not to be difficult, but I should point out that allowing users to run
> code with one click, particularly if that code is from a wiki or other
> user-submitted gallery, is just asking for trouble. How long before
> someone submits "import os, shutil;
> shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> inoffensive script?
>

I was thinking naively --with Python for Science in mind not Python for
Hacking. I accept this is a not good idea considering that harmful effect
but in this brave new world of us there is always danger involved when one
puts their hands on virtual lands --whether the code is executed
intentionally or sneaked by a conic head.


>
> > 2-) Matplotlib gallery might turn to an interactive environment where you
> > can execute the script from right within your browser and change
> parameters
> > in the same browser window. As far as I know mpl figures can now be drawn
> on
> > html canvas. This might for sure boost the number of matplotlib audience.
>
> Is there a sandboxed browser plugin? Or server plugin, depending on
> where you run the script?
>

This one is one of my aloud speculations. Only at ideas level. I will write
more, answering Fernando's reply.


>
> Anne
>



-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100914/e0b7e77c/attachment.html>

From gokhansever at gmail.com  Tue Sep 14 15:38:55 2010
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=)
Date: Tue, 14 Sep 2010 14:38:55 -0500
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
Message-ID: <AANLkTi=xb+bzc7kTWg+b=q6vSK3ZANFFfJqPOS9LSUgU@mail.gmail.com>

On Tue, Sep 14, 2010 at 1:58 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> >> 2-) Matplotlib gallery might turn to an interactive environment where
> you
> >> can execute the script from right within your browser and change
> parameters
> >> in the same browser window. As far as I know mpl figures can now be
> drawn on
> >> html canvas. This might for sure boost the number of matplotlib
> audience.
> >
> > Is there a sandboxed browser plugin? Or server plugin, depending on
> > where you run the script?
>
> This would have to be server-side, and code needs to be written.  Part
> of our interest with this explicit separation of ipython kernel and
> clients with a well-defined protocol is to make the above possible.
> But we haven't written any of the code necessary to have a browser
> client, and to serve code read from a sphinx-generated HTML page.
> Gokhan, your patches will be welcome, the infrastructure is now ready
> and waiting for you :)


Sage provides some level of interaction actually without any deployment made
on local side. Try for instance the following example on sagenb.org

from scipy import stats
import numpy as np
import matplotlib.pyplot as plt

@interact
def plot_gamma(a=(1,(1,10)), loc=(0,(0,10)), scale=(1,(1,10))):
    rv = stats.gamma(a, loc, scale)
    x = np.linspace(-1,20,1000)
    plt.plot(x,rv.pdf(x))
    plt.grid(True)
    plt.savefig('plt.png')
    plt.clf()

This one is very useful for educational and demonstrative purposes. Still
requires a bit Sage syntax manipulations to make things fully interacting on
browser.

Nice that you have matured IPython infra for implementing
such interactive functionality. I was thinking perhaps running something on
top GApss Engine but not sure they allow compiling/running C/C++ extensions
on their servers. Alternatively, like in Sage servers virtual OS'es might be
the way to go with it then possibly there will be user registration and
management issues (not sure about all specifics).

Probably, Ondrej might like experiencing with this idea :) Since he has
similar initiatives and asking help on similar topics. I am trying to
graduate myself working to solve some of my research problems and struggling
with writing especially to move on PhD. This might be a very fun Summer job
if I am wandering around jobless then.

-- 
G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100914/8748f359/attachment.html>

From fperez.net at gmail.com  Tue Sep 14 15:49:47 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 12:49:47 -0700
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTi=xb+bzc7kTWg+b=q6vSK3ZANFFfJqPOS9LSUgU@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTi=xb+bzc7kTWg+b=q6vSK3ZANFFfJqPOS9LSUgU@mail.gmail.com>
Message-ID: <AANLkTimUPtDo5ghOp1i+s1KHXqpBmDUDxhcr=LRQc_V5@mail.gmail.com>

On Tue, Sep 14, 2010 at 12:38 PM, G?khan Sever <gokhansever at gmail.com> wrote:
>
> Sage provides some level of?interaction?actually without any deployment made
> on local side. Try for instance the following example on sagenb.org
> from scipy import stats
> import numpy as np
> import matplotlib.pyplot as plt
> @interact
> def plot_gamma(a=(1,(1,10)), loc=(0,(0,10)), scale=(1,(1,10))):
> ?? ?rv = stats.gamma(a, loc, scale)
> ?? ?x = np.linspace(-1,20,1000)
> ?? ?plt.plot(x,rv.pdf(x))
> ?? ?plt.grid(True)
> ?? ?plt.savefig('plt.png')
> ?? ?plt.clf()
> This one is very useful for educational and demonstrative purposes. Still
> requires a bit Sage syntax manipulations to make things fully?interacting?on
> browser.
> Nice that you have matured IPython infra for implementing
> such?interactive?functionality. I was thinking perhaps running something on
> top GApss Engine but not sure they allow compiling/running C/C++ extensions
> on their servers. Alternatively, like in Sage servers virtual OS'es might be
> the way to go with it then possibly there will be user registration and
> management issues (not sure about all specifics).

Actually sage does have one *implicit* but very particular 'local
deployment': its notebook execution model is *strictly* tied to the
idea that the client is a web browser.  Try this code in the sage
terminal (i.e. their customized ipython, not the notebook):

sage: @interact
....: def x(a=(1, (1, 10))):
....:     print a
....:

and you'll see:

<html><!--notruncate--><div padding=6 id='div-interact-0'> <table
width=800px height=20px bgcolor='#c5c5c5'
                 cellpadding=15><tr><td bgcolor='#f9f9f9' valign=top
align=left><table><tr><td align=right><font
color="black">a&nbsp;</font></td><td><table><tr><td>
        <div id='slider-a-0' style='margin:0px; margin-left: 1.0em;
margin-right: 1.0em; width: 15.0em;'></div>

 ... lots more...

</table><div id='cell-interact-0'><?__SAGE__START>
        <table border=0 bgcolor='#white' width=100% height=100%>
        <tr><td bgcolor=white align=left
valign=top><pre><?__SAGE__TEXT></pre></td></tr>
        <tr><td  align=left valign=top><?__SAGE__HTML></td></tr>
        </table><?__SAGE__END></div></td>
                 </tr></table></div>
                 </html>
sage:


So you can see, @interact in sage does basically:

- analyze the inputs of the function
- do some basic 'type inference' and emit javascript/html controls for
each parameter.
- emit an html section that wires the above controls to repeated calls
of the decorated function as the controls are operated.

This is very cool, and it enables great functionality, but it's
hard-coded to an html/javascript client.

What we're doing is a little different, as we've built a *protocol*
that clients can use to talk to the kernel, regardless of how they are
implemented.

As the functionality matures, we'll see who contributes a
browser-based client (that will require wrapping the kernel in an http
server, obviously).  And then the question of things like @interact
will be an interesting one to think about.  @interact by nature is
creating a user interface (Mathematica's Manipulate creates Notebook
controls, sage's @interact creates HTML ones).  I'm not sure yet how
we'll approach that: having per-client implementations? A traits-style
approach where each client renders it differently?  No idea yet.

Cheers,

f


From ben.root at ou.edu  Tue Sep 14 15:57:26 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Tue, 14 Sep 2010 14:57:26 -0500
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
Message-ID: <AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>

On Tue, Sep 14, 2010 at 1:58 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Tue, Sep 14, 2010 at 11:48 AM, Anne Archibald
> <aarchiba at physics.mcgill.ca> wrote:
> > On 14 September 2010 11:08, G?khan Sever <gokhansever at gmail.com> wrote:
> >
> >> 1-) When one downloads a script from the matplotlib gallery via an
> external
> >> script (name it load_into_ipython or open_with_ipython) the contents of
> that
> >> gallery script (or any python script) can be executed locally inside an
> >> ipython session.
> >
> > Not to be difficult, but I should point out that allowing users to run
> > code with one click, particularly if that code is from a wiki or other
> > user-submitted gallery, is just asking for trouble. How long before
> > someone submits "import os, shutil;
> > shutil.deltree(os.environ['HOME'])"? Or sneaks it into some otherwise
> > inoffensive script?
>
> Very valid points.  I'm leaning more towards something like a
> combination of (hopefully) a 'copy code' button on the MPL webpages
> themselves, so users don't have to scroll/highlight a lot but would
> still do paste, execute manually, and a special %mplexample magic.
>
> This would only run examples from the mpl gallery (hardcoding the
> path), would display the code to the  user first, and would ask for
> confirmation before execution.  Since those html pages are built by
> executing those same scripts, there's a layer of sanity already built
> into it (the rmtree call would have already nuked the builder's home
> directory in the build process if it had been there).  Showing the
> code to the user and confirming execution before proceeding adds a
> final chance for the person to check her parachute before  jumping off
> the cliff.
>
> Does that sound reasonable?
>
> >> 2-) Matplotlib gallery might turn to an interactive environment where
> you
> >> can execute the script from right within your browser and change
> parameters
> >> in the same browser window. As far as I know mpl figures can now be
> drawn on
> >> html canvas. This might for sure boost the number of matplotlib
> audience.
> >
> > Is there a sandboxed browser plugin? Or server plugin, depending on
> > where you run the script?
>
> This would have to be server-side, and code needs to be written.  Part
> of our interest with this explicit separation of ipython kernel and
> clients with a well-defined protocol is to make the above possible.
> But we haven't written any of the code necessary to have a browser
> client, and to serve code read from a sphinx-generated HTML page.
> Gokhan, your patches will be welcome, the infrastructure is now ready
> and waiting for you :)
>
> Cheers,
>
> f
>

Just a crazy idea to consider that would completely bypass this whole
vulnerability issue...

Why not have an examples module that contains function calls to each
example?  On the website, we can show the source code, but also say that one
could just do:

>>> import matplotlib.examples as ex
>>> ex.bars3d_demo()

My 2 cents...

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100914/5cfb3b42/attachment.html>

From takowl at gmail.com  Tue Sep 14 19:37:36 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 15 Sep 2010 00:37:36 +0100
Subject: [IPython-dev] IPython with Py3k
Message-ID: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>

A quick update:

ipy3-preparation, my Python 2 branch tracking newkernel, now passes all nine
test suites, at least on my machine (tested at this
commit*<http://github.com/takowl/ipython/commit/22f6368352725d7e301f7af08c47982502d16dd4>).
Feel free to review the changes in this with an eye to merging them into
newkernel. It is not intended to be compatible with Python 3, but it makes
conversion simpler, partly by cutting compatibility with older Python 2.x
versions.

ipy3-newkernel (my Python 3 branch, tracking ipy3-preparation but via the
2to3 converter) is still some way from passing its test suites, although it
works well enough to be useful to me. I'll continue to work on it, although
it'll have a brief pause when I go on holiday in a couple of days. If you
want to investigate, it's at:
http://github.com/takowl/ipython/tree/ipy3-newkernel

One specific question (and the cause of a remaining failure in the
IPython.lib tests): if you enter a line with invalid syntax, Python displays
the line back to you, with a little ^ below pointing to the problem. IPython
gives that some pretty formatting. In ipy3-newkernel, I'm not getting that
display, although I still get the SyntaxError: ... line. Can anyone suggest
where I should start looking for the code for that? I think it's different
from the code for a full traceback in core/ultratb.py .

Best wishes,
Thomas

*
http://github.com/takowl/ipython/commit/22f6368352725d7e301f7af08c47982502d16dd4
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/30ecc5ae/attachment.html>

From fperez.net at gmail.com  Tue Sep 14 20:04:57 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 17:04:57 -0700
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
Message-ID: <AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>

On Tue, Sep 14, 2010 at 4:37 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> ipy3-preparation, my Python 2 branch tracking newkernel, now passes all nine
> test suites, at least on my machine (tested at this commit*). Feel free to
> review the changes in this with an eye to merging them into newkernel. It is
> not intended to be compatible with Python 3, but it makes conversion
> simpler, partly by cutting compatibility with older Python 2.x versions.

Fantastic, many thanks!!! We're 'landing' newkernel right now, so we
may wait until you return from holiday (just not enough time to do
both), but this is very, very much appreciated.  Today numpy moved to
github, we're inching closer to py3 support for ipython, the newkernel
stuff is going great.  MMh, too many good news together, I should
watch for pianos falling from windows on my way home :)

> ipy3-newkernel (my Python 3 branch, tracking ipy3-preparation but via the
> 2to3 converter) is still some way from passing its test suites, although it
> works well enough to be useful to me. I'll continue to work on it, although
> it'll have a brief pause when I go on holiday in a couple of days. If you
> want to investigate, it's at:
> http://github.com/takowl/ipython/tree/ipy3-newkernel
>
> One specific question (and the cause of a remaining failure in the
> IPython.lib tests): if you enter a line with invalid syntax, Python displays
> the line back to you, with a little ^ below pointing to the problem. IPython
> gives that some pretty formatting. In ipy3-newkernel, I'm not getting that
> display, although I still get the SyntaxError: ... line. Can anyone suggest
> where I should start looking for the code for that? I think it's different
> from the code for a full traceback in core/ultratb.py .

That is handled by showsyntaxerror, line 1506 in
core.interactiveshell.  This calls a special exception handler that
was constructed in line 1338:

        # Syntax error handler.
        self.SyntaxTB = ultratb.SyntaxTB(color_scheme='NoColor')

Let me know if you need more detials...

Cheers,

f


From fperez.net at gmail.com  Tue Sep 14 20:39:14 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 17:39:14 -0700
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
Message-ID: <AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>

On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root <ben.root at ou.edu> wrote:
>
> Why not have an examples module that contains function calls to each
> example?? On the website, we can show the source code, but also say that one
> could just do:
>
>>>> import matplotlib.examples as ex
>>>> ex.bars3d_demo()

The idea is to have the *actual code* pasted in your terminal, because
now we can easily edit complex multi-line examples directly in
ipython.  So it's not just a matter of seeing the figure results, but
mostly of having the actual source in your input buffer to play with.

Cheers,

f


From ben.root at ou.edu  Tue Sep 14 21:01:58 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 15 Sep 2010 01:01:58 +0000
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
Message-ID: <AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>

On Wed, Sep 15, 2010 at 12:39 AM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root <ben.root at ou.edu> wrote:
> >
> > Why not have an examples module that contains function calls to each
> > example?  On the website, we can show the source code, but also say that
> one
> > could just do:
> >
> >>>> import matplotlib.examples as ex
> >>>> ex.bars3d_demo()
>
> The idea is to have the *actual code* pasted in your terminal, because
> now we can easily edit complex multi-line examples directly in
> ipython.  So it's not just a matter of seeing the figure results, but
> mostly of having the actual source in your input buffer to play with.
>
> Cheers,
>
> f
>

True... but, consider this.  ipython can already display the code for a
particular module/function using the '??' idiom.  Why not have some way to
take that text and bring it into the input buffer?

I can imagine this being useful beyond matplotlib where anybody could have
their example codes easily accessed and edited.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/19c87a9b/attachment.html>

From fperez.net at gmail.com  Tue Sep 14 21:08:52 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 18:08:52 -0700
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
	<AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
Message-ID: <AANLkTinr_Di_Ey+5OV1ivCEQfiwOk-HSkuypbeVJbQsX@mail.gmail.com>

On Tue, Sep 14, 2010 at 6:01 PM, Benjamin Root <ben.root at ou.edu> wrote:
> True... but, consider this.? ipython can already display the code for a
> particular module/function using the '??' idiom.? Why not have some way to
> take that text and bring it into the input buffer?

Yes, but that's a separate issue.  The approach you propose would
likely have in ex.demo_somehting() a stub to retrieve the actual
example code as a string from a file elsewhere, because (at least
right now) the mpl examples are written as 100% standalone files, not
as functions inside of some other control module.  What you are saying
does apply to the mayavi.mlab.test_*() functions, that do serve as
examples precisely in that manner, since those *do* contain their code
inside the functions.

So for the matplotlib examples, that live in standalone files, we'd
still need something different.

> I can imagine this being useful beyond matplotlib where anybody could have
> their example codes easily accessed and edited.

Certainly!  Right now the pager is a very simple tool, but I hope that
once we put this code out we'll get contributions from enterprising Qt
coders who may improve it and add things like a button that would copy
the code from the source part of an info pane and paste it in the
interactive area, all with a single click.

We want to settle the core protocol/messaging behavior first, and once
this is ready and people test it a little, I really hope we'll get
contributions that enhance the user experience very much in this
manner.

Cheers,

f


From robert.kern at gmail.com  Tue Sep 14 21:10:38 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 14 Sep 2010 20:10:38 -0500
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
	report: happy news
In-Reply-To: <AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
	<AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
Message-ID: <i6p6eg$t7n$1@dough.gmane.org>

On 9/14/10 8:01 PM, Benjamin Root wrote:
> On Wed, Sep 15, 2010 at 12:39 AM, Fernando Perez <fperez.net
> <http://fperez.net>@gmail.com <http://gmail.com>> wrote:
>
>     On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root <ben.root at ou.edu
>     <mailto:ben.root at ou.edu>> wrote:
>      >
>      > Why not have an examples module that contains function calls to each
>      > example?  On the website, we can show the source code, but also say that one
>      > could just do:
>      >
>      >>>> import matplotlib.examples as ex
>      >>>> ex.bars3d_demo()
>
>     The idea is to have the *actual code* pasted in your terminal, because
>     now we can easily edit complex multi-line examples directly in
>     ipython.  So it's not just a matter of seeing the figure results, but
>     mostly of having the actual source in your input buffer to play with.
>
>     Cheers,
>
>     f
>
>
> True... but, consider this.  ipython can already display the code for a
> particular module/function using the '??' idiom.  Why not have some way to take
> that text and bring it into the input buffer?

Regardless of whether or not this is a good idea, it doesn't replace the 
functionality Fernando is requesting. People *will* be looking at the matplotlib 
docs on the web and copy-pasting the examples. Placing a link that they can 
click to easily, robustly, and *obviously* copy the code for pasting into the 
shell (or anywhere else!) is better than telling them to go type some magic 
commands they've probably never seen before. Magic commands that they will 
probably want to copy-paste. And so the cycle is complete.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ben.root at ou.edu  Tue Sep 14 21:23:00 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 15 Sep 2010 01:23:00 +0000
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinr_Di_Ey+5OV1ivCEQfiwOk-HSkuypbeVJbQsX@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
	<AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
	<AANLkTinr_Di_Ey+5OV1ivCEQfiwOk-HSkuypbeVJbQsX@mail.gmail.com>
Message-ID: <AANLkTimnTtnmHxU1raZEVsVVR2Hj-9D=ycN9pD9hMyqE@mail.gmail.com>

On Wed, Sep 15, 2010 at 1:08 AM, Fernando Perez <fperez.net at gmail.com>wrote:

> On Tue, Sep 14, 2010 at 6:01 PM, Benjamin Root <ben.root at ou.edu> wrote:
> > True... but, consider this.  ipython can already display the code for a
> > particular module/function using the '??' idiom.  Why not have some way
> to
> > take that text and bring it into the input buffer?
>
> Yes, but that's a separate issue.  The approach you propose would
> likely have in ex.demo_somehting() a stub to retrieve the actual
> example code as a string from a file elsewhere, because (at least
> right now) the mpl examples are written as 100% standalone files, not
> as functions inside of some other control module.  What you are saying
> does apply to the mayavi.mlab.test_*() functions, that do serve as
> examples precisely in that manner, since those *do* contain their code
> inside the functions.
>
> So for the matplotlib examples, that live in standalone files, we'd
> still need something different.
>
>
Well, my idea was predicated upon what I said previously that matplotlib
should package the examples into a useful module with function call.  This
way, one could have been able to run the demos and view the code just like
any other piece of code in matplotlib.


> > I can imagine this being useful beyond matplotlib where anybody could
> have
> > their example codes easily accessed and edited.
>
> Certainly!  Right now the pager is a very simple tool, but I hope that
> once we put this code out we'll get contributions from enterprising Qt
> coders who may improve it and add things like a button that would copy
> the code from the source part of an info pane and paste it in the
> interactive area, all with a single click.
>
> We want to settle the core protocol/messaging behavior first, and once
> this is ready and people test it a little, I really hope we'll get
> contributions that enhance the user experience very much in this
> manner.
>
>
That is a good gameplan.  What I have seen of ipython is very good and
really resolves a lot of gripes I have had with my typical python
development workflow.  Keep up the good work!

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/dc299998/attachment.html>

From ben.root at ou.edu  Tue Sep 14 21:29:19 2010
From: ben.root at ou.edu (Benjamin Root)
Date: Wed, 15 Sep 2010 01:29:19 +0000
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <i6p6eg$t7n$1@dough.gmane.org>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
	<AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
	<i6p6eg$t7n$1@dough.gmane.org>
Message-ID: <AANLkTinL60g_HhSNeHrKbJFkDvufZ5Prsj5H9fSztThb@mail.gmail.com>

On Wed, Sep 15, 2010 at 1:10 AM, Robert Kern <robert.kern at gmail.com> wrote:

> On 9/14/10 8:01 PM, Benjamin Root wrote:
> > On Wed, Sep 15, 2010 at 12:39 AM, Fernando Perez <fperez.net
> > <http://fperez.net>@gmail.com <http://gmail.com>> wrote:
> >
> >     On Tue, Sep 14, 2010 at 12:57 PM, Benjamin Root <ben.root at ou.edu
> >     <mailto:ben.root at ou.edu>> wrote:
> >      >
> >      > Why not have an examples module that contains function calls to
> each
> >      > example?  On the website, we can show the source code, but also
> say that one
> >      > could just do:
> >      >
> >      >>>> import matplotlib.examples as ex
> >      >>>> ex.bars3d_demo()
> >
> >     The idea is to have the *actual code* pasted in your terminal,
> because
> >     now we can easily edit complex multi-line examples directly in
> >     ipython.  So it's not just a matter of seeing the figure results, but
> >     mostly of having the actual source in your input buffer to play with.
> >
> >     Cheers,
> >
> >     f
> >
> >
> > True... but, consider this.  ipython can already display the code for a
> > particular module/function using the '??' idiom.  Why not have some way
> to take
> > that text and bring it into the input buffer?
>
> Regardless of whether or not this is a good idea, it doesn't replace the
> functionality Fernando is requesting. People *will* be looking at the
> matplotlib
> docs on the web and copy-pasting the examples. Placing a link that they can
> click to easily, robustly, and *obviously* copy the code for pasting into
> the
> shell (or anywhere else!) is better than telling them to go type some magic
> commands they've probably never seen before. Magic commands that they will
> probably want to copy-paste. And so the cycle is complete.
>
>
Good point.  I guess I am just a little *too* terminal-oriented.  I tend to
access the examples directly and have to copy-and-paste snippets of code,
which has many issues with indentation and empty lines.

Just the new input buffer of ipython is a significant enough feature to
stand on its own and be beneficial.  How we get text to it can be done in
many different ways.

Ben Root
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/3671fa84/attachment.html>

From fperez.net at gmail.com  Wed Sep 15 00:17:53 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 21:17:53 -0700
Subject: [IPython-dev] [matplotlib-devel] IPython (new) + matplotlib
 report: happy news
In-Reply-To: <AANLkTinL60g_HhSNeHrKbJFkDvufZ5Prsj5H9fSztThb@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTikaOPs_gbgxX5NUsW3XXEGAgGpMxeK+txXLDpG+@mail.gmail.com>
	<AANLkTin+Cnxi1UHyDyW_GcN=Q_UHFGKJsD1_xGWiOKgL@mail.gmail.com>
	<AANLkTik__o1B1QMzdTbr6s8C3=suHnmZm73ZKhYHoK62@mail.gmail.com>
	<AANLkTinevL8OP_PG_aD3B5royFroe60LrVtLzcQWOfEd@mail.gmail.com>
	<AANLkTinZa1eFGLE-RaubNZuWvjoq6sMdMYNK7OGWZjng@mail.gmail.com>
	<AANLkTindqWk5id=s8_EVuWd89nNXwVjW5nycFY6KJYoA@mail.gmail.com>
	<AANLkTinqrZDApc0Fyq32PD+pOAsNFOd=dgxfuiMwC9Pz@mail.gmail.com>
	<AANLkTinV1My5S1P6nPZTQLEFVkZ9q_eqX9TA38qjGv2Q@mail.gmail.com>
	<i6p6eg$t7n$1@dough.gmane.org>
	<AANLkTinL60g_HhSNeHrKbJFkDvufZ5Prsj5H9fSztThb@mail.gmail.com>
Message-ID: <AANLkTim7EQWtAXdvPctSh0pfiZswkKYK29b+oLSE4zEz@mail.gmail.com>

On Tue, Sep 14, 2010 at 6:29 PM, Benjamin Root <ben.root at ou.edu> wrote:
> Good point.? I guess I am just a little *too* terminal-oriented.

It's probably worth mentioning that we've gone to great lengths to try
to produce in the new console an experience that's as seamless and
fluid as possible to anyone who 'lives in a terminal' (like myself).
We want this to be 'a terminal, but better': multiline editing, inline
graphics, html documentation, popups with call tips, but all the
keyboard friendliness and raw efficiency of a terminal.  Put another
way: this should be 100% usable *without* a mouse, and you should be
more efficient with this in python than with any terminal.  If you're
not, it's a bug :)

Cheers,


f


From gael.varoquaux at normalesup.org  Wed Sep 15 01:50:58 2010
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 15 Sep 2010 07:50:58 +0200
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
	<AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
Message-ID: <20100915055058.GB30030@phare.normalesup.org>

On Tue, Sep 14, 2010 at 05:04:57PM -0700, Fernando Perez wrote:
> MMh, too many good news together, I should watch for pianos falling
> from windows on my way home :)

Nah, don't always blame it on Microsoft :)

Ga?l


From Fernando.Perez at berkeley.edu  Wed Sep 15 01:59:09 2010
From: Fernando.Perez at berkeley.edu (Fernando Perez)
Date: Tue, 14 Sep 2010 22:59:09 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
Message-ID: <AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>

Hey,

[ for the list - Brian just did a very thorough review of a ton of
recent code I wrote while he was traveling.  Many thanks go to him for
the careful and time-consuming job, I'm replying on list further so
that we have a public record of whatever design decisions arise from
this, since some of the code touches on fairly core parts of a design
we're trying to stabilize for the long haul].

On Mon, Sep 13, 2010 at 22:22, Brian Granger <ellisonbg at gmail.com> wrote:
> Fernando,
>
> I went through all of your commits over the last few weeks and did a
> code review. ?I focused on stuff in the kernel, but I did look over
> almost all of the commits. ?Overall the code looks great, fantastic
> work!
>
> Here are my comments:
>
> http://github.com/ipython/ipython/commit/60bf09757856f7769eb79c2581a86656de4d275a
>
> Does the kernel know anything about cell mode? ?If so, we need to go over this
> more carefully. Added later: I now see that the kernel does know about cell
> mode. We should talk more about this.

Yes, what we need to do is rationalize the number of 'run*' entry
points we have.  Those have accreted over time, starting with the old,
convoluted and unnecessary approach that we inherited from code.py
(push()->runsource()->runcode()).

I added run_cell separately just as a new, safe place to see if we
could get the right semantics in place.  I'm very happy with the
user-facing behavior we get, but we should all agree on whether my
happiness is justified, and separately, with what to do with the API
to keep the good behavior but with a less cahotic underpinning.

The behavior I implemented (the one I like) is that run_cell() takes a
multiline string and does:

- if it's a single statement, compiles it and runs it in 'single' mode
(tiggering sys.displayhook).

- if it contains multiple blocks, it decides:
  a. If the last one looks like a very compact statement (one or two
lines of source): all but the last are joined and executed in 'exec'
mode (no sys.displayhook) and the last is executed in 'single' mode.
  b. Otherwise, the entire cell goes in 'exec' mode, much like a script.

In practice, I really like this behavior: we get the convenience of
Out[] results for either single statements or the last thing in a
cell, which is nice, without the clutter of multiple Out[] values
coming out of a single cell.

The basic idea is: one 'cell' of input will give at most one Out[]
value, and it will correspond to the last statement in the cell.

Having used it, I think it's a good human interface.

The first question is then: do we agree on this as the behavior we
want going forward?  (independent of how it's done under the hood).
My vote is obviously yes, but I'm totally open to feedback, of course.

Now, on to the important issue of our weedy garden of run*() calls...
We currently have:

newkernel)uqbar[core]> grep 'def run' interactiveshell.py
    def run_cell(self, cell):
    def runlines(self, lines, clean=False):
    def runsource(self, source, filename='<input>', symbol='single'):
    def runcode(self, code_obj, post_execute=True):

Let's think a little about how many methods and with what semantics we
want, though obviously we need to be careful in the transition period
not to break the existing code. We can use as a marker that the new
api uses run_ names (with underscore) and we'll eventually
deprecate/remove the run* ones (no underscore) we inherited from
code.py (and to which we added runlines).

Here's a first pass at the problem.  We must keep in mind that we want
an API that's suitable for non-python clients, that are incapable of
using inputsplitter to do syntax transformations, so they can only
feed us raw 'ipython language' source (e.g. a web client).

1. execute_code(pycode, mode='exec')  # pretty much what run_code is today
  - no history management
  - pycode is either a compiled code object or python source code.  If
it doesn't just compile(), a syntax error is raised and that's it; no
source transformations are applied.
  - since this is just a wrapper around the exec() call, I think we
should rename it to execute_code()
  - if pycode is compiled code, the mode flag is ignored.  If it's
source, it determines the compilation mode (see below).

2. run_cell(ipycode):
  - transforms runs input through syntax transformer
  - manages history
  - makes actual call to execute_code()
  - exposes the semantics for single/exec modes I outlined above, but
does so by simply calling execute_mode() with the necessary mode
flags.

I'm using the loose naming convention: 'execute -> low-level,
basically a wrapper around exec()' and 'run -> higher level, manages
history, extended syntax, etc'.  I'm not crazy about the fact that our
kernel api currently uses execute() as its entry point for all
executions, but perhaps that's OK.  The messaging spec doesn't need to
match every detail of the functional API.  Dissent welcome, though.

Thoughts?

> http://github.com/ipython/ipython/commit/c3bafd129ec29601280dcab4a7bd9008ce35dd15
>
> Last summer we decided to move away from having separate files for each platform
> for modules in utils. The code in _process_* is pretty short, any reason to
> not simply put it all in a single file like the other modules in utils?

I honestly don't remember the decision from last summer, sorry :)
Could you remind me of the reasoning, if you recall?  I guess I'm a
little allergic to

if sys.platform...:
  def f()

else:
  def f()

styles...  But I agree that the diamond diagram we have with four
files (_process_common -> (_process_win, _process_posix) -> process)
isn't exactly pretty either, so I'm open to reconsidering.

We can change it back once things settle down with what Evan is doing
and I run the full tests with pexpect on win32: at that point we'll
know if the _win code needs to grow a lot.  Because once it gets big
enough, I do think the separate files are better than the if/else
approach.  So how about this: let's wait a little, and if once the
dust settles the actual files remain small enough, we'll merge them
back in.

> http://github.com/ipython/ipython/commit/566d18a2fa3f7c79c68c0597f1e4d13c34b580dd
> http://github.com/ipython/ipython/commit/380ef8fffb3cac803d291d64359bd34a1efc5935
> http://github.com/ipython/ipython/commit/94057a1f613b31d4cdec7cea2db485577581916e
>
> Beautiful feature!

Thanks :)

> We should talk about the post_execution stuff. ?See the problem it is solving
> in this case, but not sure about the solution.

Oh, I'm very much not sure about it either.  This is probably the
major thing I added that I knew we'd need to revisit; at first I just
wanted to see if we could pull it off at all.

My thinking is actually moving closer and closer to using context
managers.  We basically offer:

- pre-execution hooks
- post-execution stuff (in the ugly/temporary approach I used above)
- custom exception handlers.

That smells *a lot* like a context manager wrapping our exec calls.
But I'm a little afraid of jumping fully into context managers for
that (the semantics of nested managers, when dealing with exceptions
are highly non-ideal in python2.6, though quite a bit improved in
2.7).

I ran the idea by Min at lunch and he liked it, but it really needs
more thought.  On the one hand we're doing something *so close* to
context managers that it seems silly not to just use that.  On the
other hand I'm worried about the need for tighter exception control
than we get from the context manager protocol.

> When I first saw "paste", it was confused by its name. The word "paste" has
> a very generic meaning for all of us (as in cut/copy/paste) and I think
> something as specific as matplotlib inline figure insertion should have a more
> specific name...unless we envision the paste function becoming more generic.

Yes, paste is not ideal and I don't like it either.  I was thinking
'pastefig' earlier today: it's pretty explicit and matches the
sound/feel of 'savefig', already in mpl.  How does that sound?

> http://github.com/ipython/ipython/commit/888820b14f618dd6fee62c216aad4896694241f2
>
> I don't remember the details, but I thought that the zmqshell needed to have
> different logic in init_io, but I see that you have removed the init_io from
> zmqshell. Can you explain this to me?

Well, I just looked at the code and, as written, it was a near
duplicate of the parent.  The only difference is that it didn't use on
win32 the ANSI-enabled console, but that just adds an extra feature to
support ANSI escapes and is otherwise identical, so it didn't seem
necessary to have an overridden method.  I couldn't find a good reason
to keep it, functionally...

> http://github.com/ipython/ipython/commit/205d1241376996eb8ca5e72c7c29d4689a80fbb8
>
> Let's go over the execution model and assumptions in the kernel. I want to
> make sure that the behavior of the kernel in this respect is going to be
> stable moving forward. Most of the issue is that you have done a lot and I
> want to make sure I understand where we are at.

sure, I think I explained that one above better, and the code as its
stands now is far better documented:

http://github.com/ipython/ipython/blob/newkernel/IPython/core/interactiveshell.py#L2112

Though we'll still want to revisit all this, as per the discussion above.

> http://github.com/ipython/ipython/commit/3658d7b687325f6f16f9a0f2289f7bade51aee82
>
> This reminds me I need to add the connect_request handler to the msg_types
> in ipkernel.py.
>
> Let's rename "instant_death" to "now" everywhere as people are used to this
> word in this context (sudo shutdown -h now)

Sounds good, I'll do that in a quick commit now (no pun intended :).
I wasn't very happy with that name either, but couldn't think of a
better one.

> http://github.com/ipython/ipython/commit/0a7f662cac8f303c25ffc8ed8fe625f349c682bd
>
> Let's go over this. ?I realize that the cell stuff is important, but I am a
> bit hesitant to teach the kernel about cells in a hackish (your own words ;-)
> way. I don't want to implement things now that cause API or behavior breakage
> when we remove the hacks.

This one is OK from an api/stability perspective: it's 100 internal,
and our API would remain the same moving forward.  What's ugly is the
double-pass, combining my static analysis with Robert's block one.
The real solution is to use the new AST module, which Robert couldn't
use because he wrote that code for 2.5 and ast.py is only in 2.6.

But I don't know the ast module/use well right now, so I figured this
ugly solution would have to do for now.  It's robust, 100% internal to
our own stuff, and once we have the time (or somebody helps us) it can
be replaced with a clean, AST-based one.

> The initialization of the input_splitter in init_instance_attrs should be
> moved to somewhere else. Basically, init_instance_attrs should eventually be
> retired and anything that is traited should not be in there. ?You can have
> traits autocreate the input splitter by doing::
>
> input_splitter =
> Instance('IPython.core.inputsplitter.IPythonInputSplitter',(),{})

Ah, OK, cool.

> http://github.com/ipython/ipython/commit/e7e389c1052ceda15656f84edf8cff41f3640992
>
> I know this is not nice to say, but I think we should use a different data
> structure than a NamedTuple or dict here. Basically, we want the message
> protocol to use universal data structures. The closest thing to a NamedTuple
> is a list of tuples:
>
> data = [('type_name',type_name),('base_class',base_class), etc.]
>
> This will be both ordered and easy for both Python and Javascript to
> parse and print. ?It will also be easy for Python frontends to put the data
> back into a NamedTuple if needed. ?But the wire protocol should be universal.
> If we just send the dict, Javascript clients will loose the ordering info.

Totally agreed, no worries.  Right now only a python client could
reconstruct the order by importing the fields list, by switching to a
list-of-pairs we'll make it language-agnostic.

> http://github.com/ipython/ipython/commit/4e3685be1b53680b1894da6bba96141ed582b123
>
> Add blank line right before def _ofind(...)

yup :)

> http://github.com/ipython/ipython/commit/06dcbd4381870cd44c9b76e7a7be2c0431086264
>
> See above comment about separate _process_*.py files.

As above, let's wait on these until the dust settles, and we'll merge
them back if they stay small.

> http://github.com/ipython/ipython/commit/2be9133b2a3035f51932b3ebd9ac054a9b3e28ba
>
> Maybe rename get_user_variables to something that reflects we are getting
> the repr's of the variables?

We have these two that return reprs:

get_user_variables
eval_expressions

But in python we don't tend to add type names to function names (the
builtin is called sorted() despite the fact that it invariably returns
a list, not sorted_list()).  But I'm open to better name
suggestions...

> Shouldn't the prompt defaults in the IPythonWidget class just be in the
> traits definitions, not at the top of the file? ?They are not that long...

No object, I just didn't want to move too  much of the Qt code around,
so I added what I needed next to what was there already.

> Around L257? ?"FIXME - fperez: this should be a silent code request." Does
> this need fixing still?

Already fixed by Evan after I added the underlying support in the kernel.

> http://github.com/ipython/ipython/commit/2be9133b2a3035f51932b3ebd9ac054a9b3e28ba#L5R152
>
> Does this FIXME affect the msg spec? Same with other FIXMEs in this file.
> I just want to make sure that all of the FIXMEs in ipkernel are completely
> internal details that don't affect the msg spec or kernel behavior.

No, not the messaging spec.  It's just that we can't get rid of this
one until we stop using runlines() and disentangle our run/exec
routines.  But it's OK to leave it there, it's ugly but 100% internal
for now.  When we stop using runlines and exceptions propagate where
they should, then we can get rid of that hack.

Thanks again for the super review!  Given the mountain of code in
there, you did a tremendous job and I greatly appreciate it.

Let's try to knock out the big api ones, which are the run/exec design
questions above.   The rest is pretty easy stuff.

Cheers,

f


From fperez.net at gmail.com  Wed Sep 15 02:00:59 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 14 Sep 2010 23:00:59 -0700
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <20100915055058.GB30030@phare.normalesup.org>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
	<AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
	<20100915055058.GB30030@phare.normalesup.org>
Message-ID: <AANLkTik-3eCDguA1Y3n4sD7-WSybTiwEwOg=0YUjMEb6@mail.gmail.com>

On Tue, Sep 14, 2010 at 10:50 PM, Gael Varoquaux
<gael.varoquaux at normalesup.org> wrote:
> On Tue, Sep 14, 2010 at 05:04:57PM -0700, Fernando Perez wrote:
>> MMh, too many good news together, I should watch for pianos falling
>> from windows on my way home :)
>
> Nah, don't always blame it on Microsoft :)

Good catch :)

f


From takowl at gmail.com  Wed Sep 15 16:04:15 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 15 Sep 2010 21:04:15 +0100
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
	<AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
Message-ID: <AANLkTi=cqYvi=pQ9fWrSZXVv5BN9gBvOnFh20MLfcAuE@mail.gmail.com>

On 15 September 2010 01:04, Fernando Perez <fperez.net at gmail.com> wrote:

> Fantastic, many thanks!!! We're 'landing' newkernel right now, so we
> may wait until you return from holiday (just not enough time to do
> both), but this is very, very much appreciated.
>

No problem. I'll keep updating the code from newkernel.


> That is handled by showsyntaxerror, line 1506 in
> core.interactiveshell.  This calls a special exception handler that
> was constructed in line 1338:
>

Thanks, that's great, and I've fixed it now. It was unpacking the exception
into variables, which is no longer possible in Python 3. I think this will
be compatible with Python 2.6, so I may put it back into ipy3-preparation at
some point. One question: the code describes stuffing the filename into the
exception '(because Python's parser always uses "<string>" when reading from
a string)'. In what context is this used? Disabling it doesn't seem to make
any difference in the cases I've tried, but have I overlooked something?

Best wishes,
Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/37af8394/attachment.html>

From fperez.net at gmail.com  Wed Sep 15 16:49:05 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 15 Sep 2010 13:49:05 -0700
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <AANLkTi=cqYvi=pQ9fWrSZXVv5BN9gBvOnFh20MLfcAuE@mail.gmail.com>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
	<AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
	<AANLkTi=cqYvi=pQ9fWrSZXVv5BN9gBvOnFh20MLfcAuE@mail.gmail.com>
Message-ID: <AANLkTi=dgsHGmeE+hM6xfGmynazj0sMsP2_887YLYrMR@mail.gmail.com>

On Wed, Sep 15, 2010 at 1:04 PM, Thomas Kluyver <takowl at gmail.com> wrote:
>
> Thanks, that's great, and I've fixed it now. It was unpacking the exception
> into variables, which is no longer possible in Python 3. I think this will
> be compatible with Python 2.6, so I may put it back into ipy3-preparation at
> some point. One question: the code describes stuffing the filename into the
> exception '(because Python's parser always uses?"<string>" when reading from
> a string)'. In what context is this used? Disabling it doesn't seem to make
> any difference in the cases I've tried, but have I overlooked something?

It may matter when running code from scripts, via %run.  You can try
that...  Sorry to be a little vague, I just don't recall precisely
what it could be, and that's the price we pay for not having written
years ago a unit test that would validate that feature.

Cheers,

f


From takowl at gmail.com  Wed Sep 15 17:44:51 2010
From: takowl at gmail.com (Thomas Kluyver)
Date: Wed, 15 Sep 2010 22:44:51 +0100
Subject: [IPython-dev] IPython with Py3k
In-Reply-To: <AANLkTi=dgsHGmeE+hM6xfGmynazj0sMsP2_887YLYrMR@mail.gmail.com>
References: <AANLkTi==wB_EriLUuD_RMCiu+GU1N=wnkQTm7=Hsz0DU@mail.gmail.com>
	<AANLkTi=PJOmX-iEGrkKd9KHFXW4Wam8R4uB=0C=JH-1E@mail.gmail.com>
	<AANLkTi=cqYvi=pQ9fWrSZXVv5BN9gBvOnFh20MLfcAuE@mail.gmail.com>
	<AANLkTi=dgsHGmeE+hM6xfGmynazj0sMsP2_887YLYrMR@mail.gmail.com>
Message-ID: <AANLkTim7_-42bDKUF9Y5A3NRzK8A+=7jGb=XKfPgUPZP@mail.gmail.com>

On 15 September 2010 21:49, Fernando Perez <fperez.net at gmail.com> wrote:

> It may matter when running code from scripts, via %run.  You can try
> that...  Sorry to be a little vague, I just don't recall precisely
> what it could be, and that's the price we pay for not having written
> years ago a unit test that would validate that feature.
>

Not to worry. I tried %run, and that works fine without. I tried doctests as
well, but the doctest module prints its own error display (without pretty
colours). I've restored a simple equivalent of the code that was there for
now.

Thomas
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100915/b2458df0/attachment.html>

From fperez.net at gmail.com  Thu Sep 16 16:36:14 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 16 Sep 2010 13:36:14 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
Message-ID: <AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>

On Tue, Sep 14, 2010 at 8:21 AM, John Hunter <jdh2358 at gmail.com> wrote:
>
> How about this as an alternative: on my box, I can drag the "source
> code" link from the browser into my terminal, which by default pastes
> the URL of the referenced *.py into the terminal. ?If "run" supported
> a -w (web) option, or automatically detected that the URL starts with
> http, it could do a web run of the file. ?Of course, you may want the
> source code pasted in for illustrative purposes... To support this,
> you could add a -u (url) option to "paste" which assumes the input is
> a url, fetches it, and pastes the contents into ipython. ?So you could
> type "paste -u" and then drag the link into the terminal, and it would
> fetch it and paste the code into an input block.

Ask and ye shall receive (yes, the url was drag-dropped from the
'source code' link in the mpl page), welcome %loadpy:

http://fperez.org/tmp/iqlab_mpl_loadpy.png

Full credits go to Brian and Evan!

Cheers,

f


From benjaminrk at gmail.com  Thu Sep 16 16:56:29 2010
From: benjaminrk at gmail.com (MinRK)
Date: Thu, 16 Sep 2010 13:56:29 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
	<AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>
Message-ID: <AANLkTikkBoYfLePmPwuyH+dCn4vRbGT22j=2SNZ_OeUo@mail.gmail.com>

That's very cool.

Unrelated to %loadpy, but is anyone else bothered/confused by the fact that
the both the plot in the website and the plot embedded in the app are wrong?
 There are lines that should be blue (they don't intersect the bbox) in both
that are red.  I presume this is a bug in either the intersect calculation,
or the plot command in the example code.

-MinRK

On Thu, Sep 16, 2010 at 13:36, Fernando Perez <fperez.net at gmail.com> wrote:

> On Tue, Sep 14, 2010 at 8:21 AM, John Hunter <jdh2358 at gmail.com> wrote:
> >
> > How about this as an alternative: on my box, I can drag the "source
> > code" link from the browser into my terminal, which by default pastes
> > the URL of the referenced *.py into the terminal.  If "run" supported
> > a -w (web) option, or automatically detected that the URL starts with
> > http, it could do a web run of the file.  Of course, you may want the
> > source code pasted in for illustrative purposes... To support this,
> > you could add a -u (url) option to "paste" which assumes the input is
> > a url, fetches it, and pastes the contents into ipython.  So you could
> > type "paste -u" and then drag the link into the terminal, and it would
> > fetch it and paste the code into an input block.
>
> Ask and ye shall receive (yes, the url was drag-dropped from the
> 'source code' link in the mpl page), welcome %loadpy:
>
> http://fperez.org/tmp/iqlab_mpl_loadpy.png
>
> Full credits go to Brian and Evan!
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100916/2a1d9a21/attachment.html>

From robert.kern at gmail.com  Thu Sep 16 18:17:15 2010
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 16 Sep 2010 17:17:15 -0500
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
	<AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>
Message-ID: <i6u51b$3oe$1@dough.gmane.org>

On 9/16/10 3:36 PM, Fernando Perez wrote:
> On Tue, Sep 14, 2010 at 8:21 AM, John Hunter<jdh2358 at gmail.com>  wrote:
>>
>> How about this as an alternative: on my box, I can drag the "source
>> code" link from the browser into my terminal, which by default pastes
>> the URL of the referenced *.py into the terminal.  If "run" supported
>> a -w (web) option, or automatically detected that the URL starts with
>> http, it could do a web run of the file.  Of course, you may want the
>> source code pasted in for illustrative purposes... To support this,
>> you could add a -u (url) option to "paste" which assumes the input is
>> a url, fetches it, and pastes the contents into ipython.  So you could
>> type "paste -u" and then drag the link into the terminal, and it would
>> fetch it and paste the code into an input block.
>
> Ask and ye shall receive (yes, the url was drag-dropped from the
> 'source code' link in the mpl page), welcome %loadpy:
>
> http://fperez.org/tmp/iqlab_mpl_loadpy.png
>
> Full credits go to Brian and Evan!

I see there is an In[2] missing.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From fperez.net at gmail.com  Thu Sep 16 18:24:13 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Thu, 16 Sep 2010 15:24:13 -0700
Subject: [IPython-dev] IPython (new) + matplotlib report: happy news
In-Reply-To: <i6u51b$3oe$1@dough.gmane.org>
References: <AANLkTik8xi5XnjMc2Pw5iSXcM2hqMnEZUMY1e-4diCnd@mail.gmail.com>
	<AANLkTimraDLkLoLfktDoctQr6a3JnS9hiOv+kNicd7z4@mail.gmail.com>
	<AANLkTi=V-qeJ+cxejZju09P9VWPGtcYK8aYgwC4E6D33@mail.gmail.com>
	<AANLkTinf7OqFvNOYE3Fx3Jx9VFpCKTjsMzggqUJziJL3@mail.gmail.com>
	<i6u51b$3oe$1@dough.gmane.org>
Message-ID: <AANLkTimRYXWEihrD4Fm7=q6F5Aoqd=tzt0O6jrJQyqrP@mail.gmail.com>

On Thu, Sep 16, 2010 at 3:17 PM, Robert Kern <robert.kern at gmail.com> wrote:
> I see there is an In[2] missing.
>

row 19 of the todo spreadsheet, with my name on it :)

I'm wrapping up the previous code review, and will take this bug on
later.  It's actually not difficult, but I need a few open hours
because I need to dig into the core execution loop to really do it
right.  Brian and I spent an hour on the design last night, I'll try
to get to it on Saturday.

Silly of me to think you wouldn't notice ;)

Cheers,

f


From fperez.net at gmail.com  Fri Sep 17 19:31:00 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Fri, 17 Sep 2010 16:31:00 -0700
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
Message-ID: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>

Hi folks,

I just wanted to get a feel for what we have left to get 0.10.1 out
the door officially. Pretty much 100% of my energy and time on ipython
has gone towards the 0.11 recent work, but there's a lot of production
use of 0.10 out there.  Enthought has an EPD release planned in the
not too distant future, and it would be great to get IPython 0.10.1 in
there, since EPD is the delivery mechanism for many ipython users on
Win32 and OSX.

The big item we had in the wings was the great work Justin, Satra and
others (forgive me if I'm mis-allocating credit) have been putting in
to get SGE fully supported for 0.10.1.  Can you remind me of the
status of that?

Are there any other ready-to-go last minute fixes?  If you point me to
the right branches to pull from, I can cut an RC for 0.10.1 for final
testing, and we'll get ready to ship this in the next couple of weeks.

Thoughts, etc?  Anything else I forgot? (quite likely, with my
attention span being worse than my cat's, all the 0.11 work has mostly
erased 0.10 from my brain...)

Cheers,

f


From satra at mit.edu  Fri Sep 17 20:52:58 2010
From: satra at mit.edu (Satrajit Ghosh)
Date: Fri, 17 Sep 2010 20:52:58 -0400
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
Message-ID: <AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>

hi fernando,

you can merge the changes from justin's 0.10.1-sge branch. that contains the
sge+lsf support fixes. other than that i think you already have min's taskid
fix for the controller (i.e the clear function now accepts a task id).

cheers,

satra


On Fri, Sep 17, 2010 at 7:31 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hi folks,
>
> I just wanted to get a feel for what we have left to get 0.10.1 out
> the door officially. Pretty much 100% of my energy and time on ipython
> has gone towards the 0.11 recent work, but there's a lot of production
> use of 0.10 out there.  Enthought has an EPD release planned in the
> not too distant future, and it would be great to get IPython 0.10.1 in
> there, since EPD is the delivery mechanism for many ipython users on
> Win32 and OSX.
>
> The big item we had in the wings was the great work Justin, Satra and
> others (forgive me if I'm mis-allocating credit) have been putting in
> to get SGE fully supported for 0.10.1.  Can you remind me of the
> status of that?
>
> Are there any other ready-to-go last minute fixes?  If you point me to
> the right branches to pull from, I can cut an RC for 0.10.1 for final
> testing, and we'll get ready to ship this in the next couple of weeks.
>
> Thoughts, etc?  Anything else I forgot? (quite likely, with my
> attention span being worse than my cat's, all the 0.11 work has mostly
> erased 0.10 from my brain...)
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100917/29b0757e/attachment.html>

From fperez.net at gmail.com  Sat Sep 18 06:13:49 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 18 Sep 2010 03:13:49 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
	<AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
Message-ID: <AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>

Hey,

I've just finished up the main cleanups post-review, thanks again!

In the process I ended up implementing call signature in the calltips,
along with at least a few unittests for the object inspector (going
with the idea of trying to add unittests to any new code as much as
possible).  Now that we have calltips, I was really missing the
signature in them :)

Evan had mentioned adding more sophisticated calltips that would carry
syntax highlighting of the current argument being typed; all the info
is in there and we can extend to that later. For now I'm just
formatting the signature as a plain string, and if someone can
contribute the fancier version later, it's just a matter of changing
one flag to prevent plain string conversion and doing something
better.  But even the plain string version is very useful, I think.

On Tue, Sep 14, 2010 at 10:59 PM, Fernando Perez
<Fernando.Perez at berkeley.edu> wrote:

>> Does the kernel know anything about cell mode? ?If so, we need to go over this
>> more carefully. Added later: I now see that the kernel does know about cell
>> mode. We should talk more about this.
>
> Yes, what we need to do is rationalize the number of 'run*' entry
> points we have. ?Those have accreted over time, starting with the old,
> convoluted and unnecessary approach that we inherited from code.py
> (push()->runsource()->runcode()).

[...]

I documented the execution semantics a lot more:

http://ipython.scipy.org/doc/nightly/html/development/messaging.html#execute

We still have more work to do on this front, as discussed.  I'll try
to crack the off-by-one prompt bug and organize the core execution
loop better this weekend.

> Yes, paste is not ideal and I don't like it either. ?I was thinking
> 'pastefig' earlier today: it's pretty explicit and matches the
> sound/feel of 'savefig', already in mpl. ?How does that sound?

Done, it's now pastefig()

>> Let's rename "instant_death" to "now" everywhere as people are used to this
>> word in this context (sudo shutdown -h now)
>
> Sounds good, I'll do that in a quick commit now (no pun intended :).
> I wasn't very happy with that name either, but couldn't think of a
> better one.

Done.

>> The initialization of the input_splitter in init_instance_attrs should be
>> moved to somewhere else. Basically, init_instance_attrs should eventually be
>> retired and anything that is traited should not be in there. ?You can have
>> traits autocreate the input splitter by doing::
>>
>> input_splitter =
>> Instance('IPython.core.inputsplitter.IPythonInputSplitter',(),{})
>
> Ah, OK, cool.

Done.

>> http://github.com/ipython/ipython/commit/e7e389c1052ceda15656f84edf8cff41f3640992
>>
>> I know this is not nice to say, but I think we should use a different data
>> structure than a NamedTuple or dict here. Basically, we want the message
>> protocol to use universal data structures. The closest thing to a NamedTuple
>> is a list of tuples:
>>
>> data = [('type_name',type_name),('base_class',base_class), etc.]
>>
>> This will be both ordered and easy for both Python and Javascript to
>> parse and print. ?It will also be easy for Python frontends to put the data
>> back into a NamedTuple if needed. ?But the wire protocol should be universal.
>> If we just send the dict, Javascript clients will loose the ordering info.
>
> Totally agreed, no worries. ?Right now only a python client could
> reconstruct the order by importing the fields list, by switching to a
> list-of-pairs we'll make it language-agnostic.

I removed the namedtuple but left it as a dict for now.  The
underlying data structure was a dict and that goes fine over json, so
there's no problem with that.  Even for non-python clients, it may be
more convenient to just leave the json dict alone and ship (once) the
list of ordered names. Since that may turn out to be the cleaner
solution in the long run, and leaving the dicts alone actually made
the *current* code we do have a fair bit simpler, I decided against
the list-of-tuples for now.  Go with the simplest solution that works
and only add complexity later if we do really have a use for it.

>> http://github.com/ipython/ipython/commit/4e3685be1b53680b1894da6bba96141ed582b123
>>
>> Add blank line right before def _ofind(...)
>
> yup :)

Done.

>> http://github.com/ipython/ipython/commit/2be9133b2a3035f51932b3ebd9ac054a9b3e28ba
>>
>> Maybe rename get_user_variables to something that reflects we are getting
>> the repr's of the variables?
>
> We have these two that return reprs:
>
> get_user_variables
> eval_expressions

Done as discussed: user_variables, user_expressions.  Documented and
changed the method api to match the messaging spec.

Thanks again!

Cheers,

f


From ellisonbg at gmail.com  Sat Sep 18 12:10:46 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 18 Sep 2010 09:10:46 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
	<AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
	<AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>
Message-ID: <AANLkTimhiH3FttqKu8TpQDy-3Qf3EEN5oUhjJMV5S=-9@mail.gmail.com>

Fernando,

I just looked through everything and it looks great. Thanks for doing
this.  Things are looking really good.

Brian

On Sat, Sep 18, 2010 at 3:13 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey,
>
> I've just finished up the main cleanups post-review, thanks again!
>
> In the process I ended up implementing call signature in the calltips,
> along with at least a few unittests for the object inspector (going
> with the idea of trying to add unittests to any new code as much as
> possible). ?Now that we have calltips, I was really missing the
> signature in them :)
>
> Evan had mentioned adding more sophisticated calltips that would carry
> syntax highlighting of the current argument being typed; all the info
> is in there and we can extend to that later. For now I'm just
> formatting the signature as a plain string, and if someone can
> contribute the fancier version later, it's just a matter of changing
> one flag to prevent plain string conversion and doing something
> better. ?But even the plain string version is very useful, I think.
>
> On Tue, Sep 14, 2010 at 10:59 PM, Fernando Perez
> <Fernando.Perez at berkeley.edu> wrote:
>
>>> Does the kernel know anything about cell mode? ?If so, we need to go over this
>>> more carefully. Added later: I now see that the kernel does know about cell
>>> mode. We should talk more about this.
>>
>> Yes, what we need to do is rationalize the number of 'run*' entry
>> points we have. ?Those have accreted over time, starting with the old,
>> convoluted and unnecessary approach that we inherited from code.py
>> (push()->runsource()->runcode()).
>
> [...]
>
> I documented the execution semantics a lot more:
>
> http://ipython.scipy.org/doc/nightly/html/development/messaging.html#execute
>
> We still have more work to do on this front, as discussed. ?I'll try
> to crack the off-by-one prompt bug and organize the core execution
> loop better this weekend.
>
>> Yes, paste is not ideal and I don't like it either. ?I was thinking
>> 'pastefig' earlier today: it's pretty explicit and matches the
>> sound/feel of 'savefig', already in mpl. ?How does that sound?
>
> Done, it's now pastefig()
>
>>> Let's rename "instant_death" to "now" everywhere as people are used to this
>>> word in this context (sudo shutdown -h now)
>>
>> Sounds good, I'll do that in a quick commit now (no pun intended :).
>> I wasn't very happy with that name either, but couldn't think of a
>> better one.
>
> Done.
>
>>> The initialization of the input_splitter in init_instance_attrs should be
>>> moved to somewhere else. Basically, init_instance_attrs should eventually be
>>> retired and anything that is traited should not be in there. ?You can have
>>> traits autocreate the input splitter by doing::
>>>
>>> input_splitter =
>>> Instance('IPython.core.inputsplitter.IPythonInputSplitter',(),{})
>>
>> Ah, OK, cool.
>
> Done.
>
>>> http://github.com/ipython/ipython/commit/e7e389c1052ceda15656f84edf8cff41f3640992
>>>
>>> I know this is not nice to say, but I think we should use a different data
>>> structure than a NamedTuple or dict here. Basically, we want the message
>>> protocol to use universal data structures. The closest thing to a NamedTuple
>>> is a list of tuples:
>>>
>>> data = [('type_name',type_name),('base_class',base_class), etc.]
>>>
>>> This will be both ordered and easy for both Python and Javascript to
>>> parse and print. ?It will also be easy for Python frontends to put the data
>>> back into a NamedTuple if needed. ?But the wire protocol should be universal.
>>> If we just send the dict, Javascript clients will loose the ordering info.
>>
>> Totally agreed, no worries. ?Right now only a python client could
>> reconstruct the order by importing the fields list, by switching to a
>> list-of-pairs we'll make it language-agnostic.
>
> I removed the namedtuple but left it as a dict for now. ?The
> underlying data structure was a dict and that goes fine over json, so
> there's no problem with that. ?Even for non-python clients, it may be
> more convenient to just leave the json dict alone and ship (once) the
> list of ordered names. Since that may turn out to be the cleaner
> solution in the long run, and leaving the dicts alone actually made
> the *current* code we do have a fair bit simpler, I decided against
> the list-of-tuples for now. ?Go with the simplest solution that works
> and only add complexity later if we do really have a use for it.
>
>>> http://github.com/ipython/ipython/commit/4e3685be1b53680b1894da6bba96141ed582b123
>>>
>>> Add blank line right before def _ofind(...)
>>
>> yup :)
>
> Done.
>
>>> http://github.com/ipython/ipython/commit/2be9133b2a3035f51932b3ebd9ac054a9b3e28ba
>>>
>>> Maybe rename get_user_variables to something that reflects we are getting
>>> the repr's of the variables?
>>
>> We have these two that return reprs:
>>
>> get_user_variables
>> eval_expressions
>
> Done as discussed: user_variables, user_expressions. ?Documented and
> changed the method api to match the messaging spec.
>
> Thanks again!
>
> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Sat Sep 18 14:18:55 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 18 Sep 2010 11:18:55 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTimhiH3FttqKu8TpQDy-3Qf3EEN5oUhjJMV5S=-9@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
	<AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
	<AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>
	<AANLkTimhiH3FttqKu8TpQDy-3Qf3EEN5oUhjJMV5S=-9@mail.gmail.com>
Message-ID: <AANLkTimWJz_C8ZFBy23V7WUABN7VEg=WhaHkNpm-ZEeF@mail.gmail.com>

On Sat, Sep 18, 2010 at 9:10 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> I just looked through everything and it looks great. Thanks for doing
> this. ?Things are looking really good.

Great, thanks for the careful eyes!  Review is time and
energy-consuming, I really appreciate your thoroughness.  It's made
our codebase that much better.

I also noticed the github pull requests got much, much better:

http://github.com/blog/712-pull-requests-2-0

they pretty much fixed all the things that bothered me about their
prior system.  As we slow down the frenzy in newkernel, it will be
very nice to be able to use a stable, archived and featureful pull
request system for review.

Regards,

f


From ellisonbg at gmail.com  Sat Sep 18 14:22:37 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Sat, 18 Sep 2010 11:22:37 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTimWJz_C8ZFBy23V7WUABN7VEg=WhaHkNpm-ZEeF@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
	<AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
	<AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>
	<AANLkTimhiH3FttqKu8TpQDy-3Qf3EEN5oUhjJMV5S=-9@mail.gmail.com>
	<AANLkTimWJz_C8ZFBy23V7WUABN7VEg=WhaHkNpm-ZEeF@mail.gmail.com>
Message-ID: <AANLkTinU+Jk4gV_1zDPvcX43z0TUqBcBP9-qLaMN16_T@mail.gmail.com>

On Sat, Sep 18, 2010 at 11:18 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Sat, Sep 18, 2010 at 9:10 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>>
>> I just looked through everything and it looks great. Thanks for doing
>> this. ?Things are looking really good.
>
> Great, thanks for the careful eyes! ?Review is time and
> energy-consuming, I really appreciate your thoroughness. ?It's made
> our codebase that much better.
>
> I also noticed the github pull requests got much, much better:
>
> http://github.com/blog/712-pull-requests-2-0
>
> they pretty much fixed all the things that bothered me about their
> prior system. ?As we slow down the frenzy in newkernel, it will be
> very nice to be able to use a stable, archived and featureful pull
> request system for review.

Yes i saw that and Min and I have been using it for the pyzmq code
review.  It is pretty great.  My only complaint is that for files with
many diffs they don't show the diffs in the web page.  You have the
download the two files and do the diff by hand.  But it is vastly
better than anything else right now.

Cheers,

Brian

> Regards,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Sat Sep 18 14:23:55 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 18 Sep 2010 11:23:55 -0700
Subject: [IPython-dev] Code review (mostly for Fernando)
In-Reply-To: <AANLkTinU+Jk4gV_1zDPvcX43z0TUqBcBP9-qLaMN16_T@mail.gmail.com>
References: <AANLkTim5Nm6NezJs-CwUi_xdoqVeMj4a+hFkGpxOJCgJ@mail.gmail.com>
	<AANLkTi=HQuWN7nvYYk--qk6hEpD+o4NqL7arfZ5OAwc0@mail.gmail.com>
	<AANLkTinAVYspMkn1iP71nUkYnbcUBOp+=nRqRVZG+g7Q@mail.gmail.com>
	<AANLkTimhiH3FttqKu8TpQDy-3Qf3EEN5oUhjJMV5S=-9@mail.gmail.com>
	<AANLkTimWJz_C8ZFBy23V7WUABN7VEg=WhaHkNpm-ZEeF@mail.gmail.com>
	<AANLkTinU+Jk4gV_1zDPvcX43z0TUqBcBP9-qLaMN16_T@mail.gmail.com>
Message-ID: <AANLkTimvpzr2VSX7dHk-3fTEochOZ3-kP7Md3p7iqftd@mail.gmail.com>

On Sat, Sep 18, 2010 at 11:22 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> Yes i saw that and Min and I have been using it for the pyzmq code
> review. ?It is pretty great. ?My only complaint is that for files with
> many diffs they don't show the diffs in the web page. ?You have the
> download the two files and do the diff by hand. ?But it is vastly
> better than anything else right now.
>

Cool!  Maybe file a support ticket for that idea?  They've been really
responsive in the past, they do seem to honestly listen to people's
feedback.

Cheers,

f


From fperez.net at gmail.com  Mon Sep 20 20:17:21 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 20 Sep 2010 17:17:21 -0700
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
	<AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
Message-ID: <AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>

Hey guys,

On Fri, Sep 17, 2010 at 5:52 PM, Satrajit Ghosh <satra at mit.edu> wrote:
>
> you can merge the changes from justin's 0.10.1-sge branch. that contains the
> sge+lsf support fixes. other than that i think you already have min's taskid
> fix for the controller (i.e the clear function now accepts a task id).
>

Great, thanks for the info.  One quick question, in the diff I see
this towards the end:

-The PBS mode uses the Portable Batch System [PBS]_ to start the engines.  To us
e this mode, you first need to create a PBS script template that will be used to
 start the engines.  Here is a sample PBS script template:
+The PBS mode uses the Portable Batch System [PBS]_ to start the engines.

-.. sourcecode:: bash
+To start an ipcluster using the Portable Batch System::
+
+    $ ipcluster pbs -n 12

-    #PBS -N ipython
-    #PBS -j oe
-    #PBS -l walltime=00:10:00
-    #PBS -l nodes=${n/4}:ppn=4
-    #PBS -q parallel
+The above command will launch a PBS job array with 12 tasks using the default q
ueue. If you would like to submit the job to a different queue use the -q option
:

-    cd $$PBS_O_WORKDIR
-    export PATH=$$HOME/usr/local/bin
-    export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages
-    /usr/local/bin/mpiexec -n ${n} ipengine --logfile=$$PBS_O_WORKDIR/ipengine
+    $ ipcluster pbs -n 12 -q hpcqueue


Is it correct to remove all references to the #PBS lines?  Do  your
improvements to PBS obsolete completely the need for the manual script
writing we had before?

I just want to make sure that I understand that correctly before I
merge it, once we clarify this I'll merge and will cut an RC so we can
get it tested.

Cheers,

f


From fperez.net at gmail.com  Mon Sep 20 20:21:28 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 20 Sep 2010 17:21:28 -0700
Subject: [IPython-dev] Updated credits file
Message-ID: <AANLkTi=7ZBCUZpVtYCehcqL0GbJCS4c6SyeY_b2hw5bn@mail.gmail.com>

Hi folks,

it was just pointed out to me that I had failed to update the credits
file for a while; I did a quick scan of the authors lines in the
commits history and made this update:

http://github.com/ipython/ipython/commit/3d1d5160e091293a90dde8175d23f4bea091246b

(the names were added roughly in reverse chronological order of recent
contributions, just because that's how I scanned the log).  Please let
me know if I:

- ignored you, I'll be more than happy to add your name
- misattributed something
- missed something when mentioning you.

I really want to make sure that we credit properly everyone who
contributes to IPython in any form, so please don't be shy in
reminding us when we forget or fail to do so.  It is never out of
malice :)

Best regards,  and thanks to all who have contributed!


f


From fperez.net at gmail.com  Mon Sep 20 20:36:56 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 20 Sep 2010 17:36:56 -0700
Subject: [IPython-dev] Landing newkernel: naming entry points
Message-ID: <AANLkTikeLFRictqYU4g-dDEzGraYXHhPqaGV=H7MwefY@mail.gmail.com>

Hi folks,

I think we're mostly ready to merge the recent work done in the
'newkernel' branch back into trunk, so that it can begin to get wider
testing and polishing.  There are still things we know need work in
there, but at this point I think it's in good enough shape for at
least our -dev denizens to start playing with it.

If anyone disagrees, by all means speak up :)

One thing I'd like some feedback on is our naming policy for script
entry points.  Right now we have the following scripts installed from
newkernel (the absence of some older scripts is because their code is
in the quarantine dir awaiting porting):

uqbar[bin]> ls
ipcluster*  ipcontroller*  ipengine*  iptest*  ipython*  ipythonqt*
irunner*  pycolor*

The new zmq work is likely to quickly give us multiple new frontends:
we have the qt console (ipythonqt above), Gerardo's Qt notebook and
Omar's terminal will hopefully be updated to the kernel architecture
and be merged in, I hope Wendell's curses frontend will come, we may
get the old Wx widget and full client updated (if Gael and Laurent, or
anyone else, pitches in to bring them up to date with the new
architecture).  So we should have a rational naming scheme at least
for new names.

The ipythonqt name is up for final decision, since we haven't released
it yet, but the other names above we'll keep as-is for backwards
compatibility.  The question is how to name new script entry points
we'll necessarily develop.  Here's my current (draft) thinking:

- ipkernel: one we'll need to add for the pure-kernel part of the zmq
clients, when someone wants to create just a kernel.

- ipythonXYZ: names for 'widgets': bare-bones objects meant mostly to
be used as components embedded in other apps, though they can run on
their own.  I'm following the trail left by 'ipythonx' for the bare Wx
widget and 'ipythonqt' for the current console.

- ipython-XYZ (note the dash): these would be the names for 'main
apps'.  Gerardo's would be something like iptyhon-nb, we'd have
iptyhon-wx, ipython-curses and ipython-term for other two-process
clients yet to be finished.

How does this sound?  Any other preferences?

Once we settle this, unless I hear otherwise I'll make the necessary
adjustments to newkernel and will merge it in, so we can continue
working at the regular pace on trunk from now on.

Cheers,

f


From ellisonbg at gmail.com  Tue Sep 21 00:29:16 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Mon, 20 Sep 2010 21:29:16 -0700
Subject: [IPython-dev] Landing newkernel: naming entry points
In-Reply-To: <AANLkTikeLFRictqYU4g-dDEzGraYXHhPqaGV=H7MwefY@mail.gmail.com>
References: <AANLkTikeLFRictqYU4g-dDEzGraYXHhPqaGV=H7MwefY@mail.gmail.com>
Message-ID: <AANLkTi=p62+YJ3hb3w6AEQPtG=vJaq8seBUqZnM9rZP4@mail.gmail.com>

Fernando,

On Mon, Sep 20, 2010 at 5:36 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> I think we're mostly ready to merge the recent work done in the
> 'newkernel' branch back into trunk, so that it can begin to get wider
> testing and polishing. ?There are still things we know need work in
> there, but at this point I think it's in good enough shape for at
> least our -dev denizens to start playing with it.
>
> If anyone disagrees, by all means speak up :)
>
> One thing I'd like some feedback on is our naming policy for script
> entry points. ?Right now we have the following scripts installed from
> newkernel (the absence of some older scripts is because their code is
> in the quarantine dir awaiting porting):
>
> uqbar[bin]> ls
> ipcluster* ?ipcontroller* ?ipengine* ?iptest* ?ipython* ?ipythonqt*
> irunner* ?pycolor*
>
> The new zmq work is likely to quickly give us multiple new frontends:
> we have the qt console (ipythonqt above), Gerardo's Qt notebook and
> Omar's terminal will hopefully be updated to the kernel architecture
> and be merged in, I hope Wendell's curses frontend will come, we may
> get the old Wx widget and full client updated (if Gael and Laurent, or
> anyone else, pitches in to bring them up to date with the new
> architecture). ?So we should have a rational naming scheme at least
> for new names.
>
> The ipythonqt name is up for final decision, since we haven't released
> it yet, but the other names above we'll keep as-is for backwards
> compatibility. ?The question is how to name new script entry points
> we'll necessarily develop. ?Here's my current (draft) thinking:
>
> - ipkernel: one we'll need to add for the pure-kernel part of the zmq
> clients, when someone wants to create just a kernel.

I think this name is good.

> - ipythonXYZ: names for 'widgets': bare-bones objects meant mostly to
> be used as components embedded in other apps, though they can run on
> their own. ?I'm following the trail left by 'ipythonx' for the bare Wx
> widget and 'ipythonqt' for the current console.
>
> - ipython-XYZ (note the dash): these would be the names for 'main
> apps'. ?Gerardo's would be something like iptyhon-nb, we'd have
> iptyhon-wx, ipython-curses and ipython-term for other two-process
> clients yet to be finished.

I think this will just lead to confusion.  I don't think we want to
distinguish between the bare bones widgets-apps and the full blown
apps.  By this, I mean that we should only offer full blown apps for
our frontends.  Obviously the current ipythonqt is still bare bones on
the application side of things, but I think *it* should be the
top-level script that we should incrementally improve into a full
blown app.  That is, we should create additional widgets, application
menus, etc. and keep adding them to the mix.

But I do agree that we need to rethink how we name it.  The names need
to take into account both the toolkit (qt, wx, ...) as well as the
type of frontend (console, notebook, ...).  Some options:

ipythonqt-console
ipythonqt-nb
ipythonwx-console
ipythonwx-nb

Or

ipython-qtconsole
ipython-qtnb

ipqt-console
ipqt-nb

??? (not feeling super creative in the naming area tonight)

Cheers,

Brian

> How does this sound? ?Any other preferences?
>
> Once we settle this, unless I hear otherwise I'll make the necessary
> adjustments to newkernel and will merge it in, so we can continue
> working at the regular pace on trunk from now on.
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Tue Sep 21 01:43:55 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 20 Sep 2010 22:43:55 -0700
Subject: [IPython-dev] Landing newkernel: naming entry points
In-Reply-To: <AANLkTi=p62+YJ3hb3w6AEQPtG=vJaq8seBUqZnM9rZP4@mail.gmail.com>
References: <AANLkTikeLFRictqYU4g-dDEzGraYXHhPqaGV=H7MwefY@mail.gmail.com>
	<AANLkTi=p62+YJ3hb3w6AEQPtG=vJaq8seBUqZnM9rZP4@mail.gmail.com>
Message-ID: <AANLkTik=fTu0M-=NG8mp=vEgs4WjNnkvipWGbYCor99F@mail.gmail.com>

On Mon, Sep 20, 2010 at 9:29 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> On Mon, Sep 20, 2010 at 5:36 PM, Fernando Perez <fperez.net at gmail.com> wrote:

>> - ipkernel: one we'll need to add for the pure-kernel part of the zmq
>> clients, when someone wants to create just a kernel.
>
> I think this name is good.

OK

> I think this will just lead to confusion. ?I don't think we want to
> distinguish between the bare bones widgets-apps and the full blown
> apps. ?By this, I mean that we should only offer full blown apps for
> our frontends. ?Obviously the current ipythonqt is still bare bones on
> the application side of things, but I think *it* should be the
> top-level script that we should incrementally improve into a full
> blown app. ?That is, we should create additional widgets, application
> menus, etc. and keep adding them to the mix.
>
> But I do agree that we need to rethink how we name it. ?The names need
> to take into account both the toolkit (qt, wx, ...) as well as the
> type of frontend (console, notebook, ...). ?Some options:
>
> ipythonqt-console
> ipythonqt-nb
> ipythonwx-console
> ipythonwx-nb
>
> Or
>
> ipython-qtconsole
> ipython-qtnb

Good points, thanks.  I think I like this ipython-<toolkit><something>
style best, though I'm not feeling particularly creative with names
either.

If someone comes up with something better I'm all ears, otherwise we
can just go ahead with this suggestion of yours.

Cheers,

f


From satra at mit.edu  Tue Sep 21 10:56:00 2010
From: satra at mit.edu (Satrajit Ghosh)
Date: Tue, 21 Sep 2010 10:56:00 -0400
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
	<AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
	<AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>
Message-ID: <AANLkTi=cYi1qMSgDfShDMX-OWaDJnkzP5eDfiNp7qhNi@mail.gmail.com>

hi fernando,

the way justin set it up is that PBS/SGE/LSF obviates the need for writing a
special script. however, there is an option to provide your own script if
you want. we can augment the docs with a mini-example for your own script.

recently i realized that i would love if the ssh option for ipcluster had an
option to use my current environment similar to the way SGE/PBS does. it's
not useful when you are talking cross-platform but very useful if you are on
the same platform. part of this is motivated by the fact that my virtual
environment does not get activated over ssh (the bashrc doesn't get
executed). what do you guys think? i might have some time tonight - should i
take a crack at it?

cheers,

satra


On Mon, Sep 20, 2010 at 8:17 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hey guys,
>
> On Fri, Sep 17, 2010 at 5:52 PM, Satrajit Ghosh <satra at mit.edu> wrote:
> >
> > you can merge the changes from justin's 0.10.1-sge branch. that contains
> the
> > sge+lsf support fixes. other than that i think you already have min's
> taskid
> > fix for the controller (i.e the clear function now accepts a task id).
> >
>
> Great, thanks for the info.  One quick question, in the diff I see
> this towards the end:
>
> -The PBS mode uses the Portable Batch System [PBS]_ to start the engines.
>  To us
> e this mode, you first need to create a PBS script template that will be
> used to
>  start the engines.  Here is a sample PBS script template:
> +The PBS mode uses the Portable Batch System [PBS]_ to start the engines.
>
> -.. sourcecode:: bash
> +To start an ipcluster using the Portable Batch System::
> +
> +    $ ipcluster pbs -n 12
>
> -    #PBS -N ipython
> -    #PBS -j oe
> -    #PBS -l walltime=00:10:00
> -    #PBS -l nodes=${n/4}:ppn=4
> -    #PBS -q parallel
> +The above command will launch a PBS job array with 12 tasks using the
> default q
> ueue. If you would like to submit the job to a different queue use the -q
> option
> :
>
> -    cd $$PBS_O_WORKDIR
> -    export PATH=$$HOME/usr/local/bin
> -    export PYTHONPATH=$$HOME/usr/local/lib/python2.4/site-packages
> -    /usr/local/bin/mpiexec -n ${n} ipengine
> --logfile=$$PBS_O_WORKDIR/ipengine
> +    $ ipcluster pbs -n 12 -q hpcqueue
>
>
> Is it correct to remove all references to the #PBS lines?  Do  your
> improvements to PBS obsolete completely the need for the manual script
> writing we had before?
>
> I just want to make sure that I understand that correctly before I
> merge it, once we clarify this I'll merge and will cut an RC so we can
> get it tested.
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100921/0f2b05c2/attachment.html>

From fperez.net at gmail.com  Tue Sep 21 16:44:54 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 21 Sep 2010 13:44:54 -0700
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTi=cYi1qMSgDfShDMX-OWaDJnkzP5eDfiNp7qhNi@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
	<AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
	<AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>
	<AANLkTi=cYi1qMSgDfShDMX-OWaDJnkzP5eDfiNp7qhNi@mail.gmail.com>
Message-ID: <AANLkTinCjY+xM+xoFJ4oDuOTCD2tO_+Fuq66Xj_Qvc7T@mail.gmail.com>

Hey Satra,

On Tue, Sep 21, 2010 at 7:56 AM, Satrajit Ghosh <satra at mit.edu> wrote:
>
> the way justin set it up is that PBS/SGE/LSF obviates the need for writing a
> special script. however, there is an option to provide your own script if
> you want. we can augment the docs with a mini-example for your own script.

That would be great, I guess it could be even just leaving what was
there from the docs before the changes, indicating how to call it.

> recently i realized that i would love if the ssh option for ipcluster had an
> option to use my current environment similar to the way SGE/PBS does. it's
> not useful when you are talking cross-platform but very useful if you are on
> the same platform. part of this is motivated by the fact that my virtual
> environment does not get activated over ssh (the bashrc doesn't get
> executed). what do you guys think? i might have some time tonight - should i
> take a crack at it?

Sure, if the changes are small and self-contained.  I won't be cutting
the RC til Friday (too busy right now), so let's use that as the
cutoff date.  If you can have it done and tested by Friday, we'll roll
it into the RC.  Otherwise we'll never release :)


Cheers,

f


From fperez.net at gmail.com  Tue Sep 21 19:55:45 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 21 Sep 2010 16:55:45 -0700
Subject: [IPython-dev] Weird history bug on OSX?
Message-ID: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>

Hi folks,

in case any of you using OSX has  the foggiest clue as to what can be
happening here:

http://github.com/ipython/ipython/issues/issue/150

I'm inclined to think it's an issue specific to libedit or something
in macports, but we may be feeding it bogus data.  I've never seen the
problem on linux.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 21 19:57:40 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 21 Sep 2010 16:57:40 -0700
Subject: [IPython-dev] Weird history bug on OSX?
In-Reply-To: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>
References: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>
Message-ID: <AANLkTikwyJQ_frdV1VMx08A-a-OCwh45fwbjkyCq_HLY@mail.gmail.com>

On Tue, Sep 21, 2010 at 4:55 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
> I'm inclined to think it's an issue specific to libedit or something
> in macports, but we may be feeding it bogus data. ?I've never seen the
> problem on linux.

Duh, I should have said (since he does indicate that a readline
install fixes things): should we explicitly *require* readline on OSX?
 I know we have code that checks for libedit, but I don't know if
libedit is actually usable at all.  So should our default stance be to
require libedit being present on OSX unconditionally?

Cheers,

f


From benjaminrk at gmail.com  Tue Sep 21 21:16:26 2010
From: benjaminrk at gmail.com (MinRK)
Date: Tue, 21 Sep 2010 18:16:26 -0700
Subject: [IPython-dev] Weird history bug on OSX?
In-Reply-To: <AANLkTikwyJQ_frdV1VMx08A-a-OCwh45fwbjkyCq_HLY@mail.gmail.com>
References: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>
	<AANLkTikwyJQ_frdV1VMx08A-a-OCwh45fwbjkyCq_HLY@mail.gmail.com>
Message-ID: <AANLkTimEBM-TzZ1rRUShsuWoc-NQm=e3vmzP+EBW54o9@mail.gmail.com>

I haven't looked into how much of it is libedit itself, but at least with
the current state of IPython deployed on top of it, I consider readline a
dependency of IPython for my personal use.  IPython with libedit is not at
all acceptable in my experience.

It's been a while (IP 0.9?) since I tried it out, though.

-MinRK

On Tue, Sep 21, 2010 at 16:57, Fernando Perez <fperez.net at gmail.com> wrote:

> On Tue, Sep 21, 2010 at 4:55 PM, Fernando Perez <fperez.net at gmail.com>
> wrote:
> >
> > I'm inclined to think it's an issue specific to libedit or something
> > in macports, but we may be feeding it bogus data.  I've never seen the
> > problem on linux.
>
> Duh, I should have said (since he does indicate that a readline
> install fixes things): should we explicitly *require* readline on OSX?
>  I know we have code that checks for libedit, but I don't know if
> libedit is actually usable at all.  So should our default stance be to
> require libedit being present on OSX unconditionally?
>
> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100921/633469ba/attachment.html>

From fperez.net at gmail.com  Tue Sep 21 21:31:09 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 21 Sep 2010 18:31:09 -0700
Subject: [IPython-dev] Weird history bug on OSX?
In-Reply-To: <AANLkTimEBM-TzZ1rRUShsuWoc-NQm=e3vmzP+EBW54o9@mail.gmail.com>
References: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>
	<AANLkTikwyJQ_frdV1VMx08A-a-OCwh45fwbjkyCq_HLY@mail.gmail.com>
	<AANLkTimEBM-TzZ1rRUShsuWoc-NQm=e3vmzP+EBW54o9@mail.gmail.com>
Message-ID: <AANLkTi=g+ZPzj7_D5h9FJ_QuYXKggfWw2VXjVbu5pk0c@mail.gmail.com>

On Tue, Sep 21, 2010 at 6:16 PM, MinRK <benjaminrk at gmail.com> wrote:
> I haven't looked into how much of it is libedit itself, but at least with
> the current state of IPython deployed on top of it, I consider readline a
> dependency of IPython for my personal use. ?IPython with libedit is not at
> all acceptable in my experience.
> It's been a while (IP 0.9?) since I tried it out, though.

Sounds good, I've updated the docs to indicate that we pretty much
require readline as libedit isn't likely sufficient.

Thanks!

f


From ellisonbg at gmail.com  Tue Sep 21 21:33:59 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 21 Sep 2010 18:33:59 -0700
Subject: [IPython-dev] Weird history bug on OSX?
In-Reply-To: <AANLkTi=g+ZPzj7_D5h9FJ_QuYXKggfWw2VXjVbu5pk0c@mail.gmail.com>
References: <AANLkTineqNSPMr6wpPy7Vf_yeEdDxEyn0CLzOoHWXxWi@mail.gmail.com>
	<AANLkTikwyJQ_frdV1VMx08A-a-OCwh45fwbjkyCq_HLY@mail.gmail.com>
	<AANLkTimEBM-TzZ1rRUShsuWoc-NQm=e3vmzP+EBW54o9@mail.gmail.com>
	<AANLkTi=g+ZPzj7_D5h9FJ_QuYXKggfWw2VXjVbu5pk0c@mail.gmail.com>
Message-ID: <AANLkTikdkh6xi9uexYpuS_qai8GKAM2d6-+wzkUWW=uS@mail.gmail.com>

On Tue, Sep 21, 2010 at 6:31 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Tue, Sep 21, 2010 at 6:16 PM, MinRK <benjaminrk at gmail.com> wrote:
>> I haven't looked into how much of it is libedit itself, but at least with
>> the current state of IPython deployed on top of it, I consider readline a
>> dependency of IPython for my personal use. ?IPython with libedit is not at
>> all acceptable in my experience.
>> It's been a while (IP 0.9?) since I tried it out, though.
>
> Sounds good, I've updated the docs to indicate that we pretty much
> require readline as libedit isn't likely sufficient.

+1

> Thanks!
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From satra at mit.edu  Wed Sep 22 13:37:59 2010
From: satra at mit.edu (Satrajit Ghosh)
Date: Wed, 22 Sep 2010 13:37:59 -0400
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTinCjY+xM+xoFJ4oDuOTCD2tO_+Fuq66Xj_Qvc7T@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
	<AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
	<AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>
	<AANLkTi=cYi1qMSgDfShDMX-OWaDJnkzP5eDfiNp7qhNi@mail.gmail.com>
	<AANLkTinCjY+xM+xoFJ4oDuOTCD2tO_+Fuq66Xj_Qvc7T@mail.gmail.com>
Message-ID: <AANLkTi=ukRi_Dzgo5aSa02Nei9HCzsarg7HQZK1-sXyo@mail.gmail.com>

hey fernando,

changed, enhanced and pushed. i'll send a pull request, but it's in my
0.10.1-sge branch now.

cheers,

satra



On Tue, Sep 21, 2010 at 4:44 PM, Fernando Perez <fperez.net at gmail.com>wrote:

> Hey Satra,
>
> On Tue, Sep 21, 2010 at 7:56 AM, Satrajit Ghosh <satra at mit.edu> wrote:
> >
> > the way justin set it up is that PBS/SGE/LSF obviates the need for
> writing a
> > special script. however, there is an option to provide your own script if
> > you want. we can augment the docs with a mini-example for your own
> script.
>
> That would be great, I guess it could be even just leaving what was
> there from the docs before the changes, indicating how to call it.
>
> > recently i realized that i would love if the ssh option for ipcluster had
> an
> > option to use my current environment similar to the way SGE/PBS does.
> it's
> > not useful when you are talking cross-platform but very useful if you are
> on
> > the same platform. part of this is motivated by the fact that my virtual
> > environment does not get activated over ssh (the bashrc doesn't get
> > executed). what do you guys think? i might have some time tonight -
> should i
> > take a crack at it?
>
> Sure, if the changes are small and self-contained.  I won't be cutting
> the RC til Friday (too busy right now), so let's use that as the
> cutoff date.  If you can have it done and tested by Friday, we'll roll
> it into the RC.  Otherwise we'll never release :)
>
>
> Cheers,
>
> f
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100922/e35976c3/attachment.html>

From fperez.net at gmail.com  Wed Sep 22 13:42:19 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 22 Sep 2010 10:42:19 -0700
Subject: [IPython-dev] Justin, Satra, everyone: wrapping up 0.10.1?
In-Reply-To: <AANLkTi=ukRi_Dzgo5aSa02Nei9HCzsarg7HQZK1-sXyo@mail.gmail.com>
References: <AANLkTikPvkKxGzONYC_oqcspHtF=ff+vPd0KJFpzDL4A@mail.gmail.com>
	<AANLkTimp-PkBO_TrHSZVU0GuVMJSVAV9ajJ8q0aOpwi4@mail.gmail.com>
	<AANLkTinhjPZXaPh-9bHpSaCmD+nrdiY+1_nN+OXv10Kr@mail.gmail.com>
	<AANLkTi=cYi1qMSgDfShDMX-OWaDJnkzP5eDfiNp7qhNi@mail.gmail.com>
	<AANLkTinCjY+xM+xoFJ4oDuOTCD2tO_+Fuq66Xj_Qvc7T@mail.gmail.com>
	<AANLkTi=ukRi_Dzgo5aSa02Nei9HCzsarg7HQZK1-sXyo@mail.gmail.com>
Message-ID: <AANLkTinv5CZ1sH3gcng5bzKPXmC-kbR8F3OZJS6=Rd7S@mail.gmail.com>

On Wed, Sep 22, 2010 at 10:37 AM, Satrajit Ghosh <satra at mit.edu> wrote:
>
> changed, enhanced and pushed. i'll send a pull request, but it's in my
> 0.10.1-sge branch now.

Great, got the request. I'll take care of it Friday, a bit swamped today.

Cheers,

f


From mike at pythonlibrary.org  Thu Sep 16 10:08:23 2010
From: mike at pythonlibrary.org (Mike Driscoll)
Date: Thu, 16 Sep 2010 09:08:23 -0500
Subject: [IPython-dev] GSoC Article for PSF and my blog
Message-ID: <AANLkTinKHX3A4oTdN1SecquEOZY6thbseTNaN3+xePL9@mail.gmail.com>

Hi,

I am working on an article for the Python Software Foundation's blog,
http://pyfound.blogspot.com/, about the various Python projects that were
worked on during this year's Google Summer of Code program. They want me to
write up something about what projects were worked on and what the results
were. I found your project information here:
http://wiki.python.org/moin/SummerOfCode/2010

Anyway, since the PSF blog article will be brief, I thought I would also
write up a longer article about your projects on my personal blog as well.
The information I found on the Python wiki page was pretty brief, so I would
appreciate it if you could tell me the following:

1) What was worked on during the GSoC project
2) How many students helped with a guess at how many hours were put in
3) Your experiences being a mentor

If possible, I would like the student's perspective too. Feel free to
forward my information to them. Also feel free to opt out and I won't write
anything more than the already public info I can find. Thanks a lot for your
help!

-- 
-----------------
Mike Driscoll

Blog:   http://blog.pythonlibrary.org
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100916/5b1b67db/attachment.html>

From fperez.net at gmail.com  Tue Sep 28 03:56:31 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Sep 2010 00:56:31 -0700
Subject: [IPython-dev] Merging nk into trunk
Message-ID: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>

Hi folks,

I was planning on trying to do the merge tomorrow if I can find a gap
in the day.

Brian, you mentioned you had something you wanted to run in nk before
the merge, is that still the case?

I was just planning on renaming the entry point script according to
the previous discussion and otherwise leaving the code as-is, so we
continue working directly in trunk.

Dogfooding note: today I taught a 3-hour tutorial on intermediate
matplotlib usage at Berkeley using the new frontend, and it worked
*fantastically* well for that purpose.  The ability to directly paste
multiline blocks, explain them and display the plots, to have calltips
for info and the split pane for showing info while working, are all
really great in a teaching context.

There are still a few rough edges and some features I'd love to have
(like being able to toggle back and forth from inline to window plot
mode, which is doable but I haven't had time to implement).  But
overall, I think we have something very nice on our hands.

On Wednesday I'll present it at the Berkeley py4science group and
we'll start getting tester feedback...

Cheers,

f


From ellisonbg at gmail.com  Tue Sep 28 11:15:31 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 28 Sep 2010 08:15:31 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
Message-ID: <AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>

On Tue, Sep 28, 2010 at 12:56 AM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hi folks,
>
> I was planning on trying to do the merge tomorrow if I can find a gap
> in the day.
>
> Brian, you mentioned you had something you wanted to run in nk before
> the merge, is that still the case?

Yes, I need to make a few changes to support 2.0.7 and 2.0.8 pyzmq.  I
should be able to get to that today, but we really should do that
before it goes in.  I will ping you when it is in.

> I was just planning on renaming the entry point script according to
> the previous discussion and otherwise leaving the code as-is, so we
> continue working directly in trunk.

Sounds good.

> Dogfooding note: today I taught a 3-hour tutorial on intermediate
> matplotlib usage at Berkeley using the new frontend, and it worked
> *fantastically* well for that purpose. ?The ability to directly paste
> multiline blocks, explain them and display the plots, to have calltips
> for info and the split pane for showing info while working, are all
> really great in a teaching context.

That is great.  I am pretty much going to start using it myself for
everything as well.

> There are still a few rough edges and some features I'd love to have
> (like being able to toggle back and forth from inline to window plot
> mode, which is doable but I haven't had time to implement). ?But
> overall, I think we have something very nice on our hands.
>
> On Wednesday I'll present it at the Berkeley py4science group and
> we'll start getting tester feedback...

Very nice!

Cheers,

Brian

> Cheers,
>
> f
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Tue Sep 28 17:56:51 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Sep 2010 14:56:51 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
	<AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
Message-ID: <AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>

Hey,

On Tue, Sep 28, 2010 at 8:15 AM, Brian Granger <ellisonbg at gmail.com> wrote:
> On Tue, Sep 28, 2010 at 12:56 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>> Hi folks,
>>
>> I was planning on trying to do the merge tomorrow if I can find a gap
>> in the day.
>>
>> Brian, you mentioned you had something you wanted to run in nk before
>> the merge, is that still the case?
>
> Yes, I need to make a few changes to support 2.0.7 and 2.0.8 pyzmq. ?I
> should be able to get to that today, but we really should do that
> before it goes in. ?I will ping you when it is in.

If you can't do it today, any strong reason why this can't be done
later in trunk itself?  It's just that tomorrow we'll be demoing and
installing it with a group of people, and I'd rather not get everyone
using a git branch that will soon go obsolete.  If we merge now, you
can make those changes later in trunk itself later tomorrow or
whenever works for you...

Basically I want to give people two constant things: what to pull from
(trunk) and what the name of the entry point is.  They still know it's
testing stages and that code will change, but I'd rather not get a
bunch of users on campus (many of whom may not be experienced with
git) having to change branches the day after they install it...

Thoughts?

f


From ellisonbg at gmail.com  Tue Sep 28 18:04:11 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 28 Sep 2010 15:04:11 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
	<AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
	<AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>
Message-ID: <AANLkTikcnbjJ3SpoaiVqppVe6wofXumbu-DJEb-Ey-WB@mail.gmail.com>

On Tue, Sep 28, 2010 at 2:56 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> Hey,
>
> On Tue, Sep 28, 2010 at 8:15 AM, Brian Granger <ellisonbg at gmail.com> wrote:
>> On Tue, Sep 28, 2010 at 12:56 AM, Fernando Perez <fperez.net at gmail.com> wrote:
>>> Hi folks,
>>>
>>> I was planning on trying to do the merge tomorrow if I can find a gap
>>> in the day.
>>>
>>> Brian, you mentioned you had something you wanted to run in nk before
>>> the merge, is that still the case?
>>
>> Yes, I need to make a few changes to support 2.0.7 and 2.0.8 pyzmq. ?I
>> should be able to get to that today, but we really should do that
>> before it goes in. ?I will ping you when it is in.
>
> If you can't do it today, any strong reason why this can't be done
> later in trunk itself? ?It's just that tomorrow we'll be demoing and
> installing it with a group of people, and I'd rather not get everyone
> using a git branch that will soon go obsolete. ?If we merge now, you
> can make those changes later in trunk itself later tomorrow or
> whenever works for you...

OK, go for it.  The patch will just make it work fully with both 2.0.7
and 2.0.8.  but without the patch it still mostly works.  There are
only 2 things that are affected:

* The clean shutdown.
* A subtle pyzmq bug that we have to work around in 2.0.7.

I would go ahead and merge and ping me.  I will  work from trunk.

> Basically I want to give people two constant things: what to pull from
> (trunk) and what the name of the entry point is. ?They still know it's
> testing stages and that code will change, but I'd rather not get a
> bunch of users on campus (many of whom may not be experienced with
> git) having to change branches the day after they install it...

Yep!

Brian

> Thoughts?
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Tue Sep 28 18:21:49 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Sep 2010 15:21:49 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTikcnbjJ3SpoaiVqppVe6wofXumbu-DJEb-Ey-WB@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
	<AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
	<AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>
	<AANLkTikcnbjJ3SpoaiVqppVe6wofXumbu-DJEb-Ey-WB@mail.gmail.com>
Message-ID: <AANLkTi=MKOoz0Qk37DzyjJ7h3ywe0TYtLhdoPuLP3N9s@mail.gmail.com>

On Tue, Sep 28, 2010 at 3:04 PM, Brian Granger <ellisonbg at gmail.com> wrote:
>
> OK, go for it. ?The patch will just make it work fully with both 2.0.7
> and 2.0.8. ?but without the patch it still mostly works. ?There are
> only 2 things that are affected:
>
> * The clean shutdown.
> * A subtle pyzmq bug that we have to work around in 2.0.7.
>
> I would go ahead and merge and ping me. ?I will ?work from trunk.

Fabulous, thanks.  I'll do it now, and will ping then when done.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 28 19:18:46 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Sep 2010 16:18:46 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTi=MKOoz0Qk37DzyjJ7h3ywe0TYtLhdoPuLP3N9s@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
	<AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
	<AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>
	<AANLkTikcnbjJ3SpoaiVqppVe6wofXumbu-DJEb-Ey-WB@mail.gmail.com>
	<AANLkTi=MKOoz0Qk37DzyjJ7h3ywe0TYtLhdoPuLP3N9s@mail.gmail.com>
Message-ID: <AANLkTi=aoJEhntgE451U82fhVTdmARRZDXHEXW+imf=3@mail.gmail.com>

On Tue, Sep 28, 2010 at 3:21 PM, Fernando Perez <fperez.net at gmail.com> wrote:

> Fabulous, thanks. ?I'll do it now, and will ping then when done.

OK, done!!!  Thanks :)  I'll send a separate announcement, let's hope
I didn't screw anything up at the last minute...

I did notice after pushing that I missed the script rename for the
setuptools entry point, so rather than making a forced push on trunk I
just added that little fix as a separate commit.

So now trunk should be the place where we do any further work.  In a
week or so we can delete newkernel from github for cleanup, but I
don't want to do it right now in case someone is using it right now to
avoid causing an error on their side.

Cheers,

f


From fperez.net at gmail.com  Tue Sep 28 19:40:18 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Tue, 28 Sep 2010 16:40:18 -0700
Subject: [IPython-dev] New zeromq-based console now in trunk
Message-ID: <AANLkTimPNS0Ua2g05pk4c2CciTRy7iQJvCUze1Txd3N1@mail.gmail.com>

Hi all,

We've just pushed the 'newkernel' branch into trunk, so all that the
recent work is now fully merged.  The big user-facing news is a new
application, called 'ipython-qtconsole', that provides a rich widget
with the feel of a terminal but many improved features such as
multiline editing, syntax highlighting, graphical tooltips, inline
plots, the ability to restart the kernel without losing user history,
and more.  Attached is a screenshot showing some of these features.

At this point we know there are still rough edges, but we think the
code is in a shape good enough for developers to start testing it, and
filing tickets on our bug tracker
(http://github.com/ipython/ipython/issues) for any problems you
encounter.

This is all based on the new ZeroMQ messaging design we've been
discussing, for which updated docs are available here:

http://ipython.scipy.org/doc/nightly/html/development/messaging.html

We hope that we'll continue to refine this protocol and that now other
clients will be written against it as well (Omar and Gerardo's,
Wendell's, etc).  While we're sure there will be refinements needed,
the qt console is rich and complex enough that we're fairly confident
the core design is sound.  We put a *lot* of thought and work into it,
and all along we felt the design helped us rather than hindered us,
which is typically a sign that the basic ideas are right.

If you want to start playing with this code, you will need (in
addition to IPython trunk):

ZeroMQ and its python bindings, Pyzqm, version 2.0.8:

    * http://www.zeromq.org/local--files/area:download/zeromq-2.0.8.tar.gz
    * http://github.com/downloads/zeromq/pyzmq/pyzmq-2.0.8.tar.gz

The installation instructions for these can be found here:

    * http://www.zeromq.org/area:download
    * http://www.zeromq.org/bindings:python

PyQt:

    * http://www.riverbankcomputing.co.uk/software/pyqt/download

Once installed, run the 'ipython-qtconsole' script. Try --help to see
the different flags you can use to start it, and

When the console opens, *first* type '%guiref' to read a quick
introduction to its basic features.  We'll be happy to answer
questions here, but please make sure you have first read that
document, as we've tried to answer the key points there.  If there are
missing points in that explanation, we want to fix the docs first and
foremost.


Known limitations:

- currently the qt console can't be configured *at all* (font type,
size, etc).  It's one of our top priorities.

- you can't switch back and forth between inline figures and floating
ones (though pastefig() lets you inline select floating figures, so
it's not too bad).

- we have a bug in our execution flow that breaks history and logging
with multiline input.  Also at the very top of our list.

- you can't call subprocesses with ! if you expect to interact with
them.  This is a fundamental design choice that will stay as such.  If
you need to interact with a  subprocess, call it from a normal
terminal (plain bash or ipython if you want), but making that work
with the GUI console and other design constraints is not feasible
right now.

Credits

We'd like to acknowledge the support of Enthought Inc, that made it
possible for Brian and myself to devote a significant amount of time
over the past two months to work on this.  Evan Patterson, who many of
you have by now also met on the list or at conferences, was also part
of this effort thanks to Enthought's support; all the credit for the
beautiful Qt functionality goes to him.  He really did a remarkable
job, and we hope to have him continue participating in IPython
development in the future.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython-qtconsole.png
Type: image/png
Size: 139962 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20100928/f2c3396c/attachment.png>

From ellisonbg at gmail.com  Tue Sep 28 22:42:31 2010
From: ellisonbg at gmail.com (Brian Granger)
Date: Tue, 28 Sep 2010 19:42:31 -0700
Subject: [IPython-dev] Merging nk into trunk
In-Reply-To: <AANLkTi=aoJEhntgE451U82fhVTdmARRZDXHEXW+imf=3@mail.gmail.com>
References: <AANLkTintm_AS8sEFitodNBz6rHUFVciKAXmSq=2Zeoo6@mail.gmail.com>
	<AANLkTin7+N1FxN3OA2xmWdGETf8H3-_r58r+dyTQqeY3@mail.gmail.com>
	<AANLkTimB9NQm4bQKzDsHN3VjDt4EYfroSPNrjuaM2uvA@mail.gmail.com>
	<AANLkTikcnbjJ3SpoaiVqppVe6wofXumbu-DJEb-Ey-WB@mail.gmail.com>
	<AANLkTi=MKOoz0Qk37DzyjJ7h3ywe0TYtLhdoPuLP3N9s@mail.gmail.com>
	<AANLkTi=aoJEhntgE451U82fhVTdmARRZDXHEXW+imf=3@mail.gmail.com>
Message-ID: <AANLkTikf0L-qgc8wUFujaFaU4pnt52VmTBbF1BpH8Dp9@mail.gmail.com>

On Tue, Sep 28, 2010 at 4:18 PM, Fernando Perez <fperez.net at gmail.com> wrote:
> On Tue, Sep 28, 2010 at 3:21 PM, Fernando Perez <fperez.net at gmail.com> wrote:
>
>> Fabulous, thanks. ?I'll do it now, and will ping then when done.
>
> OK, done!!! ?Thanks :) ?I'll send a separate announcement, let's hope
> I didn't screw anything up at the last minute...

Great, I will test out and push the other changes.

> I did notice after pushing that I missed the script rename for the
> setuptools entry point, so rather than making a forced push on trunk I
> just added that little fix as a separate commit.

No problem.

> So now trunk should be the place where we do any further work. ?In a
> week or so we can delete newkernel from github for cleanup, but I
> don't want to do it right now in case someone is using it right now to
> avoid causing an error on their side.

Ok,

Brian

> Cheers,
>
> f
>



-- 
Brian E. Granger, Ph.D.
Assistant Professor of Physics
Cal Poly State University, San Luis Obispo
bgranger at calpoly.edu
ellisonbg at gmail.com


From fperez.net at gmail.com  Wed Sep 29 13:39:48 2010
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 29 Sep 2010 10:39:48 -0700
Subject: [IPython-dev] IPython 0.10.1 release candidate up for final testing
Message-ID: <AANLkTi=Lmvd5w=WPybo=9r30j-1XL8FjN6tx9_2Ov6k=@mail.gmail.com>

Hi folks,

while most of our recent energy has gone into all the zeromq-based
work for 0.11 (the code in trunk and the two GSoC projects), we
haven't abandoned the stable 0.10 series.  A lot of small bugfixes
have gone in, and recently Justin Riley, Satra Ghosh and Matthieu
Brucher have contributed Sun Grid Engine scheduling support for the
parallel computing machinery.

For those of you using the 0.10 series, here is a release candidate:

http://ipython.scipy.org/dist/testing/

Unless any problems are found with it, I will tag it as final and
release the otherwise identical code as 0.10.1 by next Monday (give or
take a day or two).

We'd appreciate testing feedback, and if anyone wants to do detailed
testing (for example a release manager for a distribution) but you
need more time, just let me know and we can wait a few extra days.

Regards,

f