From emmanuel.bacry at polytechnique.fr Thu Nov 3 06:31:27 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 03:31:27 -0700 (PDT) Subject: [IPython-dev] wxPython console Message-ID: <32771432.post@talk.nabble.com> Ilan Schnell wrote: > > As far as I know, no. Although, given the new infrastructure, > one could write a wxPython console (or in fact any type of > console) which connects to the IPython kernel. > > - Ilan > > > On Wed, Jul 13, 2011 at 3:58 PM, Piotr Zolnierczuk > wrote: >> I just saw Fernando's talk @scipy2011 I was wondering if there are any >> plans >> to support a wxPython console? >> Piotr >> -- >> >> Piotr Adam Zolnierczuk >> e-mail: piotr at zolnierczuk.net >> www:?? http://www.zolnierczuk.net >> _____________________________________ >> written on recycled electrons >> >> >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > Could you just give me some hints on how you would do that ? I'm kind of new on python/ipython. Very impressed by ipython and I would like to embed it in a wx console. Let's say I have a wx console widget ready, the wx main loop is running. Now I want to have a ipython shell object available, with stdin and stdout redirected, so that I can send to it input and get from it the output. - What is the class of such an object ? Is it InteractiveShellEmbed ? Isn't it just for "interactive" shell? - Can I make it non blocking ? (using thread each time I send input to it?) An a last but quite different question : - Would I be able to use matplotlib through it ? I kind of understood there is a way matplot lib uses a wx main_loop already running right ? Thank you very much. And please .. apologize if some questions are naive ... Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32771432.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Thu Nov 3 07:05:52 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 3 Nov 2011 11:05:52 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32771432.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> Message-ID: On 3 November 2011 10:31, Emmanuel Bacry wrote: > Let's say I have a wx console widget ready, the wx main loop is running. > Now I want to have a ipython shell object available, with stdin and stdout > redirected, > so that I can send to it input and get from it the output. > How does a Wx console widget work? I can't find a reference to it in the Wx docs. There is a protocol to run IPython in a separate process from the frontend. This is what the Qt console uses. Requests and replies are packed into JSON, and sent over ZMQ. There are message types for execution, tab completion, history, etc. http://ipython.org/ipython-doc/dev/development/messaging.html If the console widget is a terminal emulator, a simple shortcut may be to start plain ipython in that terminal. At the moment, I don't think there's a good way to embed IPython in a GUI without starting a separate process, but this is something we want to get round to. Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Thu Nov 3 08:17:47 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 05:17:47 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> Message-ID: <32772284.post@talk.nabble.com> Thank you for your prompt reply. Thomas Kluyver-2 wrote: > > On 3 November 2011 10:31, Emmanuel Bacry > wrote: > > How does a Wx console widget work? I can't find a reference to it in the > Wx > docs. > No, I don't think there is such a thing as a "standard" wx console (I kind of wrote my own). Thomas Kluyver-2 wrote: > > There is a protocol to run IPython in a separate process from the > frontend. > This is what the Qt console uses. Requests and replies are packed into > JSON, and sent over ZMQ. There are message types for execution, tab > completion, history, etc. > > http://ipython.org/ipython-doc/dev/development/messaging.html > 1- I think I understand the solution you suggest. However I don't hink it works for me. Let me tell you what ultimately I want to do : I wrote a pretty big C++ library for displaying and interacting (using wxWidget) with scientific plots. It is fast (it is able to manipulate really long data) and highly interactive. My goal is to interface it with python so that I can do computations using a console and a python interpreter in it. E.g, using numpy to compute some stuff that I display and interact with. So I want the python interpreter to be ran in the same process as my library which is using wx so that they share the data. I am not so sure this is the case in your solution ? When you run ipython, there is a python interpreter encapsulated in it right ? 2- Now wether the console is running in a different process or not is a different problem. A priori I don't care if the console runs in a separate process.. though I do a little. Let me explain why : Sometimes, when dealing with scientific data you want to display huge quantity of data in the console. You need to have a fast console (look at matlab console or R, they display a lot of data very quickly). I'm afraid that sending all the stdout through a socket will be kind of slow Thomas Kluyver-2 wrote: > > If the console widget is a terminal emulator, a simple shortcut may be to > start plain ipython in that terminal. > How would you start ipython in a terminal emulator ? (sorry for a so naive question!) Thomas Kluyver-2 wrote: > > At the moment, I don't think there's a good way to embed IPython in a GUI > without starting a separate process, but this is something we want to get > round to. > Too bad ! Thank you Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32772284.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Thu Nov 3 09:25:37 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 3 Nov 2011 13:25:37 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32772284.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: On 3 November 2011 12:17, Emmanuel Bacry wrote: > 1- I think I understand the solution you suggest. However I don't hink it > works for me. Let me tell you what ultimately I want to do : > > I wrote a pretty big C++ library for displaying and interacting (using > wxWidget) with scientific plots. It is fast (it is able to manipulate > really > long data) and highly interactive. > My goal is to interface it with python so that I can do computations using > a > console and a python interpreter in it. E.g, using numpy to compute some > stuff that I display and interact with. > So I want the python interpreter to be ran in the same process as my > library > which is using wx so that they share the data. > > I am not so sure this is the case in your solution ? > When you run ipython, there is a python interpreter encapsulated in it > right > ? > > 2- Now wether the console is running in a different process or not is a > different problem. > A priori I don't care if the console runs in a separate process.. though I > do a little. > Let me explain why : > Sometimes, when dealing with scientific data you want to display huge > quantity of data in the console. You need to have a fast console (look at > matlab console or R, they display a lot of data very quickly). > I'm afraid that sending all the stdout through a socket will be kind of > slow > > I don't think the speed of sending output to the console is a big issue. We use the Qt console for scientific work, and it's normally quite fast enough, including things like sending base-64 encoded PNGs. It does make sense, though, that you don't want to keep sending big objects between two processes. The way we've handled this is to have all the data and processing in the 'kernel', and use the frontend simply for input and display. Some version of this approach might work for you, but I don't know exactly what you're trying to do. I think it should be possible to create an InteractiveShell object (from IPython.core.interactiveshell) and use it directly. Some key points: Instantiation parameters: config: pass an IPython.config.loader.Config instance to set config values user_ns: pass a dictionary containing variables which will appear in the user namespace. (these, and all the other parameters are optional - omit them for a default IPython with an empty namespace) Methods: run_cell: Run some code as if it had been typed in at the shell. push: Pass a dictionary of variables to be added to the IPython namespace. complete: Complete some expression at a prompt (tab completion). See the docstring for details. magic: Run a magic command (see %lsmagic) Also, the InteractiveShell object has a history_manager attribute which handles input & output history. At some point, we should probably make an InteractiveShellProxy class wrapping a kernelmanager so it's easy to switch between in-process and out-of-process IPython. Hope that helps, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Thu Nov 3 09:51:43 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 06:51:43 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: <32772900.post@talk.nabble.com> Thomas Kluyver-2 wrote: > > First of all : ALL what you've said is of big help to me! And I thank you for that. Thomas Kluyver-2 wrote: > > I don't think the speed of sending output to the console is a big issue. > We use the Qt console for scientific work, and it's normally quite fast > enough, including things like sending base-64 encoded PNGs. > OK, maybe you're right. Thomas Kluyver-2 wrote: > > It does make sense, though, that you don't want to keep sending big > objects > between two processes. The way we've handled this is to have all the data > and processing in the 'kernel', and use the frontend simply for input and > display. Some version of this approach might work for you, but I don't > know > exactly what you're trying to do. > It is important to me to have very interactive display. So the display is constantly accessing the kernel data... I don't think your solution would work. Thomas Kluyver-2 wrote: > > I think it should be possible to create an InteractiveShell object (from > IPython.core.interactiveshell) and use it directly. > You mean use it directly within a wx main loop right ? Thomas Kluyver-2 wrote: > > Some key points: > > Instantiation parameters: > config: pass an IPython.config.loader.Config instance to set config values > user_ns: pass a dictionary containing variables which will appear in the > user namespace. > (these, and all the other parameters are optional - omit them for a > default > IPython with an empty namespace) > Yes I got that already, however how do I redirect output ? Thomas Kluyver-2 wrote: > > Methods: > run_cell: Run some code as if it had been typed in at the shell. > push: Pass a dictionary of variables to be added to the IPython namespace. > complete: Complete some expression at a prompt (tab completion). See the > docstring for details. > magic: Run a magic command (see %lsmagic) > OK. For complete : it actually does exactly what happens when I hit the tab key on the ipython console ? How do I manage multiple lines command ? Le me be clear : how do I know a command is not complete so I should print a prompt with a tab ? Is there a method that deals with managing all that for me ? Thomas Kluyver-2 wrote: > > Also, the InteractiveShell object has a history_manager attribute which > handles input & output history. > > At some point, we should probably make an InteractiveShellProxy class > wrapping a kernelmanager so it's easy to switch between in-process and > out-of-process IPython. > Thank you Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32772900.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Thu Nov 3 10:27:31 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 3 Nov 2011 14:27:31 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32772900.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> Message-ID: On 3 November 2011 13:51, Emmanuel Bacry wrote: > You mean use it directly within a wx main loop right ? > Yes - it's just Python methods that you call to run code and so on. I'm assuming your widget takes care of actually letting the user type things in. > Yes I got that already, however how do I redirect output ? > In Python, there are two kinds of output: firstly from "print x", for which you can simply redirect stdout - replace sys.stdout with an object with a .write() method, which will capture strings written to it and display them in your console. Secondly, if you simply enter "x" at a prompt, IPython displays the value of x at an "Out [n]:" prompt. In such cases, Python calls sys.displayhook. This is called with the object itself, rather than its repr, so you can produce more complex displays. This is how we pretty-print datastructures, and also how the notebook can display rich objects like youtube videos. Have a look at IPython.core.displayhook to see our implementation of this. If you replace the displayhook attribute of an InteractiveShell object, it will take care of using that displayhook when you call run_cell. OK. > For complete : it actually does exactly what happens when I hit the tab key > on the ipython console ? > It provides the machinery to find the completions for a given input. The frontend has to take care of catching the tab key (or some other shortcut) and applying or displaying the returned completion(s). > How do I manage multiple lines command ? > You can use an InputSplitter to determine if another line is acceptable - have a look at the code for the terminal frontend: https://github.com/ipython/ipython/blob/master/IPython/frontend/terminal/interactiveshell.py#L318 > Le me be clear : how do I know a command is not complete so I should print > a prompt with a tab ? > Is there a method that deals with managing all that for me ? > I'm not quite sure what you mean with this part, but hopefully what I've described above will help. Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Thu Nov 3 14:21:47 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 11:21:47 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> Message-ID: <32775050.post@talk.nabble.com> Thomas Kluyver-2 wrote: > > On 3 November 2011 13:51, Emmanuel Bacry > wrote: > >> You mean use it directly within a wx main loop right ? >> > > Yes - it's just Python methods that you call to run code and so on. I'm > assuming your widget takes care of actually letting the user type things > in. > > >> Yes I got that already, however how do I redirect output ? >> > > In Python, there are two kinds of output: firstly from "print x", for > which > you can simply redirect stdout - replace sys.stdout with an object with a > .write() method, which will capture strings written to it and display them > in your console. > > Secondly, if you simply enter "x" at a prompt, IPython displays the value > of x at an "Out [n]:" prompt. In such cases, Python calls sys.displayhook. > This is called with the object itself, rather than its repr, so you can > produce more complex displays. This is how we pretty-print datastructures, > and also how the notebook can display rich objects like youtube videos. > Have a look at IPython.core.displayhook to see our implementation of this. > If you replace the displayhook attribute of an InteractiveShell object, it > will take care of using that displayhook when you call run_cell. > > OK. >> For complete : it actually does exactly what happens when I hit the tab >> key >> on the ipython console ? >> > > It provides the machinery to find the completions for a given input. The > frontend has to take care of catching the tab key (or some other shortcut) > and applying or displaying the returned completion(s). > > This is very clear Thank you Thomas Kluyver-2 wrote: > >> How do I manage multiple lines command ? >> > > You can use an InputSplitter to determine if another line is acceptable - > have a look at the code for the terminal frontend: > https://github.com/ipython/ipython/blob/master/IPython/frontend/terminal/interactiveshell.py#L318 > Will it tell me what is the level of indentation I need for an eventual acceptable line ? Thomas Kluyver-2 wrote: > > Le me be clear : how do I know a command is not complete so I should > print >> a prompt with a tab ? >> Is there a method that deals with managing all that for me ? >> > > I'm not quite sure what you mean with this part, but hopefully what I've > described above will help. > Yes it does. Thank you again Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32775050.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Thu Nov 3 16:29:54 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 3 Nov 2011 20:29:54 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32775050.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> <32775050.post@talk.nabble.com> Message-ID: On 3 November 2011 18:21, Emmanuel Bacry wrote: > Will it tell me what is the level of indentation I need for an eventual > acceptable line ? > Yes, your InputSplitter instance should have an indent_spaces attribute, which is an integer multiple of 4, and applies to the next line. So if you push a line "if a > 5:" to the InputSplitter, isp.indent_spaces will increase. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Nov 3 18:23:34 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 3 Nov 2011 15:23:34 -0700 Subject: [IPython-dev] wxPython console In-Reply-To: <32772284.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: On Thu, Nov 3, 2011 at 5:17 AM, Emmanuel Bacry wrote: > Thomas Kluyver-2 wrote: >> >> At the moment, I don't think there's a good way to embed IPython in a GUI >> without starting a separate process, but this is something we want to get >> round to. >> > Too bad ! There is a way of embedding that may actually work for you, and can save you quite a bit of time in that you could actually use your current Wx library as-is, and reuse our Qt console. This example: https://github.com/ipython/ipython/blob/master/docs/examples/lib/ipkernel_wxapp.py shows how you can embed an IPython *kernel* in your own application. Given this kernel, you can then connect with any client capable of speaking the ipython protocol (right now that's the qtconsole or the notebook --with a small fix for talking to existing kernels needed), but there's also a terminal one in the works in PR #864). The point of this would be that this kernel would run in your existing Wx framework right away, and you could connect the current Qt console (also already written) to it immediately. You just need to pass the dictionary with the namespace you want the user to see. Obviously if your own Wx console has extra functionality that you need this isn't the best approach. If you want your Wx console to be able to talk to an external kernel, the easiest would be to study how the current qt console works: it's basically a matter of converting the signals and slots machiery to the equivalent Wx event handling code. Finally, we do envision being able to embed a qt/wx console into a single process, and we're not too far from the necessary architectural cleanup to make that easy. The model would be that a client (terminal, qtconsole, wx, etc) would have a kernel manager object that could be either local or remote, but with otherwise identical api. Right now we only have remote ones, and the local terminal doesn't abstract out its interactions through a kernel manager. But once that design is in place (and we're not that far from it, we just need some more refactoring to be completed), it should be effectively transparent whether a client is using a local or remote kernel; local ones wouldn't even need zmq as their 'messaging' would just be in-process method calls. Hopefully this clarifies both the short-term options for you as well as our longer-term design... Cheers, f From emmanuel.bacry at polytechnique.fr Thu Nov 3 20:24:17 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 17:24:17 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> <32775050.post@talk.nabble.com> Message-ID: <32777434.post@talk.nabble.com> Sorry to bother you again Thomas! I worked a little on what you suggested me and tried some stuff Right now I have 2 questions 1- So I ran a wx main loop, I created the InteractiveShell. I can execute commands in this shell. Now I want to be able to display the stdout of this shell in my wx console. So I need to redirect sys.stdout on the InteractiveShell as you explained to me. Is the best way to share a buffer between the two environments, and on Idle time I ask wx to empty it ? And I redirect the stdout of the InteractiveShell to print in this buffer ? 2- My second problem is more of a concern : I want, from the InteractiveShell, to open some new wx window and to be able to define some callback events for these windows. This sounds like a mess to do no ? How do I share the whole wx environment that was open in my initial environment with my InteractiveShell I hope I am clear ? Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32777434.html Sent from the IPython - Development mailing list archive at Nabble.com. From emmanuel.bacry at polytechnique.fr Thu Nov 3 20:30:54 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Thu, 3 Nov 2011 17:30:54 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: <32777457.post@talk.nabble.com> Fernando Perez wrote: > > On Thu, Nov 3, 2011 at 5:17 AM, Emmanuel Bacry > wrote: >> Thomas Kluyver-2 wrote: >>> >>> At the moment, I don't think there's a good way to embed IPython in a >>> GUI >>> without starting a separate process, but this is something we want to >>> get >>> round to. >>> >> Too bad ! > > There is a way of embedding that may actually work for you, and can > save you quite a bit of time in that you could actually use your > current Wx library as-is, and reuse our Qt console. This example: > > https://github.com/ipython/ipython/blob/master/docs/examples/lib/ipkernel_wxapp.py > > shows how you can embed an IPython *kernel* in your own application. > Given this kernel, you can then connect with any client capable of > speaking the ipython protocol (right now that's the qtconsole or the > notebook --with a small fix for talking to existing kernels needed), > but there's also a terminal one in the works in PR #864). > I just tried that but the InternalIPKernel is not found ? I'm using IPython 0.11. Where should it be ? By the way, before even asking some more questions (what you are saying seems very interesting but I'm not sure I follow), I don't quite understand the terminology here : what is a kernel ? (compared to a shell in the InteractiveShell example) Thank you for your long answer Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32777457.html Sent from the IPython - Development mailing list archive at Nabble.com. From fperez.net at gmail.com Thu Nov 3 21:24:03 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 3 Nov 2011 18:24:03 -0700 Subject: [IPython-dev] wxPython console In-Reply-To: <32777457.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> Message-ID: On Thu, Nov 3, 2011 at 5:30 PM, Emmanuel Bacry wrote: > I just tried that but the InternalIPKernel is not found ? I'm using IPython > 0.11. > Where should it be ? Ah, sorry: it's not available on 0.11. You'll need to use the git master branch to get that code. > By the way, before even asking some more questions (what you are saying > seems very interesting but I'm not sure I follow), I don't quite understand > the terminology here : what is a kernel ? (compared to a shell in the > InteractiveShell example) A kernel is basically the shell object, minus the assumption of being inside a terminal, and plus the network support to communicate over zeromq. Think of it as the shell, but on the network. Cheers, f From fperez.net at gmail.com Thu Nov 3 22:38:01 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 3 Nov 2011 19:38:01 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. Message-ID: Hi all, Yarik of neurodebian fame just reported a failure to build the docs: reading sources... [ 45%] api/generated/IPython.testing.iptest Exception occurred: File "/home/fperez/usr/lib/python2.6/site-packages/IPython/external/qt.py", line 20, in prepare_pyqt4 sip.setapi('QString', 2) ValueError: API 'QString' has already been set to version 1 Can anyone think of what is causing this? I hadn't done a full build doc recently, so I'm not sure when this snuck in, but obviously we need to sort it out before we can consider a release... Any ideas much appreciated. Cheers, f From benjaminrk at gmail.com Fri Nov 4 01:20:56 2011 From: benjaminrk at gmail.com (Min RK) Date: Thu, 3 Nov 2011 22:20:56 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. In-Reply-To: References: Message-ID: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> Should probably just add external.qt to the autodoc exclusions. -MinRK On Nov 3, 2011, at 19:38, Fernando Perez wrote: > Hi all, > > Yarik of neurodebian fame just reported a failure to build the docs: > > reading sources... [ 45%] api/generated/IPython.testing.iptest > Exception occurred: > File "/home/fperez/usr/lib/python2.6/site-packages/IPython/external/qt.py", > line 20, in prepare_pyqt4 > sip.setapi('QString', 2) > ValueError: API 'QString' has already been set to version 1 > > > Can anyone think of what is causing this? I hadn't done a full build > doc recently, so I'm not sure when this snuck in, but obviously we > need to sort it out before we can consider a release... > > Any ideas much appreciated. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From emmanuel.bacry at polytechnique.fr Fri Nov 4 06:37:50 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Fri, 4 Nov 2011 03:37:50 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> Message-ID: <32779421.post@talk.nabble.com> Fernando Perez wrote: > > On Thu, Nov 3, 2011 at 5:30 PM, Emmanuel Bacry > wrote: >> I just tried that but the InternalIPKernel is not found ? I'm using >> IPython >> 0.11. >> Where should it be ? > > Ah, sorry: it's not available on 0.11. You'll need to use the git > master branch to get that code. > >> By the way, before even asking some more questions (what you are saying >> seems very interesting but I'm not sure I follow), I don't quite >> understand >> the terminology here : what is a kernel ? (compared to a shell in the >> InteractiveShell example) > > A kernel is basically the shell object, minus the assumption of being > inside a terminal, and plus the network support to communicate over > zeromq. Think of it as the shell, but on the network. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > Thank you. I will look into this possibility. Right now I am implementing a skeleton for what suggested Thomas to see how it could work. About that I have 2 more questions : 1- Is there a module in InteractiveShell to do automatic () match, i.e., give me the '(' corresponding to a just typed in ')' ? 2- Is there a way to get interactive help while typing in the shell (? la Spyder if you are used to it) ? Thank you Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32779421.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Fri Nov 4 07:08:35 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Nov 2011 11:08:35 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32779421.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> Message-ID: On 4 November 2011 10:37, Emmanuel Bacry wrote: > 1- Is there a module in InteractiveShell to do automatic () match, i.e., > give me the '(' corresponding to a just typed in ')' ? > Not that I know of. The Qt console does it, so you could hunt out the code that it uses, but it may not be neatly packaged in a function. > 2- Is there a way to get interactive help while typing in the shell (? la > Spyder if you are used to it) ? I'm not used to spyder, but the Qt console produces tooltips when you open the bracket for a function call, describing what parameters the function takes. You could implement something similar using shell.inspector.info(x). Also, if the user enters something like "theobject.attr?", information about attr will be displayed. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Nov 4 07:16:19 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 04 Nov 2011 11:16:19 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: On 11/3/11 10:23 PM, Fernando Perez wrote: > Finally, we do envision being able to embed a qt/wx console into a > single process, and we're not too far from the necessary architectural > cleanup to make that easy. The model would be that a client > (terminal, qtconsole, wx, etc) would have a kernel manager object that > could be either local or remote, but with otherwise identical api. > > Right now we only have remote ones, and the local terminal doesn't > abstract out its interactions through a kernel manager. But once that > design is in place (and we're not that far from it, we just need some > more refactoring to be completed), it should be effectively > transparent whether a client is using a local or remote kernel; local > ones wouldn't even need zmq as their 'messaging' would just be > in-process method calls. I've done this before on 0.11. I'll see if I can update the code to reflect the current IPython codebase. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From emmanuel.bacry at polytechnique.fr Fri Nov 4 08:02:25 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Fri, 4 Nov 2011 05:02:25 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: <32779913.post@talk.nabble.com> Robert Kern-2 wrote: > > On 11/3/11 10:23 PM, Fernando Perez wrote: > > I've done this before on 0.11. I'll see if I can update the code to > reflect the > current IPython codebase. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though > it had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > I would definitely be interested ! Thank you Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32779913.html Sent from the IPython - Development mailing list archive at Nabble.com. From emmanuel.bacry at polytechnique.fr Fri Nov 4 08:05:29 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Fri, 4 Nov 2011 05:05:29 -0700 (PDT) Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> Message-ID: <32779921.post@talk.nabble.com> Thomas Kluyver-2 wrote: > > On 4 November 2011 10:37, Emmanuel Bacry > wrote: > >> 1- Is there a module in InteractiveShell to do automatic () match, i.e., >> give me the '(' corresponding to a just typed in ')' ? >> > > Not that I know of. The Qt console does it, so you could hunt out the code > that it uses, but it may not be neatly packaged in a function. > > >> 2- Is there a way to get interactive help while typing in the shell (? la >> Spyder if you are used to it) ? > > > I'm not used to spyder, but the Qt console produces tooltips when you open > the bracket for a function call, describing what parameters the function > takes. You could implement something similar using > shell.inspector.info(x). > Also, if the user enters something like "theobject.attr?", information > about attr will be displayed. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > Ok, I'll look into all that ! One more question : if I want to, will I be able to run matplotlib (non blocking) from the InteractiveShell I am using ? Will it uses the wx main loop I am using ? Emmanuel -- View this message in context: http://old.nabble.com/wxPython-console-tp32057061p32779921.html Sent from the IPython - Development mailing list archive at Nabble.com. From takowl at gmail.com Fri Nov 4 08:10:09 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Nov 2011 12:10:09 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <32779921.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> <32779921.post@talk.nabble.com> Message-ID: On 4 November 2011 12:05, Emmanuel Bacry wrote: > One more question : if I want to, will I be able to run matplotlib (non > blocking) from the InteractiveShell I am using ? Will it uses the wx main > loop I am using ? > I'm sure it's possible, but I don't know enough about matplotlib to know what you'd need to do. Try it: it might just work automatically. If not, I'm sure someone will be able to tell you more. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 12:48:53 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 09:48:53 -0700 Subject: [IPython-dev] wxPython console In-Reply-To: <32779921.post@talk.nabble.com> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> <32779921.post@talk.nabble.com> Message-ID: Hi, On Fri, Nov 4, 2011 at 5:05 AM, Emmanuel Bacry wrote: > > One more question : if I want to, will I be able to run matplotlib (non > blocking) from the InteractiveShell I am using ? Will it uses the wx main > loop I am using ? A few pointers: https://github.com/ipython/ipython/blob/master/IPython/lib/pylabtools.py https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py This is the machinery used by IPython to interact with the Wx event loop, *without threading*. We got rid of the threading code in the 0.11 series, but if you decide to use threading tricks, you can grab them from the git repo from the release 0.10.2 tag, that machinery lived in IPython/Shell.py. Cheers, f From benjaminrk at gmail.com Fri Nov 4 12:57:36 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Nov 2011 09:57:36 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. In-Reply-To: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> References: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> Message-ID: On Thu, Nov 3, 2011 at 22:20, Min RK wrote: > Should probably just add external.qt to the autodoc exclusions. > Turns out it already is, the issue is actually in test_for('IPython.external.qt') in iptest, which will raise a ValueError instead of Import/Runtime if sip API has been set (i.e. PyQt has been imported already). Is there a good reason that test_for doesn't catch all Exceptions instead of just Import/Runtime? If importing something raises an exception, it's not importable. It shouldn't matter what that exception is. Alternatively, we could turn the sip ValueError into an ImportError in external.qt. Both work, and are quick and easy fixes. > > -MinRK > > On Nov 3, 2011, at 19:38, Fernando Perez wrote: > > > Hi all, > > > > Yarik of neurodebian fame just reported a failure to build the docs: > > > > reading sources... [ 45%] api/generated/IPython.testing.iptest > > Exception occurred: > > File > "/home/fperez/usr/lib/python2.6/site-packages/IPython/external/qt.py", > > line 20, in prepare_pyqt4 > > sip.setapi('QString', 2) > > ValueError: API 'QString' has already been set to version 1 > > > > > > Can anyone think of what is causing this? I hadn't done a full build > > doc recently, so I'm not sure when this snuck in, but obviously we > > need to sort it out before we can consider a release... > > > > Any ideas much appreciated. > > > > Cheers, > > > > f > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 13:03:29 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 10:03:29 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. In-Reply-To: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> References: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> Message-ID: On Thu, Nov 3, 2011 at 10:20 PM, Min RK wrote: > Should probably just add external.qt to the autodoc exclusions. Actually it was iptest that was pulling it in via a decorator, all of external was already in the exclusions list. Fixed and pushed. Cheers, f From benjaminrk at gmail.com Fri Nov 4 13:09:30 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Nov 2011 10:09:30 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. In-Reply-To: References: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> Message-ID: But iptest still has a bug - test_for(qt) will On Fri, Nov 4, 2011 at 10:03, Fernando Perez wrote: > On Thu, Nov 3, 2011 at 10:20 PM, Min RK wrote: > > Should probably just add external.qt to the autodoc exclusions. > > Actually it was iptest that was pulling it in via a decorator, all of > external was already in the exclusions list. > Not a decorator, it was actually the test_for(external.qt) at the top, but that really is a bug (albeit a minor one). We should fix the check, so that it doesn't raise an error if qt is not usable. excluding it from autodoc is still the right answer, but not complete. -MinRK > Fixed and pushed. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 13:12:14 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 10:12:14 -0700 Subject: [IPython-dev] Docs build broken on master, Qt API issues.. In-Reply-To: References: <8B15C837-C4E3-4D51-9EB8-0C1697847B3E@gmail.com> Message-ID: On Fri, Nov 4, 2011 at 10:09 AM, MinRK wrote: > > excluding it from autodoc is still the right answer, but not complete. Feel free to push the cleaner fix, I was trying to get this quickly done for Yarik to do the Debian build but I'm in the middle of a work deadline so I've switched gears out again... Thanks! f From benjaminrk at gmail.com Fri Nov 4 14:01:32 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Nov 2011 11:01:32 -0700 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> <32779921.post@talk.nabble.com> Message-ID: A note regarding the InternalIPKernel example - it had fallen out of date with recent changes to how we connect to kernels (files instead of explicit ports). It has been fixed. On Fri, Nov 4, 2011 at 09:48, Fernando Perez wrote: > Hi, > > On Fri, Nov 4, 2011 at 5:05 AM, Emmanuel Bacry > wrote: > > > > One more question : if I want to, will I be able to run matplotlib (non > > blocking) from the InteractiveShell I am using ? Will it uses the wx main > > loop I am using ? > > A few pointers: > > https://github.com/ipython/ipython/blob/master/IPython/lib/pylabtools.py > https://github.com/ipython/ipython/blob/master/IPython/lib/inputhook.py > https://github.com/ipython/ipython/blob/master/IPython/lib/inputhookwx.py > > This is the machinery used by IPython to interact with the Wx event > loop, *without threading*. We got rid of the threading code in the > 0.11 series, but if you decide to use threading tricks, you can grab > them from the git repo from the release 0.10.2 tag, that machinery > lived in IPython/Shell.py. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 14:10:17 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 11:10:17 -0700 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32777457.post@talk.nabble.com> <32779421.post@talk.nabble.com> <32779921.post@talk.nabble.com> Message-ID: On Fri, Nov 4, 2011 at 11:01 AM, MinRK wrote: > A note regarding the InternalIPKernel example - it had fallen out of date > with recent changes to how we connect to kernels (files instead of explicit > ports). ?It has been fixed. Thanks! f From fperez.net at gmail.com Fri Nov 4 14:21:54 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 11:21:54 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? Message-ID: Hi all, can anyone confirm what I'm seeing on my system? Lately I'd been using the notebook for everything and hadn't used the qt console interactively in a while. Just now I opened it to run some throw-away codes for work, and it's basically impossible to use. There's a 2-3 second delay on any action going to the kernel such as tab completion or calltips, long enough that on tab completion the dead kernel dialog pops up. Basically it can't really be used right now. I'm on an ubuntu 11.10 box, using pyqt (no pyside installed). I'll dig into bisecting things later if need be, but right now I have some urgent work to take care of and can't, so if anyone knows/is willing to dig up what went wrong, it would be great. In its current state, the console is basically busted, so we do need to fix this... Cheers, f From benjaminrk at gmail.com Fri Nov 4 14:37:06 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Nov 2011 11:37:06 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: I certainly don't see this kind of thing, but my work machine is on 10.04. I'll check on my 11.10 machine when I get home. Are you using pylab, and if so, which backend, and does it make a difference if you switch? I wouldn't be *too* surprised if one of the backends got a bug when I made the change from classes to functions for gui integration. But Tk and Qt seem to be working fine for me. -MinRK On Fri, Nov 4, 2011 at 11:21, Fernando Perez wrote: > Hi all, > > can anyone confirm what I'm seeing on my system? Lately I'd been > using the notebook for everything and hadn't used the qt console > interactively in a while. Just now I opened it to run some throw-away > codes for work, and it's basically impossible to use. There's a 2-3 > second delay on any action going to the kernel such as tab completion > or calltips, long enough that on tab completion the dead kernel dialog > pops up. > > Basically it can't really be used right now. I'm on an ubuntu 11.10 > box, using pyqt (no pyside installed). > > I'll dig into bisecting things later if need be, but right now I have > some urgent work to take care of and can't, so if anyone knows/is > willing to dig up what went wrong, it would be great. In its current > state, the console is basically busted, so we do need to fix this... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 14:39:49 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 11:39:49 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: On Fri, Nov 4, 2011 at 11:37 AM, MinRK wrote: > Are you using pylab, and if so, which backend, and does it make a difference > if you switch? Gtk. And yes, the problem is *only* with Gtk; inline, qt, wx, tk are all ok. So actually it could be an older problem and we may not have seen this in a while, I hadn't used the gtk backend in ages. Cheers, f From benjaminrk at gmail.com Fri Nov 4 14:44:01 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Nov 2011 11:44:01 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: On Fri, Nov 4, 2011 at 11:39, Fernando Perez wrote: > On Fri, Nov 4, 2011 at 11:37 AM, MinRK wrote: > > Are you using pylab, and if so, which backend, and does it make a > difference > > if you switch? > > Gtk. And yes, the problem is *only* with Gtk; inline, qt, wx, tk are all > ok. > > So actually it could be an older problem and we may not have seen this > in a while, I hadn't used the gtk backend in ages. > Okay, good to know. Gtk does seem slower to me, but not as slow as it sounds like you are experiencing. Are you using apt matplotlib, or git? > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 4 14:45:20 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 11:45:20 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: On Fri, Nov 4, 2011 at 11:44 AM, MinRK wrote: > Okay, good to know. ?Gtk does seem slower to me, but not as slow as it > sounds like you are experiencing. ?Are you using apt matplotlib, or git? git mpl, last checkout from Oct 13. Let me rebuild mpl to see if the problem persists. f From mark.voorhies at ucsf.edu Fri Nov 4 17:05:50 2011 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Fri, 4 Nov 2011 14:05:50 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: <201111041405.51228.mark.voorhies@ucsf.edu> On Friday, November 04, 2011 11:45:20 am Fernando Perez wrote: > On Fri, Nov 4, 2011 at 11:44 AM, MinRK wrote: > > Okay, good to know. Gtk does seem slower to me, but not as slow as it > > sounds like you are experiencing. Are you using apt matplotlib, or git? > > git mpl, last checkout from Oct 13. Let me rebuild mpl to see if the > problem persists. > > f I'm not sure if I can reproduce this. If I set 'backend: GTK' in ~/.matplotlib/matplotlibrc and launch "ipython qtconsole --pylab" then matplotlib.pyplot.get_backend() reports "GTKAgg". In this configuration I am getting the dead kernel dialog, but not in a predictable way (e.g., sometimes it is enough to import matplotlib.pyplot and try to call get_backend, sometimes I need to do a bit of real work before seeing the message). The problem doesn't appear to repeat with my default (TkAgg backend) configuration. I've been using the TkAgg backend with the qtconsole for my day-to-day work for the last few weeks without seeing the dead kernel dialog or noticing slow response. Configuration: Ubuntu 10.04 32bit ipython baf566fa94e3be8c383fab4a4c628c2c08e5797e (just pulled from master) matplotlib from apt (0.99.1.2-3ubuntu1) --Mark From fperez.net at gmail.com Fri Nov 4 17:09:46 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 4 Nov 2011 14:09:46 -0700 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: <201111041405.51228.mark.voorhies@ucsf.edu> References: <201111041405.51228.mark.voorhies@ucsf.edu> Message-ID: On Fri, Nov 4, 2011 at 2:05 PM, Mark Voorhies wrote: > If I set 'backend: GTK' in ~/.matplotlib/matplotlibrc > and launch "ipython qtconsole --pylab" > then matplotlib.pyplot.get_backend() reports "GTKAgg". > > In this configuration I am getting the dead kernel dialog, > but not in a predictable way (e.g., sometimes it is enough > to import matplotlib.pyplot and try to call get_backend, > sometimes I need to do a bit of real work before seeing > the message). > > The problem doesn't appear to repeat with my default (TkAgg backend) > configuration. > > I've been using the TkAgg backend with the qtconsole for my day-to-day > work for the last few weeks without seeing the dead kernel dialog or > noticing slow response. The dead kernel dialog only appears if the delay is > 3s, and I think the typical delays I'm seeing hover right around that, so sometimes it happens and others not. But dead kernel dialog or not, there's always a very sluggish response to all actions, and for example call tooltips never appear. This does not happen in the terminal ipython (with pylab/gtk). So it's some interaction between our gtk event loop handling and the kernel code. Weird... Cheers, f From asmeurer at gmail.com Fri Nov 4 20:11:37 2011 From: asmeurer at gmail.com (Aaron Meurer) Date: Fri, 4 Nov 2011 18:11:37 -0600 Subject: [IPython-dev] Qt console unusable, any ideas? In-Reply-To: References: Message-ID: If it used to work, but doesn't now, as you suggest, you might try using git bisect to find the offending commit. Or you may find that it in fact was never working in your particular configuration. Aaron Meurer On Fri, Nov 4, 2011 at 12:21 PM, Fernando Perez wrote: > Hi all, > > can anyone confirm what I'm seeing on my system? ?Lately I'd been > using the notebook for everything and hadn't used the qt console > interactively in a while. ?Just now I opened it to run some throw-away > codes for work, and it's basically impossible to use. ?There's a 2-3 > second delay on any action going to the kernel such as tab completion > or calltips, long enough that on tab completion the dead kernel dialog > pops up. > > Basically it can't really be used right now. ?I'm on an ubuntu 11.10 > box, using pyqt (no pyside installed). > > I'll dig into bisecting things later if need be, but right now I have > some urgent work to take care of and can't, so if anyone knows/is > willing to dig up what went wrong, it would be great. ?In its current > state, the console is basically busted, so we do need to fix this... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From fperez.net at gmail.com Sat Nov 5 21:58:12 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 18:58:12 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control Message-ID: Hi folks, I wanted to start a discussion on the notebook format regarding its suitability for version control. I see the notebook format as the way in which I'll likely keep (and hopefully many others too) most of my research notes/work, and thereore it's important that it's as easy as possible to version control notebooks and use them smoothly in a version-controlled workflow. Unfortunately right now, the format we have simply doesn't fit that, mainly for two reasons: 1. The cell inputs (code and text) are stored as a single line in the json format. This means that virtually any edits anywhere in a cell will immediately produce VC conflicts. Furthermore, they are nearly impossible to resolve by hand because you have to scan very long lines by eye, and can only apply wholesale one version or the other. 2. The presence of outputs stored inside the file causes two separate problems: a) The large binary blobs make the files often quite large. b) Changes in the binary blobs can't really be inspected by hand, but tend to easily cause conflicts. To get a sense of the problem, here's the diff from a pull request made on a simple (mostly for testing purposes) repo: https://github.com/fperez/nipy-notebooks/pull/1/files That diff is more or less useless: note the huge horizontal scroll bar, and changes in inputs are impossible to understand. So I think we need to find a solution. This doesn't have to happen necessarily right away, since we're trying to put 0.12 out; I think it's OK if for now our format is mostly treated as a binary blob. But we do need to come up with a plan for the medium term. Here's my proposal, with full credit going to Yarik who suggested the idea of splitting outputs into a separate file. There are basically two changes against what we have now: 1. The notebook would *always* be split into two files, the .ipynb containing only inputs, and a companion (say .ipynbo) file with all outputs. If an output file is not available or is detected to have problems such as cell number mismatch, it is simply ignored (it can always be recreated by rerunning the notebook. 2. All inputs would be stored in a json list of strings instead of a single string. With #1, one would naturally only commit to VC the ipynb file, leaving the output ones to be always ignored. People could obviously choose to commit the output as well, at their own risk. #2 would make it much easier to get line-by-line diffs of any input (code or text). I think together, these two changes mostly solve the problems I've encountered in practice so far. I'm trying really hard to eat our own dogfood by using these tools in actual, everyday research work, so that we see the problems first. And while I think the notebook is reaching a point where it's a great working environment (even if we have a ton of ideas for improvements already and things we know need fixing), it's clear now to me that we fail pretty badly as a version-controllable format. I realize that implementing something like this will add non-trivial complexity to the format read/write code in a number of places, so if anyone sees a simpler solution to the problem, we're all ears. But we do need to figure out how to make the notebooks first-class citizens in a VC world; the (effectively) opaque binary blobs they are now just won't cut it in the long run. Thoughts, ideas? Cheers, f From fperez.net at gmail.com Sat Nov 5 22:15:09 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 19:15:09 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 6:58 PM, Fernando Perez wrote: > There are basically two changes against what we have now: Actually there's a third, though much more minor. If you look at the diff https://github.com/fperez/nipy-notebooks/pull/1/files you can see that basically every cell is marked with changes, but only because the order of the json dict fields changed. I'm not sure where that change is coming from, but it probably means we're also going to have to ensure that fields are always ordered in some canonical form (alphabetical sorting would be fine). Otherwise every single commit would generate a giant diff as all cells appear to change from field ordering, even if there isn't any actual content change. But this should be much easier to deal with, since ordering the fields doesn't change the json data structure in any way, it's just a slight annoyance to have to do custom writing ourselves instead of letting the json serializer blindly write things out. Cheers, f From benjaminrk at gmail.com Sat Nov 5 22:41:31 2011 From: benjaminrk at gmail.com (MinRK) Date: Sat, 5 Nov 2011 19:41:31 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 18:58, Fernando Perez wrote: > Hi folks, > > I wanted to start a discussion on the notebook format regarding its > suitability for version control. I see the notebook format as the way > in which I'll likely keep (and hopefully many others too) most of my > research notes/work, and thereore it's important that it's as easy as > possible to version control notebooks and use them smoothly in a > version-controlled workflow. Unfortunately right now, the format we > have simply doesn't fit that, mainly for two reasons: > > 1. The cell inputs (code and text) are stored as a single line in the > json format. This means that virtually any edits anywhere in a cell > will immediately produce VC conflicts. Furthermore, they are nearly > impossible to resolve by hand because you have to scan very long lines > by eye, and can only apply wholesale one version or the other. > > 2. The presence of outputs stored inside the file causes two separate > problems: > a) The large binary blobs make the files often quite large. > b) Changes in the binary blobs can't really be inspected by hand, but > tend to easily cause conflicts. > > To get a sense of the problem, here's the diff from a pull request > made on a simple (mostly for testing purposes) repo: > > https://github.com/fperez/nipy-notebooks/pull/1/files > > That diff is more or less useless: note the huge horizontal scroll > bar, and changes in inputs are impossible to understand. > > So I think we need to find a solution. This doesn't have to happen > necessarily right away, since we're trying to put 0.12 out; I think > it's OK if for now our format is mostly treated as a binary blob. But > we do need to come up with a plan for the medium term. > > Here's my proposal, with full credit going to Yarik who suggested the > idea of splitting outputs into a separate file. There are basically > two changes against what we have now: > > 1. The notebook would *always* be split into two files, the .ipynb > containing only inputs, and a companion (say .ipynbo) file with all > outputs. If an output file is not available or is detected to have > problems such as cell number mismatch, it is simply ignored (it can > always be recreated by rerunning the notebook. > There is a *huge* disadvantage in portability to notebooks not being single files. I think this still makes sense, though. I would treat the output as a 'cache' (along the lines of .pyc / __cache__), rather than considering the notebook itself as a multi-file format. And you should be able to embed the outputs in a single file if you want, for easier portability. Doing it this way would not require changing the notebook format, because current (output-included) notebooks would still comply with the spec. > 2. All inputs would be stored in a json list of strings instead of a > single string. > I like this - splitlines(code) / '\n'.join(lines) makes it easy. This change does mean that we need it to be nbformat v3. > > With #1, one would naturally only commit to VC the ipynb file, leaving > the output ones to be always ignored. People could obviously choose > to commit the output as well, at their own risk. #2 would make it much > easier to get line-by-line diffs of any input (code or text). > > I think together, these two changes mostly solve the problems I've > encountered in practice so far. I'm trying really hard to eat our own > dogfood by using these tools in actual, everyday research work, so > that we see the problems first. And while I think the notebook is > reaching a point where it's a great working environment (even if we > have a ton of ideas for improvements already and things we know need > fixing), it's clear now to me that we fail pretty badly as a > version-controllable format. > > I realize that implementing something like this will add non-trivial > complexity to the format read/write code in a number of places, so if > anyone sees a simpler solution to the problem, we're all ears. But we > do need to figure out how to make the notebooks first-class citizens > in a VC world; the (effectively) opaque binary blobs they are now just > won't cut it in the long run. Yes, we do need to do better. > > Thoughts, ideas? > I think this sounds like a good start, with the only change that we still allow (optionally) outputs in a single file via the download button, rather than the notebook format being canonically multifile, which is just too painful. I think the key-order issue you mention in the addendum is easily fixed by specifying `sort_keys=True` in the json dump. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 5 22:58:47 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 19:58:47 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 7:41 PM, MinRK wrote: > On Sat, Nov 5, 2011 at 18:58, Fernando Perez wrote: >> > There is a *huge* disadvantage in portability to notebooks not being single > files. ?I think this still makes > sense, though. ?I would treat the output as a 'cache' (along the lines of > .pyc / __cache__), > rather than considering the notebook?itself as a multi-file format. ?And you > should be able > to embed the outputs in a single file if you want, for easier portability. > Doing it this way would not require changing the notebook format, because > current (output-included) > notebooks?would still comply with the spec. I agree that it's a big inconvenience for everyday, non-VC use. I like the idea of making it optional, it can be a flag set in the metadata dict, that indicates whether to keep outputs in the cache or internally (and also to offer the single-file download option). > I think this sounds like a good start, with the only change that we still > allow (optionally) outputs in a single file via the download button, rather > than the notebook?format being canonically multifile, which is just too > painful. Yes, that sounds like a good balance between everyday usability and being VC-friendly. > I think the key-order issue you mention in the addendum is easily fixed by > specifying `sort_keys=True` in the json dump. That's great to hear, I thought we'd have to do the sorting manually. That's a change we should make right away then, since it doesn't change the format. People can manually remove outputs from their notebooks and this will help if they use VC a little bit (even if the singe-line cells issue isn't fixed yet). Thanks for the feedback! Cheers, f From benjaminrk at gmail.com Sat Nov 5 23:08:56 2011 From: benjaminrk at gmail.com (MinRK) Date: Sat, 5 Nov 2011 20:08:56 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 19:58, Fernando Perez wrote: > On Sat, Nov 5, 2011 at 7:41 PM, MinRK wrote: > > On Sat, Nov 5, 2011 at 18:58, Fernando Perez > wrote: > >> > > > There is a *huge* disadvantage in portability to notebooks not being > single > > files. I think this still makes > > sense, though. I would treat the output as a 'cache' (along the lines of > > .pyc / __cache__), > > rather than considering the notebook itself as a multi-file format. And > you > > should be able > > to embed the outputs in a single file if you want, for easier > portability. > > Doing it this way would not require changing the notebook format, because > > current (output-included) > > notebooks would still comply with the spec. > > I agree that it's a big inconvenience for everyday, non-VC use. I > like the idea of making it optional, it can be a flag set in the > metadata dict, that indicates whether to keep outputs in the cache or > internally (and also to offer the single-file download option). > > > > I think this sounds like a good start, with the only change that we still > > allow (optionally) outputs in a single file via the download button, > rather > > than the notebook format being canonically multifile, which is just too > > painful. > > Yes, that sounds like a good balance between everyday usability and > being VC-friendly. > > > I think the key-order issue you mention in the addendum is easily fixed > by > > specifying `sort_keys=True` in the json dump. > > That's great to hear, I thought we'd have to do the sorting manually. > That's a change we should make right away then, since it doesn't > change the format. People can manually remove outputs from their > notebooks and this will help if they use VC a little bit (even if the > singe-line cells issue isn't fixed yet). > Yes, and if I recall, for now you can just hit 'ClearAll outputs' prior to save/commit, to remove output from VCS. > > Thanks for the feedback! > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sat Nov 5 23:14:41 2011 From: benjaminrk at gmail.com (MinRK) Date: Sat, 5 Nov 2011 20:14:41 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: Shall I push the sort_keys to master? It's a one-line change in nbformat/v2/nbjson.py: diff --git a/IPython/nbformat/v2/nbjson.py b/IPython/nbformat/v2/nbjson.py index b86297d..e67bec3 100644 --- a/IPython/nbformat/v2/nbjson.py +++ b/IPython/nbformat/v2/nbjson.py @@ -48,6 +48,7 @@ class JSONWriter(NotebookWriter): def writes(self, nb, **kwargs): kwargs['cls'] = BytesEncoder kwargs['indent'] = 4 + kwargs['sort_keys'] = True return json.dumps(nb, **kwargs) Since loading from JSON is not sensitive to ordering, this can't have any effect on the content of existing notebooks, other than a reorder of keys in the file the first time it is saved. -MinRK On Sat, Nov 5, 2011 at 20:08, MinRK wrote: > > > On Sat, Nov 5, 2011 at 19:58, Fernando Perez wrote: > >> On Sat, Nov 5, 2011 at 7:41 PM, MinRK wrote: >> > On Sat, Nov 5, 2011 at 18:58, Fernando Perez >> wrote: >> >> >> >> > There is a *huge* disadvantage in portability to notebooks not being >> single >> > files. I think this still makes >> > sense, though. I would treat the output as a 'cache' (along the lines >> of >> > .pyc / __cache__), >> > rather than considering the notebook itself as a multi-file format. >> And you >> > should be able >> > to embed the outputs in a single file if you want, for easier >> portability. >> > Doing it this way would not require changing the notebook format, >> because >> > current (output-included) >> > notebooks would still comply with the spec. >> >> I agree that it's a big inconvenience for everyday, non-VC use. I >> like the idea of making it optional, it can be a flag set in the >> metadata dict, that indicates whether to keep outputs in the cache or >> internally (and also to offer the single-file download option). >> >> >> > I think this sounds like a good start, with the only change that we >> still >> > allow (optionally) outputs in a single file via the download button, >> rather >> > than the notebook format being canonically multifile, which is just too >> > painful. >> >> Yes, that sounds like a good balance between everyday usability and >> being VC-friendly. >> >> > I think the key-order issue you mention in the addendum is easily fixed >> by >> > specifying `sort_keys=True` in the json dump. >> >> That's great to hear, I thought we'd have to do the sorting manually. >> That's a change we should make right away then, since it doesn't >> change the format. People can manually remove outputs from their >> notebooks and this will help if they use VC a little bit (even if the >> singe-line cells issue isn't fixed yet). >> > > Yes, and if I recall, for now you can just hit 'ClearAll outputs' prior to > save/commit, to remove output from VCS. > > >> >> Thanks for the feedback! >> >> Cheers, >> >> f >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 5 23:14:44 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 20:14:44 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 8:08 PM, MinRK wrote: > Yes, and if I recall, for now you can just hit 'ClearAll outputs' prior to > save/commit, to remove output from VCS. Yes, that's what I've been doing in some cases. It's easy to forget and thus not ideal, but it's a workable temporary solution. That and the sorted fields will already help with a git workflow, until we implement automatic separation of output and line splitting of inputs. Cheers, f From fperez.net at gmail.com Sat Nov 5 23:20:01 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 20:20:01 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: On Sat, Nov 5, 2011 at 8:14 PM, MinRK wrote: > Since loading from JSON is not sensitive to ordering, this can't have any > effect on the content of existing notebooks, other than a reorder of keys in > the file the first time it is saved. Yes, please go ahead and push this. Thanks! f From mark.voorhies at ucsf.edu Sat Nov 5 23:39:43 2011 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Sat, 5 Nov 2011 20:39:43 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: Message-ID: <201111052039.44347.mark.voorhies@ucsf.edu> On Saturday, November 05, 2011 07:58:47 pm Fernando Perez wrote: > On Sat, Nov 5, 2011 at 7:41 PM, MinRK wrote: > > On Sat, Nov 5, 2011 at 18:58, Fernando Perez wrote: > >> > > > There is a *huge* disadvantage in portability to notebooks not being single > > files. I think this still makes > > sense, though. I would treat the output as a 'cache' (along the lines of > > .pyc / __cache__), > > rather than considering the notebook itself as a multi-file format. And you > > should be able > > to embed the outputs in a single file if you want, for easier portability. > > Doing it this way would not require changing the notebook format, because > > current (output-included) > > notebooks would still comply with the spec. > > I agree that it's a big inconvenience for everyday, non-VC use. I > like the idea of making it optional, it can be a flag set in the > metadata dict, that indicates whether to keep outputs in the cache or > internally (and also to offer the single-file download option). A complimentary idea would be to provide utility scripts for converting (unified notebook) <-> (inputs), (outputs) outside of an IPython session. These could be used by VC hooks (e.g., to strip or split outputs before committing or to strip outputs before diff-ing). The main use case that I am thinking of is keeping complete notebooks in git with hooks set up to ignore outputs for normal use, but with the ability to recover "environment dependent" outputs (e.g., when auditing an old result that is not reproducible on a new system). I am not sure how often this would be practical (since many of the interesting use cases would involve large outputs) or if it has any advantage over normal incremental backups of output files or externally marking outputs with the relevant commit SHAs (which is what I tend to do now). --Mark From fperez.net at gmail.com Sat Nov 5 23:59:09 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 20:59:09 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: <201111052039.44347.mark.voorhies@ucsf.edu> References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: On Sat, Nov 5, 2011 at 8:39 PM, Mark Voorhies wrote: > A complimentary idea would be to provide utility scripts for converting > (unified notebook) <-> (inputs), (outputs) > outside of an IPython session. > > These could be used by VC hooks (e.g., to strip or split outputs before > committing or to strip outputs before diff-ing). That would certainly be nice if one is committing outputs as well for reproducibility purposes. Though in that scenario, simply committing both the nb and the output cache file would likely give the same benefits, as one could simply instruct git to treat the caches as binary always. But more generally, we do want good command-line support for all notebook-related actions. Cheers, f From satra at mit.edu Sun Nov 6 00:12:07 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sun, 6 Nov 2011 00:12:07 -0400 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: <201111052039.44347.mark.voorhies@ucsf.edu> References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: just adding a few more random thoughts on the pile. 1. perhaps we can take a few things from gist.github.com. essentially i see the notebook and a gist as being conceptually quite similar (gist files == notebook cells). and they seem to do a pretty good job with version control there. i also very much like the list of revisions available. it's not a separate thing. 2. the notebook is a webapp. from a user perspective, the vcs integration should happen at the app level, not a secondary commit at the underlying directory/file level by the user. 3. i routinely save my notebook as a pdf to share output with folks or as a reminder to myself. primarily because in most cases, my notebook relies on data that only i have and only certain components would be useful to somebody as a pure nb. that's my way of preserving output because rerunning something like a permutation test is never going to return an exact answer (unless i set the random number generator state to be fixed). i.e. outputs themselves are separate snapshots of a notebook. 4. future: as notebooks might evolve to chapters and sections, the idea of structured version control makes quite a bit of sense. not to mention variants and derivatives of a given notebook (which something like git should handle very well). cheers, satra ps. [feature requests] i use the notebook routinely and there are lots of things i'd like to see. is there already a notebook feature request page? (e.g. clone my current -not necessarily saved- cells to a new nb) -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun Nov 6 01:03:50 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 22:03:50 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: On Sat, Nov 5, 2011 at 9:12 PM, Satrajit Ghosh wrote: > ps. [feature requests] i use the notebook routinely and there are lots of > things i'd like to see. is there already a notebook feature request page? > (e.g. clone my current -not necessarily saved- cells to a new nb) I just made one here: https://github.com/ipython/ipython/issues/977 Since most people already have a github account, it seemed like the easiest way to let anyone contribute ideas and requests. We can later filter those over time into specific issues once there's consensus on them and time/resources to implement them. Cheers, f From fperez.net at gmail.com Sun Nov 6 01:07:47 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 5 Nov 2011 22:07:47 -0700 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: On Sat, Nov 5, 2011 at 9:12 PM, Satrajit Ghosh wrote: > 2. ?the notebook is a webapp. from a user perspective, the vcs integration > should happen at the app level, not a secondary commit at the underlying > directory/file level by the user. Yes and no: while I think we should offer easy integration with VC (and things like a one-click 'publish as gist' menu entry would be great), I also don't want to lock anyone in the 'app' view of the world. If anyone has notebooks that are part of an existing project with its own version control setup, then that's where the VC belongs, not in the notebook app. For that use case, the notebooks are just another type of file, next to python scripts, text files and anything else that may be part of that project. There's no reason why users in that scenario should be forced to manage the notebooks differently from their other files in the project. Cheers, f From takowl at gmail.com Sun Nov 6 06:22:02 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 6 Nov 2011 11:22:02 +0000 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: One thought prompted by Fernando's comparison between .ipynbo files and .pyc files - Cache files cluttering up my working directory can be an annoyance, and this is why Python 3.2 has changed to using a __pycache__ directory for .pyc files. If we're splitting out output into separate files, it might be best to put them somewhere else - perhaps in the profile directory, or in a subfolder if we still want it to be simple to add them to version control. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From pablo.winant at gmail.com Sun Nov 6 08:07:30 2011 From: pablo.winant at gmail.com (Pablo Winant) Date: Sun, 6 Nov 2011 14:07:30 +0100 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: Hi, This conversation has reminded me of a nice and simple feature I have been missing when switching from the Sage notebook to the the IPython one. Attached to each Sage notebook is a DATA directory where the user can save files associated with the notebook. For instance the statistical data associated to the exploration code. Then notebooks can be exported/imported with the DATA directory so that everything is really self-contained and reproducible. Technically, I think everything is basically stored in a compressed archive. This behaviour is a bit different from the way IPython currently stores its notebook but it is not that far from having a cache directory (or a file) included with the saved notebook so I thought it could pertain to the same discussion. Pablo On Sun, Nov 6, 2011 at 12:22 PM, Thomas Kluyver wrote: > One thought prompted by Fernando's comparison between .ipynbo files and > .pyc files - Cache files cluttering up my working directory can be an > annoyance, and this is why Python 3.2 has changed to using a __pycache__ > directory for .pyc files. If we're splitting out output into separate > files, it might be best to put them somewhere else - perhaps in the profile > directory, or in a subfolder if we still want it to be simple to add them > to version control. > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Sun Nov 6 12:17:34 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sun, 6 Nov 2011 12:17:34 -0500 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: hi fernando, I also don't want to lock anyone in the 'app' view of the > world. If anyone has notebooks that are part of an existing project > with its own version control setup, then that's where the VC belongs, > not in the notebook app. fair enough. one should carefully consider if it's a file format that's just for the notebook 'app' and a general file format that could be: a) used throughout ipython (e.g., run/load blah.ipynb) and b) enable efficient vc. For that use case, the notebooks are just > another type of file, next to python scripts, text files and anything > else that may be part of that project. There's no reason why users in > that scenario should be forced to manage the notebooks differently > from their other files in the project. > but users are forced to manage vc of notebooks differently in any of the proposed scenarios. 1. one file (effectively binary) if i put it under a vc, it's simply a revision control history. no easy way to look at diffs beyond simple ones. 2. split (text + cache) if i want exact reproducibility of loading i need to have both under simultaneous version control. the chances of desynchronization are very high. and this would definitely benefit from an app-integrated vc. 3. split (keep text only) easy vc, but requires exact environment (data+libraries) for reproducibility. i would vote for allowing any of the above through configurable options as different people may have different use-cases. cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun Nov 6 12:35:40 2011 From: benjaminrk at gmail.com (MinRK) Date: Sun, 6 Nov 2011 09:35:40 -0800 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: On Sun, Nov 6, 2011 at 05:07, Pablo Winant wrote: > Hi, > > This conversation has reminded me of a nice and simple feature I have been > missing when switching from the Sage notebook to the the IPython one. > Attached to each Sage notebook is a DATA directory where the user can save > files associated with the notebook. For instance the statistical data > associated to the exploration code. Then notebooks can be exported/imported > with the DATA directory so that everything is really self-contained and > reproducible. Technically, I think everything is basically stored in a > compressed archive. > This behaviour is a bit different from the way IPython currently stores > its notebook but it is not that far from having a cache directory (or a > file) included with the saved notebook so I thought it could pertain to the > same discussion. > I think we see this sort of thing as being *easier* with IPython, because it's just the filesystem. If you put the notebook file in the folder with your data / scripts, you can just use them, track them all together with git, etc. Taking an existing project, and using it with the notebook should consist only of starting to use the notebook in the directory with the rest of your stuff. We also do not see notebooks as being 1:1 with data / python files, rather having the idea of a 'project', which is essentially a directory / repo, with code, data files, notebooks, etc. The current IPython notebook frontpage is the notebook listing for a single *project*, and in general you will be able to use multiple projects, but for now the server can only handle one at a time, and doesn't give you a view /interface for anything in that location other than notebooks, or changing the active project. > > Pablo > > > On Sun, Nov 6, 2011 at 12:22 PM, Thomas Kluyver wrote: > >> One thought prompted by Fernando's comparison between .ipynbo files and >> .pyc files - Cache files cluttering up my working directory can be an >> annoyance, and this is why Python 3.2 has changed to using a __pycache__ >> directory for .pyc files. If we're splitting out output into separate >> files, it might be best to put them somewhere else - perhaps in the profile >> directory, or in a subfolder if we still want it to be simple to add them >> to version control. >> >> Thomas >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun Nov 6 12:45:07 2011 From: benjaminrk at gmail.com (MinRK) Date: Sun, 6 Nov 2011 09:45:07 -0800 Subject: [IPython-dev] Thoughts on the notebook format for version control In-Reply-To: References: <201111052039.44347.mark.voorhies@ucsf.edu> Message-ID: On Sun, Nov 6, 2011 at 03:22, Thomas Kluyver wrote: > One thought prompted by Fernando's comparison between .ipynbo files and > .pyc files - Cache files cluttering up my working directory can be an > annoyance, and this is why Python 3.2 has changed to using a __pycache__ > directory for .pyc files. If we're splitting out output into separate > files, it might be best to put them somewhere else - perhaps in the profile > directory, or in a subfolder if we still want it to be simple to add them > to version control. If we really do consider them a cache, an output cache dir does make sense from a clutter point of view. We should think about whether we want the model for including output in vcs to be just adding the output cache to track, or turning off the cache, and inlining output into the notebook. > > > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Mon Nov 7 09:51:37 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Mon, 7 Nov 2011 15:51:37 +0100 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> Message-ID: On 3 nov. 2011 ? 15:27, Thomas Kluyver wrote : > In Python, there are two kinds of output: firstly from "print x", for which you can simply redirect stdout - replace sys.stdout with an object with a .write() method, which will capture strings written to it and display them in your console. > This one is "easy", and it works > Secondly, if you simply enter "x" at a prompt, IPython displays the value of x at an "Out [n]:" prompt. In such cases, Python calls sys.displayhook. This is called with the object itself, rather than its repr, so you can produce more complex displays. This is how we pretty-print datastructures, and also how the notebook can display rich objects like youtube videos. Have a look at IPython.core.displayhook to see our implementation of this. If you replace the displayhook attribute of an InteractiveShell object, it will take care of using that displayhook when you call run_cell. I have some problem using this last advice for redirection. Let me explain. If I override sys.stdout before creating the InteractiveShell, then it fails : /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/core/interactiveshell.pyc in init_io(self) 568 io.stdout = io.stderr = io.IOStream(self.readline._outputfile) 569 else: --> 570 io.stdout = io.IOStream(sys.stdout) 571 io.stderr = io.IOStream(sys.stderr) 572 /opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/utils/io.pyc in __init__(self, stream, fallback) 30 stream = fallback 31 else: ---> 32 raise ValueError("fallback required, but not specified") 33 self.stream = stream 34 self._swrite = stream.write ValueError: fallback required, but not specified If I override sys.stdout after creating the Interactive Shell (sharing or not the sys variable with the interactive shell I am creating), it does not crash but the display_hook does not use my newly defined sys.stdout and keeps the old one... =============== From what I understand, it looks like when the InteractiveShell is created, the sys.stdout is used to build an IOStream object (via io.stdout = io.IOStream(sys.stdout)) that will be used by the display_hook of the interactiveShell. ? Is that right? How do I redirect the printing of the default display_hook to my new sys.stdout (I don't want to go into implementing a new display_hook) ? Thank you Emmanuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Nov 7 10:23:28 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 7 Nov 2011 15:23:28 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> Message-ID: On 7 November 2011 14:51, Emmanuel Bacry wrote: > I have some problem using this last advice for redirection. Let me explain. > If I override sys.stdout before creating the InteractiveShell, then it > fails : > > Looking at the code for IPython.utils.io, it seems your stdout object needs to have both .write() and .flush() methods. With this done, the default displayhook should just work. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.buchs at gmail.com Mon Nov 7 11:58:18 2011 From: kevin.buchs at gmail.com (Kevin Buchs) Date: Mon, 7 Nov 2011 10:58:18 -0600 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: References: Message-ID: Fernando & list members On Fri, Oct 28, 2011 at 4:27 PM, Fernando Perez wrote: > Unfortunately, you're being bitten by a bug in our 0.11 version... > A few days ago I built a Windows binary installer of 0.12 which has > this bug already fixed: > http://archive.ipython.org/testing/ipython-0.12.dev.win32-setup.exe > Let us know if that helps. > The version 0.12 dev Windows installer ran fine. When I tried ipython-qtconsole.exe, it started up and displayed the following, but never came back with the prompt. Python 2.7.2 (default, Jun 12 2011, 15:08:59) [MSC v.1500 32 bit (Intel)] Type "copyright", "credits" or "license" for more information. IPython 0.12.dev -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. %guiref -> A brief reference about the graphical user interface. When I start from a command line, I see this: > python.exe ipython-qtconsole-script.pyw Traceback (most recent call last): File "", line 1, in File "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\zmq\__init__.py", line 25, in minimum_pyzmq_version, pyzmq_version)) ImportError: IPython.zmq requires pyzmq >= 2.1.4, but you have 2.1.10 So, it seems the problem still exists. -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Mon Nov 7 12:02:01 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 7 Nov 2011 17:02:01 +0000 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: References: Message-ID: On 7 November 2011 16:58, Kevin Buchs wrote: > > When I start from a command line, I see this: > > > python.exe ipython-qtconsole-script.pyw > Traceback (most recent call last): > File "", line 1, in > File > "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\zmq\__init__.py", > line 25, in > minimum_pyzmq_version, pyzmq_version)) > ImportError: IPython.zmq requires pyzmq >= 2.1.4, but you have 2.1.10 > > > So, it seems the problem still exists. > That's another bug in 0.11 - you can see from the traceback that it's importing stuff from your IPython 0.11 installation. If you install 0.12.dev so that it is imported before 0.11, it should work. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.buchs at gmail.com Mon Nov 7 15:29:47 2011 From: kevin.buchs at gmail.com (Kevin Buchs) Date: Mon, 7 Nov 2011 14:29:47 -0600 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: References: Message-ID: Yes, and with that change, all works fine. I found I even had stuff from ipython 0.10 laying around. I think I started to get the hang of the new configuration system. Thanks for documenting that further. Can someone give me a quick pointer to how I might set a default font for Qt Console that is larger (I know of C-+/-). - Kevin Buchs On Mon, Nov 7, 2011 at 11:02 AM, Thomas Kluyver wrote: > On 7 November 2011 16:58, Kevin Buchs wrote: > >> >> When I start from a command line, I see this: >> >> > python.exe ipython-qtconsole-script.pyw >> Traceback (most recent call last): >> File "", line 1, in >> File >> "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\zmq\__init__.py", >> line 25, in >> minimum_pyzmq_version, pyzmq_version)) >> ImportError: IPython.zmq requires pyzmq >= 2.1.4, but you have 2.1.10 >> >> >> So, it seems the problem still exists. >> > > That's another bug in 0.11 - you can see from the traceback that it's > importing stuff from your IPython 0.11 installation. If you install > 0.12.dev so that it is imported before 0.11, it should work. > > Thomas > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Mon Nov 7 15:49:12 2011 From: benjaminrk at gmail.com (MinRK) Date: Mon, 7 Nov 2011 12:49:12 -0800 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: References: Message-ID: On Mon, Nov 7, 2011 at 12:29, Kevin Buchs wrote: > Yes, and with that change, all works fine. I found I even had stuff from > ipython 0.10 laying around. > > I think I started to get the hang of the new configuration system. Thanks > for documenting that further. Can someone give me a quick pointer to how I > might set a default font for Qt Console that is larger (I know of C-+/-). > c.IPythonWidget.font_size = 14 # default is 0, which lets Qt pick. The actual default value will vary depending on the environment, but I think it's generally about 12. Search ipython_qtconsole_config.py for 'font', to see available options. Do `ipython profile create` to make the default config files, if you haven't already, that way you can see everything you can configure. -MinRK > - Kevin Buchs > > > On Mon, Nov 7, 2011 at 11:02 AM, Thomas Kluyver wrote: > >> On 7 November 2011 16:58, Kevin Buchs wrote: >> >>> >>> When I start from a command line, I see this: >>> >>> > python.exe ipython-qtconsole-script.pyw >>> Traceback (most recent call last): >>> File "", line 1, in >>> File >>> "C:\Python27\lib\site-packages\ipython-0.11-py2.7.egg\IPython\zmq\__init__.py", >>> line 25, in >>> minimum_pyzmq_version, pyzmq_version)) >>> ImportError: IPython.zmq requires pyzmq >= 2.1.4, but you have 2.1.10 >>> >>> >>> So, it seems the problem still exists. >>> >> >> That's another bug in 0.11 - you can see from the traceback that it's >> importing stuff from your IPython 0.11 installation. If you install >> 0.12.dev so that it is imported before 0.11, it should work. >> >> Thomas >> > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From tomspur at fedoraproject.org Tue Nov 8 04:54:53 2011 From: tomspur at fedoraproject.org (Thomas Spura) Date: Tue, 8 Nov 2011 10:54:53 +0100 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: References: Message-ID: <20111108105453.451a0a1d@fedoraproject.org> On Mon, 7 Nov 2011 12:49:12 -0800 MinRK wrote: > On Mon, Nov 7, 2011 at 12:29, Kevin Buchs > wrote: > Search ipython_qtconsole_config.py for 'font', to see available > options. Do `ipython profile create` to make the default config > files, if you haven't already, that way you can see everything you > can configure. I had no chance in looking into it, but it seems, that %install_default_config will have a traceback, when having no config files around... Tom From tomspur at fedoraproject.org Tue Nov 8 04:55:35 2011 From: tomspur at fedoraproject.org (Thomas Spura) Date: Tue, 8 Nov 2011 10:55:35 +0100 Subject: [IPython-dev] ipython qt on Windows In-Reply-To: <20111108105453.451a0a1d@fedoraproject.org> References: <20111108105453.451a0a1d@fedoraproject.org> Message-ID: <20111108105535.1e1b4bcc@fedoraproject.org> On Tue, 8 Nov 2011 10:54:53 +0100 Thomas Spura wrote: > On Mon, 7 Nov 2011 12:49:12 -0800 > MinRK wrote: > > > On Mon, Nov 7, 2011 at 12:29, Kevin Buchs > > wrote: > > > Search ipython_qtconsole_config.py for 'font', to see available > > options. Do `ipython profile create` to make the default config > > files, if you haven't already, that way you can see everything you > > can configure. > > I had no chance in looking into it, but it seems, that > %install_default_config will have a traceback, when having no config > files around... Having a bug for that would be good too ;) https://bugzilla.redhat.com/show_bug.cgi?id=751146 Tom From emmanuel.bacry at polytechnique.fr Tue Nov 8 09:45:59 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Tue, 8 Nov 2011 15:45:59 +0100 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> Message-ID: <312EE7C9-3148-43A7-9D00-F303A6312F92@polytechnique.fr> > On 7 November 2011 14:51, Emmanuel Bacry wrote: > I have some problem using this last advice for redirection. Let me explain. > If I override sys.stdout before creating the InteractiveShell, then it fails : > > > Looking at the code for IPython.utils.io, it seems your stdout object needs to have both .write() and .flush() methods. With this done, the default displayhook should just work. > > Thomas Yes, I'm sorry I could have find it myself. However I have another problem : Under wxPython I create an interactiveShell. I have a single button so that when I click on it a thread is ran in which a command is sent to the interactive shell: (I did not redirect any of stdout or sterr) interactiveshell.run_cell(command) If the command is valid, everything goes well. If not (for instance command = "ytytyutuy") then the thread never terminates and nothing is printed on stderr Emmanuel -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Nov 8 12:37:09 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 8 Nov 2011 17:37:09 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <312EE7C9-3148-43A7-9D00-F303A6312F92@polytechnique.fr> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> <312EE7C9-3148-43A7-9D00-F303A6312F92@polytechnique.fr> Message-ID: On 8 November 2011 14:45, Emmanuel Bacry wrote: > If the command is valid, everything goes well. If not (for instance > command = "ytytyutuy") then the thread never terminates and nothing is > printed on stderr I take it you've redirected stderr in the same way as stdout? Do you have it set to automatically go to the debugger (call %pdb to toggle it on and off)? If you start your application from a terminal, do you see anything appear in the terminal? Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From emmanuel.bacry at polytechnique.fr Tue Nov 8 14:52:41 2011 From: emmanuel.bacry at polytechnique.fr (Emmanuel Bacry) Date: Tue, 8 Nov 2011 20:52:41 +0100 Subject: [IPython-dev] wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> <312EE7C9-3148-43A7-9D00-F303A6312F92@polytechnique.fr> Message-ID: <0262464D-5021-4B32-8633-BACCCD4EAD79@polytechnique.fr> > On 8 November 2011 14:45, Emmanuel Bacry wrote: > If the command is valid, everything goes well. If not (for instance command = "ytytyutuy") then the thread never terminates and nothing is printed on stderr > > I take it you've redirected stderr in the same way as stdout? Do you have it set to automatically go to the debugger (call %pdb to toggle it on and off)? If you start your application from a terminal, do you see anything appear in the terminal? > > Thomas Well, no I was careful not to redirect stdout nor stderr. I am not using a debugger I do start the application from the terminal, when the command is correct, the output goes to the terminal, if it is not, the thead does not end -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Nov 8 14:59:28 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 8 Nov 2011 19:59:28 +0000 Subject: [IPython-dev] wxPython console In-Reply-To: <0262464D-5021-4B32-8633-BACCCD4EAD79@polytechnique.fr> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <32772900.post@talk.nabble.com> <312EE7C9-3148-43A7-9D00-F303A6312F92@polytechnique.fr> <0262464D-5021-4B32-8633-BACCCD4EAD79@polytechnique.fr> Message-ID: On 8 November 2011 19:52, Emmanuel Bacry wrote: > Well, no I was careful not to redirect stdout nor stderr. > I am not using a debugger > I do start the application from the terminal, when the command is correct, > the output goes to the terminal, if it is not, the thead does not end > I'm not sure, then. This comment in showtraceback() might be helpful, but I'm not sure: # WARNING: these variables are somewhat deprecated and not # necessarily safe to use in a threaded environment, but tools # like pdb depend on their existence, so let's set them. If we # find problems in the field, we'll need to revisit their use. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Nov 9 01:46:57 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Nov 2011 22:46:57 -0800 Subject: [IPython-dev] wxPython console In-Reply-To: <5B1682D4-6B0C-41F4-B7AE-972BB46F6513@polytechnique.fr> References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> <5B1682D4-6B0C-41F4-B7AE-972BB46F6513@polytechnique.fr> Message-ID: On Tue, Nov 8, 2011 at 10:33 PM, Emmanuel Bacry wrote: > > from the qt-console, can you interrupt "properly" the execution of a "user code" ? If yes, what is the mechanism ? > > (if a long computation is ran from the qt-console.... and if it is just too long for the user...) Sure, Control-. (period) sends (over zmq) a KeyboardInterrupt to the kernel like Ctrl-C would in normal, local IPython. Cheers, f From fperez.net at gmail.com Wed Nov 9 01:47:39 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Nov 2011 22:47:39 -0800 Subject: [IPython-dev] Fwd: wxPython console In-Reply-To: References: <32771432.post@talk.nabble.com> <32772284.post@talk.nabble.com> Message-ID: Forgot to cc the list... ---------- Forwarded message ---------- From: Fernando Perez Date: Tue, Nov 8, 2011 at 5:26 PM Subject: Re: [IPython-dev] wxPython console To: Emmanuel Bacry On Tue, Nov 8, 2011 at 1:37 PM, Emmanuel Bacry wrote: > 2 processes are running right ? One with the wxmainloop and the kernel and one with the qtconsole which dialog with the kernel of the first process. > I don't quite understand how the wx mainloop and the kernel coexist in the same kernel. They are in the same thread right ? So when the kernel is running the main loop is not looping any more (idle events are not executed) right ? (it is not clear where this is in the code you pointed to) > Then how can you interrupt properly the execution of ?a command run by the kernel ? > Yes, when the IPython kernel is *executing user code* the wx event loop is blocked. ?But otherwise the kernel is just sitting there, waiting for events to arrive over the network. ?Simply try running the example ./ipkernel_wxapp.py in the docs/examples/lib directory and play with it. ?A little bit of interactive experimentation should clarify things. ?Let us know if that's not enough. Cheers, f From fperez.net at gmail.com Thu Nov 10 19:13:42 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Nov 2011 16:13:42 -0800 Subject: [IPython-dev] Heads-up for notebook users: on-disk format changes for version-control friendliness Message-ID: Hi all, in light of the recent discussion [1] on using notebooks in version-controlled environments, we've made three changes to the on-disk format that should help a lot: 1. The most important one: text fields like cell inputs or printed outputs that were multi-line are now saved as separate lines in the JSON file. This means that when you change one line in a 10 line cell, the diff will now show only that line changed, instead of a change in the entire cell (which before, was stored as a big string with embedded \n characters). 2. Fields are always sorted now, so no more spurious diffs due to field reordering. 3. Each json level only indents one space instead of 4, which makes the files actually much more readable (json introduces new indentation levels a lot, so 4 spaces per level was pushing the actual content far to the right of the file). These changes should help a lot anyone sharing notebooks with colleagues over git or other version control system, but it means that if you update to master, you should also tell your colleagues to update. The changes are applied transparently when you first save your notebooks, but if your colleague doesn't have the updated version, he or she won't be able to read the files anymore. Sorry for the hassle this may introduce, but we think the benefits are significant enough to warrant this. And that's why they call it the bleeding edge, we still haven't officially released :) Anyone interested in the details can see the pull request [2] where we discussed the changes. Cheers, f [1] http://mail.scipy.org/pipermail/ipython-dev/2011-November/008351.html [2] https://github.com/ipython/ipython/pull/981 From fperez.net at gmail.com Thu Nov 10 23:23:22 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Nov 2011 20:23:22 -0800 Subject: [IPython-dev] Heads-up for notebook users: on-disk format changes for version-control friendliness In-Reply-To: References: Message-ID: Hey Satra, On Thu, Nov 10, 2011 at 8:17 PM, Satrajit Ghosh wrote: > i'm sure this will be very easy for you to add, but could you put a future > version check into the code. something along the lines of: > if file.nbformat > prog.nbformat: > ? ? Exception('notebook is saved in a newer format than currently supported > by this version of ipython. notebook format requires version >= X.Y') > cheers, We already have that in place, and once we release we'll be much stricter with version number checks. But we didn't actually bump the version number with this change (perhaps we should have, I hope it won't be too much of a hassle since right now only people running off git can use it, and it's a 'git pull' away). Currently the version string is at v2, and when we make another round of larger changes post 0.12 for features we want in that case, we'll certainly update that. Cheers, f From satra at mit.edu Thu Nov 10 23:30:07 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Thu, 10 Nov 2011 23:30:07 -0500 Subject: [IPython-dev] Heads-up for notebook users: on-disk format changes for version-control friendliness In-Reply-To: References: Message-ID: hey fernando, > We already have that in place, and once we release we'll be much stricter > with version number checks. > awesome! > But we didn't actually bump the version number with this change (perhaps > we should have, I hope it won't be too much of a hassle since right now > only people running off git can use it, and it's a 'git pull' away). > i'm ok with this, most of my collaborators currently are using the latest version anyway! (but from a principled perspective, it should change :) ) cheers, satra ps. i'll get to making the feature requests perhaps on my way to dc tomorrow. sorry i have been way beyond busy. i need to change my life! -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 11 00:32:52 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Nov 2011 21:32:52 -0800 Subject: [IPython-dev] Heads-up for notebook users: on-disk format changes for version-control friendliness In-Reply-To: References: Message-ID: On Thu, Nov 10, 2011 at 8:30 PM, Satrajit Ghosh wrote: > ps.?i'll get to making the feature requests perhaps on my way to dc > tomorrow. sorry i have been way beyond busy. i need to change my life! No worries :) Have fun at SfN! Cheers, f From benjaminrk at gmail.com Fri Nov 11 00:25:03 2011 From: benjaminrk at gmail.com (MinRK) Date: Thu, 10 Nov 2011 21:25:03 -0800 Subject: [IPython-dev] Heads-up for notebook users: on-disk format changes for version-control friendliness In-Reply-To: References: Message-ID: On Thu, Nov 10, 2011 at 20:30, Satrajit Ghosh wrote: > hey fernando, > > >> We already have that in place, and once we release we'll be much stricter >> with version number checks. >> > > awesome! > > >> But we didn't actually bump the version number with this change (perhaps >> we should have, I hope it won't be too much of a hassle since right now >> only people running off git can use it, and it's a 'git pull' away). >> > > i'm ok with this, most of my collaborators currently are using the latest > version anyway! (but from a principled perspective, it should change :) ) > I didn't bump the version for two reasons: 1. it doesn't actually change the notebook format. Every v2 notebook written to JSON prior to this change remains perfectly valid, and will continue to be so forever. The only change is that notebooks *can* ( and do when you save them via the web interface) split multi-line text into a list of lines when writing to JSON. 2. The code has not been released, and tiny changes like this should be acceptable, so that we don't proliferate a dozen notebook formats per release (we already have two, and zero releases). This does mean that users collaborating on notebooks right now have to coordinate their next pull, but that doesn't seem unreasonable when everyone affected is necessarily already running from git. But perhaps I underestimate the popularity of the notebook for collaborating at this point. We will certainly never make a change even this minuscule to a notebook format that we have actually released, and we will probably get started on v3 shortly after 0.12 is cut. -MinRK > > cheers, > > satra > > ps. i'll get to making the feature requests perhaps on my way to dc > tomorrow. sorry i have been way beyond busy. i need to change my life! > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Nov 11 14:19:31 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 11 Nov 2011 11:19:31 -0800 Subject: [IPython-dev] GitHub Issues sphinx extension Message-ID: As part of getting started on What's New for 0.12, I tweaked this bitbucket sphinx extension to point to GitHub. So, if you are referencing an IPython Issue or PR in the docs, you can do so with: :ghissue:`123` or :ghpull:`321` Rather than any need to make individual links. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 11 14:29:31 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 11 Nov 2011 11:29:31 -0800 Subject: [IPython-dev] GitHub Issues sphinx extension In-Reply-To: References: Message-ID: On Fri, Nov 11, 2011 at 11:19 AM, MinRK wrote: > As part of getting started on What's New for 0.12, I tweaked?this bitbucket > sphinx extension?to point to GitHub. [...] This is great Min, thanks a lot! Cheers, f From fperez.net at gmail.com Tue Nov 15 15:42:30 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 Nov 2011 12:42:30 -0800 Subject: [IPython-dev] Issue #1000 is filed, Thomas gets two SciPy t-shirts :) Message-ID: Hi folks, We've just been on github for a little over a year, and we already have crossed the 1000 issues threshold! https://github.com/ipython/ipython/issues/1000 Thomas filed this yesterday, and that makes him the proud winner of a couple of t-shirts :) We unfortunately don't have nice IPython-specific schwag, but Jarrod had some T-shirts from the US Scipy 2009 conference and the Scipy India 2010 one left, and kindly donated them to me, so I'm using that. It's clear that the smooth, low-friction workflow provided by github has made us a lot more productive, since despite having over 1000 issues filed, we've managed to hover with O(200) open ones and O(10) open PRs. This means it's not just a pileup of open bugs but a live pipeline where a lot of things do get closed. Ideally our open issues number would be closer to ~50 (it's unrealistic to expect it to drop to 0), but as long as it doesn't spin out of control, we're OK. Cheers, f From takowl at gmail.com Tue Nov 15 16:51:52 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 Nov 2011 21:51:52 +0000 Subject: [IPython-dev] Issue #1000 is filed, Thomas gets two SciPy t-shirts :) In-Reply-To: References: Message-ID: On 15 November 2011 20:42, Fernando Perez wrote: > Thomas filed this yesterday, and that makes him the proud winner of a > couple of t-shirts :) We unfortunately don't have nice > IPython-specific schwag, but Jarrod had some T-shirts from the US > Scipy 2009 conference and the Scipy India 2010 one left, and kindly > donated them to me, so I'm using that. > Ha, brilliant, I honestly wasn't expecting anything like that. I'll take the opportunity to say thanks to everyone working on IPython - both for building an excellent tool, and for being supportive as I got involved over about the last year. I'm looking forward to seeing how the 0.12 release goes down. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.colistete at gmail.com Wed Nov 16 07:38:26 2011 From: roberto.colistete at gmail.com (Roberto Colistete Jr.) Date: Wed, 16 Nov 2011 10:38:26 -0200 Subject: [IPython-dev] IPython for Mobile OS In-Reply-To: References: Message-ID: <4EC3AEC2.3070807@gmail.com> Hi, It is my first post to ipython-dev. I am a theoretical physicist interested a lot in Python/IPython/SymPy/NumPy/MatPlotLib/etc. Both for desktop OS and mobile OS. About mobile OS (for smartphones/tablets), IPython is packaged, easy-to-install and works well in : - Maemo 4 (Diablo) OS, e.g. on Nokia N800/N810, from 2008. Install IPython using the application manager. IPython 0.8.1 : http://maemo.org/packages/view/ipython/ - Maemo 5 (Fremantle) OS on Nokia N900, released in 11/2009 and currently difficult to buy brand new. Install IPython using the application manager. IPython for Maemo 5 has 95 thousand downloads since 2010, last version is 0.10 : http://maemo.org/packages/view/ipython/ http://maemo.dadablog.net/AppStats.php?package=ipython&os=fremantle - MeeGo 1.2 Harmattan OS on Nokia N9/N950, released in 2011, selling since September/October. IPython 0.10.2 .deb was recently released by me (as a maintainer) : http://forum.meego.com/showthread.php?p=35031 http://talk.maemo.org/showthread.php?t=79997 I will soon package and submit IPython 0.10.2 for Maemo 4 & 5. IPython 0.11 is more difficult to package for Maemo/MeeGo due to its additional dependencies. I have searched and found that IPython for Android OS doesn't work (there is no libreadline). For iOS there is IPython only jail-breaking it (Cydia) and following some manual installation steps. No IPython for Symbian OS. Am I right ? So it seems that the only smartphone selling today with IPython support is Nokia N9 (MeeGo 1.2 Harmattan OS). The repositories for Nokia N9 : http://harmattan-dev.nokia.com/unstable/beta3/Fremantle_Update7_vs_Harmattan_Beta3_content_comparison.html shows about 170 Python packages ! MeeGo Harmattan is also a developer's paradise, with more than 10 programming languages available now (via "apt-get install" or already installed) : gcc/g++ (3.4, 4.2, 4.4), gfortran 4.4, gpc (GNU Pascal 2.2), Lua 5.1, Perl 5.8, Prolog, Python 2.5/2.6/3.1, Ruby 1.8, TCL 8.4/8.5, Vala 0.12, etc. Also Qt/C++, Qt/Qt Quick, Qt/Python (PySide). I have also packaged SymPy (a Computer Algebra System, which uses IPython) for Maemo 4, Maemo 5 and MeeGo 1.2 Harmattan : http://forum.meego.com/showthread.php?t=4870 http://talk.maemo.org/showthread.php?p=1119313 SymPy interactive shell has almost 15 thousand downloads for Maemo 5 in 1 month and 1 week : http://maemo.dadablog.net/AppStats.php?package=python-sympy-ui&os=fremantle So we can see that there is interest in Python/IPython for smartphones/tablets. It would be nice to have complete Python (with IPython, etc) distribution in powerful 2011/2012 dual and quad core tablets and smartphones (using Android OS, etc), like the Asus Transformer (Prime). Best regards from Brazil, Roberto From takowl at gmail.com Wed Nov 16 07:52:18 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 Nov 2011 12:52:18 +0000 Subject: [IPython-dev] IPython for Mobile OS In-Reply-To: <4EC3AEC2.3070807@gmail.com> References: <4EC3AEC2.3070807@gmail.com> Message-ID: On 16 November 2011 12:38, Roberto Colistete Jr. < roberto.colistete at gmail.com> wrote: > About mobile OS (for smartphones/tablets), IPython is packaged, > easy-to-install and works well in : [snip] > That's awesome - thanks, Roberto. > I will soon package and submit IPython 0.10.2 for Maemo 4 & 5. IPython > 0.11 is more difficult to package for Maemo/MeeGo due to its additional > dependencies. > The basic terminal shell shouldn't have any more dependencies, as far as I'm aware. You only need ZMQ etc. if you want to use the Qt console or the parallel computing stuff. If it is feasible to package 0.11, that would be great - it's a significant upgrade from 0.10.x, and I don't think anyone's terribly interested in supporting the old code. We're also hoping to release 0.12 soon, which will further improve on 0.11. > I have searched and found that IPython for Android OS doesn't work > (there is no libreadline). For iOS there is IPython only jail-breaking > it (Cydia) and following some manual installation steps. No IPython for > Symbian OS. Am I right ? > It might be possible to use pyrepl (http://codespeak.net/pyrepl/) to provide a readline-like interface on android, depending on what the terminal there is like. Another possibility, if mobile browsers are sufficiently advanced, is to access IPython in the browser. Pythonanywhere embeds a terminal in a web page: http://www.pythonanywhere.com/try-ipython/ (down at the moment), or there's an in-browser notebook interface coming with 0.12. This is probably the only way round the restrictions for iOS platforms. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From roberto.colistete at gmail.com Wed Nov 16 08:14:08 2011 From: roberto.colistete at gmail.com (Roberto Colistete Jr.) Date: Wed, 16 Nov 2011 11:14:08 -0200 Subject: [IPython-dev] IPython for Mobile OS In-Reply-To: References: <4EC3AEC2.3070807@gmail.com> Message-ID: <4EC3B720.3040006@gmail.com> Em 16-11-2011 10:52, Thomas Kluyver escreveu: > > I will soon package and submit IPython 0.10.2 for Maemo 4 & 5. IPython > 0.11 is more difficult to package for Maemo/MeeGo due to its > additional > dependencies. > > > The basic terminal shell shouldn't have any more dependencies, as far > as I'm aware. You only need ZMQ etc. if you want to use the Qt console > or the parallel computing stuff. If it is feasible to package 0.11, > that would be great - it's a significant upgrade from 0.10.x, and I > don't think anyone's terribly interested in supporting the old code. > We're also hoping to release 0.12 soon, which will further improve on > 0.11. I have actually suffered to install IPython 0.11 on Ubuntu 10.04 64 bits due to the ZMQ dependency. But I will later give IPython 0.11 a try on MeeGo Harmattan, thanks for the suggestion. But Maemo 4/5 has Python 2.5 and IPython 0.11 needs at least Python 2.6. > > I have searched and found that IPython for Android OS doesn't work > (there is no libreadline). For iOS there is IPython only jail-breaking > it (Cydia) and following some manual installation steps. No > IPython for > Symbian OS. Am I right ? > > > It might be possible to use pyrepl (http://codespeak.net/pyrepl/) to > provide a readline-like interface on android, depending on what the > terminal there is like. Thanks for pointing this possible solution. > Another possibility, if mobile browsers are sufficiently advanced, is > to access IPython in the browser. Pythonanywhere embeds a terminal in > a web page: http://www.pythonanywhere.com/try-ipython/ (down at the > moment), or there's an in-browser notebook interface coming with 0.12. > This is probably the only way round the restrictions for iOS platforms. Very interesting this in-browser notebook interface ! Thanks, Roberto -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 16 08:58:27 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 Nov 2011 13:58:27 +0000 Subject: [IPython-dev] IPython for Mobile OS In-Reply-To: <4EC3B720.3040006@gmail.com> References: <4EC3AEC2.3070807@gmail.com> <4EC3B720.3040006@gmail.com> Message-ID: On 16 November 2011 13:14, Roberto Colistete Jr. < roberto.colistete at gmail.com> wrote: > I have actually suffered to install IPython 0.11 on Ubuntu 10.04 64 > bits due to the ZMQ dependency. But I will later give IPython 0.11 a try on > MeeGo Harmattan, thanks for the suggestion. But Maemo 4/5 has Python 2.5 > and IPython 0.11 needs at least Python 2.6. > Ah, yes, there's no easy way round the python version. Julian (CCed) has packaged IPython 0.11 for Debian/Ubuntu, and split the binary packages up so that ipython-core doesn't depend on ZMQ. I don't know how similar the process is (I see it still uses deb files and apt), but he may be able to help. > Very interesting this in-browser notebook interface ! > It is! We should stick a couple of screenshots in the docs, but for the moment there are some here: http://lighthouseinthesky.blogspot.com/2011/09/review-ipython-notebooks.html Feel free to grab the development version from github and play around with it - besides ZMQ, you need Tornado and a recent-ish version of Firefox or Chrome. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Nov 18 23:23:00 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 18 Nov 2011 20:23:00 -0800 Subject: [IPython-dev] Heads-up: no more plaintext passwords in the notebook Message-ID: Hi all, for those of you running a publicly-facing notebook server with a password, we've just merged this: https://github.com/ipython/ipython/pull/1011 Once you pull from master, you'll need to update the password in the config file, as plaintext passwords are simply not supported anymore. You can create the salted password in a python session with: from IPython.lib import passwd passwd('mypassphrase') Do NOT do this in an ipython session, as it will log the plaintext to your history!!! You can call passwd() without without arguments and it will prompt you for a password using the getpass module so it's not echoed back onto your screen. This will ensure that the plaintext password is never visible nor logged. In ipython: In [1]: from IPython.lib.security import passwd; passwd() Enter password: Verify password: Out[1]: 'sha1:e0209ae16e4e:b9a5d128ff961159b3bd66f7314d33305d60b5b0' or from the shell: longs[graphs]> python -c "from IPython.lib import passwd;print passwd()" Enter password: Verify password: sha1:3d4903d15c38:1f68a38d107ee14f6196de9c0ca8f8110662c9d3 You can then replace your current plaintext passwords with this whole string. Cheers, f From fperez.net at gmail.com Sun Nov 20 20:18:54 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 20 Nov 2011 17:18:54 -0800 Subject: [IPython-dev] Small heads-up: renamed priority 'critical' to 'blocker' in issue tracker Message-ID: we've been using the 'critical' label really to mean 'release blocker' so I changed the name to that, might as well. As always, we may in the end be forced to punt on a blocker and move it to the next milestone, but we'll make every reasonable effort to cleanly close them in the milestone they've been tagged for. Cheers, f From stefan at sun.ac.za Thu Nov 24 00:09:14 2011 From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=) Date: Wed, 23 Nov 2011 21:09:14 -0800 Subject: [IPython-dev] Participants view Message-ID: Hi all, I've implemented a proof-of-concept for an "active users" or participants viewer a-la-Google Docs: https://github.com/stefanv/ipython/compare/ipython:master...notebook_viewers When running this branch, you should see a side panel that displays a list of all users that connected to the notebook. It is still very basic, but now that the infrastructure is in place, we can come up with ideas as to exactly what should be displayed and how. Let me know! Cheers St?fan From fperez.net at gmail.com Thu Nov 24 12:19:06 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 24 Nov 2011 09:19:06 -0800 Subject: [IPython-dev] Participants view In-Reply-To: References: Message-ID: 2011/11/23 St?fan van der Walt : > When running this branch, you should see a side panel that displays a > list of all users that connected to the notebook. ?It is still very > basic, but now that the infrastructure is in place, we can come up > with ideas as to exactly what should be displayed and how. ?Let me > know! Can you post a screenshot? It will help people see what you have in mind without having to set up the branch, authentication, etc by themselves... Cheers, f From satra at mit.edu Sat Nov 26 15:58:53 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 15:58:53 -0500 Subject: [IPython-dev] quick question notebook in pylab mode Message-ID: when the figure is embedded in a notebook, it looks like that's done as a bitmap. how hard or difficult would it be to use the svg output of matplotlib and embed that? cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 16:10:55 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 13:10:55 -0800 Subject: [IPython-dev] quick question notebook in pylab mode In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 12:58 PM, Satrajit Ghosh wrote: > when the figure is embedded in a notebook, it looks like that's done as a > bitmap. how hard or difficult would it be to use the svg output of > matplotlib and embed that? Your wish is our command, Dr. Ghosh :) %config InlineBackend.figure_format = 'svg' will do it at runtime, and you can set the same value permanently in your config file with c.InlineBackend.figure_format = 'svg' we did a bunch of testing, and found that despite of the benefits of svg in many cases, there are enough others where the matplotlib-produced SVG is sub-par in quality (esp. when making figures with lots of transparent overlapping elements, arrows and text). So we decided to keep png as the default format as it was more predictable and uniform in quality. Cheers, f From satra at mit.edu Sat Nov 26 16:18:39 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 16:18:39 -0500 Subject: [IPython-dev] quick question notebook in pylab mode In-Reply-To: References: Message-ID: hi fernando, awesome. thanks! %config InlineBackend.figure_format = 'svg' > is this new? because when i put it in the first cell of my notebook it says magic config not found. cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 16:19:24 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 13:19:24 -0800 Subject: [IPython-dev] quick question notebook in pylab mode In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 1:18 PM, Satrajit Ghosh wrote: > is this new? because when i put it in the first cell of my notebook it says > magic config not found. Relatively new, just git pull. Cheers, f From satra at mit.edu Sat Nov 26 16:26:18 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 16:26:18 -0500 Subject: [IPython-dev] quick question notebook in pylab mode In-Reply-To: References: Message-ID: thanks! looks beautiful. cheers, satra On Sat, Nov 26, 2011 at 4:19 PM, Fernando Perez wrote: > On Sat, Nov 26, 2011 at 1:18 PM, Satrajit Ghosh wrote: > > is this new? because when i put it in the first cell of my notebook it > says > > magic config not found. > > Relatively new, just git pull. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 16:35:23 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 13:35:23 -0800 Subject: [IPython-dev] quick question notebook in pylab mode In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 1:26 PM, Satrajit Ghosh wrote: > thanks! looks beautiful. We strive to please :) Take care, f From satra at mit.edu Sat Nov 26 16:40:00 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 16:40:00 -0500 Subject: [IPython-dev] next question: printing Message-ID: if i have a cell that generates multiple figures (i see them in the notebook output) and in the window that print opens. but if i print to file or printer, only the figures that fit on that page with the cell gets printed. i.e. a pdf generated from the notebook doesn't reflect all the output content of the notebook. i think you can replicate it by doing the following in a single cell: figure();plot(range(1)) figure();plot(range(2)) figure();plot(range(3)) figure();plot(range(4)) and then trying to print that to a pdf using the notebook print button and then ctrl+p. (i'm using firefox on ubuntu). cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 16:51:46 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 13:51:46 -0800 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 1:40 PM, Satrajit Ghosh wrote: > > and then trying to print that to a pdf using the notebook print button and > then ctrl+p. (i'm using firefox on ubuntu). It's a browser issue: chrome handles it fine, ffox doesn't, unfortunately. Ultimately the right way to produce clean, high-quality pdf output will need to be via sphinx, but that's still a ways out. In the meantime, trying what different browsers do is the only solution we have; we're at the whim of their rendering engines. Cheers, f From satra at mit.edu Sat Nov 26 17:14:06 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 17:14:06 -0500 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: thanks fernando. cheers, satra On Sat, Nov 26, 2011 at 4:51 PM, Fernando Perez wrote: > On Sat, Nov 26, 2011 at 1:40 PM, Satrajit Ghosh wrote: > > > > and then trying to print that to a pdf using the notebook print button > and > > then ctrl+p. (i'm using firefox on ubuntu). > > It's a browser issue: chrome handles it fine, ffox doesn't, unfortunately. > > Ultimately the right way to produce clean, high-quality pdf output > will need to be via sphinx, but that's still a ways out. In the > meantime, trying what different browsers do is the only solution we > have; we're at the whim of their rendering engines. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 20:02:39 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 17:02:39 -0800 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 4:48 PM, Thomas Kluyver wrote: > > I'm a bit surprised by this - I wonder if there's some tweak to the HTML we > could make so that the contents of a cell can span more than one page. No > idea where to start looking, though. Replying back on-list... I don't either, but obviously if someone knows the intricacies of rendering in firefox and has an idea, we'd be more than happy to improve things on this front. Cheers, f From satra at mit.edu Sat Nov 26 23:04:36 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 23:04:36 -0500 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: just to add some more info. this is where life becomes fun. the svg inline option doesn't work with printing in chromium. summary: - svg works fine in firefox, but print is incomplete - svg in chromium: the notebook output looks good, but when you click on the print button, the svg output is all messed up cheers, satra On Sat, Nov 26, 2011 at 5:14 PM, Satrajit Ghosh wrote: > thanks fernando. > > cheers, > > satra > > > > On Sat, Nov 26, 2011 at 4:51 PM, Fernando Perez wrote: > >> On Sat, Nov 26, 2011 at 1:40 PM, Satrajit Ghosh wrote: >> > >> > and then trying to print that to a pdf using the notebook print button >> and >> > then ctrl+p. (i'm using firefox on ubuntu). >> >> It's a browser issue: chrome handles it fine, ffox doesn't, unfortunately. >> >> Ultimately the right way to produce clean, high-quality pdf output >> will need to be via sphinx, but that's still a ways out. In the >> meantime, trying what different browsers do is the only solution we >> have; we're at the whim of their rendering engines. >> >> Cheers, >> >> f >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Nov 26 23:24:01 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 20:24:01 -0800 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 8:04 PM, Satrajit Ghosh wrote: > just to add some more info. this is where life becomes fun. the svg inline > option doesn't work with printing in chromium. > summary: > - svg works fine in firefox, but print is incomplete > - svg in chromium: ?the notebook output looks good, but when you click on > the print button, the svg output is all messed up > Ouch... I really will be much happier when we manage to get real sphinx support in place... Anyone interested, don't hesitate to jump in! There's already a first stab on a pull request at: https://github.com/ipython/ipython/pull/873 and a fairly detailed discussion: https://github.com/ipython/ipython/issues/888 Cheers, f From fperez.net at gmail.com Sat Nov 26 23:42:52 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 20:42:52 -0800 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 8:04 PM, Satrajit Ghosh wrote: > - svg works fine in firefox, but print is incomplete > - svg in chromium: ?the notebook output looks good, but when you click on > the print button, the svg output is all messed up BTW, I'm not seeing that problem in chrome... I tried both chrome and chromium, and while they don't print identically (headers, fonts, cell outlines are all different), they both handled fine the svg of a simple plot(rand(100)). Perhaps the issue was with more complex svg? If you send me a notebook with the problem I can test it here... Cheers, f From satra at mit.edu Sat Nov 26 23:51:11 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sat, 26 Nov 2011 23:51:11 -0500 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: hey fernando, try this one. https://github.com/satra/sad/blob/master/sad_figures.ipynb cheers, satra On Sat, Nov 26, 2011 at 11:42 PM, Fernando Perez wrote: > On Sat, Nov 26, 2011 at 8:04 PM, Satrajit Ghosh wrote: > > - svg works fine in firefox, but print is incomplete > > - svg in chromium: the notebook output looks good, but when you click on > > the print button, the svg output is all messed up > > BTW, I'm not seeing that problem in chrome... I tried both chrome and > chromium, and while they don't print identically (headers, fonts, cell > outlines are all different), they both handled fine the svg of a > simple plot(rand(100)). Perhaps the issue was with more complex svg? > If you send me a notebook with the problem I can test it here... > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun Nov 27 00:02:37 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Nov 2011 21:02:37 -0800 Subject: [IPython-dev] next question: printing In-Reply-To: References: Message-ID: On Sat, Nov 26, 2011 at 8:51 PM, Satrajit Ghosh wrote: > hey fernando, try this one. > https://github.com/satra/sad/blob/master/sad_figures.ipynb OK, I see the problem too: it is indeed an issue with more complex figures, that's a bummer. Given that ffox handles them fine, the fault is probably with Chrome in this case. Well, now we know :) Cheers, f From fperez.net at gmail.com Wed Nov 30 02:07:52 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 29 Nov 2011 23:07:52 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? Message-ID: Hi folks, over the years, we've had a number of bug reports that ultimately boil down to ipython doing an extra getattr call on objects at the prompt, such as https://github.com/ipython/ipython/issues/988 https://github.com/ipython/ipython/issues/851 While this may appear odd, people do have code out there that mutates the state of objects on simple getattr calls, and so *any* introspection on such objects, no matter how carefully done, mutates them. The last example comes from Traits code, which is famous for having complex semantics attached to attribute access. Since this is a recurring problem, I'm trying to decide whether the convenience of autocall being on by default is worth against the occasional surprise of situations like these. Obviously we'd leave the functionality of autocall as-is, and people could still enable it if they wanted to. I'm only talking about whether it should be completely off by default. The reason why defaults matter a lot is that the vast majority of users never ever change them (I've seen countless long-term users with the wrong colors for a light bacgkround, the one default we can't pick for them and that leads to unreadable tracebacks, yet people live with it for years). So it's up to us to give people defaults that lead to the best experience possible. I personally had always fallen on the side of saying that autocall is a big convenience of ipython, and that code that mutates on simple attribute access is special enough that we shouldn't sacrifice that convenience for most users to accommodate such a special case. But I'm not sure anymore if that's the right balance, given the recurring nature of these reports (the above are just the two most recent ones, I've answered this question in one form or another probably 20 times or more over the years). So I'd like to hear some feedback, as we're getting ready to put 0.12 out the door soon... Cheers, f From fperez.net at gmail.com Wed Nov 30 03:07:14 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 00:07:14 -0800 Subject: [IPython-dev] IPython 0.12 beta ready for testing Message-ID: Hi everyone, we're getting close to having the code in a state we can release IPython 0.12, so at this point we'd greatly appreciate it if the more adventurous amongst you could take it for a spin. You're always welcome to run from git master (we do our best to keep master always in fully working condition) and github also offers tar/zip downloads of master at any time from: https://github.com/ipython/ipython/tarball/master https://github.com/ipython/ipython/zipball/master But we've also put here for your convenience an egg, a windows binary installer, and tar/zip downloads with the compiled documentation, in case you prefer to have it for local use: http://archive.ipython.org/testing/0.12.beta/ Please report any problems you encounter, of any kind, as we'd like to make 0.12 as solid a release as possible. Assuming nothing major turns up, we hope to put out a release candidate in a week or two with whatever small problems get reported fixed, and a few pull requests and issues we're still working on. At that point, the final 0.12 should follow in short order. Thanks in advance for any feedback you can provide us! Best, f From benjaminrk at gmail.com Wed Nov 30 03:23:59 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 00:23:59 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: I don't find myself using autocall much, but I will probably use it a lot more if I ever switch to Python 3 (maybe 2015?), if just for helping me ease out of my `print foo` habit. If it causes problems, I certainly wouldn't mind it being off by default. -MinRK On Tue, Nov 29, 2011 at 23:07, Fernando Perez wrote: > Hi folks, > > over the years, we've had a number of bug reports that ultimately boil > down to ipython doing an extra getattr call on objects at the prompt, > such as > > https://github.com/ipython/ipython/issues/988 > https://github.com/ipython/ipython/issues/851 > > While this may appear odd, people do have code out there that mutates > the state of objects on simple getattr calls, and so *any* > introspection on such objects, no matter how carefully done, mutates > them. The last example comes from Traits code, which is famous for > having complex semantics attached to attribute access. > > Since this is a recurring problem, I'm trying to decide whether the > convenience of autocall being on by default is worth against the > occasional surprise of situations like these. > > Obviously we'd leave the functionality of autocall as-is, and people > could still enable it if they wanted to. I'm only talking about > whether it should be completely off by default. The reason why > defaults matter a lot is that the vast majority of users never ever > change them (I've seen countless long-term users with the wrong colors > for a light bacgkround, the one default we can't pick for them and > that leads to unreadable tracebacks, yet people live with it for > years). So it's up to us to give people defaults that lead to the > best experience possible. > > I personally had always fallen on the side of saying that autocall is > a big convenience of ipython, and that code that mutates on simple > attribute access is special enough that we shouldn't sacrifice that > convenience for most users to accommodate such a special case. > > But I'm not sure anymore if that's the right balance, given the > recurring nature of these reports (the above are just the two most > recent ones, I've answered this question in one form or another > probably 20 times or more over the years). > > So I'd like to hear some feedback, as we're getting ready to put 0.12 > out the door soon... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 06:05:24 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 11:05:24 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: On 30 November 2011 07:07, Fernando Perez wrote: > I personally had always fallen on the side of saying that autocall is > a big convenience of ipython, and that code that mutates on simple > attribute access is special enough that we shouldn't sacrifice that > convenience for most users to accommodate such a special case. > I would still stick with this position. I often use autocall for things like "len x", and code that changes state on attribute access is doing something decidedly unusual. It should definitely be easy to switch to paranoid don't-touch-anything mode, but I don't think it's a problem often enough to be worth changing the default behaviour. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Wed Nov 30 06:18:52 2011 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Nov 2011 11:18:52 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: On 11/30/11 7:07 AM, Fernando Perez wrote: > Hi folks, > > over the years, we've had a number of bug reports that ultimately boil > down to ipython doing an extra getattr call on objects at the prompt, > such as > > https://github.com/ipython/ipython/issues/988 > https://github.com/ipython/ipython/issues/851 > > While this may appear odd, people do have code out there that mutates > the state of objects on simple getattr calls, and so *any* > introspection on such objects, no matter how carefully done, mutates > them. The last example comes from Traits code, which is famous for > having complex semantics attached to attribute access. > > Since this is a recurring problem, I'm trying to decide whether the > convenience of autocall being on by default is worth against the > occasional surprise of situations like these. > > Obviously we'd leave the functionality of autocall as-is, and people > could still enable it if they wanted to. I'm only talking about > whether it should be completely off by default. The reason why > defaults matter a lot is that the vast majority of users never ever > change them (I've seen countless long-term users with the wrong colors > for a light bacgkround, the one default we can't pick for them and > that leads to unreadable tracebacks, yet people live with it for > years). So it's up to us to give people defaults that lead to the > best experience possible. > > I personally had always fallen on the side of saying that autocall is > a big convenience of ipython, and that code that mutates on simple > attribute access is special enough that we shouldn't sacrifice that > convenience for most users to accommodate such a special case. I don't think it's *that* special of a case. There are all kinds of good reasons to set up a special case for the first time an attribute is looked up while mutating the object to let future lookups go through a different path. *Usually*, getting the attribute in the autocall code is innocuous, but sometimes it isn't. I've always been in favor of turning autocall off by default, though I can't really say that I have a strong argument beyond personal preference. I don't like to keep two parallel Python syntaxes in my head (and muscle memory!), especially when one of them is only informally specified and only works in one specific environment. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From kevin.buchs at gmail.com Wed Nov 30 08:09:55 2011 From: kevin.buchs at gmail.com (Kevin Buchs) Date: Wed, 30 Nov 2011 07:09:55 -0600 Subject: [IPython-dev] dictionaries and %run oddity Message-ID: I found a strange behavior with regard to global dictionaries and the %run command. Perhaps someone can explain this to me. Here is the scenario: 1. Create a file, x.py with these contents def a(): global b b={} b[1]=(2,3) 2. In a iPython shell execute these commands: %run x a() b which results in b being undefined at the shell level. 3. Edit x.py to contain this line at the end: a() 4. Now, in the shell, execute these commands: run x b which results in the correct value of b at the shell level. My understanding is the iPython runs %run commands in a new instance of python, but then the namespace of that instance is inherited by the iPython level namespace. With that, it would seem to matter little that the definition of a() is separated from the invocation. The unusual thing is that I found this behavior is not present if the global variable is an integer or list. So far, I have *only seen this with a dictionary.* - Kevin Buchs -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 08:25:20 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 13:25:20 +0000 Subject: [IPython-dev] dictionaries and %run oddity In-Reply-To: References: Message-ID: On 30 November 2011 13:09, Kevin Buchs wrote: > My understanding is the iPython runs %run commands in a new instance of > python, but then the namespace of that instance is inherited by the iPython > level namespace. With that, it would seem to matter little that the > definition of a() is separated from the invocation. > It's not a new instance of Python, but it is a new module, so it's a separate namespace. When the script finishes, the variables from that namespace are dumped in your interactive namespace, but that doesn't apply to variables created in its module namespace later on (which is what you're doing here). A simple workaround if you do want to see those variables is to use %run -i x.py to run it directly in the main namespace. > The unusual thing is that I found this behavior is not present if the > global variable is an integer or list. So far, I have *only seen this > with a dictionary.* > That is unusual. Can I ask you to double check this? I've tried with master and with 0.11, and I see the same behaviour (b is not in my namespace) with a dictionary, integer or list. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From kevin.buchs at gmail.com Wed Nov 30 08:56:15 2011 From: kevin.buchs at gmail.com (Kevin Buchs) Date: Wed, 30 Nov 2011 07:56:15 -0600 Subject: [IPython-dev] dictionaries and %run oddity In-Reply-To: References: Message-ID: Thomas, Thanks for your reply. What you observed about list, integer are what I see and I must retract my unusuality. I must have transposed something in my memory from the night before. There is something that is not making sense to me from your explanation. Here is my paraphrase of what I think you are saying. When I %run a command, it is given its own namespace, say N1. Any variables defined in that namespace are copied to the original namespace of my iPython session, say N0. The function definition is not copied, however. When I then invoke (not %run) the function from N0, it runs in N1. Then any variables created in N1 are NOT copies to N0. So, if I would %run the function definition and invocation in one command, then any variables defined are copied back. What does not make sense to me is if I declare a variable global in a function declared in a certain namespace, shouldn't that mean it is available in the global namespace? Kevin -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 09:28:49 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 14:28:49 +0000 Subject: [IPython-dev] dictionaries and %run oddity In-Reply-To: References: Message-ID: On 30 November 2011 13:56, Kevin Buchs wrote: > When I %run a command, it is given its own namespace, say N1. Any > variables defined in that namespace are copied to the original namespace of > my iPython session, say N0. The function definition is not copied, however. > The function object is copied (at least, a reference to it is) - it is a variable like any other. > When I then invoke (not %run) the function from N0, it runs in N1. Then > any variables created in N1 are NOT copies to N0. So, if I would %run the > function definition and invocation in one command, then any variables > defined are copied back. > Functions run in the scope where they were defined - even though your namespace has a reference to a, Python knows that a was defined in module x. If you do "from x import * " with your example, then do a(), it behaves the same way. Here it is defining an integer: In [1]: from x import * In [2]: a() In [3]: b --------------------------------------------------------------------------- NameError Traceback (most recent call last) /home/thomas/code/virtualenvs/ipy-trunk/ in () ----> 1 b NameError: name 'b' is not defined In [4]: import sys In [5]: sys.modules['x'].b Out[5]: 1000 > What does not make sense to me is if I declare a variable global in a > function declared in a certain namespace, shouldn't that mean it is > available in the global namespace? > 'global' is a bit misleading - in Python, it means global to that module. By design, there isn't a shared global namespace (well, you can modify builtins, but it's frowned upon). Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From jorgen.stenarson at bostream.nu Wed Nov 30 12:05:02 2011 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Wed, 30 Nov 2011 18:05:02 +0100 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: <4ED6623E.2020007@bostream.nu> Fernando Perez skrev 2011-11-30 08:07: > But I'm not sure anymore if that's the right balance, given the > recurring nature of these reports (the above are just the two most > recent ones, I've answered this question in one form or another > probably 20 times or more over the years). > > So I'd like to hear some feedback, as we're getting ready to put 0.12 > out the door soon... > I vote for disabling it by default. /J?rgen From warren.weckesser at enthought.com Wed Nov 30 12:50:50 2011 From: warren.weckesser at enthought.com (Warren Weckesser) Date: Wed, 30 Nov 2011 11:50:50 -0600 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 5:18 AM, Robert Kern wrote: > On 11/30/11 7:07 AM, Fernando Perez wrote: > > Hi folks, > > > > over the years, we've had a number of bug reports that ultimately boil > > down to ipython doing an extra getattr call on objects at the prompt, > > such as > > > > https://github.com/ipython/ipython/issues/988 > > https://github.com/ipython/ipython/issues/851 > > > > While this may appear odd, people do have code out there that mutates > > the state of objects on simple getattr calls, and so *any* > > introspection on such objects, no matter how carefully done, mutates > > them. The last example comes from Traits code, which is famous for > > having complex semantics attached to attribute access. > > > > Since this is a recurring problem, I'm trying to decide whether the > > convenience of autocall being on by default is worth against the > > occasional surprise of situations like these. > > > > Obviously we'd leave the functionality of autocall as-is, and people > > could still enable it if they wanted to. I'm only talking about > > whether it should be completely off by default. The reason why > > defaults matter a lot is that the vast majority of users never ever > > change them (I've seen countless long-term users with the wrong colors > > for a light bacgkround, the one default we can't pick for them and > > that leads to unreadable tracebacks, yet people live with it for > > years). So it's up to us to give people defaults that lead to the > > best experience possible. > > > > I personally had always fallen on the side of saying that autocall is > > a big convenience of ipython, and that code that mutates on simple > > attribute access is special enough that we shouldn't sacrifice that > > convenience for most users to accommodate such a special case. > > I don't think it's *that* special of a case. There are all kinds of good > reasons > to set up a special case for the first time an attribute is looked up while > mutating the object to let future lookups go through a different path. > *Usually*, getting the attribute in the autocall code is innocuous, but > sometimes it isn't. > I'll second Robert's comment. Here's a "user story" where the double evaluation of an attribute caused me headaches until I realized what was happening. We (Enthought) developed some some classes that provide interfaces to test equipment (oscilloscopes, power supplies, etc). By using properties, we developed interfaces to the instruments that were very easy to use. For example, a statement such as instr.voltage = 5 would cause a command to be sent to the instrument to set the voltage to 5. Similarly, v = instr.voltage would trigger a query to get the current voltage setting from the instrument, and return it as a floating point value. Working with such a class from within ipython is slick, but no doubt you can already see the potential problem. Using ipython as follows In [5]: instr.voltage Out [5]: 5.0 actually sent the query to the instrument twice. This can be a relatively slow query, because the communication might be over a serial line or GPIB interface. OK, so it is slower than necessary; that is a minor annoyance. The real headache occurred when I was using a property that accessed the error state of the instrument. Reading the attribute 'system_error' sent the appropriate query to the instrument, but that query also caused the instrument to reset its internal error state. It took me a long time to figure out why, when I intentionally did something wrong, the following In [8]: instr.system_error always resulted in no errors! Eventually I realized that it was triggering two queries. I wasn't aware of autocall at the time, so my solution was to always use print, e.g. In [9]: print instr.system_error So, there's another "real world" use-case where reading an attribute twice at the command prompt caused problems. (Whether that particular attribute should have been a method instead--e.g. read_and_reset_system_error()--is a separate topic, but there were good reasons for the property implementation.) Cheers, Warren > I've always been in favor of turning autocall off by default, though I > can't > really say that I have a strong argument beyond personal preference. I > don't > like to keep two parallel Python syntaxes in my head (and muscle memory!), > especially when one of them is only informally specified and only works in > one > specific environment. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 13:00:54 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 18:00:54 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: <4ED6623E.2020007@bostream.nu> References: <4ED6623E.2020007@bostream.nu> Message-ID: I think the question of multiline readline history in the terminal is relevant. We turned it off for 0.11, and had fairly loud complaints about it, so we've relented and turned it back on for 0.12. Of course, autocall will be configurable, but I'd expect more complaints about it not being on by default than about the issue with getattr affecting state. We'd also have to make changes in tab completion, since at present it will access each attribute when you do a.. I maintain that altering state on attribute access, while it's perfectly possible, is almost always a bad idea, for pretty much this reason: you can do very useful things by assuming that you're free to inspect attributes. Besides Traits, do we know what else does this? Maybe we should do the simcity fix* and just avoid attribute access on objects with traits. Or is there some way to detect without accessing it if an attribute is a property, or if __getattribute__ has been overridden? Thomas * see http://www.joelonsoftware.com/articles/APIWar.html -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Nov 30 13:23:11 2011 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 30 Nov 2011 10:23:11 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: Hi, On Wed, Nov 30, 2011 at 10:00 AM, Thomas Kluyver wrote: > I think the question of multiline readline history in the terminal is > relevant. We turned it off for 0.11, and had fairly loud complaints about > it, so we've relented and turned it back on for 0.12. Of course, autocall > will be configurable, but I'd expect more complaints about it not being on > by default than about the issue with getattr affecting state. I haven't ever used autocall for the same reason Robert gave - it felt painful to have an autocall way of typing as well as a >>> python way of typing. For example, a new user of python 3 may (in IPython) start to believe that 'print foo' is valid python. I have sometimes noticed new users going 'huh?' when autocall kicks in (followed obviously by 'oh I see'). So maybe it is best thought of as an expert setting. Best, Matthew From takowl at gmail.com Wed Nov 30 13:50:52 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 18:50:52 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 30 November 2011 18:23, Matthew Brett wrote: > For example, a new user of python 3 may (in IPython) start to believe > that 'print foo' is valid python. > I think this is potentially the case with a lot of what IPython does ('foo?' introspection, magic calls, shell access), but with autocall, we make a point of displaying the rewritten form so it's obvious what's happened. I realised I was wrong about attribute access on tab completion - a. won't actually access all attributes of a, although a.b. has to access a.b. As a compromise, why don't we allow autocall of "foo x", but not "foo.bar x" (unless autocall is set to 2)? This preserves the main uses of autocall (len, print in Python 3, etc.), without needing attribute access. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 14:20:57 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 19:20:57 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 30 November 2011 18:50, Thomas Kluyver wrote: > As a compromise, why don't we allow autocall of "foo x", but not "foo.bar > x" (unless autocall is set to 2)? This preserves the main uses of autocall > (len, print in Python 3, etc.), without needing attribute access. I've done this in my autocall-safe branch, if anyone wants to test it: https://github.com/takluyver/ipython/tree/autocall-safe Turning autocall up to 2 will still inspect attributes, but that's not the default. If we're happy with this behaviour, I'll update docs and so forth. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Wed Nov 30 14:38:16 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 30 Nov 2011 11:38:16 -0800 Subject: [IPython-dev] Problem with 0.12 beta Message-ID: When I start IPython on Ubuntu 11.04: bgadmin at csm-ru-dirac-2a:~/build/ipython/IPython/core$ ipython Python 2.7.1+ (r271:86832, Apr 11 2011, 18:13:53) Type "copyright", "credits" or "license" for more information. IPython 0.12.beta -- An enhanced Interactive Python. ? -> Introduction and overview of IPython's features. %quickref -> Quick reference. help -> Python's own help system. object? -> Details about 'object', use 'object??' for extra details. [TerminalIPythonApp] Error in loading extension: kernmagic --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /nfsopt/lib/python2.7/site-packages/IPython/core/extensions.pyc in load_extension(self, module_str) 85 if module_str not in sys.modules: 86 with prepended_to_syspath(self.ipython_extension_dir): ---> 87 __import__(module_str) 88 mod = sys.modules[module_str] 89 return self._call_load_ipython_extension(mod) ImportError: No module named kernmagic In [1]: import IPython In [2]: IPython.__file__ Out[2]: '/nfsopt/lib/python2.7/site-packages/IPython/__init__.pyc' In [3]: IPython.__version__ Out[3]: '0.12.beta' Is there a missing file? Cheers, Brian -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From fperez.net at gmail.com Wed Nov 30 14:39:44 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 11:39:44 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On Wed, Nov 30, 2011 at 11:20 AM, Thomas Kluyver wrote: > On 30 November 2011 18:50, Thomas Kluyver wrote: >> >> As a compromise, why don't we allow autocall of "foo x", but not "foo.bar >> x" (unless autocall is set to 2)? This preserves the main uses of autocall >> (len, print in Python 3, etc.), without needing attribute access. > > I've done this in my autocall-safe branch, if anyone wants to test it: > https://github.com/takluyver/ipython/tree/autocall-safe > > Turning autocall up to 2 will still inspect attributes, but that's not the > default. > > If we're happy with this behaviour, I'll update docs and so forth. While for expert users this may be a good compromise, I worry that for newbies it becomes too much to explain out of the box. The rule now becomes "ipython will automatically call functions, but only if there are aguments and they are not being called form a nested namespace but directly from the global one, so we don't trigger multiple attribute accesses". If you're explaining that to a first-time user, they're already running out the door :) It seems to me that this could instead be a new mode for autocall, and perhaps we should simply change to labels rather than numerical categories for autocall: - off (old 0): not at all - safe (no number): your proposal - smart (old 1): as before - full (old 2): as before The question remains what the default should be. There seems to be a lot of push for going to 'off' as the default, and I think the arguments carry some merit. My view of ipython has always been that it should be 'better than the plain python shell but fully transparent in being better', meaning that it should take all valid pyhton input and treat it as so, and only add convenience but not introduce special rules to worry about. In a sense, autocall fails at that because you do need to worry and there are valid cases where its behavior is fully different from plain python. I personally will keep it at 1 because I find that very useful, but I'm not the 'target audience' for this discussion. So in summary, from the feedback (thanks everyone), I'm leaning towards: - defaulting autocall to 0. - accepting Thomas' idea, if we can rework the machinery (in a backwards compatible way, so we still understand the 0,1,2 labels as synonyms for 'off', 'smart', 'full') to accept 'safe' as a new mode. How does that sound? Cheers, f From asmeurer at gmail.com Wed Nov 30 14:40:33 2011 From: asmeurer at gmail.com (Aaron Meurer) Date: Wed, 30 Nov 2011 12:40:33 -0700 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 10:50 AM, Warren Weckesser wrote: > > > On Wed, Nov 30, 2011 at 5:18 AM, Robert Kern wrote: >> >> On 11/30/11 7:07 AM, Fernando Perez wrote: >> > Hi folks, >> > >> > over the years, we've had a number of bug reports that ultimately boil >> > down to ipython doing an extra getattr call on objects at the prompt, >> > such as >> > >> > https://github.com/ipython/ipython/issues/988 >> > https://github.com/ipython/ipython/issues/851 >> > >> > While this may appear odd, people do have code out there that mutates >> > the state of objects on simple getattr calls, and so *any* >> > introspection on such objects, no matter how carefully done, mutates >> > them. ?The last example comes from Traits code, which is famous for >> > having complex semantics attached to attribute access. >> > >> > Since this is a recurring problem, I'm trying to decide whether the >> > convenience of autocall being on by default is worth against the >> > occasional surprise of situations like these. >> > >> > Obviously we'd leave the functionality of autocall as-is, and people >> > could still enable it if they wanted to. ?I'm only talking about >> > whether it should be completely off by default. ?The reason why >> > defaults matter a lot is that the vast majority of users never ever >> > change them (I've seen countless long-term users with the wrong colors >> > for a light bacgkround, the one default we can't pick for them and >> > that leads to unreadable tracebacks, yet people live with it for >> > years). ?So it's up to us to give people defaults that lead to the >> > best experience possible. >> > >> > I personally had always fallen on the side of saying that autocall is >> > a big convenience of ipython, and that code that mutates on simple >> > attribute access is special enough that we shouldn't sacrifice that >> > convenience for most users to accommodate such a special case. >> >> I don't think it's *that* special of a case. There are all kinds of good >> reasons >> to set up a special case for the first time an attribute is looked up >> while >> mutating the object to let future lookups go through a different path. >> *Usually*, getting the attribute in the autocall code is innocuous, but >> sometimes it isn't. > > > > I'll second Robert's comment. > > Here's a "user story" where the double evaluation of an attribute caused > me headaches until I realized what was happening. > > We (Enthought) developed some some classes that provide interfaces to > test equipment (oscilloscopes, power supplies, etc).? By using properties, > we developed interfaces to the instruments that were very easy to use. > For example, a statement such as > > ?? instr.voltage = 5 > > would cause a command to be sent to the instrument to set the voltage to 5. > Similarly, > > ?? v = instr.voltage > > would trigger a query to get the current voltage setting from the > instrument, > and return it as a floating point value. > > Working with such a class from within ipython is slick, but no doubt you > can already see the potential problem.? Using ipython as follows > > In? [5]: instr.voltage > Out [5]: 5.0 > > actually sent the query to the instrument twice.? This can be a relatively > slow > query, because the communication might be over a serial line or GPIB > interface. > > OK, so it is slower than necessary; that is a minor annoyance.? The real > headache occurred when I was using a property that accessed the error > state of the instrument.? Reading the attribute 'system_error' sent the > appropriate query to the instrument, but that query also caused the > instrument > to reset its internal error state.? It took me a long time to figure out > why, > when I intentionally did something wrong, the following > > In? [8]: instr.system_error > > always resulted in no errors!? Eventually I realized that it was triggering > two queries.? I wasn't aware of autocall at the time, so my solution was to > always use print, e.g. > > In? [9]: print instr.system_error > > So, there's another "real world" use-case where reading an attribute twice > at the command prompt caused problems.? (Whether that particular attribute > should have been a method instead--e.g. read_and_reset_system_error()--is > a separate topic, but there were good reasons for the property > implementation.) To me, this is not a different topic. You should not assume that an attribute will not be read multiple times. A system like IPython will need to do things like this to do its magic. There are other things that will mess this up too. I can't imagine what would happen to that system if you tried running it through a graphical debugger! On the other hand, virtually nothing would automatically do instr.system_error(), because it's kind of known that calling something changes state (in general). It's just two characters more, and shouldn't be that much different on the programatic side either, so I can't see why it was done the other way (but I haven't done much on the programatic side, so maybe it really isn't as easy). Regarding autocall, I think it should be disabled by default, not because it messes with magic things like this (so you can't mix two kinds of magic, no surprises there), but because it's very unintuitive, and unpythonic. I agree with Matthew that when I first saw this, I went "huh", and then immediately, "how do I disable this"? To put it another way, magic that literally changes the syntax of Python should perhaps not be for beginner users (I'll put ?help magic as an exception to this). But that's just my opinion as a user of IPython. Aaron Meurer > > Cheers, > > Warren > > >> >> I've always been in favor of turning autocall off by default, though I >> can't >> really say that I have a strong argument beyond personal preference. I >> don't >> like to keep two parallel Python syntaxes in my head (and muscle memory!), >> especially when one of them is only informally specified and only works in >> one >> specific environment. >> >> -- >> Robert Kern >> >> "I have come to believe that the whole world is an enigma, a harmless >> enigma >> ?that is made terrible by our own mad attempt to interpret it as though it >> had >> ?an underlying truth." >> ? -- Umberto Eco >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From asmeurer at gmail.com Wed Nov 30 14:45:03 2011 From: asmeurer at gmail.com (Aaron Meurer) Date: Wed, 30 Nov 2011 12:45:03 -0700 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On Wed, Nov 30, 2011 at 12:39 PM, Fernando Perez wrote: > On Wed, Nov 30, 2011 at 11:20 AM, Thomas Kluyver wrote: >> On 30 November 2011 18:50, Thomas Kluyver wrote: >>> >>> As a compromise, why don't we allow autocall of "foo x", but not "foo.bar >>> x" (unless autocall is set to 2)? This preserves the main uses of autocall >>> (len, print in Python 3, etc.), without needing attribute access. >> >> I've done this in my autocall-safe branch, if anyone wants to test it: >> https://github.com/takluyver/ipython/tree/autocall-safe >> >> Turning autocall up to 2 will still inspect attributes, but that's not the >> default. >> >> If we're happy with this behaviour, I'll update docs and so forth. > > While for expert users this may be a good compromise, I worry that for > newbies it becomes too much to explain out of the box. ?The rule now > becomes "ipython will automatically call functions, but only if there > are aguments and they are not being called form a nested namespace but > directly from the global one, so we don't trigger multiple attribute > accesses". ?If you're explaining that to a first-time user, they're > already running out the door :) It already is confusing, though, to some degree: In [15]: %autocall 2 Automatic calling is: Full In [16]: str 1 -------> str(1) Out[16]: 1 In [17]: 1 + str 1 File "", line 1 1 + str 1 ^ SyntaxError: invalid syntax to someone who is completely new to Python, this can be a surprise (it will also be a surprise when "str 1" doesn't work in their scripts). So personally, I don't think autocall is friendly to beginner users even as it is now, for the simple reason that it changes the syntax of Python. Aaron Meurer > > It seems to me that this could instead be a new mode for autocall, and > perhaps we should simply change to labels rather than numerical > categories for autocall: > > - off (old 0): not at all > - safe (no number): your proposal > - smart (old 1): as before > - full (old 2): as before > > The question remains what the default should be. ?There seems to be a > lot of push for going to 'off' as the default, and I think the > arguments carry some merit. ?My view of ipython has always been that > it should be 'better than the plain python shell but fully transparent > in being better', meaning that it should take all valid pyhton input > and treat it as so, and only add convenience but not introduce special > rules to worry about. ?In a sense, autocall fails at that because you > do need to worry and there are valid cases where its behavior is fully > different from plain python. > > I personally will keep it at 1 because I find that very useful, but > I'm not the 'target audience' for this discussion. > > So in summary, from the feedback (thanks everyone), I'm leaning towards: > > - defaulting autocall to 0. > > - accepting Thomas' idea, if we can rework the machinery (in a > backwards compatible way, so we still understand the 0,1,2 labels as > synonyms for 'off', 'smart', 'full') to accept 'safe' as a new mode. > > How does that sound? > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Wed Nov 30 14:53:46 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 11:53:46 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: Hey Brian, On Wed, Nov 30, 2011 at 11:38 AM, Brian Granger wrote: > > ImportError: No module named kernmagic > > In [1]: import IPython > > In [2]: IPython.__file__ > Out[2]: '/nfsopt/lib/python2.7/site-packages/IPython/__init__.pyc' > > In [3]: IPython.__version__ > Out[3]: '0.12.beta' > > Is there a missing file? Is it possible you have it listed in your profile? I grepped for 'kernmagic' in the entire source tree and it's not mentioned anywhere: (master)longs[IPython]> grin kernmagic (master)longs[IPython]> so I don't think that's coming from us... Cheers, f From fperez.net at gmail.com Wed Nov 30 14:56:10 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 11:56:10 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer wrote: > So personally, I don't think autocall is friendly to beginner users > even as it is now, for the simple reason that it changes the syntax of > Python. Yes, there seems to be consensus that autocall should be an expert tool, and therefore it would be OK to make the default be off. The defaults should be chosen to make ipython as beginner-friendly as possible (while still being more useful than the >>> shell). Experts are expected to tweak their config to suit their personal preferences (e.g., I'll always keep autocall on in 'smart' mode). Cheers, f From benjaminrk at gmail.com Wed Nov 30 14:58:22 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 11:58:22 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 11:53, Fernando Perez wrote: > Hey Brian, > > On Wed, Nov 30, 2011 at 11:38 AM, Brian Granger > wrote: > > > > ImportError: No module named kernmagic > > > > In [1]: import IPython > > > > In [2]: IPython.__file__ > > Out[2]: '/nfsopt/lib/python2.7/site-packages/IPython/__init__.pyc' > > > > In [3]: IPython.__version__ > > Out[3]: '0.12.beta' > > > > Is there a missing file? > > Is it possible you have it listed in your profile? I grepped for > 'kernmagic' in the entire source tree and it's not mentioned anywhere: > I think you are right, and this is probably EPD's doing. My guess: you installed EPD on this machine, and are not currently using it. Is this correct? EPD's default IPython profile is not a no-op - it does exactly one thing: load kernmagic. This came up on IRC a few weeks ago, for a completely baffled user. -MinRK > > (master)longs[IPython]> grin kernmagic > (master)longs[IPython]> > > so I don't think that's coming from us... > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Nov 30 15:01:32 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 12:01:32 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 11:58 AM, MinRK wrote: > > EPD's default IPython profile is not a no-op - it does exactly one thing: > load kernmagic. > This came up on IRC a few weeks ago, for a completely baffled user. Ah, good to know. It means that when users from epd update to any of our versions, they're going to have this problem. Mmh... Cheers, f From benjaminrk at gmail.com Wed Nov 30 15:12:37 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 12:12:37 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On Wed, Nov 30, 2011 at 11:56, Fernando Perez wrote: > On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer wrote: > > So personally, I don't think autocall is friendly to beginner users > > even as it is now, for the simple reason that it changes the syntax of > > Python. > > Yes, there seems to be consensus that autocall should be an expert > tool, and therefore it would be OK to make the default be off. The > defaults should be chosen to make ipython as beginner-friendly as > possible (while still being more useful than the >>> shell). Experts > are expected to tweak their config to suit their personal preferences > (e.g., I'll always keep autocall on in 'smart' mode). > I think it's exactly the opposite. autocall is very much a beginner tool, for users who have not yet committed Python's syntax to memory. When they make simple typos, autocall helps remind them by saying, "I know what you meant, but here's how you should do it in the future". Of course, the down side is that it is the same beginner users who are most hurt by the weird behavior with traits/etc, so it's a hard decision. In my experience in the classroom, I would say that the helping-hand given by autocall is quite valuable, and utterly pointless if it is not on by default, because expecting brand new users to configure IPython is a non-starter. Perhaps I underestimate the popularity of tools that suffer from this behavior among beginning users. Having this conversation on ipython-dev may not be the most illustrative, as there aren't a lot of non-expert users here. Not that it's the wrong place, but we shouldn't interpret opinions here as being those of our user base in general. -MinRK > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Nov 30 15:17:36 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 12:17:36 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 12:01, Fernando Perez wrote: > On Wed, Nov 30, 2011 at 11:58 AM, MinRK wrote: > > > > EPD's default IPython profile is not a no-op - it does exactly one thing: > > load kernmagic. > > This came up on IRC a few weeks ago, for a completely baffled user. > > Ah, good to know. It means that when users from epd update to any of > our versions, they're going to have this problem. Mmh... > Should only be *former* users of EPD, who also never edited their default profile. kernmagic will still be available within EPD, and EPD's default profile is only written if the file didn't exist before, so if they had a config file prior to firstrun in EPD, they would still not be affected. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From mark.voorhies at ucsf.edu Wed Nov 30 14:51:56 2011 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Wed, 30 Nov 2011 11:51:56 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: <201111301151.57040.mark.voorhies@ucsf.edu> On Wednesday, November 30, 2011 10:00:54 am Thomas Kluyver wrote: > I maintain that altering state on attribute access, while it's perfectly > possible, is almost always a bad idea, for pretty much this reason: you can > do very useful things by assuming that you're free to inspect attributes. I tend to agree with you. On the other hand, it's legal in the language, and there is no C++-like concept of a const method to block this sort of side effect (meaning that people will do this sort of thing both deliberately and inadvertently). Given that lack of autocall fails loudly and multiple attribute access fails silently, I think that setting the default to "fail loudly" (no autocall) makes sense. --Mark From cekees at gmail.com Wed Nov 30 15:42:41 2011 From: cekees at gmail.com (Chris Kees) Date: Wed, 30 Nov 2011 14:42:41 -0600 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: Is there a way for ipython to throw an exception or warning with "somethings wrong with your profile in path_to_profile". I had this problem too. It took me a while to figure out where the import statement was. Mine probably originated with EPD as well, but I've messed around with my profile several times working on the mpi profile and can imagine errors in profiles will happen. Chris On Wed, Nov 30, 2011 at 2:17 PM, MinRK wrote: > > > On Wed, Nov 30, 2011 at 12:01, Fernando Perez wrote: > >> On Wed, Nov 30, 2011 at 11:58 AM, MinRK wrote: >> > >> > EPD's default IPython profile is not a no-op - it does exactly one >> thing: >> > load kernmagic. >> > This came up on IRC a few weeks ago, for a completely baffled user. >> >> Ah, good to know. It means that when users from epd update to any of >> our versions, they're going to have this problem. Mmh... >> > > Should only be *former* users of EPD, who also never edited their default > profile. kernmagic will still be available within EPD, > and EPD's default profile is only written if the file didn't exist before, > so if they had a config file prior to firstrun in EPD, they would > still not be affected. > > -MinRK > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 15:59:04 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 20:59:04 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 30 November 2011 19:39, Fernando Perez wrote: > So in summary, from the feedback (thanks everyone), I'm leaning towards: > > - defaulting autocall to 0. > > - accepting Thomas' idea, if we can rework the machinery (in a > backwards compatible way, so we still understand the 0,1,2 labels as > synonyms for 'off', 'smart', 'full') to accept 'safe' as a new mode. > I would still press for having my 'safe autocall' on by default. As far as I can see, this shouldn't cause any problems with unexpected attribute access. As Min mentioned, autocall is a handy hint for beginners, and almost entirely useless if it's off by default. And I suspect that >90% of what it's used for is simple functions, like len and list - I use "list _" a lot when something returns a generator and I want to see the values. Also, for people migrating to Python 3, having "print x" just work but give a reminder will save a lot of cursing. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Nov 30 16:02:19 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 13:02:19 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: <201111301151.57040.mark.voorhies@ucsf.edu> References: <4ED6623E.2020007@bostream.nu> <201111301151.57040.mark.voorhies@ucsf.edu> Message-ID: On Wed, Nov 30, 2011 at 11:51, Mark Voorhies wrote: > On Wednesday, November 30, 2011 10:00:54 am Thomas Kluyver wrote: > > > I maintain that altering state on attribute access, while it's perfectly > > possible, is almost always a bad idea, for pretty much this reason: you > can > > do very useful things by assuming that you're free to inspect attributes. > > I tend to agree with you. On the other hand, it's legal in the language, > and > there is no C++-like concept of a const method to block this sort of side > effect > (meaning that people will do this sort of thing both deliberately and > inadvertently). > > Python is a language of unenforced convention, which allows for this kind of magic vs magic conflict. IPython's expectation that hasattr is safe is a reasonable one, as is Traits' expectation that getattr will only be called when the attribute is actually requested. But Traits' behavior invalidates hasattr, and IPython's behavior can trigger unrequested events. It would be nice if there was a corresponding `__hasattr__`, so that checking for attribute existence could bypass side effects in objects that do getattr magic that should only be triggered on *real* getattr. Given that lack of autocall fails loudly and multiple attribute access > fails silently, > I think that setting the default to "fail loudly" (no autocall) makes > sense. > This makes perfect sense, and is a good general rule for such decisions. I do think it will be newcomers using IPython to learn Python who will be most adversely affected by the change, because a SyntaxError is not helpful, while correcting the user is instructive. -MinRK > > --Mark > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Wed Nov 30 16:31:37 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 13:31:37 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 12:42, Chris Kees wrote: > Is there a way for ipython to throw an exception or warning with > "somethings wrong with your profile in path_to_profile". I had this problem > too. It took me a while to figure out where the import statement was. Mine > probably originated with EPD as well, but I've messed around with my > profile several times working on the mpi profile and can imagine errors in > profiles will happen. *Errors* in profiles do get special treatment. Add 1/0 in a config file, and you get: [TerminalIPythonApp] Exception while loading config file /Users/minrk/.ipython/profile_default/ipython_config.py Traceback (most recent call last): File "/Users/minrk/dev/ip/mine/IPython/config/application.py", line 440, in load_config_file config = loader.load_config() File "/Users/minrk/dev/ip/mine/IPython/config/loader.py", line 268, in load_config self._read_file_as_dict() File "/Users/minrk/dev/ip/mine/IPython/config/loader.py", line 321, in _read_file_as_dict py3compat.execfile(conf_filename, namespace) File "/Users/minrk/dev/ip/mine/IPython/utils/py3compat.py", line 175, in execfile __builtin__.execfile(filename, *where) File "/Users/minrk/.ipython/profile_default/ipython_config.py", line 4, in 1/0 ZeroDivisionError: integer division or modulo by zero And the missing extension error is already specially handled: [TerminalIPythonApp] Error in loading extension: dne --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /Users/minrk/dev/ip/mine/IPython/core/extensions.pyc in load_extension(self, module_str) 85 if module_str not in sys.modules: 86 with prepended_to_syspath(self.ipython_extension_dir): ---> 87 __import__(module_str) 88 mod = sys.modules[module_str] 89 return self._call_load_ipython_extension(mod) ImportError: No module named dne The only way to load an extension is by asking for it in a config file, so we have perhaps inappropriately assumed that it is obvious where the error resides. Under normal circumstances, I don't think it's possible for this error to occur without the *user* writing the bad code. The fact that EPD writes its own default config file that won't normally work outside EPD means that users are facing this issue without ever touching or even thinking about IPython config. I can easily add a 'check your config files in ...' bit to that error message. I'll push straight away if Fernando thinks that's a good idea. What it looks like right now: [TerminalIPythonApp] Error in loading extension: dne Check your config files in /Users/minrk/.ipython/profile_default --------------------------------------------------------------------------- ImportError Traceback (most recent call last) /Users/minrk/dev/ip/mine/IPython/core/extensions.pyc in load_extension(self, module_str) 85 if module_str not in sys.modules: 86 with prepended_to_syspath(self.ipython_extension_dir): ---> 87 __import__(module_str) 88 mod = sys.modules[module_str] 89 return self._call_load_ipython_extension(mod) ImportError: No module named dne -MinRK > > Chris > > On Wed, Nov 30, 2011 at 2:17 PM, MinRK wrote: > >> >> >> On Wed, Nov 30, 2011 at 12:01, Fernando Perez wrote: >> >>> On Wed, Nov 30, 2011 at 11:58 AM, MinRK wrote: >>> > >>> > EPD's default IPython profile is not a no-op - it does exactly one >>> thing: >>> > load kernmagic. >>> > This came up on IRC a few weeks ago, for a completely baffled user. >>> >>> Ah, good to know. It means that when users from epd update to any of >>> our versions, they're going to have this problem. Mmh... >>> >> >> Should only be *former* users of EPD, who also never edited their default >> profile. kernmagic will still be available within EPD, >> and EPD's default profile is only written if the file didn't exist >> before, so if they had a config file prior to firstrun in EPD, they would >> still not be affected. >> >> -MinRK >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Nov 30 16:36:31 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 30 Nov 2011 13:36:31 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 1:31 PM, MinRK wrote: > I can easily add a 'check your config files in ...' bit to that error > message. ?I'll push straight away if Fernando > thinks that's a good idea. Go for it, I'm sure it will make debugging such problems much easier, and it will help users learn about where their config stuff lives. Thanks! f From benjaminrk at gmail.com Wed Nov 30 16:42:15 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 13:42:15 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On Wed, Nov 30, 2011 at 13:36, Fernando Perez wrote: > On Wed, Nov 30, 2011 at 1:31 PM, MinRK wrote: > > I can easily add a 'check your config files in ...' bit to that error > > message. I'll push straight away if Fernando > > thinks that's a good idea. > > Go for it, I'm sure it will make debugging such problems much easier, > and it will help users learn about where their config stuff lives. > thanks, pushed. > > Thanks! > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Wed Nov 30 16:50:33 2011 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Nov 2011 21:50:33 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 11/30/11 8:12 PM, MinRK wrote: > > > On Wed, Nov 30, 2011 at 11:56, Fernando Perez > wrote: > > On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer > wrote: > > So personally, I don't think autocall is friendly to beginner users > > even as it is now, for the simple reason that it changes the syntax of > > Python. > > Yes, there seems to be consensus that autocall should be an expert > tool, and therefore it would be OK to make the default be off. The > defaults should be chosen to make ipython as beginner-friendly as > possible (while still being more useful than the >>> shell). Experts > are expected to tweak their config to suit their personal preferences > (e.g., I'll always keep autocall on in 'smart' mode). > > > I think it's exactly the opposite. autocall is very much a beginner tool, for > users who have not yet > committed Python's syntax to memory. When they make simple typos, autocall > helps remind them by saying, > "I know what you meant, but here's how you should do it in the future". But that's not really what the feature does, or what it was intended to do. It was added solely for the experienced user named Fernando who liked to type things fast without needing to use the shift keys. It was not a do-what-I-mean to correct newbie errors. There is certainly nothing in the message that implies that you *should* type things differently. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Wed Nov 30 16:53:49 2011 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Nov 2011 21:53:49 +0000 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: On 11/30/11 7:58 PM, MinRK wrote: > > > On Wed, Nov 30, 2011 at 11:53, Fernando Perez > wrote: > > Hey Brian, > > On Wed, Nov 30, 2011 at 11:38 AM, Brian Granger > wrote: > > > > ImportError: No module named kernmagic > > > > In [1]: import IPython > > > > In [2]: IPython.__file__ > > Out[2]: '/nfsopt/lib/python2.7/site-packages/IPython/__init__.pyc' > > > > In [3]: IPython.__version__ > > Out[3]: '0.12.beta' > > > > Is there a missing file? > > Is it possible you have it listed in your profile? I grepped for > 'kernmagic' in the entire source tree and it's not mentioned anywhere: > > > I think you are right, and this is probably EPD's doing. My guess: you > installed EPD on this machine, and are not currently using it. Is this correct? > > EPD's default IPython profile is not a no-op - it does exactly one thing: load > kernmagic. > > This came up on IRC a few weeks ago, for a completely baffled user. Yes. I did not ask for it to be added to EPD. Someone just added it. Let me go yell at him. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From benjaminrk at gmail.com Wed Nov 30 17:04:37 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 14:04:37 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On Wed, Nov 30, 2011 at 13:50, Robert Kern wrote: > On 11/30/11 8:12 PM, MinRK wrote: > > > > > > On Wed, Nov 30, 2011 at 11:56, Fernando Perez > > wrote: > > > > On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer > > wrote: > > > So personally, I don't think autocall is friendly to beginner > users > > > even as it is now, for the simple reason that it changes the > syntax of > > > Python. > > > > Yes, there seems to be consensus that autocall should be an expert > > tool, and therefore it would be OK to make the default be off. The > > defaults should be chosen to make ipython as beginner-friendly as > > possible (while still being more useful than the >>> shell). Experts > > are expected to tweak their config to suit their personal preferences > > (e.g., I'll always keep autocall on in 'smart' mode). > > > > > > I think it's exactly the opposite. autocall is very much a beginner > tool, for > > users who have not yet > > committed Python's syntax to memory. When they make simple typos, > autocall > > helps remind them by saying, > > "I know what you meant, but here's how you should do it in the future". > > But that's not really what the feature does, or what it was intended to > do. It > was added solely for the experienced user named Fernando who liked to type > things fast without needing to use the shift keys. It was not a > do-what-I-mean > to correct newbie errors. There is certainly nothing in the message that > implies > that you *should* type things differently. > Original developer intent doesn't seem relevant. The fact remains that it has this effect, and demonstrably makes IPython a better, friendlier tool for learning Python. Similarly, disabling it by default will make IPython better behaved in edge cases, but less friendly and less helpful for learning Python. I'm not saying we shouldn't make the change, just pointing out that there is a real class of users actively hurt by it. -MinRK > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Wed Nov 30 17:21:50 2011 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 30 Nov 2011 22:21:50 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 11/30/11 10:04 PM, MinRK wrote: > > > On Wed, Nov 30, 2011 at 13:50, Robert Kern > wrote: > > On 11/30/11 8:12 PM, MinRK wrote: > > > > > > On Wed, Nov 30, 2011 at 11:56, Fernando Perez > > >> wrote: > > > > On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer > > >> wrote: > > > So personally, I don't think autocall is friendly to beginner users > > > even as it is now, for the simple reason that it changes the syntax of > > > Python. > > > > Yes, there seems to be consensus that autocall should be an expert > > tool, and therefore it would be OK to make the default be off. The > > defaults should be chosen to make ipython as beginner-friendly as > > possible (while still being more useful than the >>> shell). Experts > > are expected to tweak their config to suit their personal preferences > > (e.g., I'll always keep autocall on in 'smart' mode). > > > > > > I think it's exactly the opposite. autocall is very much a beginner > tool, for > > users who have not yet > > committed Python's syntax to memory. When they make simple typos, autocall > > helps remind them by saying, > > "I know what you meant, but here's how you should do it in the future". > > But that's not really what the feature does, or what it was intended to do. It > was added solely for the experienced user named Fernando who liked to type > things fast without needing to use the shift keys. It was not a do-what-I-mean > to correct newbie errors. There is certainly nothing in the message that implies > that you *should* type things differently. > > > Original developer intent doesn't seem relevant. The fact remains that it has > this effect, > and demonstrably makes IPython a better, friendlier tool for learning Python. I don't think this has been convincingly demonstrated, either that it has this effect to any significant extent in the newbie population or that such an effect makes IPython a better, friendlier tool for learning Python. The original developer intent is relevant insofar as it guided the actual feature itself. As I said, there is nothing about the feature that encourages newbies to not use the feature. It just shows you what it got transformed into. The original developer intent explains why this is so: the experienced user will want to see the transformation to double-check unintended errors. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Wed Nov 30 17:25:32 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 30 Nov 2011 14:25:32 -0800 Subject: [IPython-dev] Problem with 0.12 beta In-Reply-To: References: Message-ID: MinRK, On Wed, Nov 30, 2011 at 11:58 AM, MinRK wrote: > > > On Wed, Nov 30, 2011 at 11:53, Fernando Perez wrote: >> >> Hey Brian, >> >> On Wed, Nov 30, 2011 at 11:38 AM, Brian Granger >> wrote: >> > >> > ImportError: No module named kernmagic >> > >> > In [1]: import IPython >> > >> > In [2]: IPython.__file__ >> > Out[2]: '/nfsopt/lib/python2.7/site-packages/IPython/__init__.pyc' >> > >> > In [3]: IPython.__version__ >> > Out[3]: '0.12.beta' >> > >> > Is there a missing file? >> >> Is it possible you have it listed in your profile? ?I grepped for >> 'kernmagic' in the entire source tree and it's not mentioned anywhere: > > > I think you are right, and this is probably EPD's doing. ?My guess: you > installed EPD on this machine, and are not currently using it. Is this > correct? As usual you are spot on. > EPD's default IPython profile is not a no-op - it does exactly one thing: > load kernmagic. That explains it completely then. But I do agree with Robert that EPD should be not doing this. Thanks! Cheers, Brian > This came up on IRC a few weeks ago, for a completely baffled user. > > -MinRK > >> >> >> (master)longs[IPython]> grin kernmagic >> (master)longs[IPython]> >> >> so I don't think that's coming from us... >> >> Cheers, >> >> f >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From ellisonbg at gmail.com Wed Nov 30 17:40:57 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Wed, 30 Nov 2011 14:40:57 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: LOL, I just had a research student come to my office hours with a notebook. She has been working in Python/IPython for 10 weeks now. In her notebook she had a line: plot e1 Which worked due to autocall. But she had absolutely no idea why this worked (isn't it similar to "print 10"). In her mind, IPython=Python. My options at that point: * Explain the whole autocall thing to her. No way, her cognitive load was already maxed out trying to understand the physics and code. * Silently let it be. No way, then she think that this is vald Python syntax and will start using everywhere even in library code. * Have her change it, but don't tell her exactly why. This is what I did, but she is still left with the vague sense that this syntax also works in *Python* but is not the best way of doing things. I maintain that the autocall syntax is confusing unless you take the time to really understand what is going on. And if you have done that, you are not a beginner. This confusion exists even without the potential pitfalls related to properties and attribute access. Add those pitfalls and you have significant potential for confusion. On Wed, Nov 30, 2011 at 1:50 PM, Robert Kern wrote: > On 11/30/11 8:12 PM, MinRK wrote: >> >> >> On Wed, Nov 30, 2011 at 11:56, Fernando Perez > > wrote: >> >> ? ? On Wed, Nov 30, 2011 at 11:45 AM, Aaron Meurer > ? ? > wrote: >> ? ? ?> So personally, I don't think autocall is friendly to beginner users >> ? ? ?> even as it is now, for the simple reason that it changes the syntax of >> ? ? ?> Python. >> >> ? ? Yes, there seems to be consensus that autocall should be an expert >> ? ? tool, and therefore it would be OK to make the default be off. ?The >> ? ? defaults should be chosen to make ipython as beginner-friendly as >> ? ? possible (while still being more useful than the >>> shell). ?Experts >> ? ? are expected to tweak their config to suit their personal preferences >> ? ? (e.g., I'll always keep autocall on in 'smart' mode). >> >> >> I think it's exactly the opposite. ?autocall is very much a beginner tool, for >> users who have not yet >> committed Python's syntax to memory. ?When they make simple typos, autocall >> helps remind them by saying, >> "I know what you meant, but here's how you should do it in the future". > > But that's not really what the feature does, or what it was intended to do. It > was added solely for the experienced user named Fernando who liked to type > things fast without needing to use the shift keys. It was not a do-what-I-mean > to correct newbie errors. There is certainly nothing in the message that implies > that you *should* type things differently. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- Brian E. Granger Cal Poly State University, San Luis Obispo bgranger at calpoly.edu and ellisonbg at gmail.com From benjaminrk at gmail.com Wed Nov 30 17:53:11 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 30 Nov 2011 14:53:11 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: Fair enough - I have certainly seen lots of students learn from this, but if others are confused by it then I retract my argument. -MinRK -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Wed Nov 30 17:56:05 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 30 Nov 2011 22:56:05 +0000 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: On 30 November 2011 22:40, Brian Granger wrote: > In her notebook she had a line: > > plot e1 > > Which worked due to autocall. But she had absolutely no idea why this > worked (isn't it similar to "print 10"). In her mind, IPython=Python. > My options at that point: > I think this is a technical issue as much as anything. The terminal shell clearly shows the rewritten command, and it is sent as a payload over ZMQ, but AFAIK both of our frontends ignore it. With the notebook, I imagine we could literally rewrite the cell to "plot(e1)". To my mind, that would clearly say "this is the right way to do it, but I knew what you meant." It would also work well for the copy and paste learner. In general, one of the trickiest things for new programmers is "what does this cryptic error message mean I need to change?". For the interpreter, guessing what the programmer meant is tricky. Here we have the ability to guess what the user meant and correct them. Or we can just throw an unhelpful SyntaxError. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Wed Nov 30 23:20:41 2011 From: matthew.brett at gmail.com (Matthew Brett) Date: Wed, 30 Nov 2011 20:20:41 -0800 Subject: [IPython-dev] Should we disable autocall by default from now on? In-Reply-To: References: <4ED6623E.2020007@bostream.nu> Message-ID: Hi, On Wed, Nov 30, 2011 at 2:56 PM, Thomas Kluyver wrote: > On 30 November 2011 22:40, Brian Granger wrote: >> >> In her notebook she had a line: >> >> plot e1 >> >> Which worked due to autocall. ?But she had absolutely no idea why this >> worked (isn't it similar to "print 10"). ?In her mind, IPython=Python. >> ?My options at that point: > > > I think this is a technical issue as much as anything. The terminal shell > clearly shows the rewritten command, and it is sent as a payload over ZMQ, > but AFAIK both of our frontends ignore it. With the notebook, I imagine we > could literally rewrite the cell to "plot(e1)". To my mind, that would > clearly say "this is the right way to do it, but I knew what you meant." It > would also work well for the copy and paste learner. > > In general, one of the trickiest things for new programmers is "what does > this cryptic error message mean I need to change?". For the interpreter, > guessing what the programmer meant is tricky. Here we have the ability to > guess what the user meant and correct them. Or we can just throw an > unhelpful SyntaxError. How about a default of raising a more informative error? Best, Matthew