From takowl at gmail.com Tue Feb 1 07:02:10 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 1 Feb 2011 12:02:10 +0000 Subject: [IPython-dev] Unicode in tests Message-ID: While testing the use of non-ascii characters in 0.11 recently, I found that there are things which will fail while actually using ipython, but cannot be made to fail in the test suite. Investigating, I found that for the test suite, the default encoding is somehow set to utf-8 (ascii is the default default encoding for Python 2). I don't know quite what's doing this, but it did the same with my system version of nose (0.11.1 in Ubuntu) and with nose 1.0.0 installed in the virtualenv. At present, I've worked round that by adding an entirely separate unicode_tests.py file, which has to be run separately from the main iptest test suite. https://github.com/ipython/ipython/pull/252 Does anyone have any insight on whether this is something our test suite is doing, something nose is doing, something that can be fixed, or something that we have to work around? Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From Nicolas.rougier at inria.fr Tue Feb 1 12:56:03 2011 From: Nicolas.rougier at inria.fr (Nicolas Rougier) Date: Tue, 1 Feb 2011 18:56:03 +0100 Subject: [IPython-dev] Ascii imshow Message-ID: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Hi all, I've made some time ago a kind of 'ascii imshow' function to render a numpy array directly within the console by exploiting the 256 colors capability of most modern terminal. I don't know how much interesting it can be for IPython but I found the code useful for myself. Code is located at: http://www.loria.fr/~rougier/coding/software/numpy_imshow.py (GPL license can be replaced by the BSD one if necessary) Nicolas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Feb 1 13:36:07 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 1 Feb 2011 13:36:07 -0500 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? Message-ID: Hi folks, I was just thinking, before the new display infrastructure starts getting used too much, should we adjust our convention to use __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of __html__, __svg__, etc? My concern is two-fold: - the more generic names may be already in use by something else that doesn't quite match our 'protocol', or python may in the future (though this is probably needless paranoia) choose to add them. - a more regular pattern makes it easier to collect all __repr_FOO__ methods from an object as part of our display machinery rather than matching against a hard-coded list. Thoughts? f From ellisonbg at gmail.com Tue Feb 1 14:05:52 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 1 Feb 2011 11:05:52 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: Fernando, On Tue, Feb 1, 2011 at 10:36 AM, Fernando Perez wrote: > Hi folks, > > I was just thinking, before the new display infrastructure starts > getting used too much, should we adjust our convention to use > __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of > __html__, __svg__, etc? ?My concern is two-fold: > > - the more generic names may be already in use by something else that > doesn't quite match our 'protocol', or python may in the future > (though this is probably needless paranoia) choose to add them. Yes, this is important and I have spent some time thinking about this. The collision with other project is something to think about. I can say why I made the choice I did: * Simplicity. __html__ is easier to remember that __repr_html__. I want writing these methods to be like eating candy. * There are projects that use this convention already and our usage in compatible with theirs. The biggest example is the __html__ method which Pylons, mako, Jinja2, Genshi and Django supposedly support. I thought - wow, these guys would _love_ us if there objects were immediately representable in IPython. I am definitely open to the __repr_html__ style names though. What do other people think? > - a more regular pattern makes it easier to collect all __repr_FOO__ > methods from an object as part of our display machinery rather than > matching against a hard-coded list. Yes, that is true, but it would be difficult to really use such dynamic display logic because each FOO has to be associated with a unique MIME type and formatter. I was expecting this question to come up in the review of the code. It didn't, but I am glad it is coming up now. Cheers, Brian > Thoughts? > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Tue Feb 1 14:28:11 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 11:28:11 -0800 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: On 2/1/11 4:02 AM, Thomas Kluyver wrote: > While testing the use of non-ascii characters in 0.11 recently, I found that > there are things which will fail while actually using ipython, but cannot be > made to fail in the test suite. Investigating, I found that for the test suite, > the default encoding is somehow set to utf-8 (ascii is the default default > encoding for Python 2). I don't know quite what's doing this, but it did the > same with my system version of nose (0.11.1 in Ubuntu) and with nose 1.0.0 > installed in the virtualenv. > > At present, I've worked round that by adding an entirely separate > unicode_tests.py file, which has to be run separately from the main iptest test > suite. > > https://github.com/ipython/ipython/pull/252 > > Does anyone have any insight on whether this is something our test suite is > doing, something nose is doing, something that can be fixed, or something that > we have to work around? There is no code in IPython or nose itself that does this. Can you make sure that nose is not picking up any extra plugins (print out sys.modules inside of a test suite)? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Tue Feb 1 14:33:05 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 1 Feb 2011 19:33:05 +0000 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On 1 February 2011 19:05, Brian Granger wrote: > * Simplicity. __html__ is easier to remember that __repr_html__. I > want writing these methods to be like eating candy. > I suppose the key question is: is it reasonable to expect that none of these names will have another use? I can't off the top of my head think of any reason that __html__ would produce anything but an HTML representation, and likewise for __svg__ and __png__, but it's worth thinking about. Hmm, actually... a tool which is building, say, an SVG document may have methods on the components which produce fragments (SVG which is valid within an SVG document, but isn't a document by itself). I've seen this with an svg() method, although not with an __svg__() method. Can the display code handle such fragments? How does it fall over if it attempts to display invalid svg/html/etc.? For collecting all possible repr-like methods, there could be an informal standard to put them into something like __repr_methods__. But unless it has a real world role, library writers will simply ignore it. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Tue Feb 1 14:49:44 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 11:49:44 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On 2/1/11 10:36 AM, Fernando Perez wrote: > Hi folks, > > I was just thinking, before the new display infrastructure starts > getting used too much, should we adjust our convention to use > __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of > __html__, __svg__, etc? My concern is two-fold: > > - the more generic names may be already in use by something else that > doesn't quite match our 'protocol', This is a pretty big problem particularly since there is currently no error recovery around the method call. There is a lot of room for error here. For example, the PlainTextFormatter, which uses pretty.py, specifies a print_method='__pretty__'. However, the __pretty__ method has a distinctly different signature that takes arguments. Fortunately, the __call__ was overridden to ignore the print_method and just use the pretty.py machinery directly. These kinds of mistakes are easy to make, and it bakes in a certain amount of brittleness into the system. > or python may in the future > (though this is probably needless paranoia) choose to add them. Python reserves __foo__ names for itself and requests that third party packages do not make their own. Third party packages violate this pretty frequently, but that doesn't mean it's kosher. SAGE does something sensible and uses single underscores on either side. Personally, I don't want to ever change one of my classes to get it printed nicely for IPython. That's an inversion of responsibilities that I find difficult to swallow (especially for HTML and JSON that are best done using particular libraries). Instead, I want the to write formatter functions and register them for the appropriate class. Maybe a conventional print_method could be a default. In any case, the priority as implemented is inverted; we should check for the print_method *after* the registered functions are checked. I, the IPython user, should always be able to override what the class author thought was a good idea at the time. Or who implemented an __html__ method for a different system and a different calling convention. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Tue Feb 1 16:31:02 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 1 Feb 2011 13:31:02 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: Robert, On Tue, Feb 1, 2011 at 11:49 AM, Robert Kern wrote: > On 2/1/11 10:36 AM, Fernando Perez wrote: >> Hi folks, >> >> I was just thinking, before the new display infrastructure starts >> getting used too much, should we adjust our convention to use >> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of >> __html__, __svg__, etc? ?My concern is two-fold: >> >> - the more generic names may be already in use by something else that >> doesn't quite match our 'protocol', > > This is a pretty big problem particularly since there is currently no error > recovery around the method call. There is a lot of room for error here. For > example, the PlainTextFormatter, which uses pretty.py, specifies a > print_method='__pretty__'. However, the __pretty__ method has a distinctly > different signature that takes arguments. Fortunately, the __call__ was > overridden to ignore the print_method and just use the pretty.py machinery > directly. These kinds of mistakes are easy to make, and it bakes in a certain > amount of brittleness into the system. Yes, there is definitely some brittleness right now in the design. We should definitely add additional errors checking in appropriate places. >> or python may in the future >> (though this is probably needless paranoia) choose to add them. > > Python reserves __foo__ names for itself and requests that third party packages > do not make their own. Third party packages violate this pretty frequently, but > that doesn't mean it's kosher. SAGE does something sensible and uses single > underscores on either side. I am aware of the official status of the __foo__ names, but it is tempting as i) the chances of Python using these names in minimal and ii) other products do this in a way that is consistent with our usage (the __html__ examples I gave earlier). But in terms of alternate names that don't use the __foo__ notation: _repr_html_ _html_ - problem is that SAGE uses this format with possibly different meaning. _html - problem is that sympy uses this format with possibly different meaning. ??? > Personally, I don't want to ever change one of my classes to get it printed > nicely for IPython. That's an inversion of responsibilities that I find > difficult to swallow (especially for HTML and JSON that are best done using > particular libraries). Instead, I want the to write formatter functions and > register them for the appropriate class. Maybe a conventional print_method could > be a default. > In any case, the priority as implemented is inverted; we should check for the > print_method *after* the registered functions are checked. I, the IPython user, > should always be able to override what the class author thought was a good idea > at the time. Or who implemented an __html__ method for a different system and a > different calling convention. I got the idea to look for the special print methods first and then the print function from IPython.externals.pretty itself. The first thing the pretty printer does is this: if hasattr(obj_class, '__pretty__'): return obj_class.__pretty__(obj, self, cycle) When I implemented the other printers, I wanted to make them as similar to pretty as possible, so I stuck with this ordering of priorities. I don't mind using the other priority order as long as we are consistent and modify pretty to use the same order. Is this what you would recommend Robert? Cheers, Brian > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From alex.gaynor at gmail.com Tue Feb 1 16:32:53 2011 From: alex.gaynor at gmail.com (Alex Gaynor) Date: Tue, 1 Feb 2011 16:32:53 -0500 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 4:31 PM, Brian Granger wrote: > Robert, > > On Tue, Feb 1, 2011 at 11:49 AM, Robert Kern > wrote: > > On 2/1/11 10:36 AM, Fernando Perez wrote: > >> Hi folks, > >> > >> I was just thinking, before the new display infrastructure starts > >> getting used too much, should we adjust our convention to use > >> __repr_html__, __repr_svg__, __repr_*__ as a naming pattern instead of > >> __html__, __svg__, etc? My concern is two-fold: > >> > >> - the more generic names may be already in use by something else that > >> doesn't quite match our 'protocol', > > > > This is a pretty big problem particularly since there is currently no > error > > recovery around the method call. There is a lot of room for error here. > For > > example, the PlainTextFormatter, which uses pretty.py, specifies a > > print_method='__pretty__'. However, the __pretty__ method has a > distinctly > > different signature that takes arguments. Fortunately, the __call__ was > > overridden to ignore the print_method and just use the pretty.py > machinery > > directly. These kinds of mistakes are easy to make, and it bakes in a > certain > > amount of brittleness into the system. > > Yes, there is definitely some brittleness right now in the design. We > should definitely add additional errors checking in appropriate > places. > > >> or python may in the future > >> (though this is probably needless paranoia) choose to add them. > > > > Python reserves __foo__ names for itself and requests that third party > packages > > do not make their own. Third party packages violate this pretty > frequently, but > > that doesn't mean it's kosher. SAGE does something sensible and uses > single > > underscores on either side. > > I am aware of the official status of the __foo__ names, but it is > tempting as i) the chances of Python using these names in minimal and > ii) other products do this in a way that is consistent with our usage > (the __html__ examples I gave earlier). But in terms of alternate > names that don't use the __foo__ notation: > > _repr_html_ > _html_ - problem is that SAGE uses this format with possibly different > meaning. > _html - problem is that sympy uses this format with possibly > different meaning. > ??? > > > Personally, I don't want to ever change one of my classes to get it > printed > > nicely for IPython. That's an inversion of responsibilities that I find > > difficult to swallow (especially for HTML and JSON that are best done > using > > particular libraries). Instead, I want the to write formatter functions > and > > register them for the appropriate class. Maybe a conventional > print_method could > > be a default. > > > In any case, the priority as implemented is inverted; we should check for > the > > print_method *after* the registered functions are checked. I, the IPython > user, > > should always be able to override what the class author thought was a > good idea > > at the time. Or who implemented an __html__ method for a different system > and a > > different calling convention. > > I got the idea to look for the special print methods first and then > the print function from IPython.externals.pretty itself. The first > thing the pretty printer does is this: > > if hasattr(obj_class, '__pretty__'): > return obj_class.__pretty__(obj, self, cycle) > > When I implemented the other printers, I wanted to make them as > similar to pretty as possible, so I stuck with this ordering of > priorities. I don't mind using the other priority order as long as we > are consistent and modify pretty to use the same order. Is this what > you would recommend Robert? > > Cheers, > > Brian > > > > -- > > Robert Kern > > > > "I have come to believe that the whole world is an enigma, a harmless > enigma > > that is made terrible by our own mad attempt to interpret it as though > it had > > an underlying truth." > > -- Umberto Eco > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > Be careful with __html__ (Django doesn't support it, I think all the others you named do), it's not really intended to handle HTML formatting of an object, but really just manage whether or not it's supposed to be escaped or not. Alex -- "I disapprove of what you say, but I will defend to the death your right to say it." -- Evelyn Beatrice Hall (summarizing Voltaire) "The people's good is the highest law." -- Cicero -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Feb 1 16:36:42 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 1 Feb 2011 21:36:42 +0000 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: (Apologies, I have a bad habit of hitting reply instead of reply-all, so the last two messages didn't go to the list) If GTK is changing the default encoding, presumably there isn't an easy fix from our point of view? So somehow, we need to compartmentalise the tests which use GTK from the tests which use non-ASCII characters. Is my current strategy (main test suite and unicode tests) sensible, or would it make more sense to include unicode in the main test suite, and have any tests which imported gtk separate? I think the latter could be hard to enforce as new tests are added. Or am I missing some way that the tests can be combined properly? Thomas On 1 February 2011 20:20, Robert Kern wrote: > GTK is the culprit. > > Robert Kern > On Feb 1, 2011 11:54 AM, "Thomas Kluyver" wrote: > > It has what I assume to be the standard plugins shipped with nose. The > > following are listed in sys.modules: > > > > allmodules, attrib, base, builtin, capture, collect, cover, cPickle, > > cStringIO, debug, deprecated, doctest, doctests, errorclass, > failuredetail, > > hotshot, inspect, isolate, logcapture, logging, manager, multiprocess, > nose, > > optparse, os, pdb, pickle, pkg_resources, plugintest, prof, Queue, re, > skip, > > StringIO, sys, tempfile, testid, textwrap, threading, time, traceback, > > unittest, warnings, xml, xunit > > > > They are all either showing a path within the virtualenv, or the module > > itself is None. > > > > However, the various imports in the test suite have brought in over 1000 > > modules. Could something like Zope or GTK be the offending party (just > > scanning down the list)? > > > > Thomas > > > > On 1 February 2011 19:28, Robert Kern wrote: > > > >> On 2/1/11 4:02 AM, Thomas Kluyver wrote: > >> > While testing the use of non-ascii characters in 0.11 recently, I > found > >> that > >> > there are things which will fail while actually using ipython, but > cannot > >> be > >> > made to fail in the test suite. Investigating, I found that for the > test > >> suite, > >> > the default encoding is somehow set to utf-8 (ascii is the default > >> default > >> > encoding for Python 2). I don't know quite what's doing this, but it > did > >> the > >> > same with my system version of nose (0.11.1 in Ubuntu) and with nose > >> 1.0.0 > >> > installed in the virtualenv. > >> > > >> > At present, I've worked round that by adding an entirely separate > >> > unicode_tests.py file, which has to be run separately from the main > >> iptest test > >> > suite. > >> > > >> > https://github.com/ipython/ipython/pull/252 > >> > > >> > Does anyone have any insight on whether this is something our test > suite > >> is > >> > doing, something nose is doing, something that can be fixed, or > something > >> that > >> > we have to work around? > >> > >> There is no code in IPython or nose itself that does this. Can you make > >> sure > >> that nose is not picking up any extra plugins (print out sys.modules > inside > >> of a > >> test suite)? > >> > >> -- > >> Robert Kern > >> > >> "I have come to believe that the whole world is an enigma, a harmless > >> enigma > >> that is made terrible by our own mad attempt to interpret it as though > it > >> had > >> an underlying truth." > >> -- Umberto Eco > >> > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Tue Feb 1 17:24:40 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 14:24:40 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On 2/1/11 1:32 PM, Alex Gaynor wrote: > Be careful with __html__ (Django doesn't support it, I think all the others you > named do), it's not really intended to handle HTML formatting of an object, but > really just manage whether or not it's supposed to be escaped or not. Right. It's a tool for frameworks/template libraries to implement internal SafeString/Markup/whatever objects, not for application/library authors to make HTML representations of their model objects. If a Jinja2 user is viewing a Markup object in IPython, they want to see "Markup(u'

foo

')", not an HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I think this is a good reason to avoid using __html__, at least. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Tue Feb 1 17:33:27 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 14:33:27 -0800 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: On 2/1/11 1:36 PM, Thomas Kluyver wrote: > (Apologies, I have a bad habit of hitting reply instead of reply-all, so the > last two messages didn't go to the list) > > If GTK is changing the default encoding, presumably there isn't an easy fix from > our point of view? So somehow, we need to compartmentalise the tests which use > GTK from the tests which use non-ASCII characters. Is my current strategy (main > test suite and unicode tests) sensible, or would it make more sense to include > unicode in the main test suite, and have any tests which imported gtk separate? > I think the latter could be hard to enforce as new tests are added. > > Or am I missing some way that the tests can be combined properly? The GTK tests may need to be separated. At startup, iptest currently will attempt to import gtk just to test if it exists. Perhaps it ought to exclude the GTK tests unless if they are explicitly requested. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Tue Feb 1 19:21:18 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 1 Feb 2011 16:21:18 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 2:24 PM, Robert Kern wrote: > On 2/1/11 1:32 PM, Alex Gaynor wrote: > >> Be careful with __html__ (Django doesn't support it, I think all the others you >> named do), it's not really intended to handle HTML formatting of an object, but >> really just manage whether or not it's supposed to be escaped or not. > > Right. It's a tool for frameworks/template libraries to implement internal > SafeString/Markup/whatever objects, not for application/library authors to make > HTML representations of their model objects. If a Jinja2 user is viewing a > Markup object in IPython, they want to see "Markup(u'

foo

')", not an > HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I think > this is a good reason to avoid using __html__, at least. OK, this is very useful information. Let's find a different naming convention for these. What do people prefer? Also, Robert, do you think we need to change the priority of the special method/print function logic? Brian > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From benjaminrk at gmail.com Tue Feb 1 20:13:16 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 1 Feb 2011 17:13:16 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: > On Tue, Feb 1, 2011 at 2:24 PM, Robert Kern wrote: > > On 2/1/11 1:32 PM, Alex Gaynor wrote: > > > >> Be careful with __html__ (Django doesn't support it, I think all the > others you > >> named do), it's not really intended to handle HTML formatting of an > object, but > >> really just manage whether or not it's supposed to be escaped or not. > > > > Right. It's a tool for frameworks/template libraries to implement > internal > > SafeString/Markup/whatever objects, not for application/library authors > to make > > HTML representations of their model objects. If a Jinja2 user is viewing > a > > Markup object in IPython, they want to see "Markup(u'

foo

')", not > an > > HTML-formatted "foo". The HTMLFormatter's behavior is undesirable, and I > think > > this is a good reason to avoid using __html__, at least. > > OK, this is very useful information. Let's find a different naming > convention for these. What do people prefer? > My vote might be for '__htmlrepr__', I like 'htmlrepr' more than 'repr_html' because it's 'html representation' not 'representation html', despite the tab-completion friendliness of the latter pattern. or we could use a mimetypes method that lets developers pick their own methods by returning a dict: def __mimetypes__(self): return { 'text/html' : self.htmlify, 'image/png' : lambda self: self.as_image(type='png'), } > > Also, Robert, do you think we need to change the priority of the > special method/print function logic? > > Brian > > > -- > > Robert Kern > > > > "I have come to believe that the whole world is an enigma, a harmless > enigma > > that is made terrible by our own mad attempt to interpret it as though > it had > > an underlying truth." > > -- Umberto Eco > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pivanov314 at gmail.com Tue Feb 1 21:05:31 2011 From: pivanov314 at gmail.com (Paul Ivanov) Date: Tue, 1 Feb 2011 18:05:31 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: <20110202020531.GC423@ykcyc> MinRK, on 2011-02-01 17:13, wrote: > On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: > > Let's find a different naming convention for these. What do > > people prefer? > > My vote might be for '__htmlrepr__', > I like 'htmlrepr' more than 'repr_html' because it's 'html representation' > not 'representation html', despite the tab-completion friendliness of the > latter pattern. I'd vote the other way, since obj.__repr_ is a lot nicer to use than writing a one-liner to get everything with a 'repr' substring to see in what manner you can represent a given object. Also, I don't think I'm alone in poking around an object I don't know much about using just obj., and the __repr_html__ convention would keep all of the reprs listed next to each other, instead of sprinkled about. I read __repr_html__ as "representation: html", or "represented as html", which makes sense from a course-to-fine / general-to-specific type of convention. Consider how annoying it would be if get_foo and set_foo methods of some object swapped the order of the verb and the noun and you couldn't easily figure out what all the things you could get and set were. The same goes for IPython.iplib.InteractiveShell.magic_*, which is easier to wrap one's head around than .*_magic best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From pivanov314 at gmail.com Tue Feb 1 21:05:31 2011 From: pivanov314 at gmail.com (Paul Ivanov) Date: Tue, 1 Feb 2011 18:05:31 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: <20110202020531.GC423@ykcyc> MinRK, on 2011-02-01 17:13, wrote: > On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: > > Let's find a different naming convention for these. What do > > people prefer? > > My vote might be for '__htmlrepr__', > I like 'htmlrepr' more than 'repr_html' because it's 'html representation' > not 'representation html', despite the tab-completion friendliness of the > latter pattern. I'd vote the other way, since obj.__repr_ is a lot nicer to use than writing a one-liner to get everything with a 'repr' substring to see in what manner you can represent a given object. Also, I don't think I'm alone in poking around an object I don't know much about using just obj., and the __repr_html__ convention would keep all of the reprs listed next to each other, instead of sprinkled about. I read __repr_html__ as "representation: html", or "represented as html", which makes sense from a course-to-fine / general-to-specific type of convention. Consider how annoying it would be if get_foo and set_foo methods of some object swapped the order of the verb and the noun and you couldn't easily figure out what all the things you could get and set were. The same goes for IPython.iplib.InteractiveShell.magic_*, which is easier to wrap one's head around than .*_magic best, -- Paul Ivanov 314 address only used for lists, off-list direct email at: http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 -------------- next part -------------- A non-text attachment was scrubbed... Name: signature.asc Type: application/pgp-signature Size: 198 bytes Desc: Digital signature URL: From benjaminrk at gmail.com Tue Feb 1 21:16:52 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 1 Feb 2011 18:16:52 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: <20110202020531.GC423@ykcyc> References: <20110202020531.GC423@ykcyc> Message-ID: On Tue, Feb 1, 2011 at 18:05, Paul Ivanov wrote: > MinRK, on 2011-02-01 17:13, wrote: > > On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: > > > Let's find a different naming convention for these. What do > > > people prefer? > > > > My vote might be for '__htmlrepr__', > > I like 'htmlrepr' more than 'repr_html' because it's 'html > representation' > > not 'representation html', despite the tab-completion friendliness of the > > latter pattern. > > I'd vote the other way, since obj.__repr_ is a lot nicer to use > than writing a one-liner to get everything with a 'repr' > substring to see in what manner you can represent a given object. > Also, I don't think I'm alone in poking around an object I don't > know much about using just obj., and the __repr_html__ > convention would keep all of the reprs listed next to each other, > instead of sprinkled about. > > I read __repr_html__ as "representation: html", or "represented > as html", which makes sense from a course-to-fine / > general-to-specific type of convention. Consider how annoying it > would be if get_foo and set_foo methods of some object swapped > the order of the verb and the noun and you couldn't easily figure > out what all the things you could get and set were. The same goes > for IPython.iplib.InteractiveShell.magic_*, which is easier to > wrap one's head around than .*_magic > Well put, I'm convinced. I still don't like having underscores in the middle of __x_y__ method names. I think it's ugly, but I also think PEP8 is terrible, so maybe ignore me on that point. An alternative to developers writing methods in their classes would be for the Display system to use its own lookup table for representation methods, and provide an extension API for modules, IPython, third-parties, or users to register additional methods for classes. This would be more elaborate, but would make the display system a better Python citizen, and would eliminate the possiblity of collision with other projects' private conventions. -MinRK > > best, > -- > Paul Ivanov > 314 address only used for lists, off-list direct email at: > http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+ > URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL > =QE5n > -----END PGP SIGNATURE----- > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Tue Feb 1 21:16:52 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 1 Feb 2011 18:16:52 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: <20110202020531.GC423@ykcyc> References: <20110202020531.GC423@ykcyc> Message-ID: On Tue, Feb 1, 2011 at 18:05, Paul Ivanov wrote: > MinRK, on 2011-02-01 17:13, wrote: > > On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: > > > Let's find a different naming convention for these. What do > > > people prefer? > > > > My vote might be for '__htmlrepr__', > > I like 'htmlrepr' more than 'repr_html' because it's 'html > representation' > > not 'representation html', despite the tab-completion friendliness of the > > latter pattern. > > I'd vote the other way, since obj.__repr_ is a lot nicer to use > than writing a one-liner to get everything with a 'repr' > substring to see in what manner you can represent a given object. > Also, I don't think I'm alone in poking around an object I don't > know much about using just obj., and the __repr_html__ > convention would keep all of the reprs listed next to each other, > instead of sprinkled about. > > I read __repr_html__ as "representation: html", or "represented > as html", which makes sense from a course-to-fine / > general-to-specific type of convention. Consider how annoying it > would be if get_foo and set_foo methods of some object swapped > the order of the verb and the noun and you couldn't easily figure > out what all the things you could get and set were. The same goes > for IPython.iplib.InteractiveShell.magic_*, which is easier to > wrap one's head around than .*_magic > Well put, I'm convinced. I still don't like having underscores in the middle of __x_y__ method names. I think it's ugly, but I also think PEP8 is terrible, so maybe ignore me on that point. An alternative to developers writing methods in their classes would be for the Display system to use its own lookup table for representation methods, and provide an extension API for modules, IPython, third-parties, or users to register additional methods for classes. This would be more elaborate, but would make the display system a better Python citizen, and would eliminate the possiblity of collision with other projects' private conventions. -MinRK > > best, > -- > Paul Ivanov > 314 address only used for lists, off-list direct email at: > http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+ > URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL > =QE5n > -----END PGP SIGNATURE----- > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Feb 1 21:21:43 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 1 Feb 2011 18:21:43 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: <20110202020531.GC423@ykcyc> References: <20110202020531.GC423@ykcyc> Message-ID: On Tue, Feb 1, 2011 at 6:05 PM, Paul Ivanov wrote: > MinRK, on 2011-02-01 17:13, ?wrote: >> On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: >> > Let's find a different naming convention for these. ?What do >> > people prefer? >> >> My vote might be for '__htmlrepr__', >> I like 'htmlrepr' more than 'repr_html' because it's 'html representation' >> not 'representation html', despite the tab-completion friendliness of the >> latter pattern. > > I'd vote the other way, since obj.__repr_ is a lot nicer to use > than writing a one-liner to get everything with a 'repr' > substring to see in what manner you can represent a given object. > Also, I don't think I'm alone in poking around an object I don't > know much about using just obj., and the __repr_html__ > convention would keep all of the reprs listed next to each other, > instead of sprinkled about. I agree, but do we want to avoid the double underscores, so it would be: _repr_html_ _repr_svg_ ... > I read __repr_html__ as "representation: html", or "represented > as html", which makes sense from a course-to-fine / > general-to-specific type of convention. ?Consider how annoying it > would be if get_foo and set_foo methods of some object swapped > the order of the verb and the noun and you couldn't easily figure > out what all the things you could get and set were. The same goes > for IPython.iplib.InteractiveShell.magic_*, which is easier to > wrap one's head around than .*_magic > > best, > -- > Paul Ivanov > 314 address only used for lists, ?off-list direct email at: > http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+ > URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL > =QE5n > -----END PGP SIGNATURE----- > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Tue Feb 1 21:21:43 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 1 Feb 2011 18:21:43 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: <20110202020531.GC423@ykcyc> References: <20110202020531.GC423@ykcyc> Message-ID: On Tue, Feb 1, 2011 at 6:05 PM, Paul Ivanov wrote: > MinRK, on 2011-02-01 17:13, ?wrote: >> On Tue, Feb 1, 2011 at 16:21, Brian Granger wrote: >> > Let's find a different naming convention for these. ?What do >> > people prefer? >> >> My vote might be for '__htmlrepr__', >> I like 'htmlrepr' more than 'repr_html' because it's 'html representation' >> not 'representation html', despite the tab-completion friendliness of the >> latter pattern. > > I'd vote the other way, since obj.__repr_ is a lot nicer to use > than writing a one-liner to get everything with a 'repr' > substring to see in what manner you can represent a given object. > Also, I don't think I'm alone in poking around an object I don't > know much about using just obj., and the __repr_html__ > convention would keep all of the reprs listed next to each other, > instead of sprinkled about. I agree, but do we want to avoid the double underscores, so it would be: _repr_html_ _repr_svg_ ... > I read __repr_html__ as "representation: html", or "represented > as html", which makes sense from a course-to-fine / > general-to-specific type of convention. ?Consider how annoying it > would be if get_foo and set_foo methods of some object swapped > the order of the verb and the noun and you couldn't easily figure > out what all the things you could get and set were. The same goes > for IPython.iplib.InteractiveShell.magic_*, which is easier to > wrap one's head around than .*_magic > > best, > -- > Paul Ivanov > 314 address only used for lists, ?off-list direct email at: > http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7 > > -----BEGIN PGP SIGNATURE----- > Version: GnuPG v1.4.10 (GNU/Linux) > > iEYEARECAAYFAk1Iu9kACgkQe+cmRQ8+KPeVVQCfdgts2XwnsnKosaW1J3Aae7v+ > URMAn2qm+98LxbGpnV+8Bvmr58X3lWrL > =QE5n > -----END PGP SIGNATURE----- > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Wed Feb 2 01:13:31 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 22:13:31 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On 2/1/11 4:21 PM, Brian Granger wrote: > Also, Robert, do you think we need to change the priority of the > special method/print function logic? Yes. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From robert.kern at gmail.com Wed Feb 2 01:18:30 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 01 Feb 2011 22:18:30 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: <20110202020531.GC423@ykcyc> Message-ID: On 2/1/11 6:16 PM, MinRK wrote: > An alternative to developers writing methods in their classes would be for the > Display system to use its own lookup table for representation methods, and > provide an extension API for modules, IPython, third-parties, or users to > register additional methods for classes. This would be more elaborate, but > would make the display system a better Python citizen, and would eliminate the > possiblity of collision with other projects' private conventions. The code already provides this mechanism in addition to the special method, and I think this is the one that should be promoted as the recommended way to do it. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Wed Feb 2 05:08:53 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 2 Feb 2011 10:08:53 +0000 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: Message-ID: On 2 February 2011 01:13, MinRK wrote: > or we could use a mimetypes method that lets developers pick their own > methods by returning a dict: Thinking about it, I quite like this sort of approach, but I think it would make more sense just to have a dict at the attribute, rather than a method returning a dict. It makes it much easier to change on the fly. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From walter at livinglogic.de Wed Feb 2 12:09:45 2011 From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=) Date: Wed, 02 Feb 2011 18:09:45 +0100 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: <20110202020531.GC423@ykcyc> Message-ID: <4D498FD9.8040606@livinglogic.de> On 02.02.11 03:16, MinRK wrote: > [...] > An alternative to developers writing methods in their classes would be > for the Display system to use its own lookup table for representation > methods, and provide an extension API for modules, IPython, > third-parties, or users to register additional methods for classes. > This would be more elaborate, but would make the display system a > better Python citizen, and would eliminate the possiblity of collision > with other projects' private conventions. Why not use a generic function (i.e. http://pypi.python.org/pypi/simplegeneric) for the method registry? Servus, Walter From walter at livinglogic.de Wed Feb 2 12:09:45 2011 From: walter at livinglogic.de (=?UTF-8?B?V2FsdGVyIETDtnJ3YWxk?=) Date: Wed, 02 Feb 2011 18:09:45 +0100 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: References: <20110202020531.GC423@ykcyc> Message-ID: <4D498FD9.8040606@livinglogic.de> On 02.02.11 03:16, MinRK wrote: > [...] > An alternative to developers writing methods in their classes would be > for the Display system to use its own lookup table for representation > methods, and provide an extension API for modules, IPython, > third-parties, or users to register additional methods for classes. > This would be more elaborate, but would make the display system a > better Python citizen, and would eliminate the possiblity of collision > with other projects' private conventions. Why not use a generic function (i.e. http://pypi.python.org/pypi/simplegeneric) for the method registry? Servus, Walter From robert.kern at gmail.com Wed Feb 2 12:41:17 2011 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 02 Feb 2011 09:41:17 -0800 Subject: [IPython-dev] New display methods, should we fine-tune our naming decision? In-Reply-To: <4D498FD9.8040606@livinglogic.de> References: <20110202020531.GC423@ykcyc> <4D498FD9.8040606@livinglogic.de> Message-ID: On 2/2/11 9:09 AM, Walter D?rwald wrote: > On 02.02.11 03:16, MinRK wrote: > >> [...] >> An alternative to developers writing methods in their classes would be >> for the Display system to use its own lookup table for representation >> methods, and provide an extension API for modules, IPython, >> third-parties, or users to register additional methods for classes. >> This would be more elaborate, but would make the display system a >> better Python citizen, and would eliminate the possiblity of collision >> with other projects' private conventions. > > Why not use a generic function (i.e. > http://pypi.python.org/pypi/simplegeneric) for the method registry? Generic functions (as implemented by simplegeneric) are global, and it is difficult to "unregister" functions. The dispatch algorithm in DisplayFormatter is quite similar to that of simplegeneric, though. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From sunqiang at gmail.com Thu Feb 3 07:16:55 2011 From: sunqiang at gmail.com (sunqiang) Date: Thu, 3 Feb 2011 20:16:55 +0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: thanks for the code, it's interesting. On Wed, Feb 2, 2011 at 1:56 AM, Nicolas Rougier wrote: > > > Hi all, > I've made some time ago a kind of 'ascii imshow' function to render a numpy > array directly within the console by exploiting the 256 colors capability of > most modern terminal. I don't know how much interesting it can be for > IPython but I found the code useful for myself. > Code is located > at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py > (GPL license can be replaced by the BSD one if necessary) > > Nicolas > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > From epatters at enthought.com Thu Feb 3 11:41:42 2011 From: epatters at enthought.com (Evan Patterson) Date: Thu, 3 Feb 2011 10:41:42 -0600 Subject: [IPython-dev] PySide support for the Qt frontend Message-ID: Hello all, Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here: https://github.com/epatters/ipython/tree/pyside-support This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher: https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome. Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind. Thanks, Evan From ellisonbg at gmail.com Thu Feb 3 13:49:48 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 3 Feb 2011 10:49:48 -0800 Subject: [IPython-dev] PySide support for the Qt frontend In-Reply-To: References: Message-ID: Evan, Good to hear from you. Hope things are going well. > Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here: > > https://github.com/epatters/ipython/tree/pyside-support This is great! > This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher: > > https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py This is god to know about. Overall, how ready would you say that pyside is? Is Enthought considering shipping pyside in a version of EPD soon? > I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome. > > Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind. No, your stuff should be pretty independent. We may do a rebase, but why won't you do a pull request so everyone can have a look. Thanks for doing this! Cheers, Brian > Thanks, > Evan > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Thu Feb 3 14:42:19 2011 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 03 Feb 2011 11:42:19 -0800 Subject: [IPython-dev] PySide support for the Qt frontend In-Reply-To: References: Message-ID: On 2/3/11 10:49 AM, Brian Granger wrote: > Is Enthought considering shipping pyside in a version of > EPD soon? We have no particular timeline in mind. I'd guess by EPD 7.1, PySide will have a release solid enough to include. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From epatters at enthought.com Thu Feb 3 14:42:51 2011 From: epatters at enthought.com (Evan Patterson) Date: Thu, 3 Feb 2011 13:42:51 -0600 Subject: [IPython-dev] PySide support for the Qt frontend In-Reply-To: References: Message-ID: On Feb 3, 2011, at 12:49 PM, Brian Granger wrote: > Evan, > > Good to hear from you. Hope things are going well. Thanks, Brian. Good to hear from you as well. I'll be working at Enthought for a while, which means that I may do some more work on IPython over the next six months or so. > >> Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here: >> >> https://github.com/epatters/ipython/tree/pyside-support > > This is great! > >> This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher: >> >> https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py > > This is god to know about. Overall, how ready would you say that > pyside is? Is Enthought considering shipping pyside in a version of > EPD soon? PySide is quickly approaching some kind of stability. It looks like an RC 1 release is coming a week or two, with the 1.0 release not too far after. Although the upcoming EPD release (7.0) will not include PySide, as it is being released on the 8th, the subsequent release (7.1) will. At some point, PySide will replace wx as the default toolkit, although Enthought will continue to ship wx with EPD. > >> I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome. >> >> Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind. > > No, your stuff should be pretty independent. We may do a rebase, but > why won't you do a pull request so everyone can have a look. Done. Evan > > Thanks for doing this! > > Cheers, > > Brian > >> Thanks, >> Evan >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com From dsdale24 at gmail.com Fri Feb 4 09:59:52 2011 From: dsdale24 at gmail.com (Darren Dale) Date: Fri, 4 Feb 2011 09:59:52 -0500 Subject: [IPython-dev] ipython-qtconsole and docstrings Message-ID: Hi All, This morning I updated my zmq/pyzmq and tried running ipython-qtconsole from the master branch. I had let my zeromq installation get behind ipython development, and so I'm rediscovering how awesome is the qtconsole. Having docstrings pop up at "(" is great. I noticed though, that depending on the location of the mouse pointer, the docstring may only briefly appear. It seems more likely to occur if the pointer is located inside the region where the docstring will be rendered. I'm running "ipython-qtconsole --pylab" on a Snow Leopard machine, with native (not X-based) Qt provided by MacPorts. Has anyone else noticed something similar? Who is working on the qtconsole? Is there a roadmap? Darren From takowl at gmail.com Fri Feb 4 12:16:51 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Feb 2011 17:16:51 +0000 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: Message-ID: Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to be able to get it uploaded next week. If you're running Debian or Ubuntu, and have a few minutes to spare, please check that the packaged version is working as expected. The x86 version is attached, and the amd64 version is at: http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb Thanks, Thomas ---------- Forwarded message ---------- From: Miguel Landaeta Date: 4 February 2011 14:48 Subject: Re: Debian packaging pyzmq To: Thomas Kluyver On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver wrote: > Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on > how to build one). I think the computer is 64 bit, but I'm sticking with a > 32 bit OS for now. No problem, take a look at the attached package. Cheers, -- Miguel Landaeta, miguel at miguel.cc secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/ "Faith means not wanting to know what is true." -- Nietzsche -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: python-zmq_2.0.10.1-0miguel1_i386.deb Type: application/x-debian-package Size: 280340 bytes Desc: not available URL: From benjaminrk at gmail.com Fri Feb 4 12:19:20 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Feb 2011 09:19:20 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: On Fri, Feb 4, 2011 at 06:59, Darren Dale wrote: > Hi All, > > This morning I updated my zmq/pyzmq and tried running > ipython-qtconsole from the master branch. I had let my zeromq > installation get behind ipython development, and so I'm rediscovering > how awesome is the qtconsole. Having docstrings pop up at "(" is > great. I noticed though, that depending on the location of the mouse > pointer, the docstring may only briefly appear. It seems more likely > to occur if the pointer is located inside the region where the > docstring will be rendered. I'm running "ipython-qtconsole --pylab" on > a Snow Leopard machine, with native (not X-based) Qt provided by > MacPorts. Has anyone else noticed something similar? Yes, this is my experience, also on OSX 10.6.6, with native PyQt from their website. The behavior is inconsistent, but seems more likely if the cursor is hovering over the popup, and also more likely if the popup is large. Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c', because pressing command triggers an event that clears the selection. > > Who is working on the qtconsole? Is there a roadmap? > > Darren > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg at gmail.com Fri Feb 4 12:37:54 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 4 Feb 2011 09:37:54 -0800 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: Message-ID: Thomas, This is great! Thanks Miguel, this is huge! Where is the link to the 2.0.10 of zeromq itself. I will definitely test this out. Cheers, Brian On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver wrote: > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to > be able to get it uploaded next week. If you're running Debian or Ubuntu, > and have a few minutes to spare, please check that the packaged version is > working as expected. > > The x86 version is attached, and the amd64 version is at: > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > > Thanks, > Thomas > > ---------- Forwarded message ---------- > From: Miguel Landaeta > Date: 4 February 2011 14:48 > Subject: Re: Debian packaging pyzmq > To: Thomas Kluyver > > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver wrote: >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on >> how to build one). I think the computer is 64 bit, but I'm sticking with a >> 32 bit OS for now. > > No problem, take a look at the attached package. > Cheers, > > -- > Miguel Landaeta, miguel at miguel.cc > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/ > "Faith means not wanting to know what is true." -- Nietzsche > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From takowl at gmail.com Fri Feb 4 12:49:02 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Feb 2011 17:49:02 +0000 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: Message-ID: Knew there was something I'd forget ;-) http://packages.debian.org/sid/libzmq0 Thomas On 4 February 2011 17:37, Brian Granger wrote: > Thomas, > > This is great! Thanks Miguel, this is huge! Where is the link to the > 2.0.10 of zeromq itself. I will definitely test this out. > > Cheers, > > Brian > > On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver wrote: > > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes > to > > be able to get it uploaded next week. If you're running Debian or Ubuntu, > > and have a few minutes to spare, please check that the packaged version > is > > working as expected. > > > > The x86 version is attached, and the amd64 version is at: > > > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > > > > Thanks, > > Thomas > > > > ---------- Forwarded message ---------- > > From: Miguel Landaeta > > Date: 4 February 2011 14:48 > > Subject: Re: Debian packaging pyzmq > > To: Thomas Kluyver > > > > > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver wrote: > >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers > on > >> how to build one). I think the computer is 64 bit, but I'm sticking with > a > >> 32 bit OS for now. > > > > No problem, take a look at the attached package. > > Cheers, > > > > -- > > Miguel Landaeta, miguel at miguel.cc > > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/ > > "Faith means not wanting to know what is true." -- Nietzsche > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From epatters at enthought.com Fri Feb 4 13:03:50 2011 From: epatters at enthought.com (Evan Patterson) Date: Fri, 4 Feb 2011 12:03:50 -0600 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: On Fri, Feb 4, 2011 at 11:19 AM, MinRK wrote: > On Fri, Feb 4, 2011 at 06:59, Darren Dale wrote: > > Hi All, > > > > This morning I updated my zmq/pyzmq and tried running > > ipython-qtconsole from the master branch. I had let my zeromq > > installation get behind ipython development, and so I'm rediscovering > > how awesome is the qtconsole. Having docstrings pop up at "(" is > > great. I noticed though, that depending on the location of the mouse > > pointer, the docstring may only briefly appear. It seems more likely > > to occur if the pointer is located inside the region where the > > docstring will be rendered. I'm running "ipython-qtconsole --pylab" on > > a Snow Leopard machine, with native (not X-based) Qt provided by > > MacPorts. Has anyone else noticed something similar? > > Yes, this is my experience, also on OSX 10.6.6, with native PyQt from > their website. The behavior is inconsistent, but seems more likely if > the cursor is hovering over the popup, and also more likely if the > popup is large. > This is not so much bug as it is a (mis)feature in the call tip implementation. https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/console/call_tip_widget.py The call tip autohides after a little if the mouse cursor is outside both the console window and the tooltip frame. I did this because I thought it was uncouth for an application to leave a (potentially very large) tooltip up when the mouse was elsewhere. However, this is clearly a usability issue. Perhaps a better solution would be to autohide the tooltip only if the console window loses focus in the window manager? > > Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c', > because pressing command triggers an event that clears the selection. > I'm working on a Mac more regularly now, so I should be able to see this and get it fixed soon. Evan > > > > > Who is working on the qtconsole? Is there a roadmap? > > > > Darren > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Feb 4 13:24:47 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Feb 2011 10:24:47 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: On Fri, Feb 4, 2011 at 10:03, Evan Patterson wrote: > > > On Fri, Feb 4, 2011 at 11:19 AM, MinRK wrote: >> >> On Fri, Feb 4, 2011 at 06:59, Darren Dale wrote: >> > Hi All, >> > >> > This morning I updated my zmq/pyzmq and tried running >> > ipython-qtconsole from the master branch. I had let my zeromq >> > installation get behind ipython development, and so I'm rediscovering >> > how awesome is the qtconsole. Having docstrings pop up at "(" is >> > great. I noticed though, that depending on the location of the mouse >> > pointer, the docstring may only briefly appear. It seems more likely >> > to occur if the pointer is located inside the region where the >> > docstring will be rendered. I'm running "ipython-qtconsole --pylab" on >> > a Snow Leopard machine, with native (not X-based) Qt provided by >> > MacPorts. Has anyone else noticed something similar? >> >> Yes, this is my experience, also on OSX 10.6.6, with native PyQt from >> their website. ?The behavior is inconsistent, but seems more likely if >> the cursor is hovering over the popup, and also more likely if the >> popup is large. > > This is not so much bug as it is a (mis)feature in the call tip > implementation. > > https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/console/call_tip_widget.py > > The call tip autohides after a little if the mouse cursor is outside both > the console window and the tooltip frame. I did this because I thought it > was uncouth for an application to leave a (potentially very large) tooltip > up when the mouse was elsewhere. However, this is clearly a usability issue. > Perhaps a better solution would be to autohide the tooltip only if the > console window loses focus in the window manager? Thanks for the clarification, and that makes sense as a design, but the bug is that this is not the observed behavior. The calltip is incorrectly dismissing itself when the cursor is within the tooltip frame at the time of pop-up. Based on a cursory insertion of 'print' statements, there is a surplus of 'Leave' events triggered, which results in the calltip being inappropriately dismissed while the cursor is over it. -MinRK >> >> Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c', >> because pressing command triggers an event that clears the selection. > > I'm working on a Mac more regularly now, so I should be able to see this and > get it fixed soon. > > Evan >> >> > >> > Who is working on the qtconsole? Is there a roadmap? >> > >> > Darren >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > From mark.voorhies at ucsf.edu Fri Feb 4 15:56:02 2011 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Fri, 4 Feb 2011 12:56:02 -0800 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: Message-ID: <201102041256.03222.mark.voorhies@ucsf.edu> Awesome! I can test on Debian PPC and AMD64 this weekend. Any suggested values of "working as expected" beyond running an iPython qtconsole session? --Mark On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote: > Knew there was something I'd forget ;-) > > http://packages.debian.org/sid/libzmq0 > > Thomas > > On 4 February 2011 17:37, Brian Granger wrote: > > Thomas, > > This is great! Thanks Miguel, this is huge! Where is the link to the > 2.0.10 of zeromq itself. I will definitely test this out. > > Cheers, > > Brian > > > On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver wrote: > > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and hopes to > > be able to get it uploaded next week. If you're running Debian or Ubuntu, > > and have a few minutes to spare, please check that the packaged version is > > working as expected. > > > > The x86 version is attached, and the amd64 version is at: > > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > > > > Thanks, > > Thomas > > > > ---------- Forwarded message ---------- > > From: Miguel Landaeta > > Date: 4 February 2011 14:48 > > Subject: Re: Debian packaging pyzmq > > To: Thomas Kluyver > > > > > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver wrote: > >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some pointers on > >> how to build one). I think the computer is 64 bit, but I'm sticking with a > >> 32 bit OS for now. > > > > No problem, take a look at the attached package. > > Cheers, > > > > -- > > Miguel Landaeta, miguel at miguel.cc > > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/ > > "Faith means not wanting to know what is true." -- Nietzsche > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > From takowl at gmail.com Fri Feb 4 16:08:05 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Feb 2011 21:08:05 +0000 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: <201102041256.03222.mark.voorhies@ucsf.edu> References: <201102041256.03222.mark.voorhies@ucsf.edu> Message-ID: Well, that's a good start. pyzmq people, is there a simple way to run the test suite against a separately installed version of pyzmq? Miguel, is there some standard we're aiming to confirm the package meets for its inclusion? Thomas On 4 February 2011 20:56, Mark Voorhies wrote: > Awesome! > > I can test on Debian PPC and AMD64 this weekend. Any suggested > values of "working as expected" beyond running an iPython qtconsole > session? > > --Mark > > On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote: > > Knew there was something I'd forget ;-) > > > > http://packages.debian.org/sid/libzmq0 > > > > Thomas > > > > On 4 February 2011 17:37, Brian Granger wrote: > > > > Thomas, > > > > This is great! Thanks Miguel, this is huge! Where is the link to the > > 2.0.10 of zeromq itself. I will definitely test this out. > > > > Cheers, > > > > Brian > > > > > > On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver > wrote: > > > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and > hopes to > > > be able to get it uploaded next week. If you're running Debian or > Ubuntu, > > > and have a few minutes to spare, please check that the packaged > version is > > > working as expected. > > > > > > The x86 version is attached, and the amd64 version is at: > > > > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > > > > > > Thanks, > > > Thomas > > > > > > ---------- Forwarded message ---------- > > > From: Miguel Landaeta > > > Date: 4 February 2011 14:48 > > > Subject: Re: Debian packaging pyzmq > > > To: Thomas Kluyver > > > > > > > > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver > wrote: > > >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some > pointers on > > >> how to build one). I think the computer is 64 bit, but I'm sticking > with a > > >> 32 bit OS for now. > > > > > > No problem, take a look at the attached package. > > > Cheers, > > > > > > -- > > > Miguel Landaeta, miguel at miguel.cc > > > secure email with PGP 0x7D8967E9 available at > http://keyserver.pgp.com/ > > > "Faith means not wanting to know what is true." -- Nietzsche > > > > > > > > > > > _______________________________________________ > > > IPython-dev mailing list > > > IPython-dev at scipy.org > > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > > > > > > > -- > > Brian E. Granger, Ph.D. > > Assistant Professor of Physics > > Cal Poly State University, San Luis Obispo > > bgranger at calpoly.edu > > ellisonbg at gmail.com > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Fri Feb 4 17:23:15 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 4 Feb 2011 14:23:15 -0800 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: <201102041256.03222.mark.voorhies@ucsf.edu> Message-ID: On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver wrote: > Well, that's a good start. pyzmq people, is there a simple way to run the > test suite against a separately installed version of pyzmq? With nose, you can just do: nosetests -vvs zmq.tests to run the zmq test suite from anywhere. It shouldn't matter how pyzmq/zeromq are installed, as long as your PATHs are correct. -MinRK > Miguel, is there > some standard we're aiming to confirm the package meets for its inclusion? > > Thomas > > On 4 February 2011 20:56, Mark Voorhies wrote: >> >> Awesome! >> >> I can test on Debian PPC and AMD64 this weekend. ?Any suggested >> values of "working as expected" beyond running an iPython qtconsole >> session? >> >> --Mark >> >> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote: >> > Knew there was something I'd forget ;-) >> > >> > http://packages.debian.org/sid/libzmq0 >> > >> > Thomas >> > >> > On 4 February 2011 17:37, Brian Granger wrote: >> > >> > Thomas, >> > >> > ?This is great! ?Thanks Miguel, this is huge! ?Where is the link to the >> > ?2.0.10 of zeromq itself. ?I will definitely test this out. >> > >> > ?Cheers, >> > >> > ?Brian >> > >> > >> > ?On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver >> > wrote: >> > ?> Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and >> > hopes to >> > ?> be able to get it uploaded next week. If you're running Debian or >> > Ubuntu, >> > ?> and have a few minutes to spare, please check that the packaged >> > version is >> > ?> working as expected. >> > ?> >> > ?> The x86 version is attached, and the amd64 version is at: >> > ?> >> > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb >> > ?> >> > ?> Thanks, >> > ?> Thomas >> > ?> >> > ?> ---------- Forwarded message ---------- >> > ?> From: Miguel Landaeta >> > ?> Date: 4 February 2011 14:48 >> > ?> Subject: Re: Debian packaging pyzmq >> > ?> To: Thomas Kluyver >> > ?> >> > ?> >> > ?> On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver >> > wrote: >> > ?>> Thanks Miguel. I'll need an i386 package, I'm afraid (or some >> > pointers on >> > ?>> how to build one). I think the computer is 64 bit, but I'm sticking >> > with a >> > ?>> 32 bit OS for now. >> > ?> >> > ?> No problem, take a look at the attached package. >> > ?> Cheers, >> > ?> >> > ?> -- >> > ?> Miguel Landaeta, miguel at miguel.cc >> > ?> secure email with PGP 0x7D8967E9 available at >> > http://keyserver.pgp.com/ >> > ?> "Faith means not wanting to know what is true." -- Nietzsche >> > ?> >> > ?> >> > >> > > _______________________________________________ >> > ?> IPython-dev mailing list >> > ?> IPython-dev at scipy.org >> > ?> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > ?> >> > ?> >> > >> > >> > >> > ?-- >> > ?Brian E. Granger, Ph.D. >> > ?Assistant Professor of Physics >> > ?Cal Poly State University, San Luis Obispo >> > ?bgranger at calpoly.edu >> > ?ellisonbg at gmail.com >> > >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > From takowl at gmail.com Fri Feb 4 18:26:56 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 4 Feb 2011 23:26:56 +0000 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: <201102041256.03222.mark.voorhies@ucsf.edu> Message-ID: Well, after a bit of fiddling, I can confirm that the x86 package passes the tests and runs the Qt Console frontend alright. Anyone else attempting to test on Ubuntu: the dependencies specify a debian specific version of Python. To get round this, use: sudo dpkg -i --force-depends-version Once this is uploaded to Debian, I'll put in a sync request for Ubuntu. Thomas On 4 February 2011 22:23, MinRK wrote: > On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver wrote: > > Well, that's a good start. pyzmq people, is there a simple way to run the > > test suite against a separately installed version of pyzmq? > > With nose, you can just do: > > nosetests -vvs zmq.tests > > to run the zmq test suite from anywhere. It shouldn't matter how > pyzmq/zeromq are installed, as long as your PATHs are correct. > > -MinRK > > > Miguel, is there > > some standard we're aiming to confirm the package meets for its > inclusion? > > > > Thomas > > > > On 4 February 2011 20:56, Mark Voorhies wrote: > >> > >> Awesome! > >> > >> I can test on Debian PPC and AMD64 this weekend. Any suggested > >> values of "working as expected" beyond running an iPython qtconsole > >> session? > >> > >> --Mark > >> > >> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote: > >> > Knew there was something I'd forget ;-) > >> > > >> > http://packages.debian.org/sid/libzmq0 > >> > > >> > Thomas > >> > > >> > On 4 February 2011 17:37, Brian Granger wrote: > >> > > >> > Thomas, > >> > > >> > This is great! Thanks Miguel, this is huge! Where is the link to > the > >> > 2.0.10 of zeromq itself. I will definitely test this out. > >> > > >> > Cheers, > >> > > >> > Brian > >> > > >> > > >> > On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver > >> > wrote: > >> > > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and > >> > hopes to > >> > > be able to get it uploaded next week. If you're running Debian or > >> > Ubuntu, > >> > > and have a few minutes to spare, please check that the packaged > >> > version is > >> > > working as expected. > >> > > > >> > > The x86 version is attached, and the amd64 version is at: > >> > > > >> > > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > >> > > > >> > > Thanks, > >> > > Thomas > >> > > > >> > > ---------- Forwarded message ---------- > >> > > From: Miguel Landaeta > >> > > Date: 4 February 2011 14:48 > >> > > Subject: Re: Debian packaging pyzmq > >> > > To: Thomas Kluyver > >> > > > >> > > > >> > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver > >> > wrote: > >> > >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some > >> > pointers on > >> > >> how to build one). I think the computer is 64 bit, but I'm > sticking > >> > with a > >> > >> 32 bit OS for now. > >> > > > >> > > No problem, take a look at the attached package. > >> > > Cheers, > >> > > > >> > > -- > >> > > Miguel Landaeta, miguel at miguel.cc > >> > > secure email with PGP 0x7D8967E9 available at > >> > http://keyserver.pgp.com/ > >> > > "Faith means not wanting to know what is true." -- Nietzsche > >> > > > >> > > > >> > > >> > > _______________________________________________ > >> > > IPython-dev mailing list > >> > > IPython-dev at scipy.org > >> > > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Brian E. Granger, Ph.D. > >> > Assistant Professor of Physics > >> > Cal Poly State University, San Luis Obispo > >> > bgranger at calpoly.edu > >> > ellisonbg at gmail.com > >> > > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sat Feb 5 13:23:28 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 5 Feb 2011 18:23:28 +0000 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: On 1 February 2011 22:33, Robert Kern wrote: > The GTK tests may need to be separated. At startup, iptest currently will > attempt to import gtk just to test if it exists. Perhaps it ought to > exclude the > GTK tests unless if they are explicitly requested. > What's the best way to deal with this? What tests use gtk? If we're going to need two separate runs (one for gtk, one for Unicode), it seems to me simpler to have two test scripts, rather than adding a command line switch to determine which tests will be run. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sun Feb 6 11:00:28 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 6 Feb 2011 16:00:28 +0000 Subject: [IPython-dev] Selecting lines from history Message-ID: With the new history system, the history from previous sessions is stored in the same lists as the history from the current session (input_history_raw, input_history_parsed). So, when selecting lines with magic commands %save/%edit/%macro, the numbers didn't match up to the prompt numbers in the current session (except when you start with no history). The approach I've taken to dealing with this is to add a session_offset, which is used by these cases to translate session line numbers (starting from 1) to indices in the history lists. This has the added benefit that, using a default offset of -1, we don't need a blank item at the top of the list to use 1-based indexing. In principle, you can select lines from previous sessions using negative numbers; in practice, the - is interpreted to indicate options, and the command fails. https://github.com/ipython/ipython/pull/261 Other possible approaches I can think of: - Use a dict instead of a list, with prompt numbers as keys. Previous sessions could be stored with another form of keys, perhaps tuples (-1, 1) (=first line of last session), and if desirable, we could come up with some syntax to let magic commands access these. Separating sessions would also allow a command line option to repeat the commands saved from the previous session, e.g. after a power failure. - Previous sessions go into a separate list from the current session. Again, some syntax could be devised to allow magic commands to access them. - Only load previous sessions for readline; don't allow access from magic commands until they're re-used in the current session. I believe this is what we did before the new history system. What was the rationale behind saving and reloading the history as we now do? Do we want to be able to do something like "%edit x1-x12"? That seems to have limited value when you can't immediately see the lines you're selecting (although of course the %hist command could have options to show them). Is there another reason we store history that I've overlooked? There's also the mysterious "shadow history" system, which I'm not entirely clear about the reasons for. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Sun Feb 6 15:47:55 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Sun, 6 Feb 2011 15:47:55 -0500 Subject: [IPython-dev] newparallel Message-ID: hi min, brian and fernando, i just checked out the newparallel branch from the github repo. a few questions: 1. is this the best place to look at documentation? http://minrk.github.com/ipython/parallelz/index.html (unfortunately, i could not compile the docs on my 64-bit osx machine without wx.) 2. is dag support already available in newparallel? i.e., can i add tasks with dependencies before executing them? if not, where is the best place for hacking this in? 3. fernando: is your workflow branch mergeable with this? the ability to spawn and shutdown engines as needed (especially in the context of clusters such as SGE) has come up for discussion on the nipype side a fair bit in the last few weeks. 4. is there a good place to read about the parallel architecture? cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun Feb 6 18:42:40 2011 From: benjaminrk at gmail.com (MinRK) Date: Sun, 6 Feb 2011 15:42:40 -0800 Subject: [IPython-dev] newparallel In-Reply-To: References: Message-ID: On Sun, Feb 6, 2011 at 12:47, Satrajit Ghosh wrote: > hi min, brian and fernando, > > i just checked out the newparallel branch from the github repo. a few > questions: > > 1. is this the best place to look at documentation? > > http://minrk.github.com/ipython/parallelz/index.html Yes, but I only just started updating those to match the newparallel code, so they aren't fully up to date. > > (unfortunately, i could not compile the docs on my 64-bit osx machine > without wx.) wxpython 2.9 now supports cocoa, but if you want to use it with the System default Python, you need to build from source (which takes ages). > > 2. is dag support already available in newparallel? i.e., can i add tasks > with dependencies before executing them? if not, where is the best place for > hacking this in? Yes, you can implement a DAG of dependencies, as seen here: https://github.com/ipython/ipython/blob/newparallel/examples/demo/dag/dagdeps.py. Note that you must use one of the Python schedulers (via '--scheme lru' to ipcontrollerz), because the default pure-zmq scheduler ignores dependencies. There is probably some more work to go into Dependencies, specifically regarding how to interpret job failures and unreachable DAG nodes. > > 3. fernando: is your workflow branch mergeable with this? the ability to > spawn and shutdown engines as needed (especially in the context of clusters > such as SGE) has come up for discussion on the nipype side a fair bit in the > last few weeks. The last commit of that branch has not been merged yet, but over the last week I have updated the newparallel scripts to use the IPython config system, so starting an engine is no longer the same as represented in those scripts, but it should be easy to update them. > > 4. is there a good place to read about the parallel architecture? The parallel messaging/connections documents are the best description we have of how the underlying architecture works, but they still need to be fleshed out: http://minrk.github.com/ipython/development/parallel_messages.html -MinRK > > cheers, > > satra > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > From robert.kern at gmail.com Sun Feb 6 20:01:39 2011 From: robert.kern at gmail.com (Robert Kern) Date: Sun, 06 Feb 2011 19:01:39 -0600 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: On 2011-02-05 12:23 , Thomas Kluyver wrote: > On 1 February 2011 22:33, Robert Kern > wrote: > > The GTK tests may need to be separated. At startup, iptest currently will > attempt to import gtk just to test if it exists. Perhaps it ought to exclude the > GTK tests unless if they are explicitly requested. > > > What's the best way to deal with this? What tests use gtk? I don't think there are any tests that use gtk. iptest just checks for gtk to determine whether to add IPython/lib/inputhookgtk.py to the list of module to exclude from the test suite. This can just be made unconditional and remove the test for gtk and gobject. Try the attached patch. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco -------------- next part -------------- An embedded and charset-unspecified text was scrubbed... Name: iptest-ungtk.diff URL: From mark.voorhies at ucsf.edu Sun Feb 6 23:45:24 2011 From: mark.voorhies at ucsf.edu (Mark Voorhies) Date: Sun, 6 Feb 2011 20:45:24 -0800 Subject: [IPython-dev] Fwd: Debian packaging pyzmq In-Reply-To: References: Message-ID: <201102062045.25082.mark.voorhies@ucsf.edu> I can confirm that the source package works when built for PPC. -- Built and tested on a PowerBook G4 15", freshly upgraded to Debian Squeeze. All dependencies were installed from the squeeze respository, except: libzmq-dev 2.0.10-1 from sid cython and cython-dbg 0.13.-1 from experimental Downloaded Miguel's source package from alioth, built with "debuild -us -uc", and installed with "dpkg --install python-zmq_2.0.10.1-0miguel1_powerpc.deb" nosetests -vvs zmq.tests passes all tests, with one skipped due to no python 2.7: test messages from memoryview (only valid for python >= 2.7) Built iPython from master (2ccc90d68) and tested with: ./ipython-qtconsole --pylab --paging hsplit %guiref plot(rand(100)) display(*getfigs()) --> works =) Holding off on testing on my AMD64 machine until it's upgraded to Squeeze (probably later this week). --Mark On Friday, February 04, 2011 03:26:56 pm Thomas Kluyver wrote: > Well, after a bit of fiddling, I can confirm that the x86 package passes the tests and runs the Qt Console frontend alright. > > Anyone else attempting to test on Ubuntu: the dependencies specify a debian specific version of Python. To get round this, use: > sudo dpkg -i --force-depends-version > > Once this is uploaded to Debian, I'll put in a sync request for Ubuntu. > > Thomas > > On 4 February 2011 22:23, MinRK wrote: > > On Fri, Feb 4, 2011 at 13:08, Thomas Kluyver wrote: > > Well, that's a good start. pyzmq people, is there a simple way to run the > > test suite against a separately installed version of pyzmq? > > > With nose, you can just do: > > nosetests -vvs zmq.tests > > to run the zmq test suite from anywhere. It shouldn't matter how > pyzmq/zeromq are installed, as long as your PATHs are correct. > > -MinRK > > > > Miguel, is there > > some standard we're aiming to confirm the package meets for its inclusion? > > > > Thomas > > > > On 4 February 2011 20:56, Mark Voorhies wrote: > >> > >> Awesome! > >> > >> I can test on Debian PPC and AMD64 this weekend. Any suggested > >> values of "working as expected" beyond running an iPython qtconsole > >> session? > >> > >> --Mark > >> > >> On Friday, February 04, 2011 09:49:02 am Thomas Kluyver wrote: > >> > Knew there was something I'd forget ;-) > >> > > >> > http://packages.debian.org/sid/libzmq0 > >> > > >> > Thomas > >> > > >> > On 4 February 2011 17:37, Brian Granger wrote: > >> > > >> > Thomas, > >> > > >> > This is great! Thanks Miguel, this is huge! Where is the link to the > >> > 2.0.10 of zeromq itself. I will definitely test this out. > >> > > >> > Cheers, > >> > > >> > Brian > >> > > >> > > >> > On Fri, Feb 4, 2011 at 9:16 AM, Thomas Kluyver > >> > wrote: > >> > > Miguel Landata has kindly packaged pyzmq 2.0.10.1 for Debian, and > >> > hopes to > >> > > be able to get it uploaded next week. If you're running Debian or > >> > Ubuntu, > >> > > and have a few minutes to spare, please check that the packaged > >> > version is > >> > > working as expected. > >> > > > >> > > The x86 version is attached, and the amd64 version is at: > >> > > > >> > http://alioth.debian.org/~nomadium-guest/debian/unstable/python-zmq_2.0.10.1-0miguel1_amd64.deb > >> > > > >> > > Thanks, > >> > > Thomas > >> > > > >> > > ---------- Forwarded message ---------- > >> > > From: Miguel Landaeta > >> > > Date: 4 February 2011 14:48 > >> > > Subject: Re: Debian packaging pyzmq > >> > > To: Thomas Kluyver > >> > > > >> > > > >> > > On Thu, Feb 3, 2011 at 6:30 PM, Thomas Kluyver > >> > wrote: > >> > >> Thanks Miguel. I'll need an i386 package, I'm afraid (or some > >> > pointers on > >> > >> how to build one). I think the computer is 64 bit, but I'm sticking > >> > with a > >> > >> 32 bit OS for now. > >> > > > >> > > No problem, take a look at the attached package. > >> > > Cheers, > >> > > > >> > > -- > >> > > Miguel Landaeta, miguel at miguel.cc > >> > > secure email with PGP 0x7D8967E9 available at > >> > http://keyserver.pgp.com/ > >> > > "Faith means not wanting to know what is true." -- Nietzsche > >> > > > >> > > > >> > > >> > > _______________________________________________ > >> > > IPython-dev mailing list > >> > > IPython-dev at scipy.org > >> > > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > > >> > > > >> > > >> > > >> > > >> > -- > >> > Brian E. Granger, Ph.D. > >> > Assistant Professor of Physics > >> > Cal Poly State University, San Luis Obispo > >> > bgranger at calpoly.edu > >> > ellisonbg at gmail.com > >> > > >> _______________________________________________ > >> IPython-dev mailing list > >> IPython-dev at scipy.org > >> http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > From takowl at gmail.com Mon Feb 7 09:40:06 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 7 Feb 2011 14:40:06 +0000 Subject: [IPython-dev] Unicode in tests In-Reply-To: References: Message-ID: Thanks, Robert, that did the trick. I've moved the unicode tests into the main test suite, and checked that each one fails if I undo the corresponding fix. It also caused a failure in the unicode test for inputsplitter, which I've resolved. This has all been added to the pull request. https://github.com/ipython/ipython/pull/252 Thomas On 7 February 2011 01:01, Robert Kern wrote: > On 2011-02-05 12:23 , Thomas Kluyver wrote: > >> On 1 February 2011 22:33, Robert Kern > > wrote: >> >> The GTK tests may need to be separated. At startup, iptest currently >> will >> attempt to import gtk just to test if it exists. Perhaps it ought to >> exclude the >> GTK tests unless if they are explicitly requested. >> >> >> What's the best way to deal with this? What tests use gtk? >> > > I don't think there are any tests that use gtk. iptest just checks for gtk > to determine whether to add IPython/lib/inputhookgtk.py to the list of > module to exclude from the test suite. This can just be made unconditional > and remove the test for gtk and gobject. Try the attached patch. > > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From source at sharpsteen.net Mon Feb 7 21:52:23 2011 From: source at sharpsteen.net (Charlie Sharpsteen) Date: Mon, 7 Feb 2011 18:52:23 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets Message-ID: Hello IPython developers, I was playing around with the new Qt widgets in 0.11-dev the other day and was very impressed. I tried throwing together a quick plugin to add an IPython console to the Qt-based program Quantum GIS [QGIS][1]. Creating the plugin was easy enough as QGIS runs an embedded Python interpreter and exposes a lot of PyQt4 functionality. However, I noticed a couple of things and was wondering if anyone on this list had any comments to share: - The shell is based on a stripped down version of `ipython-qtconsole` and works but is very, very slow for some reason. The is a delay of a couple of seconds between when an action is triggered and a response comes back to the terminal. The only guess I have that may explain this behavior is that QGIS is a C++ application that embeds a Python interpreter whereas ipython-qtconsole is a pure PyQt4 program. Perhaps QGIS is not letting the Python threads run as often as a pure PyQt4 program would. This could be a QGIS-specific problem, but I was wondering if anyone else had the chance to run the IPython Qt widget in an embedded interpreter and noticed similar results? - Currently, the only IPython kernel available for use with the Qt widget runs the IPython interpreter in a separate process. For QGIS, it would be very nice to have the option of running an IPython kernel inside the same process as the Qt widgets. IPython would then have direct access to the QGIS interface and would be able to inspect and operatie on the PyQt4 objects that compose the GUI providing a powerful tool for users performing spatial analysis and developers building plugins. Are there any plans to develop IPython kernels that do not run in separate processes? I spent a few days trying to build one myself by wrapping an InteractiveShell, but I feel like I am flailing a little bit when it comes to creating a kernel_manager that will integrate with the Qt frontend. If anyone has any design advice on how to implement an in-process kernel I would love to hear it. The plugin is available on [GitHub][2] under the tag `0.1` if anyone feels interested in checking it out as an IPython use case. The 0.1 version is really quite simple---about 5-10 lines in console.py that do the actual implementation. To use: - Install QGIS - Clone project as a sub-directory of ~/.qgis/python/plugins/ - Run `make` (runs PyQt4 dev tools on ui and qrc files) - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu Keep up the good work on IPython development! -Charlie [1]: http://www.qgis.org [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1 -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Mon Feb 7 23:15:38 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 7 Feb 2011 20:15:38 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: Charlie, > I was playing around with the new Qt ?widgets in 0.11-dev the other day and > was very impressed. ?I tried?throwing?together a quick plugin to add an > IPython console to the Qt-based program Quantum GIS [QGIS][1]. ?Creating the > plugin was easy enough as QGIS runs an embedded Python interpreter and > exposes a lot of PyQt4 functionality.??However, I noticed a couple of things > and was wondering if anyone on this list had any comments to share: > > - The shell is based on a stripped down version of `ipython-qtconsole` and > works but is very, very slow for some reason. ?The is a delay of a couple of > seconds between when an action is triggered and a response comes back to the > terminal. ?The only guess I have that may explain this behavior is that QGIS > is a C++ application that embeds a Python interpreter whereas > ipython-qtconsole is a pure PyQt4 program. ?Perhaps QGIS is not letting the > Python threads run as often as a pure PyQt4 program would. > This could be a QGIS-specific problem, but I was wondering if anyone else > had the chance to run the IPython Qt widget in an embedded interpreter and > noticed similar results? In general the ipython-qtconsole should be quite fast. But for that to be the case, the qt event loop has to be running. If the C++ is not letting the qt event loop run often enough, you would definitely see quite slow behavior. > - Currently, the only IPython kernel available for use with the Qt widget > runs the IPython interpreter in a?separate?process. ?For QGIS, it would be > very nice to have the option of running an IPython kernel inside the same > process as the Qt widgets. ?IPython would then have direct access to the > QGIS interface and would be able to inspect and operatie on the PyQt4 > objects that compose the GUI providing a powerful tool for users performing > spatial analysis and developers building plugins. > Are there any plans to develop IPython kernels that do not run > in?separate?processes? ?I spent a few days trying to build one myself by > wrapping an InteractiveShell, but I feel like I am flailing a little bit > when it comes to creating a kernel_manager that will integrate with the Qt > frontend. ?If anyone has any design advice on how to implement an in-process > kernel I would love to hear it. There is interest in having an in process IPython, but I honestly think it will not work very well at all. This is mostly because of the GIL. I would simply turn the model inside out. If you run the QGIS code that has all the PyQt widgets in the IPython kernel (start the kernel with the pyqt event loop running using --pylab qt) your ipython session will have all the access that it needs. IOW, don't embed IPython in your app, embed your app in IPython. If this model doesn't work for you, we can talk more about how to get started on an in process IPython widget, but it will be a significant amount of work I'm afraid. > The plugin is available on [GitHub][2] under the tag `0.1` if anyone feels > interested in checking it out as an IPython use case. ?The 0.1 version is > really quite simple---about 5-10 lines in console.py that do the actual > implementation. ?To use: > ??- Install QGIS > ??- Clone project as a sub-directory of ~/.qgis/python/plugins/ > ??- Run `make` (runs PyQt4 dev tools on ui and qrc files) > ??- Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu > > Keep up the good work on IPython development! Thanks! Brian > -Charlie > [1]: http://www.qgis.org > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1 > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Mon Feb 7 23:25:54 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 07 Feb 2011 22:25:54 -0600 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: On 2011-02-07 22:15 , Brian Granger wrote: > Charlie, >> - Currently, the only IPython kernel available for use with the Qt widget >> runs the IPython interpreter in a separate process. For QGIS, it would be >> very nice to have the option of running an IPython kernel inside the same >> process as the Qt widgets. IPython would then have direct access to the >> QGIS interface and would be able to inspect and operatie on the PyQt4 >> objects that compose the GUI providing a powerful tool for users performing >> spatial analysis and developers building plugins. >> Are there any plans to develop IPython kernels that do not run >> in separate processes? I spent a few days trying to build one myself by >> wrapping an InteractiveShell, but I feel like I am flailing a little bit >> when it comes to creating a kernel_manager that will integrate with the Qt >> frontend. If anyone has any design advice on how to implement an in-process >> kernel I would love to hear it. > > There is interest in having an in process IPython, but I honestly > think it will not work very well at all. This is mostly because of > the GIL. I don't see why. The shell may block. This is not a big deal. This is what people expect of shells embedded in apps. > I would simply turn the model inside out. If you run the > QGIS code that has all the PyQt widgets in the IPython kernel (start > the kernel with the pyqt event loop running using --pylab qt) your > ipython session will have all the access that it needs. IOW, don't > embed IPython in your app, embed your app in IPython. This is not an option for most such apps. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Mon Feb 7 23:30:04 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Mon, 7 Feb 2011 20:30:04 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: On Mon, Feb 7, 2011 at 8:25 PM, Robert Kern wrote: > On 2011-02-07 22:15 , Brian Granger wrote: >> Charlie, > >>> - Currently, the only IPython kernel available for use with the Qt widget >>> runs the IPython interpreter in a separate process. ?For QGIS, it would be >>> very nice to have the option of running an IPython kernel inside the same >>> process as the Qt widgets. ?IPython would then have direct access to the >>> QGIS interface and would be able to inspect and operatie on the PyQt4 >>> objects that compose the GUI providing a powerful tool for users performing >>> spatial analysis and developers building plugins. >>> Are there any plans to develop IPython kernels that do not run >>> in separate processes? ?I spent a few days trying to build one myself by >>> wrapping an InteractiveShell, but I feel like I am flailing a little bit >>> when it comes to creating a kernel_manager that will integrate with the Qt >>> frontend. ?If anyone has any design advice on how to implement an in-process >>> kernel I would love to hear it. >> >> There is interest in having an in process IPython, but I honestly >> think it will not work very well at all. ?This is mostly because of >> the GIL. > > I don't see why. The shell may block. This is not a big deal. This is what > people expect of shells embedded in apps. It is not the blocking of the shell that is the problem, it is the blocking of the *entire* app that is the problem. >> I would simply turn the model inside out. ?If you run the >> QGIS code that has all the PyQt widgets in the IPython kernel (start >> the kernel with the pyqt event loop running using --pylab qt) your >> ipython session will have all the access that it needs. ?IOW, don't >> embed IPython in your app, embed your app in IPython. > > This is not an option for most such apps. I do realize that. Brian > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Mon Feb 7 23:40:46 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 07 Feb 2011 22:40:46 -0600 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: On 2011-02-07 22:30 , Brian Granger wrote: > On Mon, Feb 7, 2011 at 8:25 PM, Robert Kern wrote: >> On 2011-02-07 22:15 , Brian Granger wrote: >>> Charlie, >> >>>> - Currently, the only IPython kernel available for use with the Qt widget >>>> runs the IPython interpreter in a separate process. For QGIS, it would be >>>> very nice to have the option of running an IPython kernel inside the same >>>> process as the Qt widgets. IPython would then have direct access to the >>>> QGIS interface and would be able to inspect and operatie on the PyQt4 >>>> objects that compose the GUI providing a powerful tool for users performing >>>> spatial analysis and developers building plugins. >>>> Are there any plans to develop IPython kernels that do not run >>>> in separate processes? I spent a few days trying to build one myself by >>>> wrapping an InteractiveShell, but I feel like I am flailing a little bit >>>> when it comes to creating a kernel_manager that will integrate with the Qt >>>> frontend. If anyone has any design advice on how to implement an in-process >>>> kernel I would love to hear it. >>> >>> There is interest in having an in process IPython, but I honestly >>> think it will not work very well at all. This is mostly because of >>> the GIL. >> >> I don't see why. The shell may block. This is not a big deal. This is what >> people expect of shells embedded in apps. > > It is not the blocking of the shell that is the problem, it is the > blocking of the *entire* app that is the problem. I meant "The shell may block [the app]". It would be *ideal* if the shell didn't block the app, but that doesn't mean that a blocking shell doesn't work well. Remember, every UI interaction can potentially trigger a long blocking call, unless if that part of the app is carefully written to avoid that. The GIL issue you mention applies to any GUI application written in Python, but we manage to make usable applications just fine. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From source at sharpsteen.net Tue Feb 8 13:51:22 2011 From: source at sharpsteen.net (Charlie Sharpsteen) Date: Tue, 8 Feb 2011 10:51:22 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: Thanks for the reply Brian, On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger wrote: > Charlie, > > > I was playing around with the new Qt widgets in 0.11-dev the other day > and > > was very impressed. I tried throwing together a quick plugin to add an > > IPython console to the Qt-based program Quantum GIS [QGIS][1]. Creating > the > > plugin was easy enough as QGIS runs an embedded Python interpreter and > > exposes a lot of PyQt4 functionality. However, I noticed a couple of > things > > and was wondering if anyone on this list had any comments to share: > > > > - The shell is based on a stripped down version of `ipython-qtconsole` > and > > works but is very, very slow for some reason. The is a delay of a couple > of > > seconds between when an action is triggered and a response comes back to > the > > terminal. The only guess I have that may explain this behavior is that > QGIS > > is a C++ application that embeds a Python interpreter whereas > > ipython-qtconsole is a pure PyQt4 program. Perhaps QGIS is not letting > the > > Python threads run as often as a pure PyQt4 program would. > > This could be a QGIS-specific problem, but I was wondering if anyone else > > had the chance to run the IPython Qt widget in an embedded interpreter > and > > noticed similar results? > > In general the ipython-qtconsole should be quite fast. But for that > to be the case, the qt event loop has to be running. If the C++ is > not letting the qt event loop run often enough, you would definitely > see quite slow behavior. This is what I suspect, I would probably have to spend some quality time with the C++ side of the code to be sure. > > - Currently, the only IPython kernel available for use with the Qt widget > > runs the IPython interpreter in a separate process. For QGIS, it would > be > > very nice to have the option of running an IPython kernel inside the same > > process as the Qt widgets. IPython would then have direct access to the > > QGIS interface and would be able to inspect and operatie on the PyQt4 > > objects that compose the GUI providing a powerful tool for users > performing > > spatial analysis and developers building plugins. > > Are there any plans to develop IPython kernels that do not run > > in separate processes? I spent a few days trying to build one myself by > > wrapping an InteractiveShell, but I feel like I am flailing a little bit > > when it comes to creating a kernel_manager that will integrate with the > Qt > > frontend. If anyone has any design advice on how to implement an > in-process > > kernel I would love to hear it. > > There is interest in having an in process IPython, but I honestly > think it will not work very well at all. This is mostly because of > the GIL. I would simply turn the model inside out. If you run the > QGIS code that has all the PyQt widgets in the IPython kernel (start > the kernel with the pyqt event loop running using --pylab qt) your > ipython session will have all the access that it needs. IOW, don't > embed IPython in your app, embed your app in IPython. If this model > doesn't work for you, we can talk more about how to get started on an > in process IPython widget, but it will be a significant amount of work > I'm afraid. The problem with embedding QGIS in IPython is that the main app is written in C++ and `import qgis` only works out of the box on Linux---for Windows and OS X it is a bit of a hassle to get paths set up right. Many things are wrapped via SIP and it is possible to run a lot of widgets from IPython but the main windows are not so it is currently not possible to re-launch the app under Python. QGIS currently includes a Python console that makes use of code.InteractiveInterpreter from the standard library. This console it works very well with the exception that I always find myself pounding on the tab key in vain when I forget what I can do with a random Python object. Replacing the QGIS Python console with an IPython console would be awesome and would cut out a lot of trips to the QGIS source code and various reference manuals. If an in-process kernel could be implemented that shared the same interface as the QtKernelManager, then users could easily be given the option of launching an external process for tasks where the GIL would be a bother. I am willing to take a stab at developing an in-process widget but after thrashing around a bit this weekend I feel like some guidance would help me focus my efforts. > > The plugin is available on [GitHub][2] under the tag `0.1` if anyone > feels > > interested in checking it out as an IPython use case. The 0.1 version is > > really quite simple---about 5-10 lines in console.py that do the actual > > implementation. To use: > > - Install QGIS > > - Clone project as a sub-directory of ~/.qgis/python/plugins/ > > - Run `make` (runs PyQt4 dev tools on ui and qrc files) > > - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu > > > > Keep up the good work on IPython development! > > Thanks! > > Brian > > > -Charlie > > [1]: http://www.qgis.org > > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1 > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Feb 8 14:48:44 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 11:48:44 -0800 Subject: [IPython-dev] PySide support for the Qt frontend In-Reply-To: References: Message-ID: Hi Evan, On Thu, Feb 3, 2011 at 11:42 AM, Evan Patterson wrote: > Thanks, Brian. Good to hear from you as well. I'll be working at Enthought for a while, which means that I may do some more work on IPython over the next six months or so. Great news! Glad to have you on board again, hope all is well on your side. >>> Now that several PySide bugs relevant to IPython have been fixed, my port of IPython's Qt code to PySide is working. (Note that even the most recent PySide beta 5 release will segfault sporadically. This is fixed in the PySide master branch and IPython will work with the upcoming RC 1 release.) My branch is here: >>> >>> https://github.com/epatters/ipython/tree/pyside-support >> >> This is great! Indeed, many thanks! >>> This should be fairly uncontroversial. The only thing I would like to direct attention to is the Qt API switcher: >>> >>> https://github.com/epatters/ipython/blob/pyside-support/IPython/external/qt.py >> >> This is god to know about. ?Overall, how ready would you say that >> pyside is? ?Is Enthought considering shipping pyside in a version of >> EPD soon? > > PySide is quickly approaching some kind of stability. It looks like an RC 1 release is coming a week or two, with the 1.0 release not too far after. Although the upcoming EPD release (7.0) will not include PySide, as it is being released on the 8th, the subsequent release (7.1) will. At some point, PySide will replace wx as the default toolkit, although Enthought will continue to ship wx with EPD. > >> >>> I have modeled this on the way that ETS is currently handling switching Qt APIs. Any thoughts on this, however, would be welcome. >>> >>> Also, regarding GitHub etiquette: should I merge with the master branch before I submit my pull request? My branch is now several weeks behind. >> >> No, your stuff should be pretty independent. ?We may do a rebase, but >> why won't you do a pull request so everyone can have a look. > > Done. I saw the pull request, will continue the discussion on that in there. I'm very happy to see pyside coming along as well as it seems to, it will really be a big improvement to the whole python/qt situation. Cheers, f From fperez.net at gmail.com Tue Feb 8 15:49:13 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 12:49:13 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: Hey guys, On Fri, Jan 28, 2011 at 1:02 PM, Fernando Perez wrote: > > Many thanks, Robert. ?For now this fixes at least the single-line > strings, which was likely the most common problem. ?I'll keep working > on it later, but since it was such an ugly bug I've pushed your commit > for now. A long international flight is often a useful thing, fix ready for review/merge: https://github.com/ipython/ipython/pull/265 Cheers, f From ellisonbg at gmail.com Tue Feb 8 17:37:09 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 8 Feb 2011 14:37:09 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: Great, I need to spend some time dealing with pull requests/review. Cheers, Brian On Tue, Feb 8, 2011 at 12:49 PM, Fernando Perez wrote: > Hey guys, > > On Fri, Jan 28, 2011 at 1:02 PM, Fernando Perez wrote: >> >> Many thanks, Robert. ?For now this fixes at least the single-line >> strings, which was likely the most common problem. ?I'll keep working >> on it later, but since it was such an ugly bug I've pushed your commit >> for now. > > A long international flight is often a useful thing, fix ready for review/merge: > > https://github.com/ipython/ipython/pull/265 > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From fperez.net at gmail.com Tue Feb 8 18:05:16 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 15:05:16 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: On Tue, Feb 8, 2011 at 2:37 PM, Brian Granger wrote: > Great, I need to spend some time dealing with pull requests/review. > > Cheers, > Yes, same here :) I feel terrible that we've gotten a lot of good contributions recently, and we're falling behind on reviews/feedback. I'm trying to get plugged back into everything and doing some of that as well, because I've been *terrible* lately. This recent excellent post on building open source communities: http://www.codesimplicity.com/post/open-source-community-simplified/ really made me think hard about this and how I try to organize my priorities so that, when faced with very limited time (as I inevitably am) I can still be effective with the project. This section in particular: """ Respond to contributions immediately. The Bugzilla Project has a system of code reviews that requires that all new contributions be reviewed by an experienced developer before they can become part of Bugzilla. There have been various complaints about the system over the years, but analyzing the survey data showed that people leave the project because getting a review takes too long, not because the reviews are too hard. In fact, the reviews can be as hard as you want as long as they happen almost instantly after somebody submits a contribution. People don?t (usually) mind having to revise a contribution. They even generally don?t mind revising it several times. But they do mind if they post a patch, don?t get a review for three months, and then they have to revise it, only to wait another three months to be told that they have to revise it again. It?s the delay that matters, not the level of quality control. """ spells out very well the importance of good feedback to new contributors. I feel super guilty about having dropped the ball on the wave that came from Scipy India; I hope those contributors will still keep an interest in the project. Anyway, the good thing about screwing up badly is that your next step is likely to be an improvement ;) Cheers, f From fperez.net at gmail.com Tue Feb 8 20:56:41 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 17:56:41 -0800 Subject: [IPython-dev] Making our git repo much smaller? Message-ID: Hi all, I've always been bothered by the fact that our base git repo is much bigger than seems necessary. A clean clone gives: (master)uqbar[ipython]> dsize . 4.8M ./IPython 7.2M ./docs 75M ./.git ----------------------------------------------- Total (including smaller dirs not shown above): 87M . After reading on the matplotlib-devel list a tip from Pauli Virtnanen on git compression, who suggested using: git reflog expire --expire=0 --all git prune git repack -f -a -d git gc --prune=0 I ran that on my fresh clone and now I get: (master)uqbar[ipython]> dsize . 4.8M ./IPython 7.2M ./docs 12M ./.git ----------------------------------------------- Total (including smaller dirs not shown above): 24M . This is a pretty significant reduction. What I'm wondering is: how do we push this benefit back up to github so that all new users who clone/fork get the smaller repos? I don't want to push blindly quite yet from my local copy, because I'm not sure it's OK to do that. I simply don't know enough: does it do anything, does it do what we want, any damage?... But dropping the storage area of our repo from 75M to 12M is a really significant improvement that would especially help contributors with low-bandwidth connections, so I'd like to make it happen on the github side... If anyone knows what the right step here is, I'd be grateful... Best, f From robert.kern at gmail.com Tue Feb 8 22:27:14 2011 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 08 Feb 2011 21:27:14 -0600 Subject: [IPython-dev] Making our git repo much smaller? In-Reply-To: References: Message-ID: On 2011-02-08 19:56 , Fernando Perez wrote: > I don't want to push blindly quite yet from my local copy, because I'm > not sure it's OK to do that. I simply don't know enough: does it do > anything, does it do what we want, any damage?... That sums up most of my experience with git as a whole. ;-) My brief fiddling with my IPython checkout suggests that only the repack command does the major amount of compression (although following it up with gc is just good hygiene). None of those commands ought to do damage. Pushing to github doesn't seem to affect later checkouts, though. github does seem to run a standard no-options gc on every push, but they don't seem to repack. http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From benjaminrk at gmail.com Wed Feb 9 00:04:11 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 8 Feb 2011 21:04:11 -0800 Subject: [IPython-dev] Making our git repo much smaller? In-Reply-To: References: Message-ID: I've looked into this a little bit, and as far as I can tell (though not said with any confidence), the only option you have to repack a public repo is to pull, repack, and then push to an entirely new repo and toss out the old one - less than ideal. -MinRK On Tue, Feb 8, 2011 at 19:27, Robert Kern wrote: > On 2011-02-08 19:56 , Fernando Perez wrote: > > > I don't want to push blindly quite yet from my local copy, because I'm > > not sure it's OK to do that. I simply don't know enough: does it do > > anything, does it do what we want, any damage?... > > That sums up most of my experience with git as a whole. ;-) > > My brief fiddling with my IPython checkout suggests that only the repack > command > does the major amount of compression (although following it up with gc is > just > good hygiene). None of those commands ought to do damage. Pushing to github > doesn't seem to affect later checkouts, though. github does seem to run a > standard no-options gc on every push, but they don't seem to repack. > > > http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Feb 9 00:26:59 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 21:26:59 -0800 Subject: [IPython-dev] Making our git repo much smaller? In-Reply-To: References: Message-ID: On Tue, Feb 8, 2011 at 7:27 PM, Robert Kern wrote: > > That sums up most of my experience with git as a whole. ;-) > > My brief fiddling with my IPython checkout suggests that only the repack command > does the major amount of compression (although following it up with gc is just > good hygiene). None of those commands ought to do damage. Pushing to github > doesn't seem to affect later checkouts, though. github does seem to run a > standard no-options gc on every push, but they don't seem to repack. > > http://support.github.com/discussions/feature-requests/751-garbage-collection-or-repository-repack Thanks for that tip... I'll post a new request to see what they say about running a manual repack for us, perhaps. Cheers, f From fperez.net at gmail.com Wed Feb 9 01:03:15 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 8 Feb 2011 22:03:15 -0800 Subject: [IPython-dev] Making our git repo much smaller? In-Reply-To: References: Message-ID: Hey, On Tue, Feb 8, 2011 at 9:04 PM, MinRK wrote: > I've looked into this a little bit, and as far as I can tell (though not > said with any confidence), the only option you have to repack a public repo > is to pull, repack, and then push to an entirely new repo and toss out the > old one - less than ideal. I just made a request for a repack, we'll see what happens: http://support.github.com/discussions/repos/5478-manual-repack-request-for-repo Cheers, f From fperez.net at gmail.com Wed Feb 9 14:10:46 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 9 Feb 2011 11:10:46 -0800 Subject: [IPython-dev] Making our git repo much smaller? In-Reply-To: References: Message-ID: On Tue, Feb 8, 2011 at 10:03 PM, Fernando Perez wrote: > > I just made a request for a repack, we'll see what happens: > > http://support.github.com/discussions/repos/5478-manual-repack-request-for-repo Ask and ye shall receive. They manually ran an aggressive gc pass, and now: uqbar[junk]> dsize ipython 4.8M ipython/IPython 7.2M ipython/docs 11M ipython/.git ----------------------------------------------- Total (including smaller dirs not shown above): 23M ipython So all new cloners will now have a much more reasonable download. Github is awesome :) Cheers, f From fperez.net at gmail.com Wed Feb 9 15:15:04 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 9 Feb 2011 12:15:04 -0800 Subject: [IPython-dev] newparallel In-Reply-To: References: Message-ID: Hey Satra, On Sun, Feb 6, 2011 at 12:47 PM, Satrajit Ghosh wrote: > > 3. fernando: is your workflow branch mergeable with this? the ability to > spawn and shutdown engines as needed (especially in the context of clusters > such as SGE) has come up for discussion on the nipype side a fair bit in the > last few weeks. the workflow branch was really just a trivial amount of code to show Soizic how the basic idea should work, so there isn't much in there to really merge. The more important question seems to be that we need to have a good solution for the (valid) use case of engine creation and use outside of our central scheduler, when another scheduler is in control of job creation (say SGE or anything else). I've been thinking about this a bit, and there seem to be two valid scenarios to consider: 1. nipype wants to parallelize part of a pipeline where each node is nothing but a command-line call, long enough that startup overhead is irrelevant and with no other information to be transferred from the 'head node' (the instance running nipype itself) to the workers. 2. startup time is relevant compared to execution (higher frequency execution) or there's information to be passed that would be cumbersome/impossible as command-line arguments but is available to nipy as python objects. In situation 1, it's probably not of much value to have ipython around, except for the case where you might want to debug a problematic execution. In that scenario, having the engine not terminate its execution after a problem so that you could connect to it and play with the data/variables to understand the problem could be very useful. But that could be implemented in a special debug mode, that basically runs something like namespace = {} try: exec code in namespace except: start_ipython_engine(namespace) engine_wait() The engine could only run if there's a problem with the execution, and in the normal case isn't even run. Back to #2, I think in that case there is genuine value added by having ipython around, as it gives nipype the ability to do much more flexible execution control and to be efficient in scenarios where sge/similar likely wouldn't. The trick is for us to make it very, very easy to incorporate the 'controller' parts into nipype in a way that engines can be started by the queuing system instead of by our own cluster scripts, and that those engines participate of the rest of the nipype execution as needed, coming and going as dictated by the scheduler. Cheers, f From benjaminrk at gmail.com Wed Feb 9 16:56:15 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 9 Feb 2011 13:56:15 -0800 Subject: [IPython-dev] newparallel In-Reply-To: References: Message-ID: On Wed, Feb 9, 2011 at 12:15, Fernando Perez wrote: > Hey Satra, > > On Sun, Feb 6, 2011 at 12:47 PM, Satrajit Ghosh wrote: > > > > 3. fernando: is your workflow branch mergeable with this? the ability to > > spawn and shutdown engines as needed (especially in the context of > clusters > > such as SGE) has come up for discussion on the nipype side a fair bit in > the > > last few weeks. > > the workflow branch was really just a trivial amount of code to show > Soizic how the basic idea should work, so there isn't much in there to > really merge. > That may be, but I did merge and update it anyway ;) The newparallel examples are now in docs/examples/newparallel > > The more important question seems to be that we need to have a good > solution for the (valid) use case of engine creation and use outside > of our central scheduler, when another scheduler is in control of job > creation (say SGE or anything else). > Since starting engines amounts to running shell scripts, it's not difficult to start engines with any *ix job system. I am in the process of updating the Launchers from the kernel code, in order to make this trivial on common systems. Linking engine startup to jobs can probably (but not so nicely) be done with the current setup in 3 steps: 1. submit IPython task that will start an engine (or just run this in the client) def newjob(jobID): jobfile="job-%s.sh"%jobID with open(jobfile, 'w') as f: f.write() f.write("ipenginez -c jobID=%r"%jobID) os.system('qsub %s'%jobfile) c[0].apply(newjob, my_jobid) 2. submit the real Task, with a dependency that will keep it on that engine def check_jobid(jobid): engine_jobid = globals().get('jobID', None) return engine_jobid == jobid @depend(check_jobid, my_jobid) def mytask(args): dostuff() ar = client.apply(mytask, myargs) 3. submit another task using a `follow` dependency that calls sys.exit client.apply(lambda : sys.exit(), follow=ar) > > I've been thinking about this a bit, and there seem to be two valid > scenarios to consider: > > 1. nipype wants to parallelize part of a pipeline where each node is > nothing but a command-line call, long enough that startup overhead is > irrelevant and with no other information to be transferred from the > 'head node' (the instance running nipype itself) to the workers. > > 2. startup time is relevant compared to execution (higher frequency > execution) or there's information to be passed that would be > cumbersome/impossible as command-line arguments but is available to > nipy as python objects. > > > In situation 1, it's probably not of much value to have ipython > around, except for the case where you might want to debug a > problematic execution. In that scenario, having the engine not > terminate its execution after a problem so that you could connect to > it and play with the data/variables to understand the problem could be > very useful. But that could be implemented in a special debug mode, > that basically runs something like > > namespace = {} > try: > exec code in namespace > except: > start_ipython_engine(namespace) > engine_wait() > > The engine could only run if there's a problem with the execution, and > in the normal case isn't even run. > > > Back to #2, I think in that case there is genuine value added by > having ipython around, as it gives nipype the ability to do much more > flexible execution control and to be efficient in scenarios where > sge/similar likely wouldn't. > > The trick is for us to make it very, very easy to incorporate the > 'controller' parts into nipype in a way that engines can be started by > the queuing system instead of by our own cluster scripts, and that > those engines participate of the rest of the nipype execution as > needed, coming and going as dictated by the scheduler. > It sounds like what is requested is, rather than an IPython scheduler, an SGE scheduler. That is, the Scheduler, for every Task, starts a new Engine (via qsub,etc.), runs the task on it, and shuts the engine down. This would obviously be extraordinarily slow compared to the regular Scheduler. Perhaps the new Cluster model Brian and I have been discussing will help with this - it will allow clients to make requests like 'start_engine' and 'stop_engine', which would be executed via the Launchers. -MinRK > > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gael.varoquaux at normalesup.org Wed Feb 9 17:13:53 2011 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Wed, 9 Feb 2011 23:13:53 +0100 Subject: [IPython-dev] newparallel In-Reply-To: References: Message-ID: <20110209221353.GB6115@phare.normalesup.org> On Wed, Feb 09, 2011 at 12:15:04PM -0800, Fernando Perez wrote: > In situation 1, it's probably not of much value to have ipython > around, except for the case where you might want to debug a > problematic execution. IMHO that's a hugely important point. The number one reason why I don't do much parallel work is that a significant amount of my time on research code is spent debuging. So I would say: please don't underestimate the fantastic contribution that you have been making in this direction with IPython. Take this with a grain of salt from somebody who doesn't do grid computing. Ga?l From epatters at enthought.com Thu Feb 10 12:55:16 2011 From: epatters at enthought.com (Evan Patterson) Date: Thu, 10 Feb 2011 11:55:16 -0600 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: On Feb 4, 2011, at 11:19 AM, MinRK wrote: > On Fri, Feb 4, 2011 at 06:59, Darren Dale wrote: >> Hi All, >> >> This morning I updated my zmq/pyzmq and tried running >> ipython-qtconsole from the master branch. I had let my zeromq >> installation get behind ipython development, and so I'm rediscovering >> how awesome is the qtconsole. Having docstrings pop up at "(" is >> great. I noticed though, that depending on the location of the mouse >> pointer, the docstring may only briefly appear. It seems more likely >> to occur if the pointer is located inside the region where the >> docstring will be rendered. I'm running "ipython-qtconsole --pylab" on >> a Snow Leopard machine, with native (not X-based) Qt provided by >> MacPorts. Has anyone else noticed something similar? > > Yes, this is my experience, also on OSX 10.6.6, with native PyQt from > their website. The behavior is inconsistent, but seems more likely if > the cursor is hovering over the popup, and also more likely if the > popup is large. > > Another (probably) OSX-specific glitch: You cannot copy with 'cmd-c', > because pressing command triggers an event that clears the selection. I've submitted a pull request that fixes both of these bugs. If you're aware of other, similar issues, please send those my way as well. Evan > >> >> Who is working on the qtconsole? Is there a roadmap? >> >> Darren >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From fperez.net at gmail.com Thu Feb 10 14:32:28 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Feb 2011 11:32:28 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: Hi folks, On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson wrote: > > I've submitted a pull request that fixes both of these bugs. For reference, this is Evan's pull request: https://github.com/ipython/ipython/pull/266 As per my comments there, it would be great if at least one mac user (anyone can do it, it doesn't have to be a core developer) could independently validate this. The code looks fine to me, but I don't want to approve for merging code that hasn't been actually verified in execution by at least one more person beyond its author, and I don't have a mac set up for development that I can easily use. Cheers, f From fperez.net at gmail.com Thu Feb 10 14:39:33 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Feb 2011 11:39:33 -0800 Subject: [IPython-dev] newparallel In-Reply-To: <20110209221353.GB6115@phare.normalesup.org> References: <20110209221353.GB6115@phare.normalesup.org> Message-ID: On Wed, Feb 9, 2011 at 2:13 PM, Gael Varoquaux wrote: > On Wed, Feb 09, 2011 at 12:15:04PM -0800, Fernando Perez wrote: >> In situation 1, it's probably not of much value to have ipython >> around, except for the case where you might want to debug a >> problematic execution. > > IMHO that's a hugely important point. The number one reason why I don't > do much parallel work is that a significant amount of my time on research > code is spent debuging. So I would say: please don't underestimate the > fantastic contribution that you have been making in this direction with > IPython. Thanks for the comments, Gael. Indeed that was very much my take on it, but I know you and I have similar workstyles and we really value interactive access to 'hands-on' code and data. I was just trying to be somewhat neutral towards the more classic grid computing approaches that toss interactivity overboard. But you are right that we shouldn't shy away from highlighting the value of this. And it's clear to me that with a bit of api help on our side, we can make it trivial and zero-overhead for batch-oriented systems to offer the capability to do: try: normal_execution() except: open_ipython_client() This would not even invoke ipython if things go OK, but offer the ability to debug a problematic node interactively whenever there's a problem (which is more often than the grid computing literature wants you to believe ;) So thanks for coming back to that point, it's more important than I sometimes give it credit in a misguided attempt at moderating my ipython/interactive bias :) Best, f From benjaminrk at gmail.com Thu Feb 10 14:47:48 2011 From: benjaminrk at gmail.com (MinRK) Date: Thu, 10 Feb 2011 11:47:48 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: Fix verified on my machine (OSX 10.6.6, x64 System Python 2.6.1, current release PyQt, etc.) [also posted in PR discussion thread] -MinRK On Thu, Feb 10, 2011 at 11:32, Fernando Perez wrote: > Hi folks, > > On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson > wrote: > > > > I've submitted a pull request that fixes both of these bugs. > > For reference, this is Evan's pull request: > > https://github.com/ipython/ipython/pull/266 > > As per my comments there, it would be great if at least one mac user > (anyone can do it, it doesn't have to be a core developer) could > independently validate this. The code looks fine to me, but I don't > want to approve for merging code that hasn't been actually verified in > execution by at least one more person beyond its author, and I don't > have a mac set up for development that I can easily use. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Feb 10 14:53:19 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Feb 2011 11:53:19 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Tue, Feb 1, 2011 at 9:56 AM, Nicolas Rougier wrote: > I've made some time ago a kind of 'ascii imshow' function to render a numpy > array directly within the console by exploiting the 256 colors capability of > most modern terminal. I don't know how much interesting it can be for > IPython but I found the code useful for myself. > Code is located > at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py > (GPL license can be replaced by the BSD one if necessary) Want. Now. Fabulous!! If you'd be so kind as to officialy relicense it as BSD, we'd be thrilled to have it in IPython right away. For the curious, this is what Nicolas' default example produces: http://imgur.com/iZ13v Note, this is a true terminal-only tool, it doesn't work correctly in the Qt console nor in a bare Linux console (i.e. outside X11). Both the Qt and Linux console show spurious codes and just two lines of weird colors. It would be cool to see if this can be adapted to non-xterm environments later, but for now this is great as it is. Nicolas, if you're willing to submit this as a pull request against our trunk, by adding the code as a new module named IPython/extensions/ascii_imshow.py, it would be fantastic. If you're not famliar with the github workflow let us know and we'll be happy to provide some pointers. If you can't spare the time to make a git-based pull request please let us know, it's possible someone else can collaborate with you to finish that part of the process. This file: https://github.com/ipython/ipython/blob/master/docs/source/development/template.py can serve as a template for putting your code in our standard format. Certainly do keep your original copyright statement in addition to (moving forward) the team one, and also add to the docstring an authorship line Authors -------- * Nicolas Rougier... But it's good to have all source files similarly structured and documented project-wide. In any case, great little tool, I hope we can pull it in soon! Regards, f From source at sharpsteen.net Thu Feb 10 15:02:22 2011 From: source at sharpsteen.net (Charlie Sharpsteen) Date: Thu, 10 Feb 2011 12:02:22 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez wrote: > Hi folks, > > On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson > wrote: > > > > I've submitted a pull request that fixes both of these bugs. > > For reference, this is Evan's pull request: > > https://github.com/ipython/ipython/pull/266 > > As per my comments there, it would be great if at least one mac user > (anyone can do it, it doesn't have to be a core developer) could > independently validate this. The code looks fine to me, but I don't > want to approve for merging code that hasn't been actually verified in > execution by at least one more person beyond its author, and I don't > have a mac set up for development that I can easily use. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > Verified with `ipython-qtconsole` OS X 10.6.6 with: Python 2.7.1 Qt 4.7.1 SIP/PyQt 4.12.1/4.8.3 Before: Cmd-C bug: Could not reproduce Call tips: What calltips? Home/End behavior: Reproduced as described by issue #173 After: Cmd-C bug: Still could not reproduce (good!) Call tips: Whoa! It comes with calltips?! Home/End behavior: Fixed Looks good to me. -Charlie -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Thu Feb 10 15:04:55 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 10 Feb 2011 20:04:55 +0000 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On 10 February 2011 19:53, Fernando Perez wrote: > It would be cool to see if this can be adapted to > non-xterm environments later, but for now this is great as it is. > Doing something similar for the the Qt console (and other graphics capable consoles) should be easier than making the ASCII version - scipy already has functions to turn an array into an image. http://docs.scipy.org/doc/scipy/reference/misc.html#scipy.misc.toimage Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Thu Feb 10 15:14:06 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Thu, 10 Feb 2011 15:14:06 -0500 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: also fixed on osx x64 10.6.6 system python 2.6.1 latest pyqt cheers, satra On Thu, Feb 10, 2011 at 3:02 PM, Charlie Sharpsteen wrote: > On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez wrote: > >> Hi folks, >> >> On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson >> wrote: >> > >> > I've submitted a pull request that fixes both of these bugs. >> >> For reference, this is Evan's pull request: >> >> https://github.com/ipython/ipython/pull/266 >> >> As per my comments there, it would be great if at least one mac user >> (anyone can do it, it doesn't have to be a core developer) could >> independently validate this. The code looks fine to me, but I don't >> want to approve for merging code that hasn't been actually verified in >> execution by at least one more person beyond its author, and I don't >> have a mac set up for development that I can easily use. >> >> Cheers, >> >> f >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > Verified with `ipython-qtconsole` OS X 10.6.6 with: > > Python 2.7.1 > Qt 4.7.1 > SIP/PyQt 4.12.1/4.8.3 > > Before: > > Cmd-C bug: Could not reproduce > Call tips: What calltips? > Home/End behavior: Reproduced as described by issue #173 > > After: > > Cmd-C bug: Still could not reproduce (good!) > Call tips: Whoa! It comes with calltips?! > Home/End behavior: Fixed > > Looks good to me. > > -Charlie > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Thu Feb 10 15:21:07 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Thu, 10 Feb 2011 12:21:07 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Two quick comments: * While I think this is incredibly cool, I am not sure this belongs in IPython. Two reasons for this: i) it is useful for people outside of IPython and ii) plotting/viz is outside the scope of IPython. I think a much better place for this would be a standalone module or matplotlib (imagine a terminal based backend!). * If it does go into IPython, the proper directory would be lib, not extensions. IPython/extensions is for IPython extensions that adhere to the official IPython extension API. Cheers, Brian On Thu, Feb 10, 2011 at 11:53 AM, Fernando Perez wrote: > On Tue, Feb 1, 2011 at 9:56 AM, Nicolas Rougier > wrote: >> I've made some time ago a kind of 'ascii imshow' function to render a numpy >> array directly within the console by exploiting the 256 colors capability of >> most modern terminal. I don't know how much interesting it can be for >> IPython but I found the code useful for myself. >> Code is located >> at:?http://www.loria.fr/~rougier/coding/software/numpy_imshow.py >> (GPL license can be replaced by the BSD one if necessary) > > Want. Now. Fabulous!! > > If you'd be so kind as to officialy relicense it as BSD, we'd be > thrilled to have it in IPython right away. ?For the curious, this is > what Nicolas' default example produces: > > http://imgur.com/iZ13v > > Note, this is a true terminal-only tool, it doesn't work correctly in > the Qt console nor in a bare Linux console (i.e. ?outside X11). ?Both > the Qt and Linux console show spurious codes and just two lines of > weird colors. ?It would be cool to see if this can be adapted to > non-xterm environments later, but for now this is great as it is. > > Nicolas, if you're willing to submit this as a pull request against > our trunk, by adding the code as a new module named > IPython/extensions/ascii_imshow.py, it would be fantastic. ?If you're > not famliar with the github workflow let us know and we'll be happy to > provide some pointers. > > If you can't spare the time to make a git-based pull request please > let us know, it's possible someone else can collaborate with you to > finish that part of the process. > > This file: > > https://github.com/ipython/ipython/blob/master/docs/source/development/template.py > > can serve as a template for putting your code in our standard format. > Certainly do keep your original copyright statement in addition to > (moving forward) the team one, and also add to the docstring an > authorship line > > Authors > -------- > * Nicolas Rougier... > > But it's good to have all source files similarly structured and > documented project-wide. > > In any case, great little tool, I hope we can pull it in soon! > > Regards, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From epatters at enthought.com Thu Feb 10 15:21:46 2011 From: epatters at enthought.com (Evan Patterson) Date: Thu, 10 Feb 2011 14:21:46 -0600 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: References: Message-ID: <344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com> Thanks for all the testing everyone. The fixes have been merged into master. Evan On Feb 10, 2011, at 2:14 PM, Satrajit Ghosh wrote: > also fixed on osx x64 10.6.6 system python 2.6.1 latest pyqt > > cheers, > > satra > > > On Thu, Feb 10, 2011 at 3:02 PM, Charlie Sharpsteen wrote: > On Thu, Feb 10, 2011 at 11:32 AM, Fernando Perez wrote: > Hi folks, > > On Thu, Feb 10, 2011 at 9:55 AM, Evan Patterson wrote: > > > > I've submitted a pull request that fixes both of these bugs. > > For reference, this is Evan's pull request: > > https://github.com/ipython/ipython/pull/266 > > As per my comments there, it would be great if at least one mac user > (anyone can do it, it doesn't have to be a core developer) could > independently validate this. The code looks fine to me, but I don't > want to approve for merging code that hasn't been actually verified in > execution by at least one more person beyond its author, and I don't > have a mac set up for development that I can easily use. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > Verified with `ipython-qtconsole` OS X 10.6.6 with: > > Python 2.7.1 > Qt 4.7.1 > SIP/PyQt 4.12.1/4.8.3 > > Before: > > Cmd-C bug: Could not reproduce > Call tips: What calltips? > Home/End behavior: Reproduced as described by issue #173 > > After: > > Cmd-C bug: Still could not reproduce (good!) > Call tips: Whoa! It comes with calltips?! > Home/End behavior: Fixed > > Looks good to me. > > -Charlie > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Feb 10 15:24:16 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Feb 2011 12:24:16 -0800 Subject: [IPython-dev] ipython-qtconsole and docstrings In-Reply-To: <344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com> References: <344D3E76-071E-409E-8465-A3C37E3E2007@enthought.com> Message-ID: On Thu, Feb 10, 2011 at 12:21 PM, Evan Patterson wrote: > Thanks for all the testing everyone. The fixes have been merged into master. Much appreciated from all, testers and Evan alike. Excellent. regards, f From fperez.net at gmail.com Thu Feb 10 15:43:51 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Thu, 10 Feb 2011 12:43:51 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger wrote: > * While I think this is incredibly cool, I am not sure this belongs in > IPython. ?Two reasons for this: i) it is useful for people outside of > IPython and ii) plotting/viz is outside the scope of IPython. ?I think > a much better place for this would be a standalone module or > matplotlib (imagine a terminal based backend!). While I really hope MPL gets an ascii backend at some point, that's a lot more work. I don't know if Nicolas is up for it :) I think it's OK to have it in ipython in the sense that it's small, useful code for terminal use. While it's true that we're not in the data viz business, we're in the business of making a really good interactive environment, and the terminal is one of our strengths. Something this small, left to live on its own, is much less likely to get noticed/used than if we ship it and everyone benefits from it right away. I think of this as something much like our qt console matplotlib support: something to seamlessly make ipython that much better for heavy numpy/scientific users. > * If it does go into IPython, the proper directory would be lib, not > extensions. ?IPython/extensions is for IPython extensions that adhere > to the official IPython extension API. Yes, my initial email was written with lib/, and then I backtracked and thought of it as an extension. But the api point is spot-on, so lib it is. Cheers, f From takowl at gmail.com Fri Feb 11 08:42:35 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 11 Feb 2011 13:42:35 +0000 Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1 In-Reply-To: <20110211131509.GH16708@piotro.eu> References: <20110210144324.GD16708@piotro.eu> <20110211131509.GH16708@piotro.eu> Message-ID: The bugfix for pyzmq made it into Debian experimental, and they found a couple of spelling mistakes - see below. Thomas ---------- Forwarded message ---------- From: Piotr O?arowski Date: 11 February 2011 13:15 Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1 To: Miguel Landaeta Cc: Thomas Kluyver uploaded; please send upstream a patch to fix these spelling mistakes: zmq/core/message.so Messsages ? Messages zmq/core/message.so Messsages ? Messages zmq/core/socket.so Recieve ? Receive -- http://people.debian.org/~piotr/sponsor -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Fri Feb 11 13:13:25 2011 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Fri, 11 Feb 2011 10:13:25 -0800 Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1 In-Reply-To: References: <20110210144324.GD16708@piotro.eu> <20110211131509.GH16708@piotro.eu> Message-ID: I've added another Ubuntu bug report to upgrade to this package for Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316) On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver wrote: > The bugfix for pyzmq made it into Debian experimental, and they found a > couple of spelling mistakes - see below. > > Thomas > > ---------- Forwarded message ---------- > From: Piotr O?arowski > Date: 11 February 2011 13:15 > Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1 > To: Miguel Landaeta > Cc: Thomas Kluyver > > > uploaded; please send upstream a patch to fix these spelling mistakes: > > zmq/core/message.so Messsages ? Messages > zmq/core/message.so Messsages ? Messages > zmq/core/socket.so Recieve ? Receive > -- > http://people.debian.org/~piotr/sponsor > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Erik Tollerud From benjaminrk at gmail.com Fri Feb 11 13:34:20 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 11 Feb 2011 10:34:20 -0800 Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1 In-Reply-To: References: <20110210144324.GD16708@piotro.eu> <20110211131509.GH16708@piotro.eu> Message-ID: Great, thanks! The typos are now fixed in master. -MinRK On Fri, Feb 11, 2011 at 10:13, Erik Tollerud wrote: > I've added another Ubuntu bug report to upgrade to this package for > Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316) > > On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver wrote: > > The bugfix for pyzmq made it into Debian experimental, and they found a > > couple of spelling mistakes - see below. > > > > Thomas > > > > ---------- Forwarded message ---------- > > From: Piotr O?arowski > > Date: 11 February 2011 13:15 > > Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1 > > To: Miguel Landaeta > > Cc: Thomas Kluyver > > > > > > uploaded; please send upstream a patch to fix these spelling mistakes: > > > > zmq/core/message.so Messsages ? Messages > > zmq/core/message.so Messsages ? Messages > > zmq/core/socket.so Recieve ? Receive > > -- > > http://people.debian.org/~piotr/sponsor > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > Erik Tollerud > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Fri Feb 11 17:30:43 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 11 Feb 2011 22:30:43 +0000 Subject: [IPython-dev] Fwd: RFS: DPMT: pyzmq 2.0.10.1-1 In-Reply-To: References: <20110210144324.GD16708@piotro.eu> <20110211131509.GH16708@piotro.eu> Message-ID: And Ubuntu has done the sync. That closes that chapter. Thomas On 11 February 2011 18:34, MinRK wrote: > Great, thanks! > > The typos are now fixed in master. > > -MinRK > > > On Fri, Feb 11, 2011 at 10:13, Erik Tollerud wrote: > >> I've added another Ubuntu bug report to upgrade to this package for >> Natty (https://bugs.launchpad.net/ubuntu/+source/pyzmq/+bug/717316) >> >> On Fri, Feb 11, 2011 at 5:42 AM, Thomas Kluyver wrote: >> > The bugfix for pyzmq made it into Debian experimental, and they found a >> > couple of spelling mistakes - see below. >> > >> > Thomas >> > >> > ---------- Forwarded message ---------- >> > From: Piotr O?arowski >> > Date: 11 February 2011 13:15 >> > Subject: Re: RFS: DPMT: pyzmq 2.0.10.1-1 >> > To: Miguel Landaeta >> > Cc: Thomas Kluyver >> > >> > >> > uploaded; please send upstream a patch to fix these spelling mistakes: >> > >> > zmq/core/message.so Messsages ? Messages >> > zmq/core/message.so Messsages ? Messages >> > zmq/core/socket.so Recieve ? Receive >> > -- >> > http://people.debian.org/~piotr/sponsor >> > >> > >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> >> >> >> -- >> Erik Tollerud >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Fri Feb 11 23:11:27 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 11 Feb 2011 20:11:27 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hi, On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez wrote: > On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger wrote: >> * While I think this is incredibly cool, I am not sure this belongs in >> IPython. ?Two reasons for this: i) it is useful for people outside of >> IPython and ii) plotting/viz is outside the scope of IPython. ?I think >> a much better place for this would be a standalone module or >> matplotlib (imagine a terminal based backend!). > > While I really hope MPL gets an ascii backend at some point, that's a > lot more work. ?I don't know if Nicolas is up for it :) ?I think it's > OK to have it in ipython in the sense that it's small, useful code for > terminal use. ?While it's true that we're not in the data viz > business, we're in the business of making a really good interactive > environment, and the terminal is one of our strengths. OK, if we do this, let make sure that: * There are tests. * It is fully documented in docstrings. I am mostly concerned about the fact that this type of stuff tends to get deposited with us to maintain in the future (like port to Python 3). Ideally, these things would have maintainers that are committed to helping out with things. > Something this small, left to live on its own, is much less likely to > get noticed/used than if we ship it and everyone benefits from it > right away. ?I think of this as something much like our qt console > matplotlib support: something to seamlessly make ipython that much > better for heavy numpy/scientific users. > >> * If it does go into IPython, the proper directory would be lib, not >> extensions. ?IPython/extensions is for IPython extensions that adhere >> to the official IPython extension API. > > Yes, my initial email was written with lib/, and then I backtracked > and thought of it as an extension. ?But the api point is spot-on, so > lib it is. Cheers, Brian > Cheers, > > f > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From sunqiang at gmail.com Sat Feb 12 04:25:33 2011 From: sunqiang at gmail.com (sunqiang) Date: Sat, 12 Feb 2011 17:25:33 +0800 Subject: [IPython-dev] Issue of install ipython-py3k Message-ID: Hi, After git clone git://github.com/ipython/ipython-py3k.git "python3.2 setup.py build" raise a byte/str error, I have to change line 370 of setupbase.py from "cfg_parser.set('commit hash', 'install_hash', repo_commit)" to "cfg_parser.set('commit hash', 'install_hash', repo_commit.decode())" to finish the setup. I don't know if this is my own problem. the error I am getting is: ############################################################################ ============================================================================ BUILDING IPYTHON python: 3.2rc2+ (py3k, Feb 9 2011, 01:06:54) [GCC 4.2.4 (Ubuntu 4.2.4-1ubuntu4)] platform: linux2 OPTIONAL DEPENDENCIES zope.Interface: Not found (required for parallel computing capabilities) Twisted: Not found (required for parallel computing capabilities) Foolscap: Not found (required for parallel computing capabilities) OpenSSL: Not found (required if you want security in the parallel computing capabilities) sphinx: Not found (required for building documentation) pygments: Not found (required for syntax highlighting documentation) nose: Not found (required for running the test suite) pexpect: no (required for running standalone doctests) running build running build_py copying IPython/core/display.py -> build/lib/IPython/core copying IPython/core/magic.py -> build/lib/IPython/core copying IPython/core/hooks.py -> build/lib/IPython/core copying IPython/core/formatters.py -> build/lib/IPython/core copying IPython/core/usage.py -> build/lib/IPython/core copying IPython/core/displayhook.py -> build/lib/IPython/core copying IPython/core/displaypub.py -> build/lib/IPython/core copying IPython/core/inputsplitter.py -> build/lib/IPython/core copying IPython/core/interactiveshell.py -> build/lib/IPython/core copying IPython/core/tests/test_inputsplitter.py -> build/lib/IPython/core/tests copying IPython/extensions/sympy_printing.py -> build/lib/IPython/extensions copying IPython/lib/irunner.py -> build/lib/IPython/lib copying IPython/lib/latextools.py -> build/lib/IPython/lib copying IPython/lib/pylabtools.py -> build/lib/IPython/lib copying IPython/testing/tools.py -> build/lib/IPython/testing Traceback (most recent call last): File "setup.py", line 254, in setup(**setup_args) File "/opt/python3.2/lib/python3.2/distutils/core.py", line 149, in setup dist.run_commands() File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 919, in run_commands self.run_command(cmd) File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 938, in run_command cmd_obj.run() File "/opt/python3.2/lib/python3.2/distutils/command/build.py", line 128, in run self.run_command(cmd_name) File "/opt/python3.2/lib/python3.2/distutils/cmd.py", line 315, in run_command self.distribution.run_command(command) File "/opt/python3.2/lib/python3.2/distutils/dist.py", line 938, in run_command cmd_obj.run() File "/home/romano/gitrepos/ipython-py3k/setupbase.py", line 370, in run cfg_parser.set('commit hash', 'install_hash', repo_commit) File "/opt/python3.2/lib/python3.2/configparser.py", line 1157, in set self._validate_value_types(option=option, value=value) File "/opt/python3.2/lib/python3.2/configparser.py", line 1146, in _validate_value_types raise TypeError("option values must be strings") TypeError: option values must be strings ############################################################################ From Nicolas.rougier at inria.fr Sat Feb 12 04:32:54 2011 From: Nicolas.rougier at inria.fr (Nicolas Rougier) Date: Sat, 12 Feb 2011 10:32:54 +0100 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hi, Sorry for the delay, I was away from my mail. After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ? For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it). As for the MPL terminal backend, it would be really cool (gnuplot got one) but I do have time right know to start such a thing. Nicolas On Feb 12, 2011, at 5:11 AM, Brian Granger wrote: > Hi, > > On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez wrote: >> On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger wrote: >>> * While I think this is incredibly cool, I am not sure this belongs in >>> IPython. Two reasons for this: i) it is useful for people outside of >>> IPython and ii) plotting/viz is outside the scope of IPython. I think >>> a much better place for this would be a standalone module or >>> matplotlib (imagine a terminal based backend!). >> >> While I really hope MPL gets an ascii backend at some point, that's a >> lot more work. I don't know if Nicolas is up for it :) I think it's >> OK to have it in ipython in the sense that it's small, useful code for >> terminal use. While it's true that we're not in the data viz >> business, we're in the business of making a really good interactive >> environment, and the terminal is one of our strengths. > > OK, if we do this, let make sure that: > > * There are tests. > * It is fully documented in docstrings. > > I am mostly concerned about the fact that this type of stuff tends to > get deposited with us to maintain in the future (like port to Python > 3). Ideally, these things would have maintainers that are committed > to helping out with things. > >> Something this small, left to live on its own, is much less likely to >> get noticed/used than if we ship it and everyone benefits from it >> right away. I think of this as something much like our qt console >> matplotlib support: something to seamlessly make ipython that much >> better for heavy numpy/scientific users. >> >>> * If it does go into IPython, the proper directory would be lib, not >>> extensions. IPython/extensions is for IPython extensions that adhere >>> to the official IPython extension API. >> >> Yes, my initial email was written with lib/, and then I backtracked >> and thought of it as an extension. But the api point is spot-on, so >> lib it is. > > Cheers, > > Brian > >> Cheers, >> >> f >> > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com From ellisonbg at gmail.com Sat Feb 12 13:42:38 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 12 Feb 2011 10:42:38 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Fernando, Do you think it would make sene to implement this as an object that has special print methods like __pretty__, __png__, etc. That would make it super easy to add support for other frontends. Cheers, Brian On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier wrote: > > Hi, > > Sorry for the delay, I was away from my mail. > > After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ? > > For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it). > > > As for the MPL terminal backend, it would be really cool (gnuplot got one) but I do have time right know to start such a thing. > > > Nicolas > > > On Feb 12, 2011, at 5:11 AM, Brian Granger wrote: > >> Hi, >> >> On Thu, Feb 10, 2011 at 12:43 PM, Fernando Perez wrote: >>> On Thu, Feb 10, 2011 at 12:21 PM, Brian Granger wrote: >>>> * While I think this is incredibly cool, I am not sure this belongs in >>>> IPython. ?Two reasons for this: i) it is useful for people outside of >>>> IPython and ii) plotting/viz is outside the scope of IPython. ?I think >>>> a much better place for this would be a standalone module or >>>> matplotlib (imagine a terminal based backend!). >>> >>> While I really hope MPL gets an ascii backend at some point, that's a >>> lot more work. ?I don't know if Nicolas is up for it :) ?I think it's >>> OK to have it in ipython in the sense that it's small, useful code for >>> terminal use. ?While it's true that we're not in the data viz >>> business, we're in the business of making a really good interactive >>> environment, and the terminal is one of our strengths. >> >> OK, if we do this, let make sure that: >> >> * There are tests. >> * It is fully documented in docstrings. >> >> I am mostly concerned about the fact that this type of stuff tends to >> get deposited with us to maintain in the future (like port to Python >> 3). ?Ideally, these things would have maintainers that are committed >> to helping out with things. >> >>> Something this small, left to live on its own, is much less likely to >>> get noticed/used than if we ship it and everyone benefits from it >>> right away. ?I think of this as something much like our qt console >>> matplotlib support: something to seamlessly make ipython that much >>> better for heavy numpy/scientific users. >>> >>>> * If it does go into IPython, the proper directory would be lib, not >>>> extensions. ?IPython/extensions is for IPython extensions that adhere >>>> to the official IPython extension API. >>> >>> Yes, my initial email was written with lib/, and then I backtracked >>> and thought of it as an extension. ?But the api point is spot-on, so >>> lib it is. >> >> Cheers, >> >> Brian >> >>> Cheers, >>> >>> f >>> >> >> >> >> -- >> Brian E. Granger, Ph.D. >> Assistant Professor of Physics >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu >> ellisonbg at gmail.com > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From takowl at gmail.com Sat Feb 12 14:11:34 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 12 Feb 2011 19:11:34 +0000 Subject: [IPython-dev] Issue of install ipython-py3k In-Reply-To: References: Message-ID: Thanks, Sunqiang, I've pushed the fix. Keep me posted of any more problems you find using it with Python 3.2. I probably won't upgrade until April, with the next Ubuntu release. Technical details: The value coming from a subprocess.Popen().communicate() was a byte string. This was being written to a config file, but the rewritten configparser in Python 3.2 specifically checks that the option to be set is a (unicode) str. I've actually specified that it should be decoded as ascii: the hash digest should never include any non-ascii characters, and we'll find out quickly if it's doing anything unexpected. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sat Feb 12 14:22:44 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 12 Feb 2011 11:22:44 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: Fernando, > Yes, same here :) ?I feel terrible that we've gotten a lot of good > contributions recently, and we're falling behind on reviews/feedback. > I'm trying to get plugged back into everything and doing some of that > as well, because I've been *terrible* lately. ?This recent excellent > post on building open source communities: Yes, that is the hidden side of code review and lots of (great!) contributions. It takes a ton of time just to review. Between the different projects I am involved in, I spend a significant amount of time doing review (more than I code these days). > http://www.codesimplicity.com/post/open-source-community-simplified/ > > really made me think hard about this and how I try to organize my > priorities so that, when faced with very limited time (as I inevitably > am) I can still be effective with the project. ?This section in > particular: > > """ > Respond to contributions immediately. > > The Bugzilla Project has a system of code reviews that requires that > all new contributions be reviewed by an experienced developer before > they can become part of Bugzilla. There have been various complaints > about the system over the years, but analyzing the survey data showed > that people leave the project because getting a review takes too long, > not because the reviews are too hard. In fact, the reviews can be as > hard as you want as long as they happen almost instantly after > somebody submits a contribution. I *completely* believe this. > People don?t (usually) mind having to revise a contribution. They even > generally don?t mind revising it several times. But they do mind if > they post a patch, don?t get a review for three months, and then they > have to revise it, only to wait another three months to be told that > they have to revise it again. It?s the delay that matters, not the > level of quality control. > """ Yep. > spells out very well the importance of good feedback to new contributors. > > I feel super guilty about having dropped the ball on the wave that > came from Scipy India; I hope those contributors will still keep an > interest in the project. > > Anyway, the good thing about screwing up badly is that your next step > is likely to be an improvement ;) :) One thing I think we should do is figure out a way of dividing the review workload so nothing slips through the cracks. When a review comes in, do we want to assign a "lead" reviewer that will make sure it gets pushed through in a timely manner? Sort of like journals assign reviewers to articles. Our current situation is like running a peer review journal, without an editor to manage the review proces... Cheers, Brian > Cheers, > > f > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Sat Feb 12 14:38:00 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sat, 12 Feb 2011 11:38:00 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: I think there are two possible approaches: 1. Still use the pyzmq interface through KernelManager and the SocketChannel objects, just use in process (threads) zeromq sockets. While complicated, this might be the best approach. But, it might take some GUI event loop magic to get working (because you can't run anything in a thread. 2. Get rid of the zeromq stuff entirely. This is probably easier. To do this, I would check out the subclass of InteractiveShell in IPython.zmq.zmqshell. It should be pretty easy to build a subclass of InteractiveShell that works, *but*, it leaves you without a GUI. Min, Fernando or Evan, any thoughts on this? Cheers, Brian On Tue, Feb 8, 2011 at 10:51 AM, Charlie Sharpsteen wrote: > Thanks for the reply Brian, > > On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger > wrote: > >> Charlie, >> >> > I was playing around with the new Qt widgets in 0.11-dev the other day >> and >> > was very impressed. I tried throwing together a quick plugin to add an >> > IPython console to the Qt-based program Quantum GIS [QGIS][1]. Creating >> the >> > plugin was easy enough as QGIS runs an embedded Python interpreter and >> > exposes a lot of PyQt4 functionality. However, I noticed a couple of >> things >> > and was wondering if anyone on this list had any comments to share: >> > >> > - The shell is based on a stripped down version of `ipython-qtconsole` >> and >> > works but is very, very slow for some reason. The is a delay of a >> > couple >> of >> > seconds between when an action is triggered and a response comes back to >> the >> > terminal. The only guess I have that may explain this behavior is that >> QGIS >> > is a C++ application that embeds a Python interpreter whereas >> > ipython-qtconsole is a pure PyQt4 program. Perhaps QGIS is not letting >> the >> > Python threads run as often as a pure PyQt4 program would. >> > This could be a QGIS-specific problem, but I was wondering if anyone >> > else >> > had the chance to run the IPython Qt widget in an embedded interpreter >> and >> > noticed similar results? >> >> In general the ipython-qtconsole should be quite fast. But for that >> to be the case, the qt event loop has to be running. If the C++ is >> not letting the qt event loop run often enough, you would definitely >> see quite slow behavior. > > > This is what I suspect, I would probably have to spend some quality time > with the C++ side of the code to be sure. > > > >> > - Currently, the only IPython kernel available for use with the Qt >> > widget >> > runs the IPython interpreter in a separate process. For QGIS, it would >> be >> > very nice to have the option of running an IPython kernel inside the >> > same >> > process as the Qt widgets. IPython would then have direct access to the >> > QGIS interface and would be able to inspect and operatie on the PyQt4 >> > objects that compose the GUI providing a powerful tool for users >> performing >> > spatial analysis and developers building plugins. >> > Are there any plans to develop IPython kernels that do not run >> > in separate processes? I spent a few days trying to build one myself by >> > wrapping an InteractiveShell, but I feel like I am flailing a little bit >> > when it comes to creating a kernel_manager that will integrate with the >> Qt >> > frontend. If anyone has any design advice on how to implement an >> in-process >> > kernel I would love to hear it. >> >> There is interest in having an in process IPython, but I honestly >> think it will not work very well at all. This is mostly because of >> the GIL. I would simply turn the model inside out. If you run the >> QGIS code that has all the PyQt widgets in the IPython kernel (start >> the kernel with the pyqt event loop running using --pylab qt) your >> ipython session will have all the access that it needs. IOW, don't >> embed IPython in your app, embed your app in IPython. If this model >> doesn't work for you, we can talk more about how to get started on an >> in process IPython widget, but it will be a significant amount of work >> I'm afraid. > > > The problem with embedding QGIS in IPython is that the main app is written > in C++ and `import qgis` only works out of the box on Linux---for Windows > and OS X it is a bit of a hassle to get paths set up right. Many things are > wrapped via SIP and it is possible to run a lot of widgets from IPython but > the main windows are not so it is currently not possible to re-launch the > app under Python. > > QGIS currently includes a Python console that makes use of > code.InteractiveInterpreter from the standard library. This console it works > very well with the exception that I always find myself pounding on the tab > key in vain when I forget what I can do with a random Python object. > Replacing the QGIS Python console with an IPython console would be awesome > and would cut out a lot of trips to the QGIS source code and various > reference manuals. > > If an in-process kernel could be implemented that shared the same interface > as the QtKernelManager, then users could easily be given the option of > launching an external process for tasks where the GIL would be a bother. > > I am willing to take a stab at developing an in-process widget but after > thrashing around a bit this weekend I feel like some guidance would help me > focus my efforts. > > > >> > The plugin is available on [GitHub][2] under the tag `0.1` if anyone >> feels >> > interested in checking it out as an IPython use case. The 0.1 version >> > is >> > really quite simple---about 5-10 lines in console.py that do the actual >> > implementation. To use: >> > - Install QGIS >> > - Clone project as a sub-directory of ~/.qgis/python/plugins/ >> > - Run `make` (runs PyQt4 dev tools on ui and qrc files) >> > - Launch QGIS and enable Plugin through Plugins -> Manage Plugins menu >> > >> > Keep up the good work on IPython development! >> >> Thanks! >> >> Brian >> >> > -Charlie >> > [1]: http://www.qgis.org >> > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1 >> > _______________________________________________ >> > IPython-dev mailing list >> > IPython-dev at scipy.org >> > http://mail.scipy.org/mailman/listinfo/ipython-dev >> > >> > >> >> >> >> -- >> Brian E. Granger, Ph.D. >> Assistant Professor of Physics >> Cal Poly State University, San Luis Obispo >> bgranger at calpoly.edu >> ellisonbg at gmail.com >> > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From source at sharpsteen.net Sat Feb 12 15:16:26 2011 From: source at sharpsteen.net (Charlie Sharpsteen) Date: Sat, 12 Feb 2011 12:16:26 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: On Sat, Feb 12, 2011 at 11:38 AM, Brian Granger wrote: > I think there are two possible approaches: > > 1. Still use the pyzmq interface through KernelManager and the > SocketChannel objects, just use in process (threads) zeromq sockets. > While complicated, this might be the best approach. But, it might > take some GUI event loop magic to get working (because you can't run > anything in a thread. > The problem with ZMQ for me is that the shell still has poor responsiveness when embedded in QGIS. I still don't know if this problem is specific to QGIS or would affect other C++ Qt applications that run embedded Python interpreters. I don't have another C++ app handy that does this to cross-validate with. I did some hacking over the week to try and work around the problem that included: switching the ZmqSocketChannel to a subclass of QThread, tweaking the thread run priorities, fiddling with the signal connections, and nothing gave good results. So, the problem could be: 1. With the way QGIS handles threads controlled by Python 2. With the way the ZeroMQ event loop behaves when embedded in a C++ app 3. Other I noticed that if I interact with the console window somehow, like twiddling the scroll wheel, while it is waiting then a refresh occurs that brings in the results from the external kernel. Also, I wrote a quick example in pure PyQGIS where Python is running the whole show and it is just as responsive as `ipython-qtconsole`. So, I suspect explanation #1 is still the most likely cause for this behavior. I haven't heard anything back from the QGIS-dev mailing list yet, so the ZMQ route is currently cold for me at this point. 2. Get rid of the zeromq stuff entirely. This is probably easier. > To do this, I would check out the subclass of InteractiveShell in > IPython.zmq.zmqshell. It should be pretty easy to build a subclass of > InteractiveShell that works, *but*, it leaves you without a GUI. > I also gave this a shot but creating a very primitive "kernel manager" that ran an InteractiveShell inside an OutputTrap that was subclassed to also redirect io.Term. This worked fairly well and I was able to get something working that was responsive in QGIS and supported tab-completion. There were two main difficulties I ran into going this route: - The frontend widgets seem to know a lot about the backend implementation which makes it difficult to create a new kernel manager that does something different under the hood. - The ZMQ kernel manager currently forms the base class for all kernel managers. It seems like it might be a good idea to: - Create an ABC for kernel managers that defines everything a frontend widget really needs to know in order to use them. - Replace specific references in FrontendWidgets like self.kernel_manager.xreq_channel.execute with something a little more abstract such as self.kernel_manager.execute and let the kernel manager worry about the details of which socket, channel, QSignal, etc is actually carrying the message. These are just my thoughts, and I reserve the right to be wrong :) To summarize a bit of a rambling email: embedding an InteractiveShell has proven to be a very promising route. At this point, the main question I have concerning this implementation is: - Is there a better way to set the stout/stderr streams of kernel than having the kernel manager run every call to "run_source" inside an OutputTrap? Final note: you guys are doing an awesome job with the rewrite, on the whole it has been very easy to hack on IPython without breaking everything since things are well componentized. -Charlie Min, Fernando or Evan, any thoughts on this? > > Cheers, > > Brian > > > On Tue, Feb 8, 2011 at 10:51 AM, Charlie Sharpsteen > wrote: > > Thanks for the reply Brian, > > > > On Mon, Feb 7, 2011 at 8:15 PM, Brian Granger >> wrote: > > > >> Charlie, > >> > >> > I was playing around with the new Qt widgets in 0.11-dev the other > day > >> and > >> > was very impressed. I tried throwing together a quick plugin to add > an > >> > IPython console to the Qt-based program Quantum GIS [QGIS][1]. > Creating > >> the > >> > plugin was easy enough as QGIS runs an embedded Python interpreter and > >> > exposes a lot of PyQt4 functionality. However, I noticed a couple of > >> things > >> > and was wondering if anyone on this list had any comments to share: > >> > > >> > - The shell is based on a stripped down version of `ipython-qtconsole` > >> and > >> > works but is very, very slow for some reason. The is a delay of a > >> > couple > >> of > >> > seconds between when an action is triggered and a response comes back > to > >> the > >> > terminal. The only guess I have that may explain this behavior is > that > >> QGIS > >> > is a C++ application that embeds a Python interpreter whereas > >> > ipython-qtconsole is a pure PyQt4 program. Perhaps QGIS is not > letting > >> the > >> > Python threads run as often as a pure PyQt4 program would. > >> > This could be a QGIS-specific problem, but I was wondering if anyone > >> > else > >> > had the chance to run the IPython Qt widget in an embedded interpreter > >> and > >> > noticed similar results? > >> > >> In general the ipython-qtconsole should be quite fast. But for that > >> to be the case, the qt event loop has to be running. If the C++ is > >> not letting the qt event loop run often enough, you would definitely > >> see quite slow behavior. > > > > > > This is what I suspect, I would probably have to spend some quality time > > with the C++ side of the code to be sure. > > > > > > > >> > - Currently, the only IPython kernel available for use with the Qt > >> > widget > >> > runs the IPython interpreter in a separate process. For QGIS, it > would > >> be > >> > very nice to have the option of running an IPython kernel inside the > >> > same > >> > process as the Qt widgets. IPython would then have direct access to > the > >> > QGIS interface and would be able to inspect and operatie on the PyQt4 > >> > objects that compose the GUI providing a powerful tool for users > >> performing > >> > spatial analysis and developers building plugins. > >> > Are there any plans to develop IPython kernels that do not run > >> > in separate processes? I spent a few days trying to build one myself > by > >> > wrapping an InteractiveShell, but I feel like I am flailing a little > bit > >> > when it comes to creating a kernel_manager that will integrate with > the > >> Qt > >> > frontend. If anyone has any design advice on how to implement an > >> in-process > >> > kernel I would love to hear it. > >> > >> There is interest in having an in process IPython, but I honestly > >> think it will not work very well at all. This is mostly because of > >> the GIL. I would simply turn the model inside out. If you run the > >> QGIS code that has all the PyQt widgets in the IPython kernel (start > >> the kernel with the pyqt event loop running using --pylab qt) your > >> ipython session will have all the access that it needs. IOW, don't > >> embed IPython in your app, embed your app in IPython. If this model > >> doesn't work for you, we can talk more about how to get started on an > >> in process IPython widget, but it will be a significant amount of work > >> I'm afraid. > > > > > > The problem with embedding QGIS in IPython is that the main app is > written > > in C++ and `import qgis` only works out of the box on Linux---for Windows > > and OS X it is a bit of a hassle to get paths set up right. Many things > are > > wrapped via SIP and it is possible to run a lot of widgets from IPython > but > > the main windows are not so it is currently not possible to re-launch the > > app under Python. > > > > QGIS currently includes a Python console that makes use of > > code.InteractiveInterpreter from the standard library. This console it > works > > very well with the exception that I always find myself pounding on the > tab > > key in vain when I forget what I can do with a random Python object. > > Replacing the QGIS Python console with an IPython console would be > awesome > > and would cut out a lot of trips to the QGIS source code and various > > reference manuals. > > > > If an in-process kernel could be implemented that shared the same > interface > > as the QtKernelManager, then users could easily be given the option of > > launching an external process for tasks where the GIL would be a bother. > > > > I am willing to take a stab at developing an in-process widget but after > > thrashing around a bit this weekend I feel like some guidance would help > me > > focus my efforts. > > > > > > > >> > The plugin is available on [GitHub][2] under the tag `0.1` if anyone > >> feels > >> > interested in checking it out as an IPython use case. The 0.1 version > >> > is > >> > really quite simple---about 5-10 lines in console.py that do the > actual > >> > implementation. To use: > >> > - Install QGIS > >> > - Clone project as a sub-directory of ~/.qgis/python/plugins/ > >> > - Run `make` (runs PyQt4 dev tools on ui and qrc files) > >> > - Launch QGIS and enable Plugin through Plugins -> Manage Plugins > menu > >> > > >> > Keep up the good work on IPython development! > >> > >> Thanks! > >> > >> Brian > >> > >> > -Charlie > >> > [1]: http://www.qgis.org > >> > [2]: http://www.github.com/Sharpie/qgis-ipython/tree/0.1 > >> > _______________________________________________ > >> > IPython-dev mailing list > >> > IPython-dev at scipy.org > >> > http://mail.scipy.org/mailman/listinfo/ipython-dev > >> > > >> > > >> > >> > >> > >> -- > >> Brian E. Granger, Ph.D. > >> Assistant Professor of Physics > >> Cal Poly State University, San Luis Obispo > >> bgranger at calpoly.edu > >> ellisonbg at gmail.com > >> > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://mail.scipy.org/mailman/listinfo/ipython-dev > > > > > > > > -- > Brian E. Granger, Ph.D. > Assistant Professor of Physics > Cal Poly State University, San Luis Obispo > bgranger at calpoly.edu > ellisonbg at gmail.com > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Feb 12 19:56:35 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 16:56:35 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger wrote: > > One thing I think we should do is figure out a way of dividing the > review workload so nothing slips through the cracks. ?When a review > comes in, do we want to assign a "lead" reviewer that will make sure > it gets pushed through in a timely manner? ?Sort of like journals > assign reviewers to articles. ?Our current situation is like running a > peer review journal, without an editor to manage the review proces... Yup, totally agreed. I think Thomas is going to send soon an email on this one, he has some good ideas to help out. One thing to keep in mind: currently github has no way of notifying of pull requests anyone outside of team members with write access to the repo. But they supposedly are working on it: http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list So hopefully soon we should be able to add the whole -dev list to get auto-notified of pull requests, which will help in letting people know of requests they could potentially help with. Cheers, f From fperez.net at gmail.com Sat Feb 12 19:57:12 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 16:57:12 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Fri, Feb 11, 2011 at 8:11 PM, Brian Granger wrote: > > OK, if we do this, let make sure that: > > * There are tests. > * It is fully documented in docstrings. > > I am mostly concerned about the fact that this type of stuff tends to > get deposited with us to maintain in the future (like port to Python > 3). ?Ideally, these things would have maintainers that are committed > to helping out with things. Very valid points. Cheers, f From fperez.net at gmail.com Sat Feb 12 20:01:54 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 17:01:54 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hi Nicolas, On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier wrote: > After reading the commentaries, I realize there are still some work to do before the code can be properly integrated within IPython and I will try to make a nicer version as well as fully document it (using the template.py). For the git pull, I'm not very familiar with it and I don't know the proper command to pull or push, woudl yu have any pointer on this ? > Yes, the first 4 sections of our developer guide document coding and documentation guidelines, as well as the pull request process with github: http://ipython.github.com/ipython-doc/dev/development/index.html If you've never used git/github and get confused with any of the steps, please do let us know and we'll do our best to help. While there's a little learning curve at the beginning, I think you'll find that learning to use git/github will pay off in many ways in your work. Many scientific python projects have either already moved or are in the process of moving to git and/or github (numpy, ipython, enthought.*, nipy.*, sympy, and over the next few days matplotlib and scipy, likely many others too). > For making the code to work with the QT console, it would require to parse the string sent to the console and interpret the ascii sequences. I did it some time ago for a gtk/ipython console but I'm not familiar at all with qt. (parsing code is of course available if one want to work on it). > Interestingly, the behavior of the Qt console was *identical* to that of the Linux text console (the 'bare' screen console, outside of X11). I don't know why that would be the case, just a data point for you... But for now, working in an xterm is already good. Cheers, f From sunqiang at gmail.com Sat Feb 12 20:22:53 2011 From: sunqiang at gmail.com (sunqiang) Date: Sun, 13 Feb 2011 09:22:53 +0800 Subject: [IPython-dev] Issue of install ipython-py3k In-Reply-To: References: Message-ID: Thomas, Thanks for the fix, I'll keep using ipython on python3.2 to test my little script. On Sun, Feb 13, 2011 at 3:11 AM, Thomas Kluyver wrote: > Thanks, Sunqiang, I've pushed the fix. > > Keep me posted of any more problems you find using it with Python 3.2. I > probably won't upgrade until April, with the next Ubuntu release. > > Technical details: The value coming from a subprocess.Popen().communicate() > was a byte string. This was being written to a config file, but the > rewritten configparser in Python 3.2 specifically checks that the option to > be set is a (unicode) str. I've actually specified that it should be decoded > as ascii: the hash digest should never include any non-ascii characters, and > we'll find out quickly if it's doing anything unexpected. > > Thomas > From fperez.net at gmail.com Sat Feb 12 20:40:11 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 17:40:11 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hey, Sat, Feb 12, 2011 at 10:42 AM, Brian Granger wrote: > > Do you think it would make sene to implement this as an object that > has special print methods like __pretty__, __png__, etc. ?That would > make it super easy to add support for other frontends. Mmh, interesting... Initially I thought of this as a purely functional thing, a terminal version of imshow() and nothing else. But what you're thinking of is connecting it to our extended display system... I actually do like that, especially since it would be code coming from IPython. Here's a suggestion. Considering: 1. That Nicolas is new to IPython development and not likely to know all the details of the new display design. 2. That we're still sorting out (in the other thread) precisely how we want to go about that design, after some of the concerns Robert raised. I propose that we go for now with a purely functional implementation (basically a cleaned up, documented and tested version of Nicolas current code, but with the same overall approach). This will make it easier for Nicolas to contribute his work now. But we keep this as a prime candidate to test out the display system, and once we finalize the design decisions, it can be refactored in that direction. The refactoring would be very easy, since it's basically replacing the current printing with assembling a string to be the return value of the repr. I now realize this is probably the best course of action for another reason: in its current form, the code works even with the plain terminal ipython that has no knowledge of the full display machinery. If we move it to that model, I think it wouldn't quite work in our current terminal client (correct me if I'm wrong). Does that sound reasonable? Cheers, f From fperez.net at gmail.com Sat Feb 12 21:05:30 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 18:05:30 -0800 Subject: [IPython-dev] All documentation now on Github via gh-pages Message-ID: Hi everyone, Min (who did most of the work) and I just completed moving our docs over to Github, using the gh-pages system but via the tricks we discussed before (what I wrote for datarray) that avoid polluting the main ipython repo with any static HTML content. The old documentation page on the Moin wiki is now empty and simply points to: http://ipython.github.com/ipython-doc/ This is actually all nicely generated from a real git repo: https://github.com/ipython/ipython-doc So that any core team member can easily push doc updates. Note that this repo contains actually almost zero original content (just an index.rst page and a trivial build script), all the sources are still in the doc/ directory of the IPython main repo. This repo is simply a static holder of the generated HTML/pdf so that we can take advantage of gh-pages hosting, 'git push' for publication, and repository administrative control for permissions (instead of me manually managing ssh keys on the scipy.org server, which I have no time for). Thanks to Min for adapting the previous tools and setting things up with the CSS. In summary, for contributing to the docs: - for the actual manual sources, everything is in the real ipython repo, docs/source directory like always. - to edit the index page or the css for looks, do fork/clone the -doc repo and make a pull request on that one (it will be handled just like any other pull request on the main repo, but likely with quicker turnaround since there's no major code to review there). Cheers, f From fperez.net at gmail.com Sat Feb 12 22:01:47 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 12 Feb 2011 19:01:47 -0800 Subject: [IPython-dev] Welcoming Thomas to the core/merge team Message-ID: Hi all, I just wanted to welcome Thomas Kluyver to the select club of people who mange to work on IPython instead of completing their PhDs... :) After all his great work on the py3 port, pyzmq fixes for py3 also, and work on the core, frequent code reviews, etc, it's a pleasure to have one more set of hands directly participating in merging contributions. I hope that this will also help with reducing the turnaround time for reviews and feedback when I or other traditional members bottleneck out on that strange thing that passes for a normal life. Cheers, f From Nicolas.rougier at inria.fr Sun Feb 13 07:37:44 2011 From: Nicolas.rougier at inria.fr (Nicolas Rougier) Date: Sun, 13 Feb 2011 13:37:44 +0100 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Thanks, the guide has been very helpful (I opened a bug report for a small typo). If I did thing properly, here is the summary: https://github.com/rougier/ipython/compare/master...ascii_imshow I added the possibility to display MxNx3 (RGB) or MxNx4 (RGBA) since the code is quite similar. There is no test yet since I'm not sure how to test such a visual thing. I also tried to detect terminal number of colors using curses (if available) because I don't know if IPython already got some terminal capabilities related functions (mainly, size and number of colors). Nicolas From ben.root at ou.edu Sun Feb 13 11:48:53 2011 From: ben.root at ou.edu (Benjamin Root) Date: Sun, 13 Feb 2011 10:48:53 -0600 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: On Saturday, February 12, 2011, Fernando Perez wrote: > On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger wrote: >> >> One thing I think we should do is figure out a way of dividing the >> review workload so nothing slips through the cracks. ?When a review >> comes in, do we want to assign a "lead" reviewer that will make sure >> it gets pushed through in a timely manner? ?Sort of like journals >> assign reviewers to articles. ?Our current situation is like running a >> peer review journal, without an editor to manage the review proces... > > Yup, totally agreed. ?I think Thomas is going to send soon an email on > this one, he has some good ideas to help out. > > One thing to keep in mind: currently github has no way of notifying of > pull requests anyone outside of team members with write access to the > repo. ?But they supposedly are working on it: > > http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list > > So hopefully soon we should be able to add the whole -dev list to get > auto-notified of pull requests, which will help in letting people know > of requests they could potentially help with. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > Actually, I get notified of pull requests for numpy through an rss feed. All I had to do was watch the numpy repo and add my dashboard feed to my reader. Not email, but it works. Ben Root From takowl at gmail.com Sun Feb 13 12:45:10 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 13 Feb 2011 17:45:10 +0000 Subject: [IPython-dev] Outstanding pull requests Message-ID: By way of my debut in the core team, I've prepared a list of all the open pull requests, with a very brief assessment of each, in the hope that we can start sorting them out (we're currently at 31 open requests): Magic functions documentation (several branches from SciPy India): [The first two listed here seem to have changed various bits of indentation beyond the specific things they're described as changing. This makes the diffs rather long. Do we want to copy the key changes out in new commits, using the --author option to credit them?] - cd: looks like some examples are at the bash prompt, not the ipython prompt. https://github.com/ipython/ipython/pull/233/files - profile: Has neatly documented the Python profile module. Unfortunately, IPython's %profile command does something entirely different. https://github.com/ipython/ipython/pull/232/files - logstart: Seems OK, although there's an "examples" title added for logstop with no actual examples. logstop probably doesn't need an example, but we should tidy up the "examples" title. https://github.com/ipython/ipython/pull/231/files - edit: Clean diff, but giving an example for edit is tricky, because you can't show the interactive editor. Also restates the -p option in a comment in the example. https://github.com/ipython/ipython/pull/230/files (closed 220 as duplicate) - pdef (it's confusing to use magic functions as the arguments for the example, we should have a user defined function) & colors (one example is probably enough, and the last two are wrong, using %usage instead of %colors) https://github.com/ipython/ipython/pull/229/files - who, whos, who_ls: Looks good. https://github.com/ipython/ipython/pull/228 - logon: Not quite sure what's happened here. Example seems to be a bunch of Java-ish code. It probably needs to be redone from scratch. https://github.com/ipython/ipython/pull/227 - pwd: pwd example is good, but has also added to exit (doesn't really need an example), dirs (example doesn't really show what it's for), and color (just an "examples" heading, no actual examples. https://github.com/ipython/ipython/pull/226/files - save: Clean diff, but the example and explanation aren't very clear. Probably needs to be redone from scratch. https://github.com/ipython/ipython/pull/225 - cpaste: Appears to have ripped out the actual function while writing the docs, and stitched its definition onto the magic_paste function code. Example could also be a bit clearer. https://github.com/ipython/ipython/pull/224 - colors: Same changes (& problems) as request 229. https://github.com/ipython/ipython/pull/223 - timeit: Too many equivalent examples. https://github.com/ipython/ipython/pull/222/files - alias: Already had examples, changes remove one of them. Adds two, but one (ls) is available without aliasing, and the other is inaccurate (the example shows the result of a man command). https://github.com/ipython/ipython/pull/221 - pycat: Doesn't actually use a Python file, so rather defeats the point of pycat. Not sure that an example is really helpful, as it doesn't show the colouring you get at a terminal. https://github.com/ipython/ipython/pull/219 - hist: Formatting is off (probably related to a bug in the code). Also, the "this is an example of..." line seems redundant. https://github.com/ipython/ipython/pull/218/files Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save), 232 (%profile) with "try again" and 227 (%logon), for which I'm not sure that it's even possible to do an example. The others will need further work. Fernando, any sort of feel for how many of the SciPy India contributors will still be in the loop to revise their contributions? Looking at the others: Small pull requests: - %hist output format: I've solved the same problem in a different way in pull request 261, see below. https://github.com/ipython/ipython/pull/236 - Logging unicode characters (0.10 series): Good start. See my comment there for minor concern. https://github.com/ipython/ipython/pull/249 - check_gtk: Removes a function. Mark Voorhies has tested the change and commented that it works. I'll admit to not understanding what it's about. https://github.com/ipython/ipython/pull/237 - Typos in zmq/blockingkernelmanager.py: Seems OK to me. https://github.com/ipython/ipython/pull/203/files - cProfiling ipython.py: Updated in response to Brian's concerns. Needs reviewing again. https://github.com/ipython/ipython/pull/193 - Autocomplete in Emacs: Removes the all_completions function (which apparently was broken). One other person says it works. Needs review. https://github.com/ipython/ipython/pull/204 - Unicode issues [My changes]: Tests now integrated into main test suite (thanks to help from Robert Kern). Needs review. https://github.com/ipython/ipython/pull/252 - Wildcard module [My changes]: Fixes issue 129. The bug and the fix are pretty much self contained, so shouldn't interfere with changes in other code. No feedback yet. (this is pushing what I'd call small, but shouldn't change functionality besides fixing the bug) https://github.com/ipython/ipython/pull/251 Medium pull requests: - Fixing up various magic commands with the new history system [My changes]: Revised after Fernando's comments, unit tests added. Conflicts with Madhusudan's #203, see below. https://github.com/ipython/ipython/pull/261 - %hist command bugs: This solves one of the same problems I've fixed in #261, in a different way. https://github.com/ipython/ipython/pull/203/files - Unbundling external libraries: Revised in response to Fernando's and Brian's concerns. Needs reviewing again. https://github.com/ipython/ipython/pull/191 - Pyside compatibility: Waiting for contributor to resolve bug with SVG handling. https://github.com/ipython/ipython/pull/259 - Kernel logging: Fernando has requested a final check from Robert. https://github.com/ipython/ipython/pull/264 Larger pull requests: - Magic arguments decorators. Updated in response to Fernando's comments. Needs reviewing again. https://github.com/ipython/ipython/pull/199 Not quite yet: - HTML Notebook: Gained a lot of interest, but not quite ready yet. Possible project for GSoC this year. (But if we're going to accumulate alternative frontends, should we consider putting each frontend in its own repo?) https://github.com/ipython/ipython/pull/179 - MinRK's newparallel: Work in progress. Lots of work. https://github.com/ipython/ipython/pull/254 Best wishes, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Sun Feb 13 13:10:59 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 13 Feb 2011 10:10:59 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hi, On Sat, Feb 12, 2011 at 5:40 PM, Fernando Perez wrote: > Hey, Sat, Feb 12, 2011 at 10:42 AM, Brian Granger wrote: >> >> Do you think it would make sene to implement this as an object that >> has special print methods like __pretty__, __png__, etc. ?That would >> make it super easy to add support for other frontends. > > Mmh, interesting... ?Initially I thought of this as a purely > functional thing, a terminal version of imshow() and nothing else. > But what you're thinking of is connecting it to our extended display > system... I actually do like that, especially since it would be code > coming from IPython. Great. > Here's a suggestion. ?Considering: > > 1. That Nicolas is new to IPython development and not likely to know > all the details of the new display design. > > 2. That we're still sorting out (in the other thread) precisely how we > want to go about that design, after some of the concerns Robert > raised. Definitely. > I propose that we go for now with a purely functional implementation > (basically a cleaned up, documented and tested version of Nicolas > current code, but with the same overall approach). ?This will make it > easier for Nicolas to contribute his work now. ?But we keep this as a > prime candidate to test out the display system, and once we finalize > the design decisions, it can be refactored in that direction. ?The > refactoring would be very easy, since it's basically replacing the > current printing with assembling a string to be the return value of > the repr. Sounds good. > I now realize this is probably the best course of action for another > reason: in its current form, the code works even with the plain > terminal ipython that has no knowledge of the full display machinery. > If we move it to that model, I think it wouldn't quite work in our > current terminal client (correct me if I'm wrong). Correct, but I think we could make a version that did work with a plain terminal at the same time. > Does that sound reasonable? Yep. Cheers, Brian > Cheers, > > f > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Sun Feb 13 13:48:21 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 13 Feb 2011 10:48:21 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: On Sat, Feb 12, 2011 at 4:56 PM, Fernando Perez wrote: > On Sat, Feb 12, 2011 at 11:22 AM, Brian Granger wrote: >> >> One thing I think we should do is figure out a way of dividing the >> review workload so nothing slips through the cracks. ?When a review >> comes in, do we want to assign a "lead" reviewer that will make sure >> it gets pushed through in a timely manner? ?Sort of like journals >> assign reviewers to articles. ?Our current situation is like running a >> peer review journal, without an editor to manage the review proces... > > Yup, totally agreed. ?I think Thomas is going to send soon an email on > this one, he has some good ideas to help out. > > One thing to keep in mind: currently github has no way of notifying of > pull requests anyone outside of team members with write access to the > repo. ?But they supposedly are working on it: > > http://support.github.com/discussions/pull-requests/118-send-pull-request-notifications-to-all-watchers-or-an-external-mail-list > > So hopefully soon we should be able to add the whole -dev list to get > auto-notified of pull requests, which will help in letting people know > of requests they could potentially help with. Yes, that would be nice. Cheers, Brian > Cheers, > > f > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From ellisonbg at gmail.com Sun Feb 13 14:05:11 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Sun, 13 Feb 2011 11:05:11 -0800 Subject: [IPython-dev] Welcoming Thomas to the core/merge team In-Reply-To: References: Message-ID: Welcome Thomas! Cheers, Brian On Sat, Feb 12, 2011 at 7:01 PM, Fernando Perez wrote: > Hi all, > > I just wanted to welcome Thomas Kluyver to the select club of people > who mange to work on IPython instead of completing their PhDs... :) > After all his great work on the py3 port, pyzmq fixes for py3 also, > and work on the core, frequent code reviews, etc, it's a pleasure to > have one more set of hands directly participating in merging > contributions. ?I hope that this will also help with reducing the > turnaround time for reviews and feedback when I or other traditional > members bottleneck out on that strange thing that passes for a normal > life. > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From epatters at enthought.com Sun Feb 13 14:49:17 2011 From: epatters at enthought.com (Evan Patterson) Date: Sun, 13 Feb 2011 13:49:17 -0600 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Sat, Feb 12, 2011 at 7:01 PM, Fernando Perez wrote: > Hi Nicolas, > > On Sat, Feb 12, 2011 at 1:32 AM, Nicolas Rougier > wrote: > > After reading the commentaries, I realize there are still some work to do > before the code can be properly integrated within IPython and I will try to > make a nicer version as well as fully document it (using the template.py). > For the git pull, I'm not very familiar with it and I don't know the proper > command to pull or push, woudl yu have any pointer on this ? > > > > Yes, the first 4 sections of our developer guide document coding and > documentation guidelines, as well as the pull request process with > github: > > http://ipython.github.com/ipython-doc/dev/development/index.html > > If you've never used git/github and get confused with any of the > steps, please do let us know and we'll do our best to help. While > there's a little learning curve at the beginning, I think you'll find > that learning to use git/github will pay off in many ways in your > work. Many scientific python projects have either already moved or > are in the process of moving to git and/or github (numpy, ipython, > enthought.*, nipy.*, sympy, and over the next few days matplotlib and > scipy, likely many others too). > > > For making the code to work with the QT console, it would require to > parse the string sent to the console and interpret the ascii sequences. I > did it some time ago for a gtk/ipython console but I'm not familiar at all > with qt. (parsing code is of course available if one want to work on it). > > > > Interestingly, the behavior of the Qt console was *identical* to that > of the Linux text console (the 'bare' screen console, outside of X11). > I don't know why that would be the case, just a data point for you... > > But for now, working in an xterm is already good. > The reason is that we implement the (more useful) ANSI escape sequences in the Qt console, but not any of the xterm-specific sequences. Since most terminal emulators implement them for compatibility reasons, so should we---or at least we should implement the important ones. Comparing Nicolas' code with this: http://invisible-island.net/xterm/ctlseqs/ctlseqs.html it look likes we need only implement two more sequences to get his code working. I will add this to my TODO list (but as a low priority item, I'm afraid). Evan > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun Feb 13 16:00:52 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 13:00:52 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Sun, Feb 13, 2011 at 11:49 AM, Evan Patterson wrote: > The reason is that we implement the (more useful) ANSI escape sequences in > the Qt console, but not any of the xterm-specific sequences. Since most > terminal emulators implement them for compatibility reasons, so should > we---or at least we should implement the important ones. Aha, thanks for the info, that explains it. Sure, if you can get around to it at some point and it's not hard to implement, all the better. Cheers, f From fperez.net at gmail.com Sun Feb 13 16:29:29 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 13:29:29 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: Hi Nicolas, On Sun, Feb 13, 2011 at 4:37 AM, Nicolas Rougier wrote: > Thanks, the guide has been very helpful (I opened a bug report for a small typo). Glad to hear that, and thanks for filing the report; Thomas already jumped on it and the online guide should be soon updated once Thomas sets his environment to do the gh-pages pushes (up until now only Min and I had been pushing). > If I did thing properly, here is the summary: > https://github.com/rougier/ipython/compare/master...ascii_imshow Excellent! I would suggest that you now simply click on the 'Pull Request' button so we can review your contribution for merge. It's likely to go through a few more iterations before it's merged, but once in a pull request, we have a nice centralized location for the discussion, you can keep fine-tuning it, and when ready it will go in cleanly. > I added the possibility to display MxNx3 (RGB) or MxNx4 (RGBA) since the code is quite similar. There is no test yet since I'm not sure how to test such a visual thing. Great features. The topic of testing is very important, so I want to address this in some detail, hopefully this will be useful to others as well. You are indeed correct that testing something visual is tricky. The point is to realize that printing is a side effect, and testing side effects is in general quite hard. But you can think of printing as a two-step process: a) computing the string to be printed b) actually sending that string to the terminal. While b) more or less can only be checked interactively by a human, a) can be automatically validated. Basically, when thinking of testing, you want to code so there's a clean separation between the functional computation and the part that does purely the side effects. That is, instead of: def show_foo(x): print 'abc', x, 'pqr' you would do def compute_foo(x): return 'abc %s pqr' % x def show_foo(x): print compute_foo(x) That way at least compute_foo(x) can be fully tested in a 100% functional way. And then a test can also be written for show_foo() just to ensure it doesn't crash (so that any api changes in compute_foo are correctly caught also in show_foo), but that test doesn't need to actually validate the output. In simple cases it may (such as with a doctest), but in others it may be hard, like in your situation. Let me know if this helps. If it's clear enough, we'll add it to the dev guidelines in the testing section. > I also tried to detect terminal number of colors using curses (if available) because I don't know if IPython already got some terminal capabilities related functions (mainly, size and number of colors). No idea on that front, sorry. Many thanks for your willingness to follow the (hopefully straightforward) process! We're really glad to have more contributions to the project, and I'm excited to start seeing things going more smoothly on this front. Best regards, f From fperez.net at gmail.com Sun Feb 13 17:05:49 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 14:05:49 -0800 Subject: [IPython-dev] No displayhook triggered for raw 'asdf' In-Reply-To: References: <20110126230020.GI5046@phare.normalesup.org> Message-ID: On Sun, Feb 13, 2011 at 8:48 AM, Benjamin Root wrote: > Actually, I get notified of pull requests for numpy through an rss > feed. ?All I had to do was watch the numpy repo and add my dashboard > feed to my reader. ?Not email, but it works. Ah, thanks for that tip! I've never really gotten hooked on RSS for some reason, but this is may be a good solution for many. Cheers, f From takowl at gmail.com Sun Feb 13 17:29:31 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 13 Feb 2011 22:29:31 +0000 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On 13 February 2011 21:00, Fernando Perez wrote: > Aha, thanks for the info, that explains it. Sure, if you can get > around to it at some point and it's not hard to implement, all the > better. > Though I would add that the ability to render ASCII heatmaps in the Qt console where we can display png and svg is perhaps not terribly urgent. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun Feb 13 17:30:51 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 14:30:51 -0800 Subject: [IPython-dev] Ascii imshow In-Reply-To: References: <4FB05A3E-572F-44D5-9812-F451F9F32413@inria.fr> Message-ID: On Sun, Feb 13, 2011 at 2:29 PM, Thomas Kluyver wrote: > > Though I would add that the ability to render ASCII heatmaps in the Qt > console where we can display png and svg is perhaps not terribly urgent. Agreed. Evan said it would be low on his list, so it's one of those 'for an idle moment' things. Cheers, f From takowl at gmail.com Sun Feb 13 17:51:20 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sun, 13 Feb 2011 22:51:20 +0000 Subject: [IPython-dev] Issues with building documentation Message-ID: There seem to be some issues with getting the docs to build. My guess would be that these only show up if you've not built the docs before. On Fernando's instructions, I went into docs and ran ./gh-pages.py. This tries to call "git describe", which fails because it can't find annotated tags. I peeked at the source, took a guess, and ran "./gh-pages.py dev". This gets further, but fails when it tries to copy build/html into the new repo, because I haven't yet built the docs at all. Next, I try to build the html docs with "make html". This sets out, but then falls over at: ValueError: Could not import class or module 'IPython.Extensions.PhysicalQInteractive' specified for inheritance diagram This looks like a leftover from old code - our extensions directory is now all lower case, and there's no PhysicalQInteractive module there. Is there some sort of preparation step I've overlooked? Or is this area in need of some attention? Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sun Feb 13 18:35:26 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 15:35:26 -0800 Subject: [IPython-dev] Issues with building documentation In-Reply-To: References: Message-ID: Just for reference, no worries about this. We've sorted it out on IRC. On Sun, Feb 13, 2011 at 2:51 PM, Thomas Kluyver wrote: > There seem to be some issues with getting the docs to build. My guess would > be that these only show up if you've not built the docs before. > > On Fernando's instructions, I went into docs and ran ./gh-pages.py. This > tries to call "git describe", which fails because it can't find annotated > tags. I peeked at the source, took a guess, and ran "./gh-pages.py dev". > This gets further, but fails when it tries to copy build/html into the new > repo, because I haven't yet built the docs at all. > > Next, I try to build the html docs with "make html". This sets out, but then > falls over at: > > ValueError: Could not import class or module > 'IPython.Extensions.PhysicalQInteractive' specified for inheritance diagram > > This looks like a leftover from old code - our extensions directory is now > all lower case, and there's no PhysicalQInteractive module there. > > Is there some sort of preparation step I've overlooked? Or is this area in > need of some attention? > > Thanks, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > From fperez.net at gmail.com Sun Feb 13 19:36:21 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 16:36:21 -0800 Subject: [IPython-dev] Comments on IPython Qt widgets In-Reply-To: References: Message-ID: Dear Charlie, Just joining the others to welcome you to the list! It's great to see interest and new participants with regards to the new architecture and the Qt machinery. I hope we'll be able to help yo contribute and do useful thins with ipython in the GIS domain. On Mon, Feb 7, 2011 at 6:52 PM, Charlie Sharpsteen wrote: > Hello IPython developers, > I was playing around with the new Qt ?widgets in 0.11-dev the other day and > was very impressed. ?I tried?throwing?together a quick plugin to add an > IPython console to the Qt-based program Quantum GIS [QGIS][1]. ?Creating the > plugin was easy enough as QGIS runs an embedded Python interpreter and > exposes a lot of PyQt4 functionality.??However, I noticed a couple of things > and was wondering if anyone on this list had any comments to share: > > - Currently, the only IPython kernel available for use with the Qt widget > runs the IPython interpreter in a?separate?process. ?For QGIS, it would be > very nice to have the option of running an IPython kernel inside the same > process as the Qt widgets. ?IPython would then have direct access to the > QGIS interface and would be able to inspect and operatie on the PyQt4 > objects that compose the GUI providing a powerful tool for users performing > spatial analysis and developers building plugins. > Are there any plans to develop IPython kernels that do not run > in?separate?processes? ?I spent a few days trying to build one myself by > wrapping an InteractiveShell, but I feel like I am flailing a little bit > when it comes to creating a kernel_manager that will integrate with the Qt > frontend. ?If anyone has any design advice on how to implement an in-process > kernel I would love to hear it. I actually think that this is something we do very much need and would like to have, despite the (absolutely valid) caveats that Brian raised. While it's true that an in-process kernel can block and/or crash your entire app, it can also be very useful (see Mayavi for a good example where even with rather gross hack remains extremely useful). Let me try to explain our architecture a little bit. If this is clear enough let me know, and I'll add it to our dev docs. For all of this, keep this diagram in mind: http://ipython.github.com/ipython-doc/dev/development/messaging.html In a two-process model, the Kernel is proxied on the client process by a KernelManager object. This is the object that all client code uses to communicate with the kernel, and it's the *only* object in charge of doing explicit zmq calls. The rest of ipython has no idea zmq exists, it just talks to this KernelManager object. The basic KernelManager lives in: https://github.com/ipython/ipython/blob/master/IPython/zmq/kernelmanager.py but that can't be used directly, since each front end needs to have code to specifically handle the generated messages according to its own UI constraints. So for example the Qt console has here its version that wires the message handlers to Qt's event mechanisms: https://github.com/ipython/ipython/blob/master/IPython/frontend/qt/kernelmanager.py With this in mind, what would be needed is an object that exposes the kernelmanager interface but runs in-process instead of out of process, and is otherwise the same. Then, we could compose for Qt two KM's: one based on an out-of-process, zmq-based class (what we have today) and another one based on such an in-process KM that doesn't use zmq. In this manner, the code above would function identically, modulo some modifications so it doesn't mention external kernels on shutdown and somesuch. Let us know if this makes sense, I think it would be great to get something like this implemented in full. > Keep up the good work on IPython development! Thanks for the kind words! We hope to have you on board for the project. The new architecture is actually what we think is a fairly good and clean design, so contributions like what you're discussing would be great to have. Regards, f From fperez.net at gmail.com Sun Feb 13 20:01:31 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 13 Feb 2011 17:01:31 -0800 Subject: [IPython-dev] Outstanding pull requests In-Reply-To: References: Message-ID: Hi Thomas, I just wanted to thank you *enormously* for doing this. I have to switch gears out of ipython right now for some deadlines, but later in the week I'll return to this, and I'll try to flush a few of the easy ones out of the way. Some comments below. On Sun, Feb 13, 2011 at 9:45 AM, Thomas Kluyver wrote: > By way of my debut in the core team, I've prepared a list of all the open > pull requests, with a very brief assessment of each, in the hope that we can > start sorting them out (we're currently at 31 open requests): > > Magic functions documentation (several branches from SciPy India): > [The first two listed here seem to have changed various bits of indentation > beyond the specific things they're described as changing. This makes the > diffs rather long. Do we want to copy the key changes out in new commits, > using the --author option to credit them?] Good idea. These docstring requests were done by a number of teams at scipy india who were at the very beginning of their python/programming experience, so they are more likely to have beginner's problems. It's a real bummer that I dropped the ball in providing them with early feedback, but I had a ton of travel right after and just got overrun with life... So definitely crediting with --author but cleaning up the commit is a good strategy for those with unnecessary noise. > - cd: looks like some examples are at the bash prompt, not the ipython > prompt. https://github.com/ipython/ipython/pull/233/files > - profile: Has neatly documented the Python profile module. Unfortunately, > IPython's %profile command does something entirely different. > https://github.com/ipython/ipython/pull/232/files > - logstart: Seems OK, although there's an "examples" title added for logstop > with no actual examples. logstop probably doesn't need an example, but we > should tidy up the "examples" title. > https://github.com/ipython/ipython/pull/231/files > - edit: Clean diff, but giving an example for edit is tricky, because you > can't show the interactive editor. Also restates the -p option in a comment > in the example. https://github.com/ipython/ipython/pull/230/files (closed > 220 as duplicate) > - pdef (it's confusing to use magic functions as the arguments for the > example, we should have a user defined function) & colors (one example is > probably enough, and the last two are wrong, using %usage instead of > %colors) https://github.com/ipython/ipython/pull/229/files > - who, whos, who_ls: Looks good. https://github.com/ipython/ipython/pull/228 > - logon: Not quite sure what's happened here. Example seems to be a bunch of > Java-ish code. It probably needs to be redone from scratch. > https://github.com/ipython/ipython/pull/227 > - pwd: pwd example is good, but has also added to exit (doesn't really need > an example), dirs (example doesn't really show what it's for), and color > (just an "examples" heading, no actual examples. > https://github.com/ipython/ipython/pull/226/files > - save: Clean diff, but the example and explanation aren't very clear. > Probably needs to be redone from scratch. > https://github.com/ipython/ipython/pull/225 > - cpaste: Appears to have ripped out the actual function while writing the > docs, and stitched its definition onto the magic_paste function code. > Example could also be a bit clearer. > https://github.com/ipython/ipython/pull/224 > - colors: Same changes (& problems) as request 229. > https://github.com/ipython/ipython/pull/223 > - timeit: Too many equivalent examples. > https://github.com/ipython/ipython/pull/222/files > - alias: Already had examples, changes remove one of them. Adds two, but one > (ls) is available without aliasing, and the other is inaccurate (the example > shows the result of a man command). > https://github.com/ipython/ipython/pull/221 > - pycat: Doesn't actually use a Python file, so rather defeats the point of > pycat. Not sure that an example is really helpful, as it doesn't show the > colouring you get at a terminal. https://github.com/ipython/ipython/pull/219 > - hist: Formatting is off (probably related to a bug in the code). Also, the > "this is an example of..." line seems redundant. > https://github.com/ipython/ipython/pull/218/files > > Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save), 232 > (%profile) with "try again" and 227 (%logon), for which I'm not sure that > it's even possible to do an example. > > The others will need further work. Fernando, any sort of feel for how many > of the SciPy India contributors will still be in the loop to revise their > contributions? Thanks for going through all of this! Unfortunately due to the delay, and the fact that many of them were first-timers to pretty much everything, I'm not too hopeful. I think if there are some where we can easily finish up what they did with minimal work, it would be a nice gesture. But those that require significant rework, just dropping a comment would be sufficient. If they do pop up again and come back with improvements, then we can engage them better for the future. In my defense, I was trying to handle a sprint where 60! people wanted to contribute, most of which had never coded a single line of Python. So I tried to do something where at least they would learn during the day, even if it wasn't realistic to expect actual final contributions to the codebase to come from everyone. > Looking at the others: > > Small pull requests: > - %hist output format: I've solved the same problem in a different way in > pull request 261, see below. https://github.com/ipython/ipython/pull/236 > - Logging unicode characters (0.10 series): Good start. See my comment there > for minor concern. https://github.com/ipython/ipython/pull/249 > - check_gtk: Removes a function. Mark Voorhies has tested the change and > commented that it works. I'll admit to not understanding what it's about. > https://github.com/ipython/ipython/pull/237 > - Typos in zmq/blockingkernelmanager.py: Seems OK to me. > https://github.com/ipython/ipython/pull/203/files > - cProfiling ipython.py: Updated in response to Brian's concerns. Needs > reviewing again. https://github.com/ipython/ipython/pull/193 > - Autocomplete in Emacs: Removes the all_completions function (which > apparently was broken). One other person says it works. Needs review. > https://github.com/ipython/ipython/pull/204 > - Unicode issues [My changes]: Tests now integrated into main test suite > (thanks to help from Robert Kern). Needs review. > https://github.com/ipython/ipython/pull/252 > - Wildcard module [My changes]: Fixes issue 129. The bug and the fix are > pretty much self contained, so shouldn't interfere with changes in other > code. No feedback yet. (this is pushing what I'd call small, but shouldn't > change functionality besides fixing the bug) > https://github.com/ipython/ipython/pull/251 > > Medium pull requests: > - Fixing up various magic commands with the new history system [My changes]: > Revised after Fernando's comments, unit tests added. Conflicts with > Madhusudan's #203, see below. https://github.com/ipython/ipython/pull/261 > - %hist command bugs: This solves one of the same problems I've fixed in > #261, in a different way. https://github.com/ipython/ipython/pull/203/files > - Unbundling external libraries: Revised in response to Fernando's and > Brian's concerns. Needs reviewing again. > https://github.com/ipython/ipython/pull/191 > - Pyside compatibility: Waiting for contributor to resolve bug with SVG > handling. https://github.com/ipython/ipython/pull/259 > - Kernel logging: Fernando has requested a final check from Robert. > https://github.com/ipython/ipython/pull/264 > > Larger pull requests: > - Magic arguments decorators. Updated in response to Fernando's comments. > Needs reviewing again. https://github.com/ipython/ipython/pull/199 > > Not quite yet: > - HTML Notebook: Gained a lot of interest, but not quite ready yet. Possible > project for GSoC this year. (But if we're going to accumulate alternative > frontends, should we consider putting each frontend in its own repo?) > https://github.com/ipython/ipython/pull/179 > - MinRK's newparallel: Work in progress. Lots of > work.https://github.com/ipython/ipython/pull/254 Ok, thanks again for this analysis. Over the next few days, I suggest that anyone willing to 'take ownership' of one just pitch in and add a comment to the corresponding one, mentioning they'll keep an eye on it. That way, we can split this load and claw our way back to a small number of pending ones. Best, f From fperez.net at gmail.com Mon Feb 14 03:36:19 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 14 Feb 2011 00:36:19 -0800 Subject: [IPython-dev] Configuring logging for the kernel? Message-ID: Howdy, I've merged: https://github.com/ipython/ipython/pull/264 after Omar took Robert's feedback into account. But I'm not very experienced with the logging module, so I'm wondering what the best pattern should be for enabling logging conveniently at the command-line level. Right now, applications like the Qt console have to pass flags to the kernel (and there's some duplication there we should remove, like the code in ipythonqt.py and entry_point that creates the parsers). We can pass a --log-level flag to the kernel entry point script easily, that's not a problem. My question has to do with how to cleanly and conveniently let the kernel log to the screen or a file, keeping in mind that the kernel will be running in a separate process. The default approach to configuring logging for libraries is to let the application using a library set its own handlers, but in this case the app (say a Qt console) is calling the kernel in another process, so it can't pass it any handlers; in fact it can't pass the kernel anything other than command-line flags. The code path is somewhat elaborate, because options have to pass from the user-facing client through the kernel manager and ultimately to the actual kernel object (in ipkernel.py). I admit I played with it for a while and got a bit lost, so I'm looking for some ideas on this one. In addition, we need to keep in mind that all of the Qt console code is still in need of a Configurable restructuring, but that can be handled separately from this. If anyone for now needs to re-activate message printing to debug low-level kernel behavior, this snippet in ipkernel.py, below the logger definition line, will do: logger.addHandler(logging.StreamHandler()) logger.setLevel(logging.DEBUG) But what I want to do is come up with a cleaner solution for the long run... Cheers, f From robert.kern at gmail.com Mon Feb 14 12:23:55 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 14 Feb 2011 11:23:55 -0600 Subject: [IPython-dev] Configuring logging for the kernel? In-Reply-To: References: Message-ID: On 2/14/11 2:36 AM, Fernando Perez wrote: > Howdy, > > I've merged: > > https://github.com/ipython/ipython/pull/264 > > after Omar took Robert's feedback into account. But I'm not very > experienced with the logging module, so I'm wondering what the best > pattern should be for enabling logging conveniently at the > command-line level. Right now, applications like the Qt console have > to pass flags to the kernel (and there's some duplication there we > should remove, like the code in ipythonqt.py and entry_point that > creates the parsers). We can pass a --log-level flag to the kernel > entry point script easily, that's not a problem. > > My question has to do with how to cleanly and conveniently let the > kernel log to the screen or a file, keeping in mind that the kernel > will be running in a separate process. The default approach to > configuring logging for libraries is to let the application using a > library set its own handlers, but in this case the app (say a Qt > console) is calling the kernel in another process, so it can't pass it > any handlers; in fact it can't pass the kernel anything other than > command-line flags. > > The code path is somewhat elaborate, because options have to pass from > the user-facing client through the kernel manager and ultimately to > the actual kernel object (in ipkernel.py). I admit I played with it > for a while and got a bit lost, so I'm looking for some ideas on this > one. In addition, we need to keep in mind that all of the Qt console > code is still in need of a Configurable restructuring, but that can be > handled separately from this. For now, just add command line arguments enough to use logging.basicConfig() in the kernel process, and don't worry about any more complicated logging scenarios. Just the filename/stream and level arguments are sufficient, I think. Untested, but probably sufficient: parser.add_argument('--log-file', default='stderr', help='The file to send logging information to [default: stderr]') parser.add_argument('--log-level', default='ERROR', choices=['DEBUG', 'INFO', 'WARNING', 'ERROR', 'CRITICAL'], help='The log level [default: ERROR]') args = parser.parse_args() kwds = {} do_logging = True if args.log_file.lower() == 'none': do_logging = False elif args.log_file == 'stdout': kwds['stream'] = sys.stdout elif args.log_file == 'stderr': kwds['stream'] = sys.stderr else: kwds['filename'] = args.log_file if do_logging: kwds['level'] = getattr(logging, args.log_level) logging.basicConfig(**kwds) else: root_logger = logging.getLogger() class NullHandler(logging.Handler): def emit(self, record): pass root_logger.addHandler(NullHandler()) The more general solution would be to make logging Configurable and allow the passing of a config file via the command line parameters. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From fperez.net at gmail.com Mon Feb 14 13:36:40 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 14 Feb 2011 10:36:40 -0800 Subject: [IPython-dev] Configuring logging for the kernel? In-Reply-To: References: Message-ID: Hi Robert, On Mon, Feb 14, 2011 at 9:23 AM, Robert Kern wrote: > For now, just add command line arguments enough to use logging.basicConfig() in > the kernel process, and don't worry about any more complicated logging > scenarios. Just the filename/stream and level arguments are sufficient, I think. > Untested, but probably sufficient: OK, many thanks. That's along the lines of what I had, I was just a little tired to finish it up and decided to get some feedback before putting more time into it, in case I was off-base. Indeed, we'll want to make it properly configurable later on, but for now something like this should suffice. If nobody beats me to it with a coded pull request, I'll try to work on it in the next couple of days. Cheers, f From benjaminrk at gmail.com Mon Feb 14 16:32:03 2011 From: benjaminrk at gmail.com (MinRK) Date: Mon, 14 Feb 2011 13:32:03 -0800 Subject: [IPython-dev] Outstanding pull requests In-Reply-To: References: Message-ID: Thanks for all this great work. I'll pitch in and merge some of these. I just have a clarifying question for direct-commit/pull-request policy. For instance, I was looking at %who, and the docstrings look great, but I noticed that the behavior in the code is actually wrong: out = [i for i in out if type(i).__name__ in typeset] should be: out = [i for i in out if type(user_ns[i]).__name__ in typeset] (since `out` is the list of *names*, not variables, so it behaved as if everything had type 'str'). Which, first of all, is excellent, since docstrings should help us find bugs like this one has, but here's my question: What is the policy on tiny bugs like this? Should I do a one-line pull-request, or just push a fix directly to master (with test, of course)? -MinRK On Sun, Feb 13, 2011 at 17:01, Fernando Perez wrote: > Hi Thomas, > > I just wanted to thank you *enormously* for doing this. I have to > switch gears out of ipython right now for some deadlines, but later in > the week I'll return to this, and I'll try to flush a few of the easy > ones out of the way. Some comments below. > > On Sun, Feb 13, 2011 at 9:45 AM, Thomas Kluyver wrote: > > By way of my debut in the core team, I've prepared a list of all the open > > pull requests, with a very brief assessment of each, in the hope that we > can > > start sorting them out (we're currently at 31 open requests): > > > > Magic functions documentation (several branches from SciPy India): > > [The first two listed here seem to have changed various bits of > indentation > > beyond the specific things they're described as changing. This makes the > > diffs rather long. Do we want to copy the key changes out in new commits, > > using the --author option to credit them?] > > Good idea. These docstring requests were done by a number of teams at > scipy india who were at the very beginning of their python/programming > experience, so they are more likely to have beginner's problems. > > It's a real bummer that I dropped the ball in providing them with > early feedback, but I had a ton of travel right after and just got > overrun with life... > > So definitely crediting with --author but cleaning up the commit is a > good strategy for those with unnecessary noise. > > > - cd: looks like some examples are at the bash prompt, not the ipython > > prompt. https://github.com/ipython/ipython/pull/233/files > > - profile: Has neatly documented the Python profile module. > Unfortunately, > > IPython's %profile command does something entirely different. > > https://github.com/ipython/ipython/pull/232/files > > - logstart: Seems OK, although there's an "examples" title added for > logstop > > with no actual examples. logstop probably doesn't need an example, but we > > should tidy up the "examples" title. > > https://github.com/ipython/ipython/pull/231/files > > - edit: Clean diff, but giving an example for edit is tricky, because you > > can't show the interactive editor. Also restates the -p option in a > comment > > in the example. https://github.com/ipython/ipython/pull/230/files(closed > > 220 as duplicate) > > - pdef (it's confusing to use magic functions as the arguments for the > > example, we should have a user defined function) & colors (one example is > > probably enough, and the last two are wrong, using %usage instead of > > %colors) https://github.com/ipython/ipython/pull/229/files > > - who, whos, who_ls: Looks good. > https://github.com/ipython/ipython/pull/228 > > - logon: Not quite sure what's happened here. Example seems to be a bunch > of > > Java-ish code. It probably needs to be redone from scratch. > > https://github.com/ipython/ipython/pull/227 > > - pwd: pwd example is good, but has also added to exit (doesn't really > need > > an example), dirs (example doesn't really show what it's for), and color > > (just an "examples" heading, no actual examples. > > https://github.com/ipython/ipython/pull/226/files > > - save: Clean diff, but the example and explanation aren't very clear. > > Probably needs to be redone from scratch. > > https://github.com/ipython/ipython/pull/225 > > - cpaste: Appears to have ripped out the actual function while writing > the > > docs, and stitched its definition onto the magic_paste function code. > > Example could also be a bit clearer. > > https://github.com/ipython/ipython/pull/224 > > - colors: Same changes (& problems) as request 229. > > https://github.com/ipython/ipython/pull/223 > > - timeit: Too many equivalent examples. > > https://github.com/ipython/ipython/pull/222/files > > - alias: Already had examples, changes remove one of them. Adds two, but > one > > (ls) is available without aliasing, and the other is inaccurate (the > example > > shows the result of a man command). > > https://github.com/ipython/ipython/pull/221 > > - pycat: Doesn't actually use a Python file, so rather defeats the point > of > > pycat. Not sure that an example is really helpful, as it doesn't show the > > colouring you get at a terminal. > https://github.com/ipython/ipython/pull/219 > > - hist: Formatting is off (probably related to a bug in the code). Also, > the > > "this is an example of..." line seems redundant. > > https://github.com/ipython/ipython/pull/218/files > > > > Whew. Of these, I recommend merging 228 (%who), and closing 225 (%save), > 232 > > (%profile) with "try again" and 227 (%logon), for which I'm not sure that > > it's even possible to do an example. > > > > The others will need further work. Fernando, any sort of feel for how > many > > of the SciPy India contributors will still be in the loop to revise their > > contributions? > > Thanks for going through all of this! Unfortunately due to the delay, > and the fact that many of them were first-timers to pretty much > everything, I'm not too hopeful. I think if there are some where we > can easily finish up what they did with minimal work, it would be a > nice gesture. But those that require significant rework, just > dropping a comment would be sufficient. If they do pop up again and > come back with improvements, then we can engage them better for the > future. > > In my defense, I was trying to handle a sprint where 60! people wanted > to contribute, most of which had never coded a single line of Python. > So I tried to do something where at least they would learn during the > day, even if it wasn't realistic to expect actual final contributions > to the codebase to come from everyone. > > > Looking at the others: > > > > Small pull requests: > > - %hist output format: I've solved the same problem in a different way in > > pull request 261, see below. https://github.com/ipython/ipython/pull/236 > > - Logging unicode characters (0.10 series): Good start. See my comment > there > > for minor concern. https://github.com/ipython/ipython/pull/249 > > - check_gtk: Removes a function. Mark Voorhies has tested the change and > > commented that it works. I'll admit to not understanding what it's about. > > https://github.com/ipython/ipython/pull/237 > > - Typos in zmq/blockingkernelmanager.py: Seems OK to me. > > https://github.com/ipython/ipython/pull/203/files > > - cProfiling ipython.py: Updated in response to Brian's concerns. Needs > > reviewing again. https://github.com/ipython/ipython/pull/193 > > - Autocomplete in Emacs: Removes the all_completions function (which > > apparently was broken). One other person says it works. Needs review. > > https://github.com/ipython/ipython/pull/204 > > - Unicode issues [My changes]: Tests now integrated into main test suite > > (thanks to help from Robert Kern). Needs review. > > https://github.com/ipython/ipython/pull/252 > > - Wildcard module [My changes]: Fixes issue 129. The bug and the fix are > > pretty much self contained, so shouldn't interfere with changes in other > > code. No feedback yet. (this is pushing what I'd call small, but > shouldn't > > change functionality besides fixing the bug) > > https://github.com/ipython/ipython/pull/251 > > > > Medium pull requests: > > - Fixing up various magic commands with the new history system [My > changes]: > > Revised after Fernando's comments, unit tests added. Conflicts with > > Madhusudan's #203, see below. > https://github.com/ipython/ipython/pull/261 > > - %hist command bugs: This solves one of the same problems I've fixed in > > #261, in a different way. > https://github.com/ipython/ipython/pull/203/files > > - Unbundling external libraries: Revised in response to Fernando's and > > Brian's concerns. Needs reviewing again. > > https://github.com/ipython/ipython/pull/191 > > - Pyside compatibility: Waiting for contributor to resolve bug with SVG > > handling. https://github.com/ipython/ipython/pull/259 > > - Kernel logging: Fernando has requested a final check from Robert. > > https://github.com/ipython/ipython/pull/264 > > > > Larger pull requests: > > - Magic arguments decorators. Updated in response to Fernando's comments. > > Needs reviewing again. https://github.com/ipython/ipython/pull/199 > > > > Not quite yet: > > - HTML Notebook: Gained a lot of interest, but not quite ready yet. > Possible > > project for GSoC this year. (But if we're going to accumulate alternative > > frontends, should we consider putting each frontend in its own repo?) > > https://github.com/ipython/ipython/pull/179 > > - MinRK's newparallel: Work in progress. Lots of > > work.https://github.com/ipython/ipython/pull/254 > > Ok, thanks again for this analysis. > > Over the next few days, I suggest that anyone willing to 'take > ownership' of one just pitch in and add a comment to the corresponding > one, mentioning they'll keep an eye on it. That way, we can split > this load and claw our way back to a small number of pending ones. > > Best, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Mon Feb 14 17:53:15 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 14 Feb 2011 14:53:15 -0800 Subject: [IPython-dev] Outstanding pull requests In-Reply-To: References: Message-ID: On Mon, Feb 14, 2011 at 1:32 PM, MinRK wrote: > Thanks for all this great work. ?I'll pitch in and merge some of these. Many thanks! > I just have a clarifying question for direct-commit/pull-request policy. > For instance, I was looking at %who, and the docstrings look great, but I > noticed that the behavior in the code is actually wrong: > ?? ??? ? ? ?out = [i for i in out if type(i).__name__ in typeset] > should be: > ?? ? ? ? ? ?out = [i for i in out if type(user_ns[i]).__name__ in typeset] > (since `out` is the list of names, not variables, so it behaved as if > everything had type 'str'). ?Which, first of all, is excellent, since > docstrings should help us find bugs like this one has, but here's my > question: > What is the policy on tiny bugs like this? ?Should I do a one-line > pull-request, or just push a fix directly to master (with test, of course)? I've always felt that a certain level of trust, and willingness to fix little slip-ups without getting upset if/when they happen, is the best policy. Core developers should feel free to make small merges like these when they are confident that there's no danger. I realize that it's always possible to make a mistake and to have a small change that inadvertently introduces an error. But I also deeply dislike 'zero tolerance' style approaches to anything, where human thought and judgment are eliminated in favor of blind policy. Especially since it's true that even *with* review we can still make mistakes (and that's OK, we'll just fix them), I don't want the idea of review to become a straitjacket that forces every last action in the project to require six rubber stamps. Since an absolutist policy would be just the *illusion* of 'no mistakes', we might as well live with the reality and find an approach that balances things to maximize the efficiency, fluidity and the fun of working on the project for all involved [1]. Paraphrasing the famous quote on simplicity, we need just enough process and policy to help things run smoothly, but no more :) So my take is: use your judgment, and when you are confident that a *small* fix is obvious, otherwise innocuous, and that making it directly will just save everyone time for more complex reviews, go ahead. When in doubt for any reason, do ask for review. And needless to say, large or complex changes of any kind, or even small ones that entail important design decisions, should always be put up for review. Cheers, f [1] The fun part *is* important. As a largely volunteer effort, we need to ensure that people *enjoy* working on IPython (myself included :). From takowl at gmail.com Mon Feb 14 18:14:23 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Mon, 14 Feb 2011 23:14:23 +0000 Subject: [IPython-dev] Outstanding pull requests In-Reply-To: References: Message-ID: On 14 February 2011 22:53, Fernando Perez wrote: > I've always felt that a certain level of trust, and willingness to fix > little slip-ups without getting upset if/when they happen, is the best > policy. Core developers should feel free to make small merges like > these when they are confident that there's no danger. > I'll second that. Even in purely utilitarian terms, the time spent finding and undoing the occasional mistaken commit is bound to be far less than the time saved by not having to wait for a seal of approval on every little change. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From gokhansever at gmail.com Tue Feb 15 13:15:13 2011 From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_Sever?=) Date: Tue, 15 Feb 2011 11:15:13 -0700 Subject: [IPython-dev] Link broken on github main-page Message-ID: Hi, The link is broken on the line starting with "website"... Overview ======== Welcome to IPython. Our full documentation can be found in the ``docs/dist`` subdirectory in ``.html`` and ``.pdf`` formats, also available online at our `website . The ``docs/source`` directory contains the plaintext version of these manuals. None of the documentation seems updated to reflect github transition (from http://ipython.github.com/ipython-doc/) Is this still in so experimental stage? -- G?khan -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Tue Feb 15 13:38:33 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 15 Feb 2011 10:38:33 -0800 Subject: [IPython-dev] Link broken on github main-page In-Reply-To: References: Message-ID: The link is broken because GitHub doesn't support using '.txt' as a reST extension, so it's interpreting the wrong text as a link. We should probably rename README.txt to README.rst, and link to the new doc page. Is there a reason to *not* use .rst instead of .txt? Thanks for the note. -MinRK On Tue, Feb 15, 2011 at 10:15, G?khan Sever wrote: > Hi, > > The link is broken on the line starting with "website"... > > > Overview > ======== > > Welcome to IPython. Our full documentation can be found in the ``docs/dist`` > subdirectory in ``.html`` and ``.pdf`` formats, also available online at our > `website . The ``docs/source`` directory contains > the plaintext version of these manuals. > > > None of the documentation seems updated to reflect github transition (from > http://ipython.github.com/ipython-doc/) > > Is this still in so experimental stage? > > -- > G?khan > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Tue Feb 15 14:41:53 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 Feb 2011 19:41:53 +0000 Subject: [IPython-dev] Examples for magic commands Message-ID: OK, we've now got through most of the magic command examples from SciPy India. Min merged one, I've amalgamated several into pull request 272, and closed a few more. There are three left: #219 (pycat) and #230 (edit): I don't think an example adds much to either of these. The first case involves colour, which we can't put in the docstrings. The second revolves around an editor which won't be in the example. #221 has several problems, and there were already some examples included in it, so I think it's OK without any changes. Should I go ahead and close those? Or do people think those commands would benefit from examples? Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Tue Feb 15 15:05:35 2011 From: benjaminrk at gmail.com (MinRK) Date: Tue, 15 Feb 2011 12:05:35 -0800 Subject: [IPython-dev] Examples for magic commands In-Reply-To: References: Message-ID: On Tue, Feb 15, 2011 at 11:41, Thomas Kluyver wrote: > OK, we've now got through most of the magic command examples from SciPy > India. Min merged one, I've amalgamated several into pull request 272, and > closed a few more. There are three left: > > #219 (pycat) and #230 (edit): I don't think an example adds much to either > of these. The first case involves colour, which we can't put in the > docstrings. The second revolves around an editor which won't be in the > example. > > #221 has several problems, and there were already some examples included in > it, so I think it's OK without any changes. > > Should I go ahead and close those? Or do people think those commands would > benefit from examples? > I'm okay on all of those, but I don't know about other people. Thanks for doing all of this, you are a hero! -MinRK > > Thanks, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Feb 15 18:18:02 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 15 Feb 2011 15:18:02 -0800 Subject: [IPython-dev] Examples for magic commands In-Reply-To: References: Message-ID: Thomas, Thanks for your work on this. > #219 (pycat) and #230 (edit): I don't think an example adds much to either > of these. The first case involves colour, which we can't put in the > docstrings. The second revolves around an editor which won't be in the > example. > > #221 has several problems, and there were already some examples included in > it, so I think it's OK without any changes. > > Should I go ahead and close those? Or do people think those commands would > benefit from examples? I am fine with just closing these. Cheers, Brian > Thanks, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From takowl at gmail.com Tue Feb 15 18:57:09 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 15 Feb 2011 23:57:09 +0000 Subject: [IPython-dev] pyzmq 2.0.10.1 available in unstable In-Reply-To: References: Message-ID: Thanks, Miguel. On 15 February 2011 23:19, Miguel Landaeta wrote: > Hello Thomas, > > FYI, pyzmq 2.0.10.1 has reached unstable. > > http://packages.debian.org/unstable/python-zmq > > Cheers, > > -- > Miguel Landaeta, miguel at miguel.cc > secure email with PGP 0x7D8967E9 available at http://keyserver.pgp.com/ > "Faith means not wanting to know what is true." -- Nietzsche > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg at gmail.com Tue Feb 15 22:48:11 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Tue, 15 Feb 2011 19:48:11 -0800 Subject: [IPython-dev] My schedule for the next week Message-ID: Hi, I am excited to see all the activity going on. I wanted to let you all know that I will be mostly occupied with a conference until mid next week. I am going to try to catch up on things while I am travelling, but I may be on/off a lot of the next week. Cheers, Brian -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From fperez.net at gmail.com Wed Feb 16 01:52:38 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 15 Feb 2011 22:52:38 -0800 Subject: [IPython-dev] My schedule for the next week In-Reply-To: References: Message-ID: Hi everyone, On Tue, Feb 15, 2011 at 7:48 PM, Brian Granger wrote: > > > I am excited to see all the activity going on. ?I wanted to let you > all know that I will be mostly occupied with a conference until mid > next week. ?I am going to try to catch up on things while I am > travelling, but I may be on/off a lot of the next week. and in a perfect storm of bad timing, I'm in the exact same boat, off to the East Coast for the same period to a (different) conference :) I'll do my best to catch up on the list and github, but will likely fall behind, sorry... Cheers, f From fperez.net at gmail.com Wed Feb 16 03:02:47 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 16 Feb 2011 00:02:47 -0800 Subject: [IPython-dev] Examples for magic commands In-Reply-To: References: Message-ID: On Tue, Feb 15, 2011 at 12:05 PM, MinRK wrote: > > Thanks for doing all of this, you are a hero! Indeed! From takowl at gmail.com Wed Feb 16 17:10:27 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Wed, 16 Feb 2011 22:10:27 +0000 Subject: [IPython-dev] History Message-ID: I've been having a think about how we store history: At present: Commands entered are stored in two lists (raw and "translated" - i.e. turning magic commands into function calls). These are persisted to disk at the next input every time 60 seconds elapse, with storage in a JSON file, which is reloaded into the same lists (and into readline history) when starting IPython. Each command entered is also immediately persisted in the "shadow history", a collection of files in .ipython managed by the pickleshare DB. The output objects are also stored in a dictionary by prompt number, but I'm less concerned with that here. Uses: - Readline history (getting previous commands via up arrow) - Various magic commands (save, macro, hist) can access ranges of input, using the prompt numbers from the current session. - %hist -g allows searching shadow and current history with glob syntax. - %rep can access ranges of this session's history, or single lines from shadow history. (These are all I've found so far - please let me know if there are others) I'd like to propose simplifying this somewhat. Specifically, I think it makes sense to store history on disk just once. I suggest an SQLite database. SQLite has been in the standard library since 2.5, is lightweight, simple and widely used (e.g. Firefox's history). More specifically, I envisage: - We keep the session lists of raw and translated history as they are. - Each command is stored instantly, so we do away with the need for an autosave timer thread. A crash at any stage should leave your entire history intact up to the last command completed. - We store only raw history on disk (I think raw history is what we're looking for 90% of the time). If we want translated history, we redo the translation on the fly (this should require minimal computation, unless I've missed something). - History is indexed by session number and prompt number. This provides a sensible behaviour if we have two IPython shells open together - the second one to be opened will be the latter session (and will be able to access commands entered in the other session as soon as they are completed). - For magic commands, accessing a line from a previous session could look like "-1#9" (9th line of immediately previous session). - On starting IPython, we load the last (~40 lines/~2 sessions) from the database into readline history. Thoughts? Have I overlooked some key reason we use the existing system? Is there a better alternative to SQLite? Would you design it differently? I've not written any code for this yet, so I'm open to ideas. But if people think that makes sense, I'm volunteering to make it happen. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Wed Feb 16 22:07:17 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 16 Feb 2011 19:07:17 -0800 Subject: [IPython-dev] Examples for magic commands In-Reply-To: References: Message-ID: <4D5C90E5.1010302@gmail.com> On 02/15/2011 03:18 PM, Brian Granger wrote: >> > Should I go ahead and close those? Or do people think those commands would >> > benefit from examples? > I am fine with just closing these. No objection from me on closing them either. It would be great to drop them a short word of thanks and why it's being closed, since these are first-time contributors and even if their work wasn't accepted we want to be kind to newbies... And indeed, many many thanks Thomas for this rather heroic push to flush our accumulation of requests. Way to land in the project with a splash :) Best, f From fperez.net at gmail.com Thu Feb 17 00:54:04 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 16 Feb 2011 21:54:04 -0800 Subject: [IPython-dev] Link broken on github main-page In-Reply-To: References: Message-ID: On Tue, Feb 15, 2011 at 10:38 AM, MinRK wrote: > The link is broken because GitHub doesn't support using '.txt' as a reST > extension, so it's interpreting the wrong text as a link. ?We should > probably rename README.txt to README.rst, and link to the new doc page. > Is there a reason to *not* use .rst instead of .txt? Well, docutils does suggest using txt as the reST extension, their argument being that they ultimately are text files. And on more gui-oriented platforms, it was deemed as nicer for new users who could double-click on a .txt file and get automatically a text editor instead of a dialog about what application to use. So historically, we went with txt. But I have no strong feelings on the matter, so if you guys prefer to switch over to rst so the github setup is smoother, I won't oppose it in any way. Cheers, f From benjaminrk at gmail.com Thu Feb 17 01:14:48 2011 From: benjaminrk at gmail.com (MinRK) Date: Wed, 16 Feb 2011 22:14:48 -0800 Subject: [IPython-dev] Link broken on github main-page In-Reply-To: References: Message-ID: On Wed, Feb 16, 2011 at 21:54, Fernando Perez wrote: > On Tue, Feb 15, 2011 at 10:38 AM, MinRK wrote: > > The link is broken because GitHub doesn't support using '.txt' as a reST > > extension, so it's interpreting the wrong text as a link. We should > > probably rename README.txt to README.rst, and link to the new doc page. > > Is there a reason to *not* use .rst instead of .txt? > > Well, docutils does suggest using txt as the reST extension, their > argument being that they ultimately are text files. And on more > gui-oriented platforms, it was deemed as nicer for new users who could > double-click on a .txt file and get automatically a text editor > instead of a dialog about what application to use. So historically, > we went with txt. > That makes sense. Basic reST with headers, sections, emphasis and blocks is perfectly readable as nicely formatted plaintext. However, to me it seems like if you want a file to read as plaintext as well as reST, you shouldn't use the markup that is less readable like `link text `_, and you can use .rst as an extension when you do use the extended reST bits. My vote: either a) remove the hidden links for better .txt readability (and correct GH link parsing) or b) rename to .rst so it looks nice on GitHub I lean towards b), but if there really is a significant number of people who actually read READMEs but are foiled by a .rst extension, then we can stick to a). > > But I have no strong feelings on the matter, so if you guys prefer to > switch over to rst so the github setup is smoother, I won't oppose it > in any way. > > Cheers, > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Thu Feb 17 01:22:55 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Wed, 16 Feb 2011 22:22:55 -0800 Subject: [IPython-dev] Link broken on github main-page In-Reply-To: References: Message-ID: On Wed, Feb 16, 2011 at 10:14 PM, MinRK wrote: > > I lean towards b), but if there really is a significant number of people who > actually read READMEs but are foiled by a .rst extension, then we can stick > to a). I'm OK with b) too... f From takowl at gmail.com Thu Feb 17 05:45:30 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 17 Feb 2011 10:45:30 +0000 Subject: [IPython-dev] Examples for magic commands In-Reply-To: <4D5C90E5.1010302@gmail.com> References: <4D5C90E5.1010302@gmail.com> Message-ID: On 17 February 2011 03:07, Fernando Perez wrote: > It would be great to drop > them a short word of thanks and why it's being closed, since these are > first-time contributors and even if their work wasn't accepted we want > to be kind to newbies... > I tried to do something along these lines with each one I closed. I hope it sounded sincere - I found it a bit tricky to convey that we're grateful for their interest in contributing, even though we don't want the specific changes. I should add that what you did at SciPy India was an impressive feat: you got a large group of entirely new contributors, had them download the development codebase, make changes, and submit those changes as pull requests that, although not perfect, mostly made sense. Hopefully it's a good indication that there's relatively little technical friction for new contributors. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans_meine at gmx.net Thu Feb 17 12:53:59 2011 From: hans_meine at gmx.net (Hans Meine) Date: Thu, 17 Feb 2011 18:53:59 +0100 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: <201102171853.59724.hans_meine@gmx.net> Am Mittwoch, 16. Februar 2011, um 23:10:27 schrieb Thomas Kluyver: > I'd like to propose simplifying this somewhat. Specifically, I think it > makes sense to store history on disk just once. I suggest an SQLite > database. +0.9 (it will become a little bit harder to directly access the history, compared to the plain text files at least) > - We keep the session lists of raw and translated history as they are. > - Each command is stored instantly, so we do away with the need for an > autosave timer thread. A crash at any stage should leave your entire > history intact up to the last command completed. Why "completed"? I would assume that a crashing command will still appear in the DB?! > - We store only raw history on disk (I think raw history is what we're > looking for 90% of the time). If we want translated history, we redo the > translation on the fly (this should require minimal computation, unless > I've missed something). Not sure about this one - sounds fragile, I guess there was a reason to store the processed commands; what about processing which depends on currently active aliases etc.? My fear is that this will become difficult to recompute. OTOH, most of the time, the two commands are absolutely equal. What about two DB columns, but leaving the processed command empty if it is equal to the raw command? This could save a lot of space, and it would be trivial to create a view which provides the two full columns. > - History is indexed by session number and prompt number. Sounds sensible. > This provides a > sensible behaviour if we have two IPython shells open together - the second > one to be opened will be the latter session (and will be able to access > commands entered in the other session as soon as they are completed). I would still expect only local commands to appear in the normal readline history, and in the default %history output. IMO there should be optional modes (something along %history --all) which list the other shells' history. > - For magic commands, accessing a line from a previous session could look > like "-1#9" (9th line of immediately previous session). Hmm, what about concurrent sessions? (Versus "previous" sessions which have been quit.) > - On starting IPython, we load the last (~40 lines/~2 sessions) from the > database into readline history. Are you kidding me? One session equals roughly 1400 lines for me. :-) That's a good question, how much history should be available from "previous" sessions. > Thoughts? You wrote "will be able to access commands entered in the other session as soon as they are completed" - does this require ipython to constantly read from the DB? No, obviously there should be a history API that directly operates on the database. Only for readline, we would be reading some lines from the DB initially, right? HTH, Hans From takowl at gmail.com Thu Feb 17 13:46:52 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Thu, 17 Feb 2011 18:46:52 +0000 Subject: [IPython-dev] History In-Reply-To: <201102171853.59724.hans_meine@gmx.net> References: <201102171853.59724.hans_meine@gmx.net> Message-ID: On 17 February 2011 17:53, Hans Meine wrote: > +0.9 > (it will become a little bit harder to directly access the history, > compared > to the plain text files at least) > True, but I don't think that's a common use case (correct me if I'm wrong). And there are tools to access SQLite databases if you need to, e.g. http://sqlitebrowser.sourceforge.net/ Why "completed"? I would assume that a crashing command will still appear > in > the DB?! > My mistake. For some reason, I thought that input was stored in history after execution. Checking the code, it's stored before, so a crashing command would be saved. That should have been obvious. Not sure about this one - sounds fragile, I guess there was a reason to > store > the processed commands; what about processing which depends on currently > active aliases etc.? My fear is that this will become difficult to > recompute. > OTOH, most of the time, the two commands are absolutely equal. What about > two > DB columns, but leaving the processed command empty if it is equal to the > raw > command? This could save a lot of space, and it would be trivial to create > a > view which provides the two full columns. > Good point, I hadn't thought of temporary aliases (I've never used them); and good solution. While we're on the subject, what does translation actually do? %magic commands, !system commands, aliases; anything else? > I would still expect only local commands to appear in the normal readline > history, and in the default %history output. IMO there should be optional > modes (something along %history --all) which list the other shells' > history. > Yes, that's what I'd envisage. At present, %history -g is used for a glob search over all the history; passing it no pattern is the way to list history from all sessions. The readline history wouldn't be updated from other sessions - so commands you enter in shell 1 before starting shell 2 will be loaded into readline for shell 2, but then only shell 2 commands would be added to that. Starting a third shell would not add anything to readline in shell 2. Hmm, what about concurrent sessions? (Versus "previous" sessions which > have > been quit.) > Interesting idea, but more complicated to implement. You'd have to somehow have separate shells signal their presence. I think this is a separate consideration - I'm looking at what we can get for free by restructuring the history. Are you kidding me? One session equals roughly 1400 lines for me. :-) > > That's a good question, how much history should be available from > "previous" > sessions. > But how much of that do you access through readline? It can of course be user configurable, but we should set a sensible default. 40 is on the low side. I assume readline is fairly fast: maybe a couple of hundred is a good limit? > You wrote "will be able to access commands entered in the other session as > soon as they are completed" - does this require ipython to constantly read > from the DB? No, obviously there should be a history API that directly > operates on the database. > Sorry, I should have been clearer about what I meant by access. Those commands will be in the database immediately, and accessible with magic commands (e.g. "%rep -1#9" or "%hist -g"), which will call the history API to read the database. Other than those commands, IPython would only read the database on startup, to push lines into readline. Thanks for the feedback, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Thu Feb 17 14:31:38 2011 From: benjaminrk at gmail.com (MinRK) Date: Thu, 17 Feb 2011 11:31:38 -0800 Subject: [IPython-dev] History In-Reply-To: References: <201102171853.59724.hans_meine@gmx.net> Message-ID: Hi, I like this idea. It's a bit unfortunate that we are starting this discussion while Brian and Fernando are at conferences for a week, but we have good ideas when we have them. It's possible there will be an objection based on ideas of non-Python interfaces, but I think those are generally implemented via history_request messages, which would still work and thus not be a problem. On Thu, Feb 17, 2011 at 10:46, Thomas Kluyver wrote: > On 17 February 2011 17:53, Hans Meine wrote: > >> +0.9 >> (it will become a little bit harder to directly access the history, >> compared >> to the plain text files at least) >> > > True, but I don't think that's a common use case (correct me if I'm wrong). > And there are tools to access SQLite databases if you need to, e.g. > http://sqlitebrowser.sourceforge.net/ > > Why "completed"? I would assume that a crashing command will still appear >> in >> the DB?! >> > > My mistake. For some reason, I thought that input was stored in history > after execution. Checking the code, it's stored before, so a crashing > command would be saved. That should have been obvious. > > Not sure about this one - sounds fragile, I guess there was a reason to >> store >> the processed commands; what about processing which depends on currently >> active aliases etc.? My fear is that this will become difficult to >> recompute. > > >> OTOH, most of the time, the two commands are absolutely equal. What about >> two >> DB columns, but leaving the processed command empty if it is equal to the >> raw >> command? This could save a lot of space, and it would be trivial to >> create a >> view which provides the two full columns. >> > I don't think there's any real penalty to storing both (other than doubling the size of a very small file), so might as well. > > Good point, I hadn't thought of temporary aliases (I've never used them); > and good solution. While we're on the subject, what does translation > actually do? %magic commands, !system commands, aliases; anything else? > > >> I would still expect only local commands to appear in the normal readline >> history, and in the default %history output. IMO there should be optional >> modes (something along %history --all) which list the other shells' >> history. >> > > Yes, that's what I'd envisage. At present, %history -g is used for a glob > search over all the history; passing it no pattern is the way to list > history from all sessions. > > The readline history wouldn't be updated from other sessions - so commands > you enter in shell 1 before starting shell 2 will be loaded into readline > for shell 2, but then only shell 2 commands would be added to that. Starting > a third shell would not add anything to readline in shell 2. > > Hmm, what about concurrent sessions? (Versus "previous" sessions which >> have >> been quit.) >> > > Interesting idea, but more complicated to implement. You'd have to somehow > have separate shells signal their presence. I think this is a separate > consideration - I'm looking at what we can get for free by restructuring the > history. > I think the question here might be: what would the history look like at startup if I've been running several sessions simultaneously, and I start a new one? IP0: foo='bar' IP1: a=1 IP2: b=2 IP1: c=3 IP2: d=4 i.e. what is the 'previous' session? Is it 'most recently closed' or 'most recently started' or 'most recently executed'? You seemed to suggest above that IP3's readline history at startup should go d,c,b,a,foo (as I continue to press up) Is that correct? I like this approach, but there's also value in keeping related commands (same session) adjacent to each other, so I don't think it's an obvious choice. In most real cases, this probably doesn't make much difference. > > Are you kidding me? One session equals roughly 1400 lines for me. :-) >> >> That's a good question, how much history should be available from >> "previous" >> sessions. >> > > But how much of that do you access through readline? It can of course be > user configurable, but we should set a sensible default. 40 is on the low > side. I assume readline is fairly fast: maybe a couple of hundred is a good > limit? > I would think a default should be something like 500 at the very least. I use ctrl-r all the time, and basically expect that if I remember typing it, IPython should remember as well. If you are just using arrow-keys, then more than a few dozen isn't particularly valuable, but reverse-i-search should find anything I remember well enough to look for. -MinRK > > >> You wrote "will be able to access commands entered in the other session as >> soon as they are completed" - does this require ipython to constantly read >> from the DB? No, obviously there should be a history API that directly >> operates on the database. >> > > Sorry, I should have been clearer about what I meant by access. Those > commands will be in the database immediately, and accessible with magic > commands (e.g. "%rep -1#9" or "%hist -g"), which will call the history API > to read the database. Other than those commands, IPython would only read the > database on startup, to push lines into readline. > > Thanks for the feedback, > Thomas > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From satra at mit.edu Thu Feb 17 14:36:59 2011 From: satra at mit.edu (Satrajit Ghosh) Date: Thu, 17 Feb 2011 14:36:59 -0500 Subject: [IPython-dev] History In-Reply-To: References: <201102171853.59724.hans_meine@gmx.net> Message-ID: hi thomas and hans, just a few thoughts. (it will become a little bit harder to directly access the history, compared >> to the plain text files at least) >> > > True, but I don't think that's a common use case (correct me if I'm wrong). > And there are tools to access SQLite databases if you need to, e.g. > http://sqlitebrowser.sourceforge.net/ > history is currently in json files and yes it does get long, but reading json is now part of the standard library. we can definitely save space by replacing identical commands between raw and processed with only one copy. when fernando and i worked on this in december, we increased the save history limit in the config. > > >> Hmm, what about concurrent sessions? (Versus "previous" sessions which >> have >> been quit.) >> > > Interesting idea, but more complicated to implement. You'd have to somehow > have separate shells signal their presence. I think this is a separate > consideration - I'm looking at what we can get for free by restructuring the > history. > we do have concurrent history as long as the clients connect to the same kernel. this is in fact what we implemented between the qt client and the terminal client or combinations thereof. but this will be different from using separate kernels. > > Are you kidding me? One session equals roughly 1400 lines for me. :-) >> >> That's a good question, how much history should be available from >> "previous" >> sessions. >> > > But how much of that do you access through readline? It can of course be > user configurable, but we should set a sensible default. 40 is on the low > side. I assume readline is fairly fast: maybe a couple of hundred is a good > limit? > are we really in memory trouble these days? or is this more a question of load time? cheers, satra -------------- next part -------------- An HTML attachment was scrubbed... URL: From hans_meine at gmx.net Fri Feb 18 08:06:33 2011 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 18 Feb 2011 14:06:33 +0100 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: <201102181406.33551.hans_meine@gmx.net> Am Donnerstag, 17. Februar 2011, um 20:36:59 schrieb Satrajit Ghosh: > are we really in memory trouble these days? or is this more a question of > load time? As anecdotal evidence, I once had the problem that it took ages for new (bash) shells to start up (sth. > 15s). It took me quite some debugging (strace etc.) to understand that the problem was in fact the readline history, not (directly) my startup files. I like having a history limit of 150,000 commands. Why not? Same as MinRK: As long as I dimly remember that I solved a particular problem on the commandline in the middle of last year, why should my computer have forgotten about it? (Yes, privacy issues might arise, but there are countermeasures here, too.) But I could very well live with magic commands for this use case, i.e. with limited readline history. Am Donnerstag, 17. Februar 2011, um 20:31:38 schrieb MinRK: > I don't think there's any real penalty to storing both (other than doubling > the size of a very small file), so might as well. Here, you're assuming that the file is "very small", while AFAICS it might be at least some megabytes. So why have this redundancy? Ciao, Hans From takowl at gmail.com Fri Feb 18 08:37:06 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 18 Feb 2011 13:37:06 +0000 Subject: [IPython-dev] History In-Reply-To: <201102181406.33551.hans_meine@gmx.net> References: <201102181406.33551.hans_meine@gmx.net> Message-ID: On 18 February 2011 13:06, Hans Meine wrote: > I like having a history limit of 150,000 commands. Why not? Same as > MinRK: > As long as I dimly remember that I solved a particular problem on the > commandline in the middle of last year, why should my computer have > forgotten > about it? (Yes, privacy issues might arise, but there are countermeasures > here, too.) > > But I could very well live with magic commands for this use case, i.e. with > limited readline history. > I'll ensure that this can all be tuned through a config file. I've gone with loading the last 1000 into readline history for now, and I've not yet implemented any limit on the database size. > Here, you're assuming that the file is "very small", while AFAICS it might > be > at least some megabytes. So why have this redundancy? > I tested it briefly yesterday evening. After about 140 commands, storing both raw and translated all the time, the database was about 14kB; i.e. about 100 bytes per line. If we assume the lines in my test were half what's typical, 150,000 commands would be 30MB. In disk space terms, that's not worth worrying about (unless we do IPython for embedded systems ;-)). Here's the code: https://github.com/takluyver/ipython/tree/sqlite-history It's still very much a work in progress - I haven't hooked up all the magic commands to access previous sessions. But it writes to the database, reloads lines into readline (in the order I described), and searching the database with "%hist -g thing" works. I'll make a pull request once it's a bit more complete. As an aside, there are a few odds and ends left using the pickleshare DB - on my system, there's still syscmdlist and dhist (directory history). I've got nothing against pickleshare - it seems to work perfectly well - but is it worth looking at simplifying those odds and ends of storage? It looks like it's using pickle to dump to/load from file: is it simpler to call pickle directly? Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Fri Feb 18 10:48:56 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 18 Feb 2011 10:48:56 -0500 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: Hi all, sorry that this will be brief and not very thought-through, but I can only sneak in short periods while at the conference... Many thanks to Thomas for getting this work going! But I think, since we now have a bit more manpower and good momentum going, it's worth thinking a little about the key points we want to hit so we end up with something really solid. Comments below... On Wed, Feb 16, 2011 at 5:10 PM, Thomas Kluyver wrote: > - Each command is stored instantly, so we do away with the need for an > autosave timer thread. A crash at any stage should leave your entire history > intact up to the last command completed. Instant saving has one problem: frequent disk usage prevents hard drives from spinnning down when on battery. The idea of an auto-save thread on a timer with a user-controllable delay has the advantage that the user can control their power consumption profile to fit their needs. On an international flight when you're trying to squeeze every last bit of your battery, this matters a lot. We don't want to turn ipython into the thing that eats up your battery life to death just by virtue of running very simple interactive commands that in principle are purely CPU/memory resident, but because we generate lots of sideband disk activity. So we should keep this consideration in mind in the design. If we don't think about it now, it will be much harder to retrofit a decent power profile later on. > - We store only raw history on disk (I think raw history is what we're > looking for 90% of the time). If we want translated history, we redo the > translation on the fly (this should require minimal computation, unless I've > missed something). > I've been having a think about how we store history: > > At present: Commands entered are stored in two lists (raw and "translated" - > i.e. turning magic commands into function calls). These are persisted to > disk at the next input every time 60 seconds elapse, with storage in a JSON > file, which is reloaded into the same lists (and into readline history) when > starting IPython. Each command entered is also immediately persisted in the > "shadow history", a collection of files in .ipython managed by the > pickleshare DB. The output objects are also stored in a dictionary by prompt > number, but I'm less concerned with that here. > > Uses: > - Readline history (getting previous commands via up arrow) > - Various magic commands (save, macro, hist) can access ranges of input, > using the prompt numbers from the current session. > - %hist -g allows searching shadow and current history with glob syntax. > - %rep can access ranges of this session's history, or single lines from > shadow history. > (These are all I've found so far - please let me know if there are others) No, we must store the translated history for two reasons: 1. Some translations are dynamic and context-dependent, so they can not be recomputed later (though these are the minority). 2. More importantly, the translation process is relatively cpu intensive, while disk space is the absolutely cheapest resource in existence (at least at the data storage volumes we're talking about here). So it makes sense to store on disk these results once we have computed them, rather than recomputing them all over later on reload. > - History is indexed by session number and prompt number. This provides a > sensible behaviour if we have two IPython shells open together - the second > one to be opened will be the latter session (and will be able to access > commands entered in the other session as soon as they are completed). > - For magic commands, accessing a line from a previous session could look > like "-1#9" (9th line of immediately previous session). > - On starting IPython, we load the last (~40 lines/~2 sessions) from the > database into readline history. > > Thoughts? Have I overlooked some key reason we use the existing system? Is > there a better alternative to SQLite? Would you design it differently? I've > not written any code for this yet, so I'm open to ideas. But if people think > that makes sense, I'm volunteering to make it happen. Finally, but importantly, I'm somewhat reluctant to go to sqlite until we've fully shown that accomplishing our design goals is hard/impossible with a simple json persisted data structure. While sqlite is indeed lightweight and available to us, it's also a more complex api to program against than simply storing a list/dict on disk. I'm OK accepting that complexity price *if we need it*, but I'm not convinced we do yet. Specifically, we need to answer: what is precisely the functionality that we want, that is hard/impossible to implement on json and easy/possible with sqlite? Cheers, f From fperez.net at gmail.com Fri Feb 18 11:06:26 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 18 Feb 2011 11:06:26 -0500 Subject: [IPython-dev] History In-Reply-To: References: <201102171853.59724.hans_meine@gmx.net> Message-ID: Hi Tomas, On Thu, Feb 17, 2011 at 1:46 PM, Thomas Kluyver wrote: > Good point, I hadn't thought of temporary aliases (I've never used them); > and good solution. While we're on the subject, what does translation > actually do? %magic commands, !system commands, aliases; anything else? Our input processing has been fairly cleaned up, but it could still use some more refactoring to put it all in one place so it's easier to understand. There are two parts to it: a) 'static' transformations that can be applied to any input string without knowing about the user's state/namespace. b) dynamic ones that are only applied to single-line input, and that are context-dependent. The code that does the input processing of type (a) is here: https://github.com/ipython/ipython/blob/master/IPython/core/inputsplitter.py And the single-line prefiltering of type (b) is here: https://github.com/ipython/ipython/blob/master/IPython/core/prefilter.py (b) is only called in one place, by the run_single_line method: https://github.com/ipython/ipython/blob/master/IPython/core/interactiveshell.py#L2188 I hope this helps, keep asking and we'll try to explain things out... Cheers, f From ellisonbg at gmail.com Fri Feb 18 12:01:59 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 18 Feb 2011 09:01:59 -0800 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: I too am a bit hesitant to go the sqlite route on this without some careful thinking. Cheers, Brian On Fri, Feb 18, 2011 at 7:48 AM, Fernando Perez wrote: > Hi all, > > sorry that this will be brief and not very thought-through, but I can > only sneak in short periods while at the conference... > > Many thanks to Thomas for getting this work going! ?But I think, since > we now have a bit more manpower and good momentum going, it's worth > thinking a little about the key points we want to hit so we end up > with something really solid. ?Comments below... > > On Wed, Feb 16, 2011 at 5:10 PM, Thomas Kluyver wrote: > >> - Each command is stored instantly, so we do away with the need for an >> autosave timer thread. A crash at any stage should leave your entire history >> intact up to the last command completed. > > Instant saving has one problem: frequent disk usage prevents hard > drives from spinnning down when on battery. ?The idea of an auto-save > thread on a timer with a user-controllable delay has the advantage > that the user can control their power consumption profile to fit their > needs. > > On an international flight when you're trying to squeeze every last > bit of your battery, this matters a lot. ?We don't want to turn > ipython into the thing that eats up your battery life to death just by > virtue of running very simple interactive commands that in principle > are purely CPU/memory resident, but because we generate lots of > sideband disk activity. > > So we should keep this consideration in mind in the design. ?If we > don't think about it now, it will be much harder to retrofit a decent > power profile later on. > >> - We store only raw history on disk (I think raw history is what we're >> looking for 90% of the time). If we want translated history, we redo the >> translation on the fly (this should require minimal computation, unless I've >> missed something). >> I've been having a think about how we store history: >> >> At present: Commands entered are stored in two lists (raw and "translated" - >> i.e. turning magic commands into function calls). These are persisted to >> disk at the next input every time 60 seconds elapse, with storage in a JSON >> file, which is reloaded into the same lists (and into readline history) when >> starting IPython. Each command entered is also immediately persisted in the >> "shadow history", a collection of files in .ipython managed by the >> pickleshare DB. The output objects are also stored in a dictionary by prompt >> number, but I'm less concerned with that here. >> >> Uses: >> - Readline history (getting previous commands via up arrow) >> - Various magic commands (save, macro, hist) can access ranges of input, >> using the prompt numbers from the current session. >> - %hist -g allows searching shadow and current history with glob syntax. >> - %rep can access ranges of this session's history, or single lines from >> shadow history. >> (These are all I've found so far - please let me know if there are others) > > No, we must store the translated history for two reasons: > > 1. Some translations are dynamic and context-dependent, so they can > not be recomputed later (though these are the minority). > > 2. More importantly, the translation process is relatively cpu > intensive, while disk space is the absolutely cheapest resource in > existence (at least at the data storage volumes we're talking about > here). ?So it makes sense to store on disk these results once we have > computed them, rather than recomputing them all over later on reload. > >> - History is indexed by session number and prompt number. This provides a >> sensible behaviour if we have two IPython shells open together - the second >> one to be opened will be the latter session (and will be able to access >> commands entered in the other session as soon as they are completed). >> - For magic commands, accessing a line from a previous session could look >> like "-1#9" (9th line of immediately previous session). >> - On starting IPython, we load the last (~40 lines/~2 sessions) from the >> database into readline history. >> >> Thoughts? Have I overlooked some key reason we use the existing system? Is >> there a better alternative to SQLite? Would you design it differently? I've >> not written any code for this yet, so I'm open to ideas. But if people think >> that makes sense, I'm volunteering to make it happen. > > Finally, but importantly, I'm somewhat reluctant to go to sqlite until > we've fully shown that accomplishing our design goals is > hard/impossible with a simple json persisted data structure. ?While > sqlite is indeed lightweight and available to us, it's also a more > complex api to program against than simply storing a list/dict on > disk. ?I'm OK accepting that complexity price *if we need it*, but I'm > not convinced we do yet. ?Specifically, we need to answer: what is > precisely the functionality that we want, that is hard/impossible to > implement on json and easy/possible with sqlite? > > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From fperez.net at gmail.com Fri Feb 18 11:54:40 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 18 Feb 2011 11:54:40 -0500 Subject: [IPython-dev] History In-Reply-To: References: <201102171853.59724.hans_meine@gmx.net> Message-ID: Hi, On Thu, Feb 17, 2011 at 2:31 PM, MinRK wrote: >>> Hmm, what about concurrent sessions? ?(Versus "previous" sessions which >>> have >>> been quit.) >> >> Interesting idea, but more complicated to implement. You'd have to somehow >> have separate shells signal their presence. I think this is a separate >> consideration - I'm looking at what we can get for free by restructuring the >> history. > > I think the question here might be: what would the history look like at > startup if I've been running several sessions simultaneously, and I start a > new one? > > IP0: foo='bar' > IP1: a=1 > IP2: b=2 > IP1: c=3 > > IP2: d=4 > > i.e. what is the 'previous' session? ?Is it 'most recently closed' or 'most > recently started' or 'most recently executed'? ?You seemed to suggest above > that IP3's readline history at startup should go d,c,b,a,foo (as I continue > to press up) Is that correct? ?I like this approach, but there's also value > in keeping related commands (same session) adjacent to each other, so I > don't think it's an obvious choice. ?In most real cases, this probably > doesn't > make much difference. This is really the key problem to think about, and I'm not sure I have a full solution even in my head yet. For now, a few thoughts... In a sense, the history problem can be seen similar to the evolution of a git repository, where you start with the master branch (the history of your first session ever). If you close and open a single session again (but no concurrent ones), then each of these operations is akin to a new commit, just adding to the old history. But if a concurrent session is opened, this is like making a new branch. However, from a user experience standpoint, we don't want to expose this kind of complexity in everyday work: people shouldn't need to think of doing git branch merges to manage their history :) And in actual usage, you tend to think of your history as a linear object running back in time. But as we've seen clearly, it's not linear, it's a DAG. So the question is: how to best flatten it, so that the user experience is automatic and shows a reasonable linearization of the DAG, and yet it's useful and powerful (better than today's behavior where a lot of clobbering happens from concurrent sessions)? If we start keeping all the sessions around, though, after a while it will get very unwieldy to manage this, as we'll probably need to use uuids to tag them, and tracking multiple sessions like this will be a pain. But we can't fully linearize on every save operation, because we'd lose the ability to track open concurrent sessions. I have to run to a talk soon... Just as a sketch of how it could work: there's a notion of a master session (to borrow git's terminology deliberately); on opening ipython, it marks it as in use (locks). If a second session is opened (it sees the active one as locked), a branch is made, tagged with a uuid and timestamp. On startup, if there are branches available they get merged into the current view, but grouped together with most recent available first. And only up to 5 branches are allowed to be 'in flight', so that the system doesn't grow too complicated. We may want to think about a few commands for users to manage this manually if they want... Profiles could allow long-lived branches: there's a 'master' per-profile, but otherwise the same logic as above applies. Incomplete ideas as food for thought, but I have to run now... >>> Are you kidding me? ?One session equals roughly 1400 lines for me. :-) >>> >>> That's a good question, how much history should be available from >>> "previous" >>> sessions. >> >> But how much of that do you access through readline? It can of course be >> user configurable, but we should set a sensible default. 40 is on the low >> side. I assume readline is fairly fast: maybe a couple of hundred is a good >> limit? > > I would think a default should be something like 500 at the very least. ?I > use ctrl-r all the time, and basically expect that if I remember typing it, > IPython?should remember as well. ?If you are just using arrow-keys, then > more than a few dozen isn't particularly valuable, but reverse-i-search > should find anything I remember well enough to look for. Yes, Ctrl-r should have O(1000) lines of history at least available to it. It's easy to use, fast, and very powerful. Many people live and die by it;l even if it's a somewhat more advanced readline feature, for power users it's a very important one. Cheers, f From robert.kern at gmail.com Fri Feb 18 12:11:49 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 18 Feb 2011 11:11:49 -0600 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 2/18/11 9:48 AM, Fernando Perez wrote: > Finally, but importantly, I'm somewhat reluctant to go to sqlite until > we've fully shown that accomplishing our design goals is > hard/impossible with a simple json persisted data structure. While > sqlite is indeed lightweight and available to us, it's also a more > complex api to program against than simply storing a list/dict on > disk. I'm OK accepting that complexity price *if we need it*, but I'm > not convinced we do yet. Specifically, we need to answer: what is > precisely the functionality that we want, that is hard/impossible to > implement on json and easy/possible with sqlite? Well, if the computation resources devoted to translating raw input to their executable form is important, then I suspect the serialization to JSON of the whole history every time it gets saved would often dominate it (translation happens rarely; loading/saving happens often). SQLite is safer. SQLite allows concurrency. SQLite allows us to store absolutely everything we have ever typed, but only load what we want to use in a session. SQLite is more easily extended to also record outputs. You get a basic notebook format practically for free! :-) -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg at gmail.com Fri Feb 18 13:06:11 2011 From: ellisonbg at gmail.com (Brian Granger) Date: Fri, 18 Feb 2011 10:06:11 -0800 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On Fri, Feb 18, 2011 at 9:11 AM, Robert Kern wrote: > On 2/18/11 9:48 AM, Fernando Perez wrote: > >> Finally, but importantly, I'm somewhat reluctant to go to sqlite until >> we've fully shown that accomplishing our design goals is >> hard/impossible with a simple json persisted data structure. ?While >> sqlite is indeed lightweight and available to us, it's also a more >> complex api to program against than simply storing a list/dict on >> disk. ?I'm OK accepting that complexity price *if we need it*, but I'm >> not convinced we do yet. ?Specifically, we need to answer: what is >> precisely the functionality that we want, that is hard/impossible to >> implement on json and easy/possible with sqlite? > > Well, if the computation resources devoted to translating raw input to their > executable form is important, then I suspect the serialization to JSON of the > whole history every time it gets saved would often dominate it (translation > happens rarely; loading/saving happens often). > > SQLite is safer. > SQLite allows concurrency. > SQLite allows us to store absolutely everything we have ever typed, but only > load what we want to use in a session. Absolutely, I think these points are *very* important and make sqlite worth thinking about. > SQLite is more easily extended to also record outputs. You get a basic notebook > format practically for free! ?:-) Some thoughts about this: * The history is a chronological picture of the command that happen on a particular engine session. * The notebook is a "spatial" picture of a subset of commands that is completely independent of a particular session. * The history can contain things that are not in a notebook (from a terminal IPython) and a notebook can contain things not in a history. * In the past, we have envisioned the notebook server as being a completely separate process/server than the kernel. For examples, you could imagine a centralized notebook server for a group that is used by frontends/kernels running all over the world. * Notebooks are inherently multiuser (notebooks in a notebook server are associated with users, they can be shared, etc.). But...it is an interesting idea to combine the kernel and notebook server. There could be some significant benefits for this, but it is definitely not the model we have been thinking. Any ideas about how this would work? One other thought: possibly, something like the history storage format should be abstracted through an API that we could implement using sqlite/json/mongodb/etc. Cheers, Brian > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -- Brian E. Granger, Ph.D. Assistant Professor of Physics Cal Poly State University, San Luis Obispo bgranger at calpoly.edu ellisonbg at gmail.com From robert.kern at gmail.com Fri Feb 18 13:44:39 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 18 Feb 2011 12:44:39 -0600 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: [OT: *please* stop Ccing me. I am on the list!] On 2/18/11 12:06 PM, Brian Granger wrote: > On Fri, Feb 18, 2011 at 9:11 AM, Robert Kern wrote: >> SQLite is more easily extended to also record outputs. You get a basic notebook >> format practically for free! :-) > > Some thoughts about this: > > * The history is a chronological picture of the command that happen on > a particular engine session. > * The notebook is a "spatial" picture of a subset of commands that is > completely independent of a particular session. > * The history can contain things that are not in a notebook (from a > terminal IPython) and a notebook can contain things not in a history. > * In the past, we have envisioned the notebook server as being a > completely separate process/server than the kernel. For examples, you > could imagine a centralized notebook server for a group that is used > by frontends/kernels running all over the world. > * Notebooks are inherently multiuser (notebooks in a notebook server > are associated with users, they can be shared, etc.). > > But...it is an interesting idea to combine the kernel and notebook > server. There could be some significant benefits for this, but it is > definitely not the model we have been thinking. Any ideas about how > this would work? The only thing that is fundamentally "multiuser" about a notebook is that one usually makes them to share with people. That sharing can take many forms, is frequently just for them to view, and often the only "other person" is yourself, six months later. None of that necessitates (or is even facilitated) by a server model, so I think that is a very limiting view and substantial overhead. Swapping files has worked pretty well for Mathematica, the Ur-notebook. To use a buzzword, a more "social" notebook interaction like you are talking about is something that's valuable and does require a server, but this is something built on top of the fundamental notebook idea; it's not the notebook itself. I think if you are chasing just the social features, you're missing out on a large segment of use cases that can't support the overhead of a server. As my smiley and "basic" indicated, I don't think that it's really a notebook, but it is an extension feature that I've wanted for a long time: record everything I execute and everything that's output. Less of a Mathematica notebook than a lab notebook. Given that all-inclusive history, I can write tools that let me extract the stuff I have been doing now and make a proper notebook whenever you get around to implementing one. :-) Mostly, I hate needing to make decisions before I start. Don't make me decide whether I want this session to be a notebook or not before I even begin typing. > One other thought: possibly, something like the history storage > format should be abstracted through an API that we could implement > using sqlite/json/mongodb/etc. I think you just need to flesh out the HistoryManager interface a little. I don't think it's worth making another interface. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Fri Feb 18 14:14:18 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 18 Feb 2011 19:14:18 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 18 February 2011 15:48, Fernando Perez wrote: > Instant saving has one problem: frequent disk usage prevents hard > drives from spinnning down when on battery. The idea of an auto-save > thread on a timer with a user-controllable delay has the advantage > that the user can control their power consumption profile to fit their > needs. > In the default configuration, I think we already have disk access on each command as it's written to the shadow history. Admittedly that can be disabled without affecting the main function of the program. We could write a powersaver configuration, which disables the immediate writes to the database, but registers an atexit handler to squirt the session in when IPython is closed. I certainly want to cater for people trying to eke out every last bit of battery life, but at the same time I don't want all the users who aren't relying on a battery to miss out on what I hope is a simpler, more powerful design for history. Summarising your other points: >Store translated history Indeed - I've already changed my mind on this thanks to comments from other people. My prototype is storing both raw and translated history >Advantages of SQLite? Robert's given a few. While JSON is a great tool, I don't think it's the "one obvious way to do it" here - you have to parse it all to read any of it, which is slow for a large file, and can lose your entire history if one byte gets out of place. Neither is pickleshare, which is essentially a key-value database, not designed to store a sequence. SQLite can essentially combine what we're trying to do with both, and do it "right". Also, I don't think the way we're reloading the history at present is tidy. It gets lumped in with the current session, which causes the indexing problems I already discovered. I've worked round those with session_offset, but it's really a workaround. I could rethink the saving/reloading with JSON, but when I tried thinking about it, I just decided JSON wasn't the best tool for the job. >Input processing Thanks for the info :) >Ordering multiple sessions: While it's worth some thought, let's not lose too much sleep over the order. As we've determined, if you want to go more than a few lines back in the history, you search it somehow, in which case order doesn't matter much. I never actually knew about Ctrl-R: I just type the first few letters of what I want, then press up. There are all sorts of options we could have for named sessions, for repeating previous sessions, and so on, but I've not explored any of that yet. As before, profiles have, for now, entirely separate history files - I don't know if we want to keep that, or integrate them into one to allow accessing history from other profiles. >Tracking concurrent sessions: Sessions can clearly leave some signal of their existence on disk, but if they crash without clearing it up, other shells will see a zombie shell still there. I don't think there's a payoff worth the trouble of tracking 'live' shells: we can just share the database with whatever other shells are around. >Readline history - 1000 lines OK, I've made the prototype load 1000 lines by default. We can always up that if we want. (Robert/Brian) > Notebooks [Snip] OK, Robert's clarified now that he meant something more like a logbook than the UI option we've called a notebook. Robert: storing the output is a bit trickier. We can store the repr of it easily enough, which might be what you want. We could pickle output items and store them, but that could quickly make the database file very large. If you want to save a session with output, I believe you can use "%hist -of session.txt". (Brian) > Consistent API, alternative backends Parts of the history_manager API can easily be abstracted - like store_inputs and get_history. But there's a more conceptual difference in what I'm doing, too: I'm getting rid of the idea of saving and loading history, in favour of a model where it's immediately persisted. We can expand that to include delayed persistence (as I've suggested above for extending battery life), but we have to choose whether our API is written in terms of "save" and "load", or in terms of "push cache to persistent store". Also, while I'm all in favour of clean, well documented APIs, I think storing history is the sort of thing that we as the creators should get right, and users should never have to worry about. If someone wants to write their own history manager for some reason, that's fine, but I don't think we need to design around the assumption that people will. Let's focus on writing a history system, not on an interface for possible history systems. Whew! This seems to be a thread of epic e-mails. Thanks everyone for your thoughts, and I hope I've answered any questions people had. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Feb 18 14:34:43 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 18 Feb 2011 13:34:43 -0600 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 2/18/11 1:14 PM, Thomas Kluyver wrote: > >Ordering multiple sessions: > While it's worth some thought, let's not lose too much sleep over the order. As > we've determined, if you want to go more than a few lines back in the history, > you search it somehow, in which case order doesn't matter much. Right. Just pick a simple rule that keeps sessions intact, say ordered by the start time of the session. The only thing to avoid is interleaving inputs from different sessions. > (Robert/Brian) > Notebooks > [Snip] OK, Robert's clarified now that he meant something more like a logbook > than the UI option we've called a notebook. > > Robert: storing the output is a bit trickier. We can store the repr of it easily > enough, which might be what you want. We could pickle output items and store > them, but that could quickly make the database file very large. I don't mean the actual objects, just whatever is given by the DisplayFormatter system and the displayhook, i.e. whatever would get sent to a remote frontend. I'm happy to have it be an extension functionality disabled by default, but I do think that it should live in the same SQLite DB to get the references right. > If you want to > save a session with output, I believe you can use "%hist -of session.txt". Not what I'm looking for, but that functionality could be more easily (and efficiently) backed by the datastore I'm asking for. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Fri Feb 18 16:50:50 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 18 Feb 2011 21:50:50 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 18 February 2011 19:34, Robert Kern wrote: > I don't mean the actual objects, just whatever is given by the > DisplayFormatter > system and the displayhook, i.e. whatever would get sent to a remote > frontend. > I'm happy to have it be an extension functionality disabled by default, but > I do > think that it should live in the same SQLite DB to get the references > right. I don't have a problem with that, although my vote does go to having it disabled by default. It's not altogether trivial, though, because we currently store the input before running it, so the output isn't available at the same time. Possible ways round that: - Keep output in a separate table, indexed in the same way as input. This is my preferred option, because it means the main history system doesn't need to be aware of the optional output history. There's a slightly greater chance of a mismatch between input and output, but the indexing is simple enough that it shouldn't be a problem. - Store input, then execute, and update the row with output (if any) - I don't much like the idea of doing update statements after every command. - Store input after execution - undesirable, because a command that crashes IPython will not be stored. - Store input after execution only when output history is turned on - ugly idea. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Feb 18 17:10:40 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 18 Feb 2011 16:10:40 -0600 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: [Please do not Cc me. I am on the list!] On 2/18/11 3:50 PM, Thomas Kluyver wrote: > On 18 February 2011 19:34, Robert Kern > wrote: > > I don't mean the actual objects, just whatever is given by the DisplayFormatter > system and the displayhook, i.e. whatever would get sent to a remote frontend. > I'm happy to have it be an extension functionality disabled by default, but I do > think that it should live in the same SQLite DB to get the references right. > > I don't have a problem with that, although my vote does go to having it disabled > by default. I vote the same way, too. :-) > It's not altogether trivial, though, because we currently store the > input before running it, so the output isn't available at the same time. That's fine. It's necessarily a separate API although I do think that it should be part of the HistoryManager. > Possible ways round that: > - Keep output in a separate table, indexed in the same way as input. This is my > preferred option, because it means the main history system doesn't need to be > aware of the optional output history. There's a slightly greater chance of a > mismatch between input and output, but the indexing is simple enough that it > shouldn't be a problem. This sounds fine. With (session, line) as the foreign key, it seems straightforward to keep it synchronous. The only suggestion I would make is to make the store_inputs() method (and a putative store_outputs() method) take the execution_count explicitly rather than taking it from the shell. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Fri Feb 18 17:37:44 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Fri, 18 Feb 2011 22:37:44 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 18 February 2011 22:10, Robert Kern wrote: > [Please do not Cc me. I am on the list!] > Sorry. If I hit reply, it e-mails you directly (GMail's not so good at mailing lists), so I'm in the habit of doing reply all. I thought the list was smart enough to see your e-mail address and only send it once. I'll try to remember to take other addresses out. > This sounds fine. With (session, line) as the foreign key, it seems > straightforward to keep it synchronous. The only suggestion I would make is > to > make the store_inputs() method (and a putative store_outputs() method) take > the > execution_count explicitly rather than taking it from the shell. > OK, excellent. And yes, it's not a bad idea to take the execution_count/line number explicitly. Done. Coming back to Fernando's point about disk writes, I've implemented a very simple cache (21 extra lines :) ). It can be turned on simply by setting ip.history_manager.db_cache_size to the number of lines you want to store before writing to disk. So, if you set it to 19, it will write on every 20th command. For now, I've left the default at 0 - instant writes. And on exit, any remaining cache is written, so you can set it to a very high value if you want to minimise disk access. Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From takowl at gmail.com Sat Feb 19 16:54:29 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Sat, 19 Feb 2011 21:54:29 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 18 February 2011 22:10, Robert Kern wrote: > This sounds fine. With (session, line) as the foreign key, it seems > straightforward to keep it synchronous. The only suggestion I would make is > to > make the store_inputs() method (and a putative store_outputs() method) take > the > execution_count explicitly rather than taking it from the shell. > Basic implementation here: https://github.com/takluyver/ipython/commit/15fbffa0d924765461c9a1d539d49daf70bc5315 The only bit missing is that I don't know off the top of my head where to call the store_outputs method. I can have a dig in the code, or if someone would like to point me to the best place, I'd be grateful. In the meantime, I've tried calling the function manually, and it works as expected. Thanks, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From benjaminrk at gmail.com Sun Feb 20 18:02:55 2011 From: benjaminrk at gmail.com (MinRK) Date: Sun, 20 Feb 2011 15:02:55 -0800 Subject: [IPython-dev] GitHub Issues viewer Message-ID: Hello, Much as many of us are fond of GitHub, one of the site's major shortcomings is the Issues page, especially if you have as many issues as IPython does. However, one of the great features of GitHub is its json api, which means that anyone can write a better UI to the issue tracker, and various people have had a run at this. One I have found useful is actually a demo of Cappucino, an OSX-centric webapp framework. Far from perfect, but it does handle the long list much better. You can see IPython's Issues by visiting: http://githubissues.heroku.com/#ipython/ipython And, if you are on OSX, you can run it as a standalone app, downloaded from here: http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/ -MinRK From fperez.net at gmail.com Sun Feb 20 20:50:11 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 20 Feb 2011 20:50:11 -0500 Subject: [IPython-dev] GitHub Issues viewer In-Reply-To: References: Message-ID: On Sun, Feb 20, 2011 at 6:02 PM, MinRK wrote: > > You can see IPython's Issues by visiting: > http://githubissues.heroku.com/#ipython/ipython Nice! Thanks for the link, very useful. Cheers, f From ccordoba12 at gmail.com Sun Feb 20 22:13:45 2011 From: ccordoba12 at gmail.com (=?ISO-8859-1?Q?Carlos_C=F3rdoba?=) Date: Sun, 20 Feb 2011 22:13:45 -0500 Subject: [IPython-dev] GitHub Issues viewer In-Reply-To: References: Message-ID: <4D61D869.9080905@gmail.com> Quite cool app indeed! For those using Chrome, they can play with it after installing it from Chrome Web Store, at https://chrome.google.com/webstore/detail/oaianekphgmiibhdffnoimngeolnailp Carlos El 20/02/11 18:02, MinRK escribi?: > Hello, > > Much as many of us are fond of GitHub, one of the site's major > shortcomings is the Issues page, especially if you have as many issues > as IPython does. However, one of the great features of GitHub is its > json api, which means that anyone can write a better UI to the issue > tracker, and various people have had a run at this. One I have found > useful is actually a demo of Cappucino, an OSX-centric webapp > framework. Far from perfect, but it does handle the long list much > better. > > You can see IPython's Issues by visiting: > http://githubissues.heroku.com/#ipython/ipython > > And, if you are on OSX, you can run it as a standalone app, downloaded > from here: > http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/ > > -MinRK > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev From kunal.t2 at gmail.com Mon Feb 21 09:00:04 2011 From: kunal.t2 at gmail.com (kunal) Date: Mon, 21 Feb 2011 19:30:04 +0530 Subject: [IPython-dev] GitHub Issues viewer In-Reply-To: References: Message-ID: <4D626FE4.2010107@gmail.com> On 02/21/2011 07:20 AM, Fernando Perez wrote: > On Sun, Feb 20, 2011 at 6:02 PM, MinRK wrote: >> You can see IPython's Issues by visiting: >> http://githubissues.heroku.com/#ipython/ipython > Nice! Thanks for the link, very useful. This looks neat ! +1 much more use-able than the default github issue tracker. > Cheers, > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev -- regards ------- Kunal Ghosh Dept of Computer Sc.& Engineering. Sir MVIT Bangalore,India permalink: member.acm.org/~kunal.t2 Blog:kunalghosh.wordpress.com Website:www.kunalghosh.net46.net From robert.kern at gmail.com Mon Feb 21 15:31:37 2011 From: robert.kern at gmail.com (Robert Kern) Date: Mon, 21 Feb 2011 14:31:37 -0600 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 2/19/11 3:54 PM, Thomas Kluyver wrote: > On 18 February 2011 22:10, Robert Kern > wrote: > > This sounds fine. With (session, line) as the foreign key, it seems > straightforward to keep it synchronous. The only suggestion I would make is to > make the store_inputs() method (and a putative store_outputs() method) take the > execution_count explicitly rather than taking it from the shell. > > > Basic implementation here: > https://github.com/takluyver/ipython/commit/15fbffa0d924765461c9a1d539d49daf70bc5315 > > The only bit missing is that I don't know off the top of my head where to call > the store_outputs method. I can have a dig in the code, or if someone would like > to point me to the best place, I'd be grateful. In the meantime, I've tried > calling the function manually, and it works as expected. The DisplayHook.log_output() method would be the right place to do it. Eventually, it would be nice to expand this to capture all of the different formats and the printed outputs, but just storing the text/plain format for displayhook'ed objects is enough for now. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From takowl at gmail.com Tue Feb 22 14:15:54 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 22 Feb 2011 19:15:54 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On 21 February 2011 23:19, Robert Kern wrote: > Now you just have to make HistoryManager a Configurable so we can > enable it from the configuration. > I think this does the trick: https://github.com/takluyver/ipython/commit/eca335fc6efe0f479774e2a93cf95f730342bf91 I've tested uncommenting the lines in ipython_config.py, and they have the desired effects. Thanks for your help with the config system and traitlets, Thomas -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Feb 22 16:39:39 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 22 Feb 2011 13:39:39 -0800 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: Quick question: does the work you've done in any way impact/supersede/collide with this one: https://github.com/ipython/ipython/pull/235 ? Cheers f From takowl at gmail.com Tue Feb 22 16:49:42 2011 From: takowl at gmail.com (Thomas Kluyver) Date: Tue, 22 Feb 2011 21:49:42 +0000 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: At a glance, yes, I'm afraid so. It looks like he's solved the same thing I've already got a pull request open for. I should have looked at the pull requests before I started coding. He's added an "all history" option to the %history magic command, which is more than I did. However, I'm still hoping to persuade you (and others) of the benefits of the SQLite history system I'm currently working on. In that case, that work should supersede both that pull request and my open history-related pull request. I'm just sorting out APIs, but I should have a pull request ready before long (next couple of days). Best wishes, Thomas On 22 February 2011 21:39, Fernando Perez wrote: > Quick question: does the work you've done in any way > impact/supersede/collide with this one: > > https://github.com/ipython/ipython/pull/235 > > ? > > Cheers > > f > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Tue Feb 22 16:53:02 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 22 Feb 2011 13:53:02 -0800 Subject: [IPython-dev] GitHub Issues viewer In-Reply-To: References: Message-ID: Hey, On Sun, Feb 20, 2011 at 3:02 PM, MinRK wrote: > > You can see IPython's Issues by visiting: > http://githubissues.heroku.com/#ipython/ipython > > And, if you are on OSX, you can run it as a standalone app, downloaded > from here: > http://cappuccino.org/discuss/2010/05/13/github-issues-cappuccino-app-desktop-and-web/ > I added the link in our dev docs on the wiki: http://ipython.scipy.org/moin/Developer_Zone Cheers, f From fperez.net at gmail.com Tue Feb 22 16:55:43 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Tue, 22 Feb 2011 13:55:43 -0800 Subject: [IPython-dev] History In-Reply-To: References: Message-ID: On Tue, Feb 22, 2011 at 1:49 PM, Thomas Kluyver wrote: > > However, I'm still hoping to persuade you (and others) of the benefits of > the SQLite history system I'm currently working on. In that case, that work > should supersede both that pull request and my open history-related pull > request. I'm just sorting out APIs, but I should have a pull request ready > before long (next couple of days). OK, no problem. I have to say, I've reread the discussion and I'm warming up to the idea of the sqlite approach quite a bit. Robert's points are very valid, and I like the idea of having a programatically accessible store of all inputs (and possibly output reprs) that users can get data from in any way they see fit. And as Robert says, this is a global store, complementary to any more document-oriented notebook idea. Cheers, f From robert.kern at gmail.com Fri Feb 25 19:40:33 2011 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 25 Feb 2011 18:40:33 -0600 Subject: [IPython-dev] Delete pyzmq 2.0.10-1 on PyPI? Message-ID: Brian, Min, Can you delete or hide the pyzmq 2.0.10-1 release on PyPI? Due to the way PyPI sorts releases, it is coming up as the latest release of pyzmq. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From benjaminrk at gmail.com Fri Feb 25 19:50:13 2011 From: benjaminrk at gmail.com (MinRK) Date: Fri, 25 Feb 2011 16:50:13 -0800 Subject: [IPython-dev] Delete pyzmq 2.0.10-1 on PyPI? In-Reply-To: References: Message-ID: Done, thanks for catching that one. I had missed it, since easy_install was correctly finding 2.0.10.1. -MinRK On Fri, Feb 25, 2011 at 16:40, Robert Kern wrote: > Brian, Min, > > Can you delete or hide the pyzmq 2.0.10-1 release on PyPI? Due to the way PyPI > sorts releases, it is coming up as the latest release of pyzmq. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless enigma > ?that is made terrible by our own mad attempt to interpret it as though it had > ?an underlying truth." > ? -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > From fperez.net at gmail.com Sat Feb 26 19:38:26 2011 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 26 Feb 2011 16:38:26 -0800 Subject: [IPython-dev] Fwd: [GitHub] SQLite history [ipython/ipython GH-282] In-Reply-To: References: Message-ID: Hi folks, Thomas just created this pull request for the sqlite-based history. I'll try to have a look soon, but this is large and important enough that more sets of eyes would be very, very welcome. And many thanks to Thomas for implementing all this! Best, f ---------- Forwarded message ---------- From: takluyver Date: Sat, Feb 26, 2011 at 3:11 PM Subject: [GitHub] SQLite history [ipython/ipython GH-282] To: fperez.net at gmail.com This was discussed previously on the mailing list. The code will probably still need some refining, but I think it's working well enough to put it forward. There are two major changes involved: History is now stored on disk as an SQLite database. This replaces both the JSON history and the "shadow history" database. By default, the input is stored as each line is entered, but (as Fernando suggested), there is a configurable attribute, db_cache_size, which lets you determine how often (in number of commands) it will write to disk. Very large values will cause all input to be written when IPython quits. A separate configurable attribute, db_log_output, causes the plaintext repr of output to be written to a separate table in the file. Secondly, whereas history was previously treated as one long stream of commands, it is now stored by "sessions", i.e. runs of IPython. So, if you are particularly fond of the fifth command you gave IPython in one session, the next time you start IPython, you can refer to the fifth command of the last session (the syntax for magic commands is `~1/5`). Sessions are consecutively numbered from when they start, so interleaved commands from two shells open at the same time will be kept separate. This requires somewhat more complicated interfaces. There are now four main interfaces to retrieve history from HistoryManager: * `get_history`: Retrieves specified ranges of history, either from the database, or the current session. * `get_hist_from_rangestr`: Is a convenience method for magic functions, which accepts the "~1/3-6 ?12" style range specifications. * `get_hist_tail`: Gets the last n items from the database, ordered by session and line. Used to populate readline (and the equivalent in the Qt console). * `get_hist_search`: Searches the database using glob style matching (i.e. wildcards, not regexen). All of these return iterators over 3-tuples, `(session, lineno, item)`. If asked for output, `item` will be a 2-tuple of `(input, output)`. I've also given `%hist` the same range specification parsing as `%save`, `%macro` and `%edit` (it would previously only accept one or two numbers separated by a space). A single numeric argument, such as `%hist 10` previously fetched the last 10 lines; now it gets the 10th line of the current session. So I've added a new option, `-l` (for last), which works with a single numeric argument. This is marginally less convenient if that's what you want, but I think the consistency among magic functions retrieving history is a good thing. https://github.com/ipython/ipython/pull/282