From prabhu at aero.iitb.ac.in Thu Oct 1 16:48:57 2009 From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran) Date: Fri, 02 Oct 2009 02:18:57 +0530 Subject: [IPython-dev] [ANN] SciPy India conference in Dec. 2009 Message-ID: <4AC515B9.4070604@aero.iitb.ac.in> Greetings, The first "Scientific Computing with Python" conference in India (http://scipy.in) will be held from December 12th to 17th, 2009 at the Technopark in Trivandrum, Kerala, India (http://www.technopark.org/). The theme of the conference will be "Scientific Python in Action" with respect to application and teaching. We are pleased to have Travis Oliphant, the creator and lead developer of numpy (http://numpy.scipy.org) as the keynote speaker. Here is a rough schedule of the conference: Sat. Dec. 12 (conference) Sun. Dec. 13 (conference) Mon. Dec. 14 (tutorials) Tues. Dec. 15 (tutorials) Wed. Dec. 16 (sprint) Thu. Dec. 17 (sprint) The tutorial sessions will have two tracks, one specifically for teachers and one for the general public. There are no registration fees. Please register at: http://scipy.in The call for papers will be announced soon. This conference is organized by the FOSSEE project (http://fossee.in) funded by the Ministry of Human Resources and Development's National Mission on Education (NME) through Information and Communication Technology (ICT) jointly with SPACE-Kerala (http://www.space-kerala.org). Regards, Prabhu Ramachandran and Jarrod Millman From robert.kern at gmail.com Thu Oct 8 14:58:26 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 08 Oct 2009 13:58:26 -0500 Subject: [IPython-dev] Status of ipy_pretty and ipy_traits_completer? Message-ID: Why are they in quarantine/ and deathrow/, respectively? There are several reasons one could be in either one. There should be a comment or a README that explains why they were placed there so contributors know what to do to remedy the situation or users prepare for the module's removal. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Thu Oct 8 16:42:33 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 8 Oct 2009 13:42:33 -0700 Subject: [IPython-dev] Status of ipy_pretty and ipy_traits_completer? In-Reply-To: References: Message-ID: <6ce0ac130910081342v41685578t3811a777633e1e87@mail.gmail.com> Robert, I agree it would be useful to have a README in those directories, but I wasn't entirely neglectful: >From the "changes/whatsnew" docs: * :mod:`Extensions` has been moved to :mod:`extensions` and all existing extensions have been moved to either :mod:`IPython.quarantine` or :mod:`IPython.deathrow`. :mod:`IPython.quarantine` contains modules that we plan on keeping but that need to be updated. :mod:`IPython.deathrow` contains modules that are either dead or that should be maintained as third party libraries. More details about this can be found :ref:`here `. >From module_reorg: * :mod:`IPython.deathrow`. This is for code that is outdated, untested, rotting, or that belongs in a separate third party project. Eventually all this code will either i) be revived by someone willing to maintain it with tests and docs and re-included into IPython or 2) be removed from IPython proper, but put into a separate third-party Python package. No new code will be allowed here. * :mod:`IPython.quarantine`. This is for code that doesn't meet IPython's standards, but that we plan on keeping. To be moved out of this sub-package a module needs to have approval of the core IPython developers, tests and documentation. Based on this, I would say the following: * ipy_pretty should be cleaned up, ported to the new extension API and put back into extensions. * ipy_traits_completer should be cleaned up, ported to the new extension API and shipped with traits itself. The new extension API supports loading extensions from any module in sys.path (it just imports them). Cheers, Brian On Thu, Oct 8, 2009 at 11:58 AM, Robert Kern wrote: > Why are they in quarantine/ and deathrow/, respectively? There are several > reasons one could be in either one. There should be a comment or a README > that > explains why they were placed there so contributors know what to do to > remedy > the situation or users prepare for the module's removal. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 8 16:58:54 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 08 Oct 2009 15:58:54 -0500 Subject: [IPython-dev] Status of ipy_pretty and ipy_traits_completer? In-Reply-To: <6ce0ac130910081342v41685578t3811a777633e1e87@mail.gmail.com> References: <6ce0ac130910081342v41685578t3811a777633e1e87@mail.gmail.com> Message-ID: On 2009-10-08 15:42 PM, Brian Granger wrote: > Robert, > > I agree it would be useful to have a README in those directories, but I > wasn't entirely > neglectful: > > From the "changes/whatsnew" docs: ... > From module_reorg: ... I read those. I asked because those documents give a number of listed reasons a module could be in either directory. They don't tell me which reason you thought applied to each module. For example, ipy_traits_completer could plausibly be in deathrow because it didn't have a maintainer (my submitted patches notwithstanding) or because you thought it belonged in a separate package. The appropriate next action depends on which reason you had in mind. > Based on this, I would say the following: > > * ipy_pretty should be cleaned up, What needs to be cleaned up? -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Thu Oct 8 18:25:50 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 8 Oct 2009 15:25:50 -0700 Subject: [IPython-dev] Status of ipy_pretty and ipy_traits_completer? In-Reply-To: References: <6ce0ac130910081342v41685578t3811a777633e1e87@mail.gmail.com> Message-ID: <6ce0ac130910081525x5ab6a6e5ve0d45f5612f1bee1@mail.gmail.com> On Thu, Oct 8, 2009 at 1:58 PM, Robert Kern wrote: > On 2009-10-08 15:42 PM, Brian Granger wrote: > > Robert, > > > > I agree it would be useful to have a README in those directories, but I > > wasn't entirely > > neglectful: > > > > From the "changes/whatsnew" docs: > ... > > From module_reorg: > ... > > I read those. I asked because those documents give a number of listed > reasons a > module could be in either directory. They don't tell me which reason you > thought > applied to each module. For example, ipy_traits_completer could plausibly > be in > deathrow because it didn't have a maintainer (my submitted patches > notwithstanding) or because you thought it belonged in a separate package. > The > appropriate next action depends on which reason you had in mind. > > Ah, Yes, you are absolutely right, especially about the reasons something may be in deathrow. I sorted through the extension modules putting them into either quarantine or deathrow, simply because most (if not all) of them are currently broken because of the API changes that have been made. In many cases, I had no idea what the status of various extensions (are they used by anyone, were they broken before the API changes) was and I made a very rough guess about where it should go (quarantine or deathrow). My *intention* is that when people see their favorite extension in quarantine or deathrow, that *have* to come to the list to help us figure out what needs to happen. Over time, that is how we can figure out which extensions "matter" and which have outlived their life or belong elsewhere. For ipy_pretty and ipy_traits_completer it is easy, because I know what they do and that they are still used. But for many other ones, I personally have *no idea* what there status is, so it really needs to be discussed on list. Summary: I want people to have to ask on list about the status of things in quarantine/deathrow. > Based on this, I would say the following: > > > > * ipy_pretty should be cleaned up, > > What needs to be cleaned up? > > It needs to be converted to the new extension API. Here are the docs for that: (this is in docs/source/config/ipython.txt) :attr:`c.Global.extensions` A list of strings, each of which is an importable IPython extension. An IPython extension is a regular Python module or package that has a :func:`load_in_ipython(ip)` method. This method gets called when the extension is loaded with the currently running :class:`~IPython.core.iplib.InteractiveShell` as its only argument. You can put your extensions anywhere they can be imported but we add the :file:`extensions` subdirectory of the ipython directory to ``sys.path`` during extension loading, so you can put them there as well. Extensions are not executed in the user's interactive namespace and they must be pure Python code. Extensions are the recommended way of customizing :command:`ipython`. The easy part is this, if you remove the ipapi stuff and instead all the following to the bottom: def load_in_ipython(ip): ip.set_hook('result_display', pretty_result_display, priority=99) The slightly more involved part is that the way the config variables are handled will need to be changed. I don't mind doing this as an example if you can wait until tomorrow (my son's 4th B-day is today!). But, one question: once the extension is activated, who does a user activate a custom displayer for a type? By calling for_type/for_type_by_name (which are injected into the userns)? It would be great to convert this to the new API to show how it all works. Cheers, Brian -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 8 18:42:28 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 08 Oct 2009 17:42:28 -0500 Subject: [IPython-dev] Status of ipy_pretty and ipy_traits_completer? In-Reply-To: <6ce0ac130910081525x5ab6a6e5ve0d45f5612f1bee1@mail.gmail.com> References: <6ce0ac130910081342v41685578t3811a777633e1e87@mail.gmail.com> <6ce0ac130910081525x5ab6a6e5ve0d45f5612f1bee1@mail.gmail.com> Message-ID: On 2009-10-08 17:25 PM, Brian Granger wrote: > Summary: I want people to have to ask on list about the status of things in > quarantine/deathrow. A sentence or two in the organization doc to that effect would work well. But tomorrow on a not-your-son's-birth-day. :-) > The slightly more involved part is that the way the config variables are > handled will > need to be changed. I don't mind doing this as an example if you can > wait until tomorrow > (my son's 4th B-day is today!). But, one question: once > the extension is activated, who does a user activate a custom displayer > for a type? > By calling for_type/for_type_by_name (which are injected into the userns)? I assumed people would usually do configure this in what used to be ipy_user_conf.py. They would import IPython.external.pretty and call the functions there. I don't think for_type/for_type_by_name are particularly useful in the userns. A pair of push/pop %magics that manages a stack built on top of for_type would be pretty useful, though. Push a verbose printer to look at particular type in depth, then pop it off to go back to the less noisy mode. Or vice versa: push a summ...ng printer for lists, for example, to get a look at "the big picture" in a larger object hierarchy. But without the ability to pop, I don't see a reason to put for_type into the namespace. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Fri Oct 9 14:00:59 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 9 Oct 2009 11:00:59 -0700 Subject: [IPython-dev] How to handle extensions Message-ID: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> We have started to get questions about we will handle ipython extensions with all the API changes that have happened. There are some unresolved issues that I wanted to bring up here. In the past, IPython.extensions was a general dumping ground for IPython code. It contained many types of things: * Part of the core IPython functionality. * Third party extensions. * Many other things... To force us to deal with these things I have moved everything from IPython.extensions to either IPython.deathrow or IPython.quarantine. The purposes of these 2 sub-packages are documented here: http://bazaar.launchpad.net/~ipython-dev/ipython/trunk/annotate/head%3A/docs/source/development/reorg.txt We now have an official extension API that is described here: http://bazaar.launchpad.net/~ipython-dev/ipython/trunk/annotate/head%3A/docs/source/config/ipython.txt For things to be moved back into IPython.extensions they need to adhere to this new API. But...not all things that adhere to this API should be let into IPython.extensions. Here is my current thinking, but I wanted to get feedback from others. The context of this is that a user wants to know if ipy_profile_zope.py could be put back into extensions. Here is my thought... * To in extensions, it has to be an extension. Profiles are something completely different now. A profile is simply a config file with a special name. Profiles can load extensions though. We currently have a place for custom profiles (IPython.config.profile). * To be in IPython.extensions or IPython.config.profile a module has to satisfy a few things: - There has to be willingness amongst the core IPython developers to maintain the code. - The code needs to be reviewed (tests, docs, etc.) - If the code should be in IPython.core, it shouldn't be in extensions. - If an extension/profile can be distributed as a third party package, it should be. Thus, the custom completer for enthough.traits should ship with enthought.traits. Things related to numpy should ship with numpy. The only case where an extension/profile should be included with IPython is if i) there is no third party package that it makes sense to ship it with and ii) the majority of IPython users want the functionality but iii) the functionality shouldn't go into IPython.core. These decisions will need to be made on a case by case basis as people start asking about their favorite extensions being in quarantine or deathrow. But, I wanted to see what people though about the overall approach. Cheers, Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Oct 9 14:55:40 2009 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 09 Oct 2009 13:55:40 -0500 Subject: [IPython-dev] How to handle extensions In-Reply-To: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> Message-ID: On 2009-10-09 13:00 PM, Brian Granger wrote: > * To be in IPython.extensions or IPython.config.profile a module has to > satisfy a few things: > > - There has to be willingness amongst the core IPython developers to > maintain the code. > - The code needs to be reviewed (tests, docs, etc.) > - If the code should be in IPython.core, it shouldn't be in extensions. > - If an extension/profile can be distributed as a third party package, > it should be. Thus, > the custom completer for enthough.traits should ship with > enthought.traits. Things related > to numpy should ship with numpy. There is one issue with this. You want to be able to configure some things for a particular package in your configuration files at startup, but not actually import the package, which may be expensive. With care, many of the package-specific extensions can be written in such a way as to be enabled at startup but not import the package until it absolutely must (in which case the user has probably already imported it himself). For example, I submitted a patch to ipy_traits_completer that replace isinstance(obj, HasTraits) checks for one that looked for superclasses named "HasTraits" before import HasTraits and doing the real isinstance() check. Now, it wouldn't be too oppressive to put ipy_traits_completer into enthought.traits because we keep the __init__s clear (except for namespace package stuff, natch). But one couldn't put IPython support code into numpy without needing to import numpy itself to activate it in the configuration. It might be worth having an ipy_extensions package distributed separately from IPython that collects these extensions to reduce the burden on the IPython developers and the quality disparity with the rest of code. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Fri Oct 9 16:56:09 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 9 Oct 2009 13:56:09 -0700 Subject: [IPython-dev] How to handle extensions In-Reply-To: References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> Message-ID: <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> On Fri, Oct 9, 2009 at 11:55 AM, Robert Kern wrote: > On 2009-10-09 13:00 PM, Brian Granger wrote: > > > * To be in IPython.extensions or IPython.config.profile a module has to > > satisfy a few things: > > > > - There has to be willingness amongst the core IPython developers to > > maintain the code. > > - The code needs to be reviewed (tests, docs, etc.) > > - If the code should be in IPython.core, it shouldn't be in extensions. > > - If an extension/profile can be distributed as a third party package, > > it should be. Thus, > > the custom completer for enthough.traits should ship with > > enthought.traits. Things related > > to numpy should ship with numpy. > > This is an important point and I want to understand the issues and fix things now... > There is one issue with this. You want to be able to configure some things > for a > particular package in your configuration files at startup, but not actually > import the package, which may be expensive. With the changes I have in my local branch this is possible. You simply enter the config information in the config files, but don't list the extensions in the config field of the extensions that should be loaded at startup. Then you have a number of choices for loading the extension: get_ipython().load_extension('package.myextension') or %load_ext package.myextension > With care, many of the > package-specific extensions can be written in such a way as to be enabled > at > startup but not import the package until it absolutely must (in which case > the > user has probably already imported it himself). For example, I submitted a > patch > to ipy_traits_completer that replace isinstance(obj, HasTraits) checks for > one > that looked for superclasses named "HasTraits" before import HasTraits and > doing > the real isinstance() check. > > The question of a lazy loading of an extension is a good point. For now, let's pretend that an extension can be imported/loaded without any side effects (it is not in a package that has an __init__ that imports everything). Then lazy importing is easy right? You can just make sure that the package (traits/numpy/etc) isn't imported when the extension is imported, but later, when the extension is first used. But don't you think that if someone has explicitly enabled a numpy based extension, it is because they are using numpy (so importing it at startup would happen anyways). If a user didn't want to *always* have the numpy extension loaded, they could simple maintain a numpy profile that loaded numpy. But if you really need lazy importing, I think it is possible still... > Now, it wouldn't be too oppressive to put ipy_traits_completer into > enthought.traits because we keep the __init__s clear (except for namespace > package stuff, natch). But one couldn't put IPython support code into numpy > without needing to import numpy itself to activate it in the configuration. > > Yes, for packages that have lots of things in __init__, this is a problem. For something as common as numpy, I think shipping it in IPython.extensions may make sense. > It might be worth having an ipy_extensions package distributed separately > from > IPython that collects these extensions to reduce the burden on the IPython > developers and the quality disparity with the rest of code. > This is a good point. I have envisioned something like a meta-project on launchpad, similar to this for Twisted related projects: https://launchpad.net/tx That way the development of all the extensions/profiles can proceed independently. I could even imagine that other things might eventually be moved outside the core to a place like this (GUI frontends, etc.) Cheers, Brian > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Fri Oct 9 18:07:05 2009 From: robert.kern at gmail.com (Robert Kern) Date: Fri, 9 Oct 2009 17:07:05 -0500 Subject: [IPython-dev] How to handle extensions In-Reply-To: <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> Message-ID: <3d375d730910091507v760a7aa6y357eb6a2b025e069@mail.gmail.com> On Fri, Oct 9, 2009 at 15:56, Brian Granger wrote: > > > On Fri, Oct 9, 2009 at 11:55 AM, Robert Kern wrote: >> >> On 2009-10-09 13:00 PM, Brian Granger wrote: >> >> > * To be in IPython.extensions or IPython.config.profile a module has to >> > satisfy a few things: >> > >> > - There has to be willingness amongst the core IPython developers to >> > maintain the code. >> > - The code needs to be reviewed (tests, docs, etc.) >> > - If the code should be in IPython.core, it shouldn't be in extensions. >> > - If an extension/profile can be distributed as a third party package, >> > it should be. ?Thus, >> > the custom completer for enthough.traits should ship with >> > enthought.traits. ?Things related >> > to numpy should ship with numpy. >> > > This is an important point and I want to understand the issues and fix > things now... > >> >> There is one issue with this. You want to be able to configure some things >> for a >> particular package in your configuration files at startup, but not >> actually >> import the package, which may be expensive. > > With the changes I have in my local branch this is possible.? You simply > enter the config > information in the config files, but don't list the extensions in the config > field of the extensions that should be loaded at startup.? Then you have a > number of > choices for loading the extension: > > get_ipython().load_extension('package.myextension') > > or > > %load_ext package.myextension That certainly helps. >> With care, many of the >> package-specific extensions can be written in such a way as to be enabled >> at >> startup but not import the package until it absolutely must (in which case >> the >> user has probably already imported it himself). For example, I submitted a >> patch >> to ipy_traits_completer that replace isinstance(obj, HasTraits) checks for >> one >> that looked for superclasses named "HasTraits" before import HasTraits and >> doing >> the real isinstance() check. > > The question of a lazy loading of an extension is a good point.? For now, > let's pretend > that an extension can be imported/loaded without any side effects (it is not > in a package > that has an __init__ that imports everything).? Then lazy importing is easy > right? > You can just make sure that the package (traits/numpy/etc) isn't imported > when the > extension is imported, but later, when the extension is first used. It depends strongly on what the extension does. If the extension adds some %magics, then yes; you just implement the %magic methods to do the imports inside instead of at the module level. However, for things like completers, you need an entirely different strategy. You can't do isinstance() checks. > But don't you think that if someone has explicitly enabled a numpy based > extension, > it is because they are using numpy (so importing it at startup would happen > anyways). > > If a user didn't want to *always* have the numpy extension loaded, they > could > simple maintain a numpy profile that loaded numpy. Not at all. I have many IPython sessions that go on for hundreds and hundreds of lines, and I change what I am doing frequently inside of them. I may start a session to do one thing and then switch to do something else. Or I realize that I need a capability after I've already started IPython. For example, without the new GUI hooks, I would frequently start up a plain IPython shell, do a bunch of computations, then curse myself when I suddenly realize that I really need to plot something. Ctrl-D, $ ipython -wthread, Up-Up-Up . Anything that lets me avoid having to know exactly what I am going to do when I start up the shell is really, really helpful. Profiles have always been basically useless to my personal workflow because of this. Having a quick way to enable an extension by name via %load_ext is really helpful, but for a lot of things, you just want them to be always available. If I need to do "%load_ext ipy_traits_completer" after every time I do "from enthought.traits.api import *", I'm just going to get annoyed by the needlessly repetitive typing. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Fri Oct 9 19:00:20 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 9 Oct 2009 16:00:20 -0700 Subject: [IPython-dev] How to handle extensions In-Reply-To: <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> Message-ID: <6ce0ac130910091600o175c7f2en5f2807b0ac3e8d6c@mail.gmail.com> Robert, Can you have a look at this branch and let me know how it looks: https://code.launchpad.net/~ipython-dev/ipython/new-extensions I have ported the pretty extension to the new api and moved it back into extensions. It is not quite done, but I have to run out for a while. I have changed the extension API a bit to make this easier. Let me know what you think. Cheers, Brian On Fri, Oct 9, 2009 at 1:56 PM, Brian Granger wrote: > > > On Fri, Oct 9, 2009 at 11:55 AM, Robert Kern wrote: > >> On 2009-10-09 13:00 PM, Brian Granger wrote: >> >> > * To be in IPython.extensions or IPython.config.profile a module has to >> > satisfy a few things: >> > >> > - There has to be willingness amongst the core IPython developers to >> > maintain the code. >> > - The code needs to be reviewed (tests, docs, etc.) >> > - If the code should be in IPython.core, it shouldn't be in extensions. >> > - If an extension/profile can be distributed as a third party package, >> > it should be. Thus, >> > the custom completer for enthough.traits should ship with >> > enthought.traits. Things related >> > to numpy should ship with numpy. >> >> > This is an important point and I want to understand the issues and fix > things now... > > >> There is one issue with this. You want to be able to configure some >> things for a >> particular package in your configuration files at startup, but not >> actually >> import the package, which may be expensive. > > > With the changes I have in my local branch this is possible. You simply > enter the config > information in the config files, but don't list the extensions in the > config > field of the extensions that should be loaded at startup. Then you have a > number of > choices for loading the extension: > > get_ipython().load_extension('package.myextension') > > or > > %load_ext package.myextension > > >> With care, many of the >> package-specific extensions can be written in such a way as to be enabled >> at >> startup but not import the package until it absolutely must (in which case >> the >> user has probably already imported it himself). For example, I submitted a >> patch >> to ipy_traits_completer that replace isinstance(obj, HasTraits) checks for >> one >> that looked for superclasses named "HasTraits" before import HasTraits and >> doing >> the real isinstance() check. >> >> > The question of a lazy loading of an extension is a good point. For now, > let's pretend > that an extension can be imported/loaded without any side effects (it is > not in a package > that has an __init__ that imports everything). Then lazy importing is easy > right? > You can just make sure that the package (traits/numpy/etc) isn't imported > when the > extension is imported, but later, when the extension is first used. > > But don't you think that if someone has explicitly enabled a numpy based > extension, > it is because they are using numpy (so importing it at startup would happen > anyways). > > If a user didn't want to *always* have the numpy extension loaded, they > could > simple maintain a numpy profile that loaded numpy. > > But if you really need lazy importing, I think it is possible still... > > >> Now, it wouldn't be too oppressive to put ipy_traits_completer into >> enthought.traits because we keep the __init__s clear (except for namespace >> package stuff, natch). But one couldn't put IPython support code into >> numpy >> without needing to import numpy itself to activate it in the >> configuration. >> >> > Yes, for packages that have lots of things in __init__, this is a problem. > > > For something as common as numpy, I think shipping it in IPython.extensions > may make sense. > > > >> It might be worth having an ipy_extensions package distributed separately >> from >> IPython that collects these extensions to reduce the burden on the IPython >> developers and the quality disparity with the rest of code. >> > > This is a good point. I have envisioned something like a meta-project on > launchpad, > similar to this for Twisted related projects: > > https://launchpad.net/tx > > That way the development of all the extensions/profiles can proceed > independently. > I could even imagine that other things might eventually be moved outside > the core > to a place like this (GUI frontends, etc.) > > Cheers, > > Brian > > >> >> -- >> Robert Kern >> >> "I have come to believe that the whole world is an enigma, a harmless >> enigma >> that is made terrible by our own mad attempt to interpret it as though it >> had >> an underlying truth." >> -- Umberto Eco >> >> _______________________________________________ >> IPython-dev mailing list >> IPython-dev at scipy.org >> http://mail.scipy.org/mailman/listinfo/ipython-dev >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Mon Oct 12 17:05:30 2009 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Mon, 12 Oct 2009 14:05:30 -0700 Subject: [IPython-dev] debug magic broken in dev trunk? Message-ID: I've recently moved from .10 to the dev branch in hopes of testing this new GUI system that's been mentioned here a few times, and I've got most things working great. One serious hitch, though - the 'debug' magic doesn't seem to be working - when it should bring me to the debugger prompt, simply nothing happens (see below for an example session). Is this a serious bug, or a temporary/nightly problem? From hans_meine at gmx.net Tue Oct 13 04:14:19 2009 From: hans_meine at gmx.net (Hans Meine) Date: Tue, 13 Oct 2009 10:14:19 +0200 Subject: [IPython-dev] How to handle extensions In-Reply-To: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> Message-ID: <200910131014.21553.hans_meine@gmx.net> On Friday 09 October 2009 20:00:59 Brian Granger wrote: > - If an extension/profile can be distributed as a third party package, it > should be. Thus, > the custom completer for enthough.traits should ship with enthought.traits. > Things related > to numpy should ship with numpy. Although this sounds good, one needs to make sure that this does not create installation problems. E.g. if the ipython extension is only installed if ipython is found, the installation order may suddenly become relevant. Or packagers may start to make the extension optional (e.g. with Gentoo USE flags, or by putting them into an additional package), which would make things less convenient for the user IMO. Finally, I would expect "import numpy" to activate all relevant extensions in ipython (otherwise, where's the convenience?), but then we need to put code into 3rd party packages to check for the IPython runtime - would that be good design? Nice greetings, Hans From hans_meine at gmx.net Tue Oct 13 04:20:08 2009 From: hans_meine at gmx.net (Hans Meine) Date: Tue, 13 Oct 2009 10:20:08 +0200 Subject: [IPython-dev] How to handle extensions In-Reply-To: <3d375d730910091507v760a7aa6y357eb6a2b025e069@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <6ce0ac130910091356q42f3dd6j6100551cbde2a2d8@mail.gmail.com> <3d375d730910091507v760a7aa6y357eb6a2b025e069@mail.gmail.com> Message-ID: <200910131020.13132.hans_meine@gmx.net> On Saturday 10 October 2009 00:07:05 Robert Kern wrote: > . Anything that lets me avoid having to know > exactly what I am going to do when I start up the shell is really, > really helpful. Profiles have always been basically useless to my > personal workflow because of this. My main use of profiles is to have a shortcut to predefine a whole set of imports. This is very useful if your software is using IPython as the main controlling/debugging/experimentation frontend, like mine, although it has a more or less complete (but always slightly restrictive) PyQt GUI for the most common interactive purposes. Using a profile, I can simply type "geomap", which is a shell alias for "ipython -qthread -p geomap" (yes, the -qthread would not be necessary with the new design anymore), to start my software. (I thought this could be interesting information for Brian et al., when thinking about design issues.) Have a nice day, Hans From erik.tollerud at gmail.com Tue Oct 13 04:38:05 2009 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Tue, 13 Oct 2009 01:38:05 -0700 Subject: [IPython-dev] debug magic broken in dev trunk? In-Reply-To: References: Message-ID: Whoops! Forgot to include the example session: In [2]: debug ERROR: No traceback has been produced, nothing to debug. In [3]: raise Exception --------------------------------------------------------------------------- Exception Traceback (most recent call last) /home/erik/ in () Exception: In [4]: debug In [5]: On Mon, Oct 12, 2009 at 2:05 PM, Erik Tollerud wrote: > I've recently moved from .10 to the dev branch in hopes of testing > this new GUI system that's been mentioned here a few times, and I've > got most things working great. ?One serious hitch, though - the > 'debug' magic doesn't seem to be working - when it should bring me to > the debugger prompt, simply nothing happens (see below for an example > session). ?Is this a serious bug, or a temporary/nightly problem? > -- Erik Tollerud Graduate Student Center For Cosmology Department of Physics and Astronomy 2142 Frederick Reines Hall University of California, Irvine Office Phone: (949)824-2587 Cell: (651)307-9409 etolleru at uci.edu http://ps.uci.edu/~etolleru From robert.kern at gmail.com Tue Oct 13 11:26:53 2009 From: robert.kern at gmail.com (Robert Kern) Date: Tue, 13 Oct 2009 10:26:53 -0500 Subject: [IPython-dev] How to handle extensions In-Reply-To: <200910131014.21553.hans_meine@gmx.net> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> Message-ID: On 2009-10-13 03:14 AM, Hans Meine wrote: > On Friday 09 October 2009 20:00:59 Brian Granger wrote: >> - If an extension/profile can be distributed as a third party package, it >> should be. Thus, >> the custom completer for enthough.traits should ship with enthought.traits. >> Things related >> to numpy should ship with numpy. > > Although this sounds good, one needs to make sure that this does not create > installation problems. E.g. if the ipython extension is only installed if > ipython is found, the installation order may suddenly become relevant. > Or packagers may start to make the extension optional (e.g. with Gentoo USE > flags, or by putting them into an additional package), which would make things > less convenient for the user IMO. Packagers simply shouldn't do that. Optional components == bad news. > Finally, I would expect "import numpy" to activate all relevant extensions in > ipython (otherwise, where's the convenience?), but then we need to put code > into 3rd party packages to check for the IPython runtime - would that be good > design? I would really hope that packages wouldn't do that. Even if I have the package, I may not want their extensions. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ondrej at certik.cz Wed Oct 14 14:26:02 2009 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 14 Oct 2009 11:26:02 -0700 Subject: [IPython-dev] IPython.ipapi stopped working Message-ID: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> Hi, with the latest ipython bzr trunk: $ bin/doctest ============================= test process starts ============================== executable: /usr/bin/python (2.6.2-final-0) Traceback (most recent call last): File "bin/doctest", line 30, in "blacklist": blacklist}) File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 140, in doctest dtest = t.test() File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in test self.test_file(f) File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 382, in test_file setup_pprint() File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 48, in setup_pprint init_printing(sstrrepr) File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line 14, in init_printing ip = IPython.ipapi.get() AttributeError: 'module' object has no attribute 'ipapi' That happens because there is no "ipapi" in IPython: In [1]: import IPython.ipapi ------------------------------------------------------------ Traceback (most recent call last): File "", line 1, in ImportError: No module named ipapi A quick experimentation shows that this could work: In [1]: import IPython.core.ipapi In [2]: So applying this patch: $ git diff diff --git a/sympy/interactive/__init__.py b/sympy/interactive/__init__.py index 5e253b8..d0cc9fe 100644 --- a/sympy/interactive/__init__.py +++ b/sympy/interactive/__init__.py @@ -11,7 +11,7 @@ def init_printing(stringify_func): try: import IPython - ip = IPython.ipapi.get() + ip = IPython.core.ipapi.get() if ip is not None: def result_display(self, arg): moves us a bit further, but not enough: $ bin/doctest ============================= test process starts ============================== executable: /usr/bin/python (2.6.2-final-0) Traceback (most recent call last): File "bin/doctest", line 30, in "blacklist": blacklist}) File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 140, in doctest dtest = t.test() File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in test self.test_file(f) File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 382, in test_file setup_pprint() File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 48, in setup_pprint init_printing(sstrrepr) File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line 14, in init_printing ip = IPython.core.ipapi.get() File "/home/ondrej/usr/lib/python/IPython/core/ipapi.py", line 31, in get most_recent = insts[0] IndexError: list index out of range Any ideas how to fix this? I can't run doctests anymore due to this bug, unless I install old ipython again... Ondrej From ondrej at certik.cz Wed Oct 14 14:27:42 2009 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 14 Oct 2009 11:27:42 -0700 Subject: [IPython-dev] IPython.ipapi stopped working In-Reply-To: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> References: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> Message-ID: <85b5c3130910141127j52982999i25fcd186225e64f7@mail.gmail.com> On Wed, Oct 14, 2009 at 11:26 AM, Ondrej Certik wrote: > Hi, > > with the latest ipython bzr trunk: > > $ bin/doctest > ============================= test process starts ============================== > executable: ? /usr/bin/python ?(2.6.2-final-0) > > Traceback (most recent call last): > ?File "bin/doctest", line 30, in > ? ?"blacklist": blacklist}) > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 140, in doctest > ? ?dtest = t.test() > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in test > ? ?self.test_file(f) > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 382, in test_file > ? ?setup_pprint() > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 48, in setup_pprint > ? ?init_printing(sstrrepr) > ?File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line > 14, in init_printing > ? ?ip = IPython.ipapi.get() > AttributeError: 'module' object has no attribute 'ipapi' > > > That happens because there is no "ipapi" in IPython: > > In [1]: import IPython.ipapi > ------------------------------------------------------------ > Traceback (most recent call last): > ?File "", line 1, in > ImportError: No module named ipapi > > > A quick experimentation shows that this could work: > > In [1]: import IPython.core.ipapi > > In [2]: > > So applying this patch: > > $ git diff > diff --git a/sympy/interactive/__init__.py b/sympy/interactive/__init__.py > index 5e253b8..d0cc9fe 100644 > --- a/sympy/interactive/__init__.py > +++ b/sympy/interactive/__init__.py > @@ -11,7 +11,7 @@ def init_printing(stringify_func): > ? ? try: > ? ? ? ? import IPython > > - ? ? ? ?ip = IPython.ipapi.get() > + ? ? ? ?ip = IPython.core.ipapi.get() > > ? ? ? ? if ip is not None: > ? ? ? ? ? ? def result_display(self, arg): > > > moves us a bit further, but not enough: > > > $ bin/doctest > ============================= test process starts ============================== > executable: ? /usr/bin/python ?(2.6.2-final-0) > > Traceback (most recent call last): > ?File "bin/doctest", line 30, in > ? ?"blacklist": blacklist}) > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 140, in doctest > ? ?dtest = t.test() > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in test > ? ?self.test_file(f) > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 382, in test_file > ? ?setup_pprint() > ?File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 48, in setup_pprint > ? ?init_printing(sstrrepr) > ?File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line > 14, in init_printing > ? ?ip = IPython.core.ipapi.get() > ?File "/home/ondrej/usr/lib/python/IPython/core/ipapi.py", line 31, in get > ? ?most_recent = insts[0] > IndexError: list index out of range > > > > > Any ideas how to fix this? I can't run doctests anymore due to this > bug, unless I install old ipython again... And I also can't run isympy: $ bin/isympy Traceback (most recent call last): File "bin/isympy", line 153, in main() File "bin/isympy", line 147, in main init_session('ipython', **args) File "sympy/interactive/__init__.py", line 98, in init_session ip = IPython.ipapi.get() AttributeError: 'module' object has no attribute 'ipapi' so this is *extremely* annoying. We might have used the IPython api's improperly though. Ondrej From robert.kern at gmail.com Wed Oct 14 14:45:50 2009 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 14 Oct 2009 13:45:50 -0500 Subject: [IPython-dev] [sympy] IPython.ipapi stopped working In-Reply-To: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> References: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> Message-ID: <3d375d730910141145udb4a246o22fe1391c0622b54@mail.gmail.com> On Wed, Oct 14, 2009 at 13:26, Ondrej Certik wrote: > Any ideas how to fix this? I can't run doctests anymore due to this > bug, unless I install old ipython again... IPython's trunk is undergoing a major reorganization. Go back to the last release. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Wed Oct 14 15:06:27 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 14 Oct 2009 12:06:27 -0700 Subject: [IPython-dev] IPython.ipapi stopped working In-Reply-To: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> References: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> Message-ID: <6ce0ac130910141206q46f1ddb5ld9fef904bbed539d@mail.gmail.com> Ondrej, Yes, as Robert said, trunk is in a semi-unstable state right now. Most things do work, but the APIs have changed. I can help you update the sympy things to the new APIs, but it is probably a little early. Other than isympy, in what other ways does sympy use IPython? I can help you update the things in sympy. I have already created a sympy profile for ipython that does what isympy does: http://bazaar.launchpad.net/~ipython-dev/ipython/trunk/annotate/head%3A/IPython/config/profile/ipython_config_sympy.py Also, I have implemented the top-level embed function you requested. If you don't mind going back to the latest release, I will try to get sympy working with the new stuff. Cheers, Brian On Wed, Oct 14, 2009 at 11:26 AM, Ondrej Certik wrote: > Hi, > > with the latest ipython bzr trunk: > > $ bin/doctest > ============================= test process starts > ============================== > executable: /usr/bin/python (2.6.2-final-0) > > Traceback (most recent call last): > File "bin/doctest", line 30, in > "blacklist": blacklist}) > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 140, in doctest > dtest = t.test() > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in > test > self.test_file(f) > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 382, in test_file > setup_pprint() > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 48, in setup_pprint > init_printing(sstrrepr) > File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line > 14, in init_printing > ip = IPython.ipapi.get() > AttributeError: 'module' object has no attribute 'ipapi' > > > That happens because there is no "ipapi" in IPython: > > In [1]: import IPython.ipapi > ------------------------------------------------------------ > Traceback (most recent call last): > File "", line 1, in > ImportError: No module named ipapi > > > A quick experimentation shows that this could work: > > In [1]: import IPython.core.ipapi > > In [2]: > > So applying this patch: > > $ git diff > diff --git a/sympy/interactive/__init__.py b/sympy/interactive/__init__.py > index 5e253b8..d0cc9fe 100644 > --- a/sympy/interactive/__init__.py > +++ b/sympy/interactive/__init__.py > @@ -11,7 +11,7 @@ def init_printing(stringify_func): > try: > import IPython > > - ip = IPython.ipapi.get() > + ip = IPython.core.ipapi.get() > > if ip is not None: > def result_display(self, arg): > > > moves us a bit further, but not enough: > > > $ bin/doctest > ============================= test process starts > ============================== > executable: /usr/bin/python (2.6.2-final-0) > > Traceback (most recent call last): > File "bin/doctest", line 30, in > "blacklist": blacklist}) > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 140, in doctest > dtest = t.test() > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line 368, in > test > self.test_file(f) > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 382, in test_file > setup_pprint() > File "/home/ondrej/repos/sympy/sympy/utilities/runtests.py", line > 48, in setup_pprint > init_printing(sstrrepr) > File "/home/ondrej/repos/sympy/sympy/interactive/__init__.py", line > 14, in init_printing > ip = IPython.core.ipapi.get() > File "/home/ondrej/usr/lib/python/IPython/core/ipapi.py", line 31, in get > most_recent = insts[0] > IndexError: list index out of range > > > > > Any ideas how to fix this? I can't run doctests anymore due to this > bug, unless I install old ipython again... > > Ondrej > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondrej at certik.cz Wed Oct 14 15:13:43 2009 From: ondrej at certik.cz (Ondrej Certik) Date: Wed, 14 Oct 2009 12:13:43 -0700 Subject: [IPython-dev] [sympy] Re: IPython.ipapi stopped working In-Reply-To: <6ce0ac130910141206q46f1ddb5ld9fef904bbed539d@mail.gmail.com> References: <85b5c3130910141126p45ff3270kab867c7ddc2763a8@mail.gmail.com> <6ce0ac130910141206q46f1ddb5ld9fef904bbed539d@mail.gmail.com> Message-ID: <85b5c3130910141213j364585d5seeea13f38bf9aec0@mail.gmail.com> On Wed, Oct 14, 2009 at 12:06 PM, Brian Granger wrote: > Ondrej, > > Yes, as Robert said, trunk is in a semi-unstable state right now.? Most > things do work, but > the APIs have changed.? I can help you update the sympy things to the new > APIs, but it is > probably a little early. > > Other than isympy, in what other ways does sympy use IPython? > > I can help you update the things in sympy. > > I have already created a sympy profile for ipython that does what isympy > does: > > http://bazaar.launchpad.net/~ipython-dev/ipython/trunk/annotate/head%3A/IPython/config/profile/ipython_config_sympy.py > > Also, I have implemented the top-level embed function you requested.? If you > don't mind going > back to the latest release, I will try to get sympy working with the new > stuff. Excellent, thanks so much for the help. I will use the latest release till then. The only reason I use the bzr trunk is because of the IPython.embed() feature. I think we should use the IPython's api in the correct way -- e.g. I think just using the profile as you did. The only way we use IPython is by trying to import it in sympy/interactive/ and set the pretty printing in it. And then in isympy. Ondrej From ellisonbg.net at gmail.com Wed Oct 14 19:20:17 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 14 Oct 2009 16:20:17 -0700 Subject: [IPython-dev] How to handle extensions In-Reply-To: References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> Message-ID: <6ce0ac130910141620l5e53ef1cs3fe93d89537224cc@mail.gmail.com> Robert, I finished up the pretty extension. Everything seems to work fine. Do you want to try this branch out before I merge it into trunk? Thanks, Brian On Tue, Oct 13, 2009 at 8:26 AM, Robert Kern wrote: > On 2009-10-13 03:14 AM, Hans Meine wrote: > > On Friday 09 October 2009 20:00:59 Brian Granger wrote: > >> - If an extension/profile can be distributed as a third party package, > it > >> should be. Thus, > >> the custom completer for enthough.traits should ship with > enthought.traits. > >> Things related > >> to numpy should ship with numpy. > > > > Although this sounds good, one needs to make sure that this does not > create > > installation problems. E.g. if the ipython extension is only installed > if > > ipython is found, the installation order may suddenly become relevant. > > Or packagers may start to make the extension optional (e.g. with Gentoo > USE > > flags, or by putting them into an additional package), which would make > things > > less convenient for the user IMO. > > Packagers simply shouldn't do that. Optional components == bad news. > > > Finally, I would expect "import numpy" to activate all relevant > extensions in > > ipython (otherwise, where's the convenience?), but then we need to put > code > > into 3rd party packages to check for the IPython runtime - would that be > good > > design? > > I would really hope that packages wouldn't do that. Even if I have the > package, > I may not want their extensions. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From dwf at cs.toronto.edu Thu Oct 15 10:19:42 2009 From: dwf at cs.toronto.edu (David Warde-Farley) Date: Thu, 15 Oct 2009 10:19:42 -0400 Subject: [IPython-dev] How to handle extensions In-Reply-To: References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> Message-ID: <9707FDC9-00CF-429B-A040-0FDFDC0F3AB4@cs.toronto.edu> (originally sent to Robert, argh! was expecting munged reply-to header) On 13-Oct-09, at 11:26 AM, Robert Kern wrote: > > I would really hope that packages wouldn't do that. Even if I have > the package, > I may not want their extensions. +1. Two things: a) I might not have any use for their package interactively, in which case it has the potential to slow down IPython for me if it's always loading them (less of a problem if they're only loaded on import from the interactive prompt). b) Even if I do use their package interactively, their extensions may not fit my brain. David From ellisonbg.net at gmail.com Thu Oct 15 12:51:55 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 15 Oct 2009 09:51:55 -0700 Subject: [IPython-dev] How to handle extensions In-Reply-To: <9707FDC9-00CF-429B-A040-0FDFDC0F3AB4@cs.toronto.edu> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> <9707FDC9-00CF-429B-A040-0FDFDC0F3AB4@cs.toronto.edu> Message-ID: <6ce0ac130910150951s635ecaf7xf24c0a220faeb1e9@mail.gmail.com> > > I would really hope that packages wouldn't do that. Even if I have > > the package, > > I may not want their extensions. > > +1. Two things: > > a) I might not have any use for their package interactively, in which > case it has the potential to slow down IPython for me if it's always > loading them (less of a problem if they're only loaded on import from > the interactive prompt). > b) Even if I do use their package interactively, their extensions may > not fit my brain. > > Yes, the the loading of extensions should always be in the control of the user. Another way of saying this is that an IPython extension should be loaded only if one of the following happens: * It is listed in the extensions list of the config file. * The user calls %load_ext to load it. The only case that is different is in the case of something like isympy, which is a custom version of ipython that loads various things needed by sympy. But in that case, it is a *different program* that is being run. Cheers, Brian > David > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From robert.kern at gmail.com Thu Oct 15 15:58:43 2009 From: robert.kern at gmail.com (Robert Kern) Date: Thu, 15 Oct 2009 14:58:43 -0500 Subject: [IPython-dev] How to handle extensions In-Reply-To: <6ce0ac130910150951s635ecaf7xf24c0a220faeb1e9@mail.gmail.com> References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> <9707FDC9-00CF-429B-A040-0FDFDC0F3AB4@cs.toronto.edu> <6ce0ac130910150951s635ecaf7xf24c0a220faeb1e9@mail.gmail.com> Message-ID: On 2009-10-15 11:51 AM, Brian Granger wrote: > > > I would really hope that packages wouldn't do that. Even if I have > > the package, > > I may not want their extensions. > > +1. Two things: > > a) I might not have any use for their package interactively, in which > case it has the potential to slow down IPython for me if it's always > loading them (less of a problem if they're only loaded on import from > the interactive prompt). > b) Even if I do use their package interactively, their extensions may > not fit my brain. > > > Yes, the the loading of extensions should always be in the control of > the user. > Another way of saying this is that an IPython extension should be loaded > only > if one of the following happens: > > * It is listed in the extensions list of the config file. > * The user calls %load_ext to load it. > > The only case that is different is in the case of something like isympy, > which is a custom version of ipython that loads various things needed > by sympy. But in that case, it is a *different program* that is being run. * Or it's activated by a profile. I guess that amounts to calling %load_ext, but I think it's worth breaking out as its own item. -- Robert Kern "I have come to believe that the whole world is an enigma, a harmless enigma that is made terrible by our own mad attempt to interpret it as though it had an underlying truth." -- Umberto Eco From ellisonbg.net at gmail.com Thu Oct 15 19:05:42 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Thu, 15 Oct 2009 16:05:42 -0700 Subject: [IPython-dev] How to handle extensions In-Reply-To: References: <6ce0ac130910091100s6e4fc414q9c1aa8eae8160567@mail.gmail.com> <200910131014.21553.hans_meine@gmx.net> <9707FDC9-00CF-429B-A040-0FDFDC0F3AB4@cs.toronto.edu> <6ce0ac130910150951s635ecaf7xf24c0a220faeb1e9@mail.gmail.com> Message-ID: <6ce0ac130910151605i68befc9cy53046ed89bd976f9@mail.gmail.com> > > * It is listed in the extensions list of the config file. > > * The user calls %load_ext to load it. > > > > The only case that is different is in the case of something like isympy, > > which is a custom version of ipython that loads various things needed > > by sympy. But in that case, it is a *different program* that is being > run. > > * Or it's activated by a profile. > > With the new config system, profile==config file is True, so this is covered. All you have to do is list the extension in c.Global.extensions in ipython_config.py or your profile and you are set. Cheers, Brian > I guess that amounts to calling %load_ext, but I think it's worth breaking > out > as its own item. > > -- > Robert Kern > > "I have come to believe that the whole world is an enigma, a harmless > enigma > that is made terrible by our own mad attempt to interpret it as though it > had > an underlying truth." > -- Umberto Eco > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg.net at gmail.com Sat Oct 17 14:18:32 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 17 Oct 2009 11:18:32 -0700 Subject: [IPython-dev] Merging only 1 commit from branch to trunk Message-ID: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> Hi, I have a branch that I am doing a lot of work on. I just fixed the %debug magic in that branch and would like to merge that to trunk. But the rest of my branch is not ready for merging. Are there any bzr experts here who know the best way of doing this? Thanks Brian -------------- next part -------------- An HTML attachment was scrubbed... URL: From fperez.net at gmail.com Sat Oct 17 14:40:09 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sat, 17 Oct 2009 11:40:09 -0700 Subject: [IPython-dev] Merging only 1 commit from branch to trunk In-Reply-To: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> References: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> Message-ID: Sorry, just heading out for a conference, so very short... On Sat, Oct 17, 2009 at 11:18 AM, Brian Granger wrote: > I have a branch that I am doing a lot of work on.? I just fixed the %debug > magic in that branch and > would like to merge that to trunk.? But the rest of my branch is not ready > for merging. > > Are there any bzr experts here who know the best way of doing this? > I don't think it's possible to do it cleanly. If I recall correctly (this is going from memory) if you do that, you lose the automatic revision merge tracking. You'll still be able to merge from that branch, but afterwards you'll always have to do all merges with manual revision ids. There may be a trick to do this without creating manual (and error-prone) work afterwards, but I don't know of it. Cheers, f From vivainio at gmail.com Sun Oct 18 04:00:46 2009 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 18 Oct 2009 11:00:46 +0300 Subject: [IPython-dev] Merging only 1 commit from branch to trunk In-Reply-To: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> References: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> Message-ID: <46cb515a0910180100w82c236bk68b900152b488830@mail.gmail.com> On Sat, Oct 17, 2009 at 9:18 PM, Brian Granger wrote: > I have a branch that I am doing a lot of work on.? I just fixed the %debug > magic in that branch and > would like to merge that to trunk.? But the rest of my branch is not ready > for merging. > > Are there any bzr experts here who know the best way of doing this? http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#cherrypicking However, as a general DVCS rule, bugfixes should usually be done first at the oldest branch that will receive the bufig - often this is the "old release maintenance" branch. Then, it can be merged to trunk cleanly (as old branches should always be in a state where they have been merged to trunk), -- Ville M. Vainio http://tinyurl.com/vainio From fperez.net at gmail.com Sun Oct 18 19:11:01 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 18 Oct 2009 16:11:01 -0700 Subject: [IPython-dev] Merging only 1 commit from branch to trunk In-Reply-To: <46cb515a0910180100w82c236bk68b900152b488830@mail.gmail.com> References: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> <46cb515a0910180100w82c236bk68b900152b488830@mail.gmail.com> Message-ID: 2009/10/18 Ville M. Vainio : > http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#cherrypicking I think this is what I remembered, and unless I'm misunderstanding something, it has the problem I alluded to, as indicated by this sentence: """ Unlike a normal merge, Bazaar does not currently track cherrypicks. In particular, the changes look like a normal commit and the (internal) revision history of the changes from the other branch is lost. In many cases where they are useful (see above), this is not a major problem because there are good reasons why a full merge should never be done at a later time. In other cases, additional conflicts will need to be resolved when the changes are merged again. """ This means to me that once you've cherrypicked from branch A into branch B, then you'll never be able to do a merge(A->B), because the cherrypicked commits are likely to cause conflicts. That's precisely the problem I alluded to, am I misunderstanding something? Cheers, f From vivainio at gmail.com Mon Oct 19 01:55:36 2009 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 19 Oct 2009 08:55:36 +0300 Subject: [IPython-dev] Merging only 1 commit from branch to trunk In-Reply-To: References: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> <46cb515a0910180100w82c236bk68b900152b488830@mail.gmail.com> Message-ID: <46cb515a0910182255i519358cfhda609be42a91ddf0@mail.gmail.com> On Mon, Oct 19, 2009 at 2:11 AM, Fernando Perez wrote: > 2009/10/18 Ville M. Vainio : >> http://doc.bazaar-vcs.org/bzr.dev/en/user-guide/index.html#cherrypicking > This means to me that once you've cherrypicked from branch A into > branch B, then you'll never be able to do a merge(A->B), because the > cherrypicked commits are likely to cause conflicts. > > That's precisely the problem I alluded to, am I misunderstanding something? You are perfectly able to do a merge - you'll just have to resolve the conflict where it happens. After that is done, the branches are fully mergeable again (and the rest of the changes are merged normally). -- Ville M. Vainio http://tinyurl.com/vainio From fperez.net at gmail.com Mon Oct 19 02:04:37 2009 From: fperez.net at gmail.com (Fernando Perez) Date: Sun, 18 Oct 2009 23:04:37 -0700 Subject: [IPython-dev] Merging only 1 commit from branch to trunk In-Reply-To: <46cb515a0910182255i519358cfhda609be42a91ddf0@mail.gmail.com> References: <6ce0ac130910171118w540f97ecx93ae9b699e477e19@mail.gmail.com> <46cb515a0910180100w82c236bk68b900152b488830@mail.gmail.com> <46cb515a0910182255i519358cfhda609be42a91ddf0@mail.gmail.com> Message-ID: 2009/10/18 Ville M. Vainio : > > You are perfectly able to do a merge - you'll just have to resolve the > conflict where it happens. After that is done, the branches are fully > mergeable again (and the rest of the changes are merged normally). Ah, OK. I thought it was a situation similar to that with svn (in the old days, I think svn-merge is better now) where you'd have to track merged revisions by hand separately, and do all subsequent merges with explicit, manual revision markers. This isn't ideal if the cherrypicked commit is large (the conflict resolution could be a fair bit of work) but it's not the end of the world either, and at least it's a one-time deal. Cheers, f From dsdale24 at gmail.com Sun Oct 25 09:33:14 2009 From: dsdale24 at gmail.com (Darren Dale) Date: Sun, 25 Oct 2009 09:33:14 -0400 Subject: [IPython-dev] please expose ipython_console_highlighting for third parties Message-ID: Would it be possible to expose the ipython_console_highlighting sphinx extension in the ipython package, so third parties could use it without having to include it in their own projects? From jorgen.stenarson at bostream.nu Wed Oct 28 15:49:44 2009 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Wed, 28 Oct 2009 20:49:44 +0100 Subject: [IPython-dev] iptest is crashing on trunk Message-ID: <4AE8A058.6070907@bostream.nu> Hi, I have just installed python2.6 and ipython from trunk on windows xp. Is the problem below a known issue? /J?rgen C:\python\external\ipython-trunk\IPython>python scripts\iptest Traceback (most recent call last): File "scripts\iptest", line 8, in iptest.main() File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", line 315, in main run_iptestall() File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", line 278, in run_iptestall runners = make_runners() File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", line 264, in make_runners runners.update(trial_runners) UnboundLocalError: local variable 'trial_runners' referenced before assignment From ellisonbg.net at gmail.com Wed Oct 28 17:52:35 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 28 Oct 2009 14:52:35 -0700 Subject: [IPython-dev] iptest is crashing on trunk In-Reply-To: <4AE8A058.6070907@bostream.nu> References: <4AE8A058.6070907@bostream.nu> Message-ID: <6ce0ac130910281452o5d6d3df7hb18d9af8803532bd@mail.gmail.com> The test suite is currently more or less completely broken. I am currently doing some work for MSFT on IPython.kernel so that part of the test suite is working. Fernando has written some very nice testing code that will really simplify all our testing stuff...but we haven't integrated it yet. This will definitely have to happen before the next release, but I don't have any time to spend on this right now. For now, here is my best advice. If you are writing new code, use unittest and run your tests using nosetests on the specific file. Cheers, Brian On Wed, Oct 28, 2009 at 12:49 PM, J?rgen Stenarson < jorgen.stenarson at bostream.nu> wrote: > Hi, > > I have just installed python2.6 and ipython from trunk on windows xp. > Is the problem below a known issue? > > /J?rgen > > C:\python\external\ipython-trunk\IPython>python scripts\iptest > Traceback (most recent call last): > File "scripts\iptest", line 8, in > iptest.main() > File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", > line 315, in main > run_iptestall() > File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", > line 278, in run_iptestall > runners = make_runners() > File "c:\python\external\ipython-trunk\IPython\testing\iptest.py", > line 264, in make_runners > runners.update(trial_runners) > UnboundLocalError: local variable 'trial_runners' referenced before > assignment > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://mail.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From erik.tollerud at gmail.com Fri Oct 30 16:09:02 2009 From: erik.tollerud at gmail.com (Erik Tollerud) Date: Fri, 30 Oct 2009 13:09:02 -0700 Subject: [IPython-dev] numpy dreload problems specific? Message-ID: Is everyone encountering https://bugs.launchpad.net/ipython/+bug/306805 on ipython 0.10, or is this something that's particular to certain installations? I see it (as I reported in a comment to the bug), but I'm wondering if this is low priority because its somehow site-specific... From ellisonbg.net at gmail.com Fri Oct 30 18:46:52 2009 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 30 Oct 2009 15:46:52 -0700 Subject: [IPython-dev] Before a patch for LSF support In-Reply-To: References: <6ce0ac130908120258y66547ee5x77b4453beae5fecc@mail.gmail.com> <6ce0ac130908120917m6dd5a68fq67bd22fade7ce47@mail.gmail.com> <6ce0ac130908130110y7402edfbs18480f22c1fd4092@mail.gmail.com> <6ce0ac130909080949wd076e5fh946febd26729ffc7@mail.gmail.com> Message-ID: <6ce0ac130910301546q780cc67dv52e46cf12515019f@mail.gmail.com> Matthieu, I have spent the last month completely refactoring ipcluster. This work should make it easier to add support for LSF. This work should be merged into trunk in the next few weeks. But, I think you are facing some unique issues described below... I'm trying to get ipython working with LSF. I did manage to submit the > job and let it run on the nodes. > The first issue I face is that $HOME is not the same location on the > LSF nodes than on the main computer. This is an issue that should be > tackled with the refactoring of ipython (adding the possibility of > setting the path for the controller and a different pass for the > engines, if I understood ipython correctly). > When is this an issue? When finding the .ipython directory? When importing ipython (PYTHONPATH)? I think this one can be solved without making any changes to IPython, but if we need to we can. > Meanwhile, I've tried to launch ipcluster from an LSF node (I have an > SSH access too for debugging purposes). So in this case, I don't have > a $HOME problem. This time, it's the ipcontroller-mec.furl file that > has a 127.0.0.1 as the controller IP. I've tried to replace it with > the node name, but I also a Conenction Refused error in the engines > logs when I try to create a MultiEngineClient. > Ahh, yes, this type of thing can be subtle to get working - especially on clusters with multiple tcp/ip interfaces. With my refactored work, these things can easily be configured. There are two things to look at: --engine-ip = set's the ip address (or hostname) that the controller will listen on. Default is probably fine. --engine-location = set's the ip address (or hostname) that will appear in the furl. This is the ip address that the engine will try to connect on. I would try changing this one. > I hope I summed up my issue properly. > Yes, with all the changes going on in trunk, this might take a while to get figured out, but I think it is doable. Cheers, Brian > Cheers, > > Matthieu > -- > Information System Engineer, Ph.D. > Website: http://matthieu-brucher.developpez.com/ > Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 > LinkedIn: http://www.linkedin.com/in/matthieubrucher > -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthieu.brucher at gmail.com Fri Oct 30 19:11:30 2009 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sat, 31 Oct 2009 00:11:30 +0100 Subject: [IPython-dev] Before a patch for LSF support In-Reply-To: <6ce0ac130910301546q780cc67dv52e46cf12515019f@mail.gmail.com> References: <6ce0ac130908120917m6dd5a68fq67bd22fade7ce47@mail.gmail.com> <6ce0ac130908130110y7402edfbs18480f22c1fd4092@mail.gmail.com> <6ce0ac130909080949wd076e5fh946febd26729ffc7@mail.gmail.com> <6ce0ac130910301546q780cc67dv52e46cf12515019f@mail.gmail.com> Message-ID: Hi Brian! 2009/10/30 Brian Granger : > Matthieu, > > I have spent the last month completely refactoring ipcluster.? This work > should make it easier to add support for LSF.? This work should be merged > into trunk in the next few weeks. Excellent piece of news! I hope I'll be able to have a proper Internet access then (I've moved ino another flat, but my Internet provider is kind of sloppy...) to update my modifications and send the appropriate patch. But, I think you are facing some unique > issues described below... Indeed, it's mainly because I want to test my futur patch properly :| > When is this an issue?? When finding the .ipython directory?? When importing > ipython (PYTHONPATH)?? I think this one can be solved without making any > changes > to IPython, but if we need to we can. When finding the .ipython folder for the furl file. It's not in the $HOME folder of the nodes, but in the $HOME folder of the controller. Fo the moment, I copy the furl file on the fly, but this isn't a viable approach :| > Ahh, yes, this type of thing can be subtle to get working - especially on > clusters with multiple > tcp/ip interfaces.? With my refactored work, these things can easily be > configured.? There are two things to look at: > > --engine-ip = set's the ip address (or hostname) that the controller will > listen on.? Default is probably fine. > --engine-location = set's the ip address (or hostname) that will appear in > the furl.? This is the ip address that > the engine will try to connect on.? I would try changing this one. OK, I'll test this. >> I hope I summed up my issue properly. > > Yes, with all the changes going on in trunk, this might take a while to get > figured out, but I think it is doable. I hope I'll be able to find the time to figure it out (and the Internet access as well :|) Cheers, Matthieu -- Information System Engineer, Ph.D. Website: http://matthieu-brucher.developpez.com/ Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92 LinkedIn: http://www.linkedin.com/in/matthieubrucher