From hans_meine at gmx.net Fri Feb 1 11:50:40 2008 From: hans_meine at gmx.net (Hans Meine) Date: Fri, 1 Feb 2008 17:50:40 +0100 Subject: [IPython-dev] Opening files via ipython cmdline args? Message-ID: <200802011750.40339.hans_meine@gmx.net> Hi! I have defined a multitude of ipython profiles for various purposes, e.g. one that loads fig.py module for DOM-like processing of XFig files (*). Then, the first command I usually issue is # ipython -p figpy In [1]: f = fig.File("contraction.fig") In [2]: _ I have defined "figpy" as a shell alias for "ipython -p figpy", but it would be even cooler if I could write # figpy contraction.fig and be greeted with an IPython env with "f" being already setup. AFAICS, direct loading of files from the commandline could be useful for many other cases, too (e.g. pytables, image analysis, ...). Is this possible today? I know I can access sys.argv from ipy_profile_....py, but how can I tell ipython to ignore the filename afterwards? See the attached try of removing the entry from sys.argv, but ipython seems to have copied the args beforehand: # ipython -p figpy contraction.fig ------------------------------------------------------------ File "contraction.fig", line 9 1200 2 ^ SyntaxError: invalid syntax WARNING: Failure executing file: In [1]: Ciao, / / /--/ / / ANS * http://kogs-www.informatik.uni-hamburg.de/~meine/software/figpy/ -------------- next part -------------- A non-text attachment was scrubbed... Name: ipy_profile_figpy.py Type: application/x-python Size: 233 bytes Desc: not available URL: From vivainio at gmail.com Fri Feb 1 11:51:14 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 1 Feb 2008 18:51:14 +0200 Subject: [IPython-dev] Non-blocking interact() for GUI/IPython1 use Message-ID: <46cb515a0802010851k2c97e526jfa77b6aca8054074@mail.gmail.com> In the interest of making ipython trunk more likely to fit into the ipython1 framework, I made the blocking raw_input read_eval_print optional. I created def interact_prompt(self): ... def interact_handle_input(self,line): ... These can be used to create GUIs (that of course want to handle their own input-waiting) for IPython. Here is a demo on how to use it, but of course this demo uses plain old readline and blocking raw_input: def interact_with_readline(self): """ Demo of using interact_handle_input, interact_prompt This is the main read-eval-print loop. If you need to implement your own (e.g. for GUI), it should work like this. """ self.readline_startup_hook(self.pre_readline) while not self.exit_now: self.interact_prompt() if self.more: self.rl_do_indent = True else: self.rl_do_indent = False line = raw_input_original().decode(self.stdin_encoding) self.interact_handle_input(line) Currently, interact_with_readline() is not called in IPython, but it works pretty much exactly like the standard interact(). -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Fri Feb 1 12:27:18 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 1 Feb 2008 19:27:18 +0200 Subject: [IPython-dev] Fwd: Opening files via ipython cmdline args? In-Reply-To: <46cb515a0802010926v3d01ef49icb6d7f02894f7289@mail.gmail.com> References: <200802011750.40339.hans_meine@gmx.net> <46cb515a0802010926v3d01ef49icb6d7f02894f7289@mail.gmail.com> Message-ID: <46cb515a0802010927ufc6b909od7fbaba367501e26@mail.gmail.com> And replying to list as well: On Feb 1, 2008 6:50 PM, Hans Meine wrote: > AFAICS, direct loading of files from the commandline could be useful for many > other cases, too (e.g. pytables, image analysis, ...). Is this possible > today? Yes, use -c: ipython -p figpy -i -c "f = fig.File('contraction.fig')" -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Fri Feb 1 13:23:04 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 1 Feb 2008 11:23:04 -0700 Subject: [IPython-dev] Non-blocking interact() for GUI/IPython1 use In-Reply-To: <46cb515a0802010851k2c97e526jfa77b6aca8054074@mail.gmail.com> References: <46cb515a0802010851k2c97e526jfa77b6aca8054074@mail.gmail.com> Message-ID: On Feb 1, 2008 9:51 AM, Ville M. Vainio wrote: > In the interest of making ipython trunk more likely to fit into the > ipython1 framework, I made the blocking raw_input read_eval_print > optional. Thanks! Some of the internal object layout for ipython may change, since right now the main Shell object has a jillion things in it that will be broken into subcomponents, but this is definitely the line of thought we need to follow. Hey Ville, what do you think of targetting mid-March for an 0.8.3 release that we could consider the baseline for integration? How's your calendar looking for that? At that point, we'll have had a bit of time to look more at the ipython1 code: Stefan vdW, Min and possibly Brian are coming to the Enthought/scipy/sage meeting Feb 29, and the Paris scipy/nipy sprint is March 17, so making a release in-between those two dates would be perfect, giving us that release as a reference point for the Paris sprint and an April or May target date for an alpha of the ipython1-based main code. How does that sound? Cheers, f From walters at verbum.org Fri Feb 1 16:10:21 2008 From: walters at verbum.org (Colin Walters) Date: Fri, 1 Feb 2008 16:10:21 -0500 Subject: [IPython-dev] Fwd: iPipe and Hotwire Shell In-Reply-To: References: <47A1B7CC.2060906@livinglogic.de> Message-ID: [have now taught GMail to use my right email so I don't get caught in the spamtrap] On Jan 31, 2008 6:58 AM, Walter D?rwald wrote: > > Currently ipipe's ifile class is a thin wrapper around Jason Orendorff's > path class (actually it's a subclass). Looking more closely, this class is very different from Hotwire's File object, which is designed for asynchronous use in a GUI, gathering all information it can in one go from a separate thread; then property access from the main thread never touches disk. >The only thing ifile does is add > the properties that will we used when displaying the output and > implementing the method __xattrs__() which tells ipipe which attributes > to display Ah, I see. The problem of which attributes to display is something I haven't really tackled yet in Hotwire. Right now the way it works is essentially that the renderers define it. But it would be good to share this, because then Python module authors could use it generically. Maybe call it __dispattrs__ to make it more clear it's related to display? > hotwire propably does its own command line parsing, ipipe tables just > implement the __or__/__ror__ magic methods. But thats a superficial > difference, ipipe could switch to command line parsing, or hotwire might > add the required magic. Right, as you mention ipipe really is Python, just using the __or__ trick. This may sound a bit heretical here but I'm actually thinking of the HotwirePipe language as independent of Python; in the long term I think what we really want in a system shell is that you can seamlessly use whichever (sane) language you want - but using a common, shared class library on top of a common, shared runtime (bytecode, VM etc.). That's why I called the Hotwire builtin which takes a snippet of Python code "py-map", and not just "map". And that the everyday tools you use in that shell are just really thin convenience wrappers around that common shared library. I'm dreaming, I know... But let me try to put together a strawman here for the near-term future =) Hotwire builtins can become wrappers on top of IPipe library "i" functions. There's a close correspondence between the names of Hotwire builtins and the names of IPipe functions. The Hotwire language uses "-" in names (following Unix tradition), but we can bidirectionally map from/to "_" for IPipe functions. Options to Hotwire builtins would map to keyword arguments in IPipe functions. For example, the LsBuiltin "-a" option maps to a hypothetical "all=True" keyword argument for ils. Hotwire builtin arguments map to "*args"-style argument captures for IPipe functions. IPipe would gain the concept of a multithreaded pipe (you need this, otherwise the system can deadlock), joined by iterable queues. I can try to do a bit of work to untangle this from the rest of the Hotwire core. There is a bit of a question as to how this should map to IPipe because in Hotwire there is the concept of GStreamer-style negotation between components; for example, if you do: /bin/ls -al | /bin/grep x This expands to a pipe between two SysBuiltin threads, each spawning a system binary. But, we don't really want to parse the data in between as Unicode lines (the default I/O for SysBuiltin) - it would be pure overhead, because /bin/grep like all Unix binaries takes byte arrays. So the first SysBuiltin says to the second "I can output bytearray/chunked", and the second says "I can accept bytearray/chunked", and thus they just pass "str" objects. Though, thinking about it a bit more in this case I should really have the first actually pass the file descriptor for the OS-level pipe to the second to use as stdin, so the Hotwire process isn't involved at all. If we wanted to support something like that in IPipe, it'd need to change probably from being Python to being a language of its own. To implement the above, we also need to share the underlying object library, such as File and Process objects. Probably what I should really do is try to submit this stuff to Python upstream... > Unfortunately I'm on OS X, so I can't really test hotwire, so I'm > limited to reading source and looking at screenshots. Yeah, I need to put together a package with the native GTK+-OSX bits and PyGTK. > As Fernando stated, we can't use any of the GPL licensed parts of > hotwire, but if I've read COPYING/COPYING.API/COPYING.UI correctly, that > only affects the GUI parts. Right, all the stuff I'm talking about here is under a MIT-style license which is basically the same as the IPython license. From ellisonbg.net at gmail.com Fri Feb 1 16:38:06 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 1 Feb 2008 14:38:06 -0700 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: References: Message-ID: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> Hello all, I picked a bad week to be away from email.... It is great to see that so many people are excited about pushing ipython to the next level. I have a number of comments about various aspects of this: Fernando: > - We'll try to keep compatibility with existing APIs (like ipapi) > where feasible, but there are no promises. We'll break backwards > compatibility wherever we need to in order to do the right thing. I think this needs to be clarified. The current ipython1 (purposely) makes no attemps to be API compatible with the ipython trunk. In fact, I would say that in most cases it is _extremely_ different that what is in ipython. The reason for this is that the abstractions needed in ipython1 are _extremely_ different that those in ipython trunk. Thus, the APIs are also very different. Walter: > Porting ipipe/ibrowse to ipython1 would be simple, once we have a > display hook extension mechanism. Except for the hook that invokes the > display object, there are no other dependencies. I don't know much about ipipe, but in IPython1 there is no such thing as a display hook - and there won't ever be - not in the same sense that there is in ipython trunk. I think we will be able to implement ipipe in ipython1, but the implementation will look completely different. In particular, in ipython1, things are divided between the "core" which knows nothing about how things are displayed and the frontend, which makes all the display related decisions. My feeling is that ipipe will have to be implemented in the various frontends rather than in the core. This is good though as it will make it possible to create nice GUI based ipipe widgets in the GUI frontends. But it does mean that the implementation will look really different. Ville: > don't see why not. We can call both, because it is a priority to > retain API compatibility with the trunk and the next-gen IPython > (generics, hooks and ipapi are all part of the API). I think to say that with ipython1 "it is a priority to retain API compatibility with the trunk" is a misinterpretation of Fernando's earlier statement. Many things in the ipython trunk API are by definition impossible to carry over the ipython1 because the assumptions and abstractions are so different. Of course, where we can we will try to maintain API compatibility, but I think those cases will be few and far between. The important point is feature parity. This is what I would say our priority must be as we think about how ipython trunk and ipython1 will be related. But achieving feature parity in the context of a system (ipython1) that has completely different abstractions will most definitely require API changes. This is why I think it will be important to have everyone hacking away at ipython1. There are a lot of basic design decisions/discussions needed because things are so different. To get ready for this, the best thing would be for everyone to begin looking at the new core, http://projects.scipy.org/ipython/ipython/browser/ipython1/trunk/ipython1/core to see how things in ipython can be mapped onto ipython1. Over the next few weeks, I am going to try to get ipython1 ready to "have a party" Cheers, Brian From vivainio at gmail.com Fri Feb 1 17:00:31 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 2 Feb 2008 00:00:31 +0200 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> Message-ID: <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> On Feb 1, 2008 11:38 PM, Brian Granger wrote: > I think to say that with ipython1 "it is a priority to retain API > compatibility with the trunk" is a misinterpretation of Fernando's > earlier statement. Many things in the ipython trunk API are by > definition impossible to carry over the ipython1 because the > assumptions and abstractions are so different. Of course, where we > can we will try to maintain API compatibility, but I think those cases > will be few and far between. Why are they so different? trunk/ is like the core, and a frontend will just manage the read / display functionality. If we just change all the printing to be some "write" function that can dump stuff to frontend, and raw_input to get stuff from front end, what else is different? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Fri Feb 1 18:04:35 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 1 Feb 2008 16:04:35 -0700 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> Message-ID: <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> > > I think to say that with ipython1 "it is a priority to retain API > > compatibility with the trunk" is a misinterpretation of Fernando's > > earlier statement. Many things in the ipython trunk API are by > > definition impossible to carry over the ipython1 because the > > assumptions and abstractions are so different. Of course, where we > > can we will try to maintain API compatibility, but I think those cases > > will be few and far between. > > Why are they so different? Good question One of the core assumptions in ipython trunk is that the frontend (stdout/stdin/terminal/readline/etc) has full immediate and complete access to things living in the users namespace. A few examples: 1. Tab complettion. Readline detects a tab and calls the tab completion machinery which directly looks at things in the users namespace. 2. Prompt display. IPython prompts can include info from the users namespace. 3. Traceback printing. Tracebacks are formatted (ACSII coloring, etc) by ipython at the moment the exception is raised, when the full state of the interpreter is available to look at. This is the main assumption that gets broken in ipython1. In ipython1, the frontend has no direct access to anything in the user's namespace. Instead the frontend only has access to a standardized API (methods, not attributes). A perfect example is Tab completion. Here is how it will work with ipython1. Let's say a user types the following: In [1]: foo. # and then hits TAB When the frontend detects the TAB event, it will read the current input as a string "foo." It will then do something like: interpreter.complete("foo.") This will return a list of strings that are the possible completions. It will then be up to the frontend to display those completions in whatever way is best. While this doesn't look too bad there is a lot of subtlety going on underneath the hood. For example, the complete method could actually trigger a network call to ipython core/engine running on a remote system. At the time the complete call gets made, the engine could be executing some blocking C code the user had started in a previous line. In this case, the tab completion can't happen until that C code finishes (threads won't help). So, then, in a case like that, the complete method will need to raise an exception to reflect the fact that TAB completion can't happen right now. Summary, TAB completion is still possible, but the API/design will look completely different. More specifically the functionality will need to be decomposed into 1) a python method that encapsulates the core behavior and and can happen over a network 2) the frontend code that calls that method and the appropriate time and displays the result. Another example: Prompts. If a frontend wants a prompt to include information from an object in the users namespace, the frontend will have to get the object (over the network possibly) each time it wants to update the information: a = interpreter.pull('a') This gets the object named 'a' from the users namespace. Again, this can potentially be a non-local (network) call, that is not completed immediately. Displaying things. The core of ipython1 will not be able to make any decisions about how things (stdout, tracebacks, etc) are formatted. This is because a single instance of the core could be simultaneously connected to frontends that need things formatted in completely different ways. You could have a Javascript frontend, a terminal based frontend and a Qt app all talking to the same instance of the core. Because of this, the core will need to return output in a "unformatted format." (xml, python dict, etc) that can transformed by the frontends into whatever form they need. Because of this, the various parts of ipython that make decisions about formatting will need to be moved to the frontend and hooked up to varous methods of the interpreter. This is what I mean that the "core" won't know anything about display hooks. They will still exist, but look very different. Does this help give a better idea about why the APIs will look so different? Brian > trunk/ is like the core, and a frontend will just manage the read / > display functionality. > > If we just change all the printing to be some "write" function that > can dump stuff to frontend, and raw_input to get stuff from front end, > what else is different? From fperez.net at gmail.com Sat Feb 2 01:03:35 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 1 Feb 2008 23:03:35 -0700 Subject: [IPython-dev] Wiki Dead Links - 0 In-Reply-To: References: Message-ID: 2008/1/31 Nathan Johnson : > > http://ipython.scipy.org/moin/PyReadline/Intro > Link " > Download installer from here. > " is broken. > > Link " > Questions and bug reports can be directed to the ipython-user mail list. > " too. Thanks for your report, fixed those. Cheers, f From vivainio at gmail.com Sat Feb 2 03:33:55 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 2 Feb 2008 10:33:55 +0200 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> Message-ID: <46cb515a0802020033n78ef19ecnb35bd7dca77fba97@mail.gmail.com> On Feb 2, 2008 1:04 AM, Brian Granger wrote: > 2. Prompt display. IPython prompts can include info from the users namespace. I think the prompt calculation should be done in the back-end. Isn't it trivial for the frontend to ask for the prompt string from the back end (even if it doesn't happen by merely writing out the characters)? > 3. Traceback printing. Tracebacks are formatted (ACSII coloring, > etc) by ipython at the moment the exception is raised, when the full > state of the interpreter is available to look at. Why can't the back end just render the exception string, and let the display be handled by front end? We can change the color escape sequences to something more "universal", if need be (foo, NOTE, whatever). > whatever way is best. While this doesn't look too bad there is a lot > of subtlety going on underneath the hood. For example, the complete > method could actually trigger a network call to ipython core/engine > running on a remote system. At the time the complete call gets made, > the engine could be executing some blocking C code the user had > started in a previous line. In this case, the tab completion can't > happen until that C code finishes (threads won't help). So, then, in > a case like that, the complete method will need to raise an exception > to reflect the fact that TAB completion can't happen right now. If we have the part that communicates with the front end in a separate thread, this won't be a problem. Completion can ALWAYS happen. This is probably easy stuff, apart from ctrl+c handling. > Prompts. If a frontend wants a prompt to include information from an > object in the users namespace, the frontend will have to get the > object (over the network possibly) each time it wants to update the > information: > > a = interpreter.pull('a') > > This gets the object named 'a' from the users namespace. Again, this > can potentially be a non-local (network) call, that is not completed > immediately. If we ask for the whole prompt from the back end, we know when it will be fully ready. Here, the complications are in the front end. > Displaying things. The core of ipython1 will not be able to make any > decisions about how things (stdout, tracebacks, etc) are formatted. > This is because a single instance of the core could be simultaneously > connected to frontends that need things formatted in completely > different ways. You could have a Javascript frontend, a terminal > based frontend and a Qt app all talking to the same instance of the > core. Because of this, the core will need to return output in a > "unformatted format." (xml, python dict, etc) that can transformed by > the frontends into whatever form they need. Yeah, this is what I referred to earlier. Changing what the back end blurts out to xml should be pretty easy, and a trivial front end implementation (a readline one) can just do string replace to convert it to the current ansi codes. > to varous methods of the interpreter. This is what I mean that the > "core" won't know anything about display hooks. They will still > exist, but look very different. I think the display hooks should still be in core, but return strings instead of direct printing. > Does this help give a better idea about why the APIs will look so different? Yes. Meetings these expectations seems like a pretty fun project, though :-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From edreamleo at yahoo.com Sat Feb 2 09:58:54 2008 From: edreamleo at yahoo.com (Edward Ream) Date: Sat, 2 Feb 2008 06:58:54 -0800 (PST) Subject: [IPython-dev] Please end my confusion :-) Message-ID: <532749.66245.qm@web57502.mail.re1.yahoo.com> Ville has been implicitly demonstrating, by various calm answers to my questions, that IPShellEmbed? hides some amazing technology. Over and over, I have thought something would be difficult or impossible to do, only to have Ville tell me that, no, everything I want to do can be done very easily. http://leo.zwiki.org/ConfusedAboutIPythonEmedding discusses my present state of confusion. I would appreciate any help in understanding what is going on. The essence of my confusion is that it appears that Leo can pretend that only one process is involved, and I have no idea how that magic could be possible. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at yahoo.com Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping From walter at livinglogic.de Sat Feb 2 16:20:54 2008 From: walter at livinglogic.de (Walter =?iso-8859-1?Q?D=F6rwald?=) Date: Sat, 2 Feb 2008 22:20:54 +0100 (CET) Subject: [IPython-dev] iPipe and Hotwire Shell In-Reply-To: References: <47A1B7CC.2060906@livinglogic.de> Message-ID: <63688.89.54.32.106.1201987254.squirrel@isar.livinglogic.de> Colin Walters sagte: > On Jan 31, 2008 6:58 AM, Walter D?rwald wrote: >> >> Currently ipipe's ifile class is a thin wrapper around Jason Orendorff's >> path class (actually it's a subclass). > > Looking more closely, this class is very different from Hotwire's File > object, which is designed for asynchronous use in a GUI, gathering all > information it can in one go from a separate thread; then property > access from the main thread never touches disk. Indeed. In ifile almost *every* property access touches disk. >> The only thing ifile does is add >> the properties that will we used when displaying the output and >> implementing the method __xattrs__() which tells ipipe which attributes >> to display > > Ah, I see. The problem of which attributes to display is something I > haven't really tackled yet in Hotwire. Right now the way it works is > essentially that the renderers define it. So is the basic output mode of hotwire a table or something else? Another thing I noticed is that in hotwire the "container" (e.g. LsBuiltin etc.) tells the system what kind of objects it's going to produce. In ipipe the display object displays whatever comes out of the pipe. This makes it possible to display anything that is a sequence or iterator. > But it would be good to share this, because then Python module authors > could use it generically. Maybe call it __dispattrs__ to make it more > clear it's related to display? Why not? Then we'd have to rename the generic function xattrs() too. What xattrs() produces is a list of Descriptor objects, which are more flexible than simple attribute names. Python 2.6 will allow to customize what the dir() function returns by implementing the __dir__() magic method. So maybe we should go that road (or at least use it as a fallback)? Another advantage of xattrs() is that it's a generic function, so you can configure the display attributes for classes you didn't write without having to monkeypatch the class, i.e. you can do stuff like: @ipipe.xattrs.when_type(complex): def xattrs_compres(obj, mode="default"): return ("real", "imag") >> hotwire propably does its own command line parsing, ipipe tables just >> implement the __or__/__ror__ magic methods. But thats a superficial >> difference, ipipe could switch to command line parsing, or hotwire might >> add the required magic. > > Right, as you mention ipipe really is Python, just using the __or__ > trick. This may sound a bit heretical here but I'm actually thinking > of the HotwirePipe language as independent of Python; No problem. Actually I'd prefer if I could type: >>> ils ~ instead of having to type >>> ils("~") > in the long term > I think what we really want in a system shell is that you can > seamlessly use whichever (sane) language you want - but using a > common, shared class library on top of a common, shared runtime > (bytecode, VM etc.). Sounds like .NET. > That's why I called the Hotwire builtin which > takes a snippet of Python code "py-map", and not just "map". And that > the everyday tools you use in that shell are just really thin > convenience wrappers around that common shared library. I'm dreaming, > I know... > > But let me try to put together a strawman here for the near-term future =) > > Hotwire builtins can become wrappers on top of IPipe library "i" > functions. There's a close correspondence between the names of > Hotwire builtins and the names of IPipe functions. The Hotwire > language uses "-" in names (following Unix tradition), but we can > bidirectionally map from/to "_" for IPipe functions. Most exiting ipipe names are single words anyway, so there's no mapping required. > Options to Hotwire builtins would map to keyword arguments in IPipe > functions. For example, the LsBuiltin "-a" option maps to a > hypothetical "all=True" keyword argument for ils. Hotwire builtin > arguments map to "*args"-style argument captures for IPipe functions. This sounds like a great idea. Configuration could be done by an optparse option parser. The long option names could be the names of the keyword arguments. However we still have the generator/threading mismatch and the "container declares the output" mismatch. > IPipe would gain the concept of a multithreaded pipe (you need this, > otherwise the system can deadlock), joined by iterable queues. As long as both sides of this look like normal generator this would be *really* useful. The ipipe object iteration could stay a simple iterator/generator, the piping operator would pipe this into a threading queue. Getting the results out of the queue should be a simple iterator/generator again. > I can > try to do a bit of work to untangle this from the rest of the Hotwire > core. There is a bit of a question as to how this should map to IPipe > because in Hotwire there is the concept of GStreamer-style negotation > between components; for example, if you do: > > /bin/ls -al | /bin/grep x > > This expands to a pipe between two SysBuiltin threads, each spawning a > system binary. But, we don't really want to parse the data in between > as Unicode lines (the default I/O for SysBuiltin) - it would be pure > overhead, because /bin/grep like all Unix binaries takes byte arrays. > So the first SysBuiltin says to the second "I can output > bytearray/chunked", and the second says "I can accept > bytearray/chunked", and thus they just pass "str" objects. Though, > thinking about it a bit more in this case I should really have the > first actually pass the file descriptor for the OS-level pipe to the > second to use as stdin, so the Hotwire process isn't involved at all. So when two system commands are adjacent in the pipe encoding/decoding or even passing Python string objects around could be optimized away. But that should be a specials case that doesn't affect Python tools in the pipe. > If we wanted to support something like that in IPipe, it'd need to > change probably from being Python to being a language of its own. Currently ipipe doesn't do anything like this (calling external commands). The only command that executes external code is ix() (which could greatly benefit from a threaded implementation) > To implement the above, we also need to share the underlying object > library, such as File and Process objects. Probably what I should > really do is try to submit this stuff to Python upstream... > >> Unfortunately I'm on OS X, so I can't really test hotwire, so I'm >> limited to reading source and looking at screenshots. > > Yeah, I need to put together a package with the native GTK+-OSX bits and PyGTK. That would be really helpful. >> As Fernando stated, we can't use any of the GPL licensed parts of >> hotwire, but if I've read COPYING/COPYING.API/COPYING.UI correctly, that >> only affects the GUI parts. > > Right, all the stuff I'm talking about here is under a MIT-style > license which is basically the same as the IPython license. Good! Servus, Walter From ellisonbg.net at gmail.com Sat Feb 2 17:59:55 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 2 Feb 2008 15:59:55 -0700 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <46cb515a0802020033n78ef19ecnb35bd7dca77fba97@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> <46cb515a0802020033n78ef19ecnb35bd7dca77fba97@mail.gmail.com> Message-ID: <6ce0ac130802021459s606c52e7ub93c71f0933d839@mail.gmail.com> > > 2. Prompt display. IPython prompts can include info from the users namespace. > > I think the prompt calculation should be done in the back-end. Isn't > it trivial for the frontend to ask for the prompt string from the back > end (even if it doesn't happen by merely writing out the characters)? This is sort of possible. We could have a method in the api called something like get_next_prompt. But, in practice, this won't work well. The reason is that the backend could be busy at the moment you want a prompt. And threads won't help this in any way. I probably need to explain a bit more about how the core works for this to make sense. Anytime you make a call to the core the call _may_ be put into a queue if the core is busy. This is a standard FIFO queue. The reason this happens is to enable a user to continue working on new input lines while the old ones compute. If you are familiar with Mathematica, you can edit and submit new input cells while old ones work. But, when the queue is active, everything goes through the queue - tab completion, prompt stuff. etc. This is done to make sure the entire thing is deterministic. Now there will be times where the queue is not active, but our API needs to be designed for the general case where a queue may be present. There is also another complication with prompts. The architecture already supports multiple users connecting to the same ipython engine/core process. All of the users frontends will have access to the current prompt number, but, you can imagine that the frontend has to in this case make decisions about what the prompts look like - the core simply can't as is knows nothing about how many user are connected. So, get_next_prompt would be easy to implement, but it behavior will not be quite what you really need to reliably create prompts in the frontends. > > 3. Traceback printing. Tracebacks are formatted (ACSII coloring, > > etc) by ipython at the moment the exception is raised, when the full > > state of the interpreter is available to look at. > > Why can't the back end just render the exception string, and let the > display be handled by front end? We can change the color escape > sequences to something more "universal", if need be (foo, > NOTE, whatever). That is exactly what we have in mind. In this case it shouldn't be too difficult. But, the API is different in a sense as the actual traceback formatter will be something that lives in the frontend. > > whatever way is best. While this doesn't look too bad there is a lot > > of subtlety going on underneath the hood. For example, the complete > > method could actually trigger a network call to ipython core/engine > > running on a remote system. At the time the complete call gets made, > > the engine could be executing some blocking C code the user had > > started in a previous line. In this case, the tab completion can't > > happen until that C code finishes (threads won't help). So, then, in > > a case like that, the complete method will need to raise an exception > > to reflect the fact that TAB completion can't happen right now. > > If we have the part that communicates with the front end in a separate > thread, this won't be a problem. Completion can ALWAYS happen. This is > probably easy stuff, apart from ctrl+c handling. When we started out on this stuff 3 years ago (my gosh, I can't believe it has been so long), that is what we thought. We actually had an initial prototype that used threads in this way.....then Fernando gave an embarassing demo of it at a conference. Here is the problem. Python threads are only useful in two cases: 1. When running pure python code. 2. When running extension code that releases the GIL The killer problem is that there is lots of extension code that doesn't release the GIL. This is a problem because this code is not IPython code, it is user code that we don't have any control over. Furthermore, we can't even reliably detect when they are about to call such code. Thus, when we have used threads in this manner the system simply locks up anytime non-GIL releasing code is run by the user - this is what happened in Fernando's talk before we solved these problems. But, there is another more important reason that tab completion can't always happen. I can give you an example - remember that every command that is sent to the core _may_ be put into a queue and executed later. In [1]: run_some_long_calculation() # the core starts working on this and it will take 5 minutes In [2]: a = Thing() # The user gets this prompt immediately and enters this command. # because the first command is still executing, this command is put on the # queue. It will be executed when the first one is done - in 5 minutes. In [3]: a. # Again, the user gets this prompt immediately. They want to tab complete # on a, but it is truly impossible as the command to create a has not been # run yet. As you can see, the other really fundamental change in ipython1 is that everything _may_ be completely asynchronous. This requires a huge paradigm shift in terms of thinking. I should say that Fernando, Min and myself have spent months of our time struggling to figure out the best model for all of this. One conclusion we have made is that what we are trying to do is extremely difficult. It is very likely that our model/architecture will need to be changed further - especially to support things that the new frontends will want/need. But, the fundamental notion of things being possibly asynchronous is probably here to stay. With that said, it will be possible to make a terminal based frontend that is not asynchronous and that always tab completes immediately. > Yeah, this is what I referred to earlier. Changing what the back end > blurts out to xml should be pretty easy, and a trivial front end > implementation (a readline one) can just do string replace to convert > it to the current ansi codes. Yep. > > to varous methods of the interpreter. This is what I mean that the > > "core" won't know anything about display hooks. They will still > > exist, but look very different. > > I think the display hooks should still be in core, but return strings > instead of direct printing. Yes, this is more of what it looks like. Currently when you call the method to run code in the core, it gives you back a python dict of various things (stdout, stderr, etc.). The display hooks could possible manipulate those dictionaries. But, it is important to remember that multple users (with different display desires) could be connected to the same core instance. Thus, I think that much of the display logic will need to be the frontends. > > Does this help give a better idea about why the APIs will look so different? > > Yes. Meetings these expectations seems like a pretty fun project, though :-) I think it will be a blast. Thanks to everyone's great work IPython is already extremely cool. It we can pull this new stuff off, the resulting system will be unparalleled in terms of interactive computing environments. I can't wait to use the new stuff myself :) Brian > > -- > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > From vivainio at gmail.com Sun Feb 3 17:03:49 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 4 Feb 2008 00:03:49 +0200 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <6ce0ac130802021459s606c52e7ub93c71f0933d839@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> <46cb515a0802020033n78ef19ecnb35bd7dca77fba97@mail.gmail.com> <6ce0ac130802021459s606c52e7ub93c71f0933d839@mail.gmail.com> Message-ID: <46cb515a0802031403xed54de7k9536404ea680ecfe@mail.gmail.com> Very busy now, but few points. > sense. Anytime you make a call to the core the call _may_ be put into > a queue if the core is busy. This is a standard FIFO queue. The > reason this happens is to enable a user to continue working on new > input lines while the old ones compute. If you are familiar with I'm not sure this should be the default behaviour. I like my stuff synchronous, most of the time. I think there should be the synchronous mode (where there are no problems with tab completion), and async mode (where the delays with tab completion can occur on problematic non-GIL-releasing code). > Mathematica, you can edit and submit new input cells while old ones > work. But, when the queue is active, everything goes through the > queue - tab completion, prompt stuff. etc. This is done to make sure > the entire thing is deterministic. Tab completion does not need to be deterministic IMO. It's a very "interactive" thing, and you can press tab a couple of times while you wait for the completion to be "ready". Or the frontend can retry the completion when it knows the queue is empty. But I still think the completions should be served immediately in a thread. > There is also another complication with prompts. The architecture > already supports multiple users connecting to the same ipython > engine/core process. All of the users frontends will have access to > the current prompt number, but, you can imagine that the frontend has > to in this case make decisions about what the prompts look like - the > core simply can't as is knows nothing about how many user are > connected. The frontend can just send the Itpl string to the core which expands it. Of course a "generate_prompt" like hook would happen in the core, but if that is what the user wants that's what he will get. > But, there is another more important reason that tab completion can't > always happen. I can give you an example - remember that every > command that is sent to the core _may_ be put into a queue and > executed later. > > In [1]: run_some_long_calculation() # the core starts > working on this and it will take 5 minutes > > In [2]: a = Thing() # The user gets this prompt > immediately and enters this command. > # because the first > command is still executing, this command is put on the > # queue. It will be > executed when the first one is done - in 5 minutes. > > In [3]: a. # Again, the user gets this > prompt immediately. They want to tab complete > # on a, but it is truly > impossible as the command to create a has not been > # run yet. > > As you can see, the other really fundamental change in ipython1 is > that everything _may_ be completely asynchronous. This requires a > huge paradigm shift in terms of thinking. (see the solution above - retry until the queue is empty). > > I think the display hooks should still be in core, but return strings > > instead of direct printing. > > Yes, this is more of what it looks like. Currently when you call the > method to run code in the core, it gives you back a python dict of > various things (stdout, stderr, etc.). The display hooks could > possible manipulate those dictionaries. But, it is important to > remember that multple users (with different display desires) could be > connected to the same core instance. Thus, I think that much of the > display logic will need to be the frontends. We can't pass "live" complicated objects to the frontend (pickling wont work), so I think the issue is twofold: - The display hooks on server output those dicts (generics.result_display can return those). - Frontend does whatever it wants with that dict. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Sun Feb 3 17:15:17 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 4 Feb 2008 00:15:17 +0200 Subject: [IPython-dev] Please end my confusion :-) In-Reply-To: <532749.66245.qm@web57502.mail.re1.yahoo.com> References: <532749.66245.qm@web57502.mail.re1.yahoo.com> Message-ID: <46cb515a0802031415v7fcb1217lbcb83ec69e17c88e@mail.gmail.com> On Feb 2, 2008 4:58 PM, Edward Ream wrote: > Ville has been implicitly demonstrating, by various calm > answers to my questions, that IPShellEmbed? hides some > amazing technology. Over and over, I have thought something There is nothing amazing in there. The magic is on Leo side - the ipshell() call, which blocks, does not block the Leo event loop. That makes it all work so easily. Does Leo run the scripts in separate threads? > The essence of my confusion is that it appears that Leo can > pretend that only one process is involved, and I have no > idea how that magic could be possible. One one process is involved, the Leo process (that happens to run IPython as well). -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Sun Feb 3 17:54:57 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sun, 3 Feb 2008 15:54:57 -0700 Subject: [IPython-dev] [IPython-user] Development plans update In-Reply-To: <46cb515a0802031403xed54de7k9536404ea680ecfe@mail.gmail.com> References: <6ce0ac130802011338w4c6d2013r5e32b11542fe2b8c@mail.gmail.com> <46cb515a0802011400g109a8af1mf1514e1e8a691fe4@mail.gmail.com> <6ce0ac130802011504h4736404fp422e0d3fd475b4f6@mail.gmail.com> <46cb515a0802020033n78ef19ecnb35bd7dca77fba97@mail.gmail.com> <6ce0ac130802021459s606c52e7ub93c71f0933d839@mail.gmail.com> <46cb515a0802031403xed54de7k9536404ea680ecfe@mail.gmail.com> Message-ID: <6ce0ac130802031454g43bf26cu6776a7818a505f6a@mail.gmail.com> > > sense. Anytime you make a call to the core the call _may_ be put into > > a queue if the core is busy. This is a standard FIFO queue. The > > reason this happens is to enable a user to continue working on new > > input lines while the old ones compute. If you are familiar with > > I'm not sure this should be the default behaviour. I like my stuff > synchronous, most of the time. I think there should be the synchronous > mode (where there are no problems with tab completion), and async mode > (where the delays with tab completion can occur on problematic > non-GIL-releasing code). At some level I agree. While all the operations _may_ be asynchronous, it will be up to each frontend whether or not it exposes that to the user. If a frontend desires, it can make everything synchronous again. In a terminal based frontend, this would surely be the default. In GUI frontends, you could imagine that the user could even choose which behavior is exhibited. But the APIs do need to be designed with the possibility of things being asynchronous. > > Mathematica, you can edit and submit new input cells while old ones > > work. But, when the queue is active, everything goes through the > > queue - tab completion, prompt stuff. etc. This is done to make sure > > the entire thing is deterministic. > > Tab completion does not need to be deterministic IMO. It's a very > "interactive" thing, and you can press tab a couple of times while you > wait for the completion to be "ready". Or the frontend can retry the > completion when it knows the queue is empty. But I still think the > completions should be served immediately in a thread. Yes, at some level, it will be up to the frontends to decide how to handle these cases. Retrying when the queue is empty or printing some sort of message to the user "tab completion is currently not available, please try again in a second" are reasonable options. But, the I do disagree about the deterministic aspect of things. Once you sacrifice that in a complex, asynchronous system, things become a nightmare. Also, threads don't solve this problem. > > There is also another complication with prompts. The architecture > > already supports multiple users connecting to the same ipython > > engine/core process. All of the users frontends will have access to > > the current prompt number, but, you can imagine that the frontend has > > to in this case make decisions about what the prompts look like - the > > core simply can't as is knows nothing about how many user are > > connected. > > The frontend can just send the Itpl string to the core which expands > it. Of course a "generate_prompt" like hook would happen in the core, > but if that is what the user wants that's what he will get. That is a great idea. There just needs to be some fallback behavior if the core can't generate a prompt at that second, but the frontend doesn't want to wait. > > But, there is another more important reason that tab completion can't > > always happen. I can give you an example - remember that every > > command that is sent to the core _may_ be put into a queue and > > executed later. > > > > In [1]: run_some_long_calculation() # the core starts > > working on this and it will take 5 minutes > > > > In [2]: a = Thing() # The user gets this prompt > > immediately and enters this command. > > # because the first > > command is still executing, this command is put on the > > # queue. It will be > > executed when the first one is done - in 5 minutes. > > > > In [3]: a. # Again, the user gets this > > prompt immediately. They want to tab complete > > # on a, but it is truly > > impossible as the command to create a has not been > > # run yet. > > > > As you can see, the other really fundamental change in ipython1 is > > that everything _may_ be completely asynchronous. This requires a > > huge paradigm shift in terms of thinking. (see the solution above - retry until the queue is empty). Very reasonable. > > > I think the display hooks should still be in core, but return strings > > > instead of direct printing. > > > > Yes, this is more of what it looks like. Currently when you call the > > method to run code in the core, it gives you back a python dict of > > various things (stdout, stderr, etc.). The display hooks could > > possible manipulate those dictionaries. But, it is important to > > remember that multple users (with different display desires) could be > > connected to the same core instance. Thus, I think that much of the > > display logic will need to be the frontends. > > We can't pass "live" complicated objects to the frontend (pickling > wont work), so I think the issue is twofold: Very true. This is especially relevant for the traceback formatting when you actually want to inspect the stack , etc. in real time. > - The display hooks on server output those dicts > (generics.result_display can return those). > - Frontend does whatever it wants with that dict. I think we will have to have something like this. Thanks for the comments, it helps to see how these things might look. Also, I hope this clarifies what I meant when I said that the APIs will be different, but the feature will be the same (or better!). Brian > -- > > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > From edreamleo at yahoo.com Tue Feb 5 12:54:25 2008 From: edreamleo at yahoo.com (Edward Ream) Date: Tue, 5 Feb 2008 09:54:25 -0800 (PST) Subject: [IPython-dev] Leo's ipython plugin is complete Message-ID: <806244.17500.qm@web57516.mail.re1.yahoo.com> I just updated the Cookbook/LeoIPythonBridge: http://ipython.scipy.org/moin/Cookbook/LeoIpythonBridge Here it is: Leo's ipython plugin provides two-way interaction between IPython and Leo. This plugin is now essentially complete. For complete user-level documentation, see: http://webpages.charter.net/edreamleo/IPythonBridge.html For detailed (exhaustive would be a better word) of the design history of this plugin, including all the Aha's, see http://sourceforge.net/forum/forum.php?thread_id=1911662&forum_id=10226 Executive summary: - Executing the start-ipython command (in Leo) starts IPython in Leo's console. The start-ipython command also injects an object called leox (for Leo Interface) into the IPython namespace that gives IPython scripts access to Leo's g and c objects. - Using these g and c objects, IPython scripts can do *anything* that Leo scripts can do! They can access data in Leo files, including Leo settings, insert and delete nodes, open or delete .leo files, etc. Thus, the leox object is clearly the simplest, most general, most powerful interface between Leo and IPython. In short, Leo's ipython plugin now goes way beyond my fondest hopes. Edward P.S. I would like to state once again that none of this would have been possible without Ville's help. EKR -------------------------------------------------------------------- Edward K. Ream email: edreamleo at yahoo.com Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- ____________________________________________________________________________________ Looking for last minute shopping deals? Find them fast with Yahoo! Search. http://tools.search.yahoo.com/newsearch/category.php?category=shopping From vivainio at gmail.com Tue Feb 5 13:29:57 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 5 Feb 2008 20:29:57 +0200 Subject: [IPython-dev] Leo's ipython plugin is complete In-Reply-To: <806244.17500.qm@web57516.mail.re1.yahoo.com> References: <806244.17500.qm@web57516.mail.re1.yahoo.com> Message-ID: <46cb515a0802051029y513e20dfsbc57a2883c01fd71@mail.gmail.com> On Feb 5, 2008 7:54 PM, Edward Ream wrote: > In short, Leo's ipython plugin now goes way beyond my fondest hopes. I would like to stress that this is extremely cool stuff, and provides higher & deeper level of IPython integration than what any other editor can boast about. Notably, this allows using Leo as an "IPython workbook", where you rapidly create and experiment with new functions, deal with data that is stored in Leo nodes (both for input and output), create more functions to deal with that data, etc. And all the time, all the functions and data are persisted inside Leo nodes, so you don't need to think about where to store stuff, cluttering your work directories with files, etc. And your IPython hacks will no longer be "throwaway" stuff, unless you really want it. As my thesis project (if all the bureaucracy clears out), I am about to create a convenient "workbook" approach to using IPython and Leo, introducing lots of convenience stuff and fast approaches to deal with data & code (without having to dwell into details of either IPython or Leo scripting). All of it will end up as an IPython extension (trunk at first), most of the Leo side is in place already. It's all quite mindblowing stuff, really. And all of it is possible *now*. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From edreamleo at yahoo.com Tue Feb 5 14:35:06 2008 From: edreamleo at yahoo.com (Edward Ream) Date: Tue, 5 Feb 2008 11:35:06 -0800 (PST) Subject: [IPython-dev] Leo's ipython plugin is complete Message-ID: <252850.95677.qm@web57504.mail.re1.yahoo.com> > Notably, this allows using Leo as an "IPython workbook", where you rapidly create and experiment with new functions, deal with data that is stored in Leo nodes (both for input and output), create more functions to deal with that data, etc. And all the time, all the functions and data are persisted inside Leo nodes, so you don't need to think about where to store stuff, cluttering your work directories with files, etc. And your IPython hacks will no longer be "throwaway" stuff, unless you really want it. Now might be a good time to mention this page on Leo's wiki: http://leo.zwiki.org/WhatMakesLeoSpecial After more than 10 years of trying to understand and explain why Leo works, it dawned on me that Leo's DOM (Document Object Model) is what sets Leo apart. Clearly, this realization has relevance for Leo as an IPython notebook. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at yahoo.com Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- ____________________________________________________________________________________ Never miss a thing. Make Yahoo your home page. http://www.yahoo.com/r/hs From antont at an.org Wed Feb 6 04:09:15 2008 From: antont at an.org (Toni Alatalo) Date: Wed, 06 Feb 2008 11:09:15 +0200 Subject: [IPython-dev] Leo's ipython plugin is complete In-Reply-To: <252850.95677.qm@web57504.mail.re1.yahoo.com> References: <252850.95677.qm@web57504.mail.re1.yahoo.com> Message-ID: <47A9793B.8000104@an.org> Edward Ream kirjoitti: > it dawned on me that Leo's DOM (Document Object Model) is what sets Leo apart. > Clearly, this realization has relevance for Leo as an IPython notebook. This sounds very interesting indeed. Some of you may remember that I participated in the Summer of Code work on IPython notebooks back in .. 2005 it was. Back then there were some discussions about document models: the initial idea (in http://ipython.scipy.org/google_soc/ipnb_google_soc.pdf) was to use Python, but that was then rejected ("Python is not Lisp" :) and XML was used instead. IIRC documents were processed by making docbook xml out of the 'notebook xml's, and I experimented with using the docutils document model (~rest) too. The thing in the notebook XML is/was that it includes IPython in&out nodes, with the executable code. I remember that Leo was discussed then too, and actually quite recently when I once more encountered a need to write Python notebooks, I tried Leo again and learned to use it a bit. It certainly seemed like something I could use, but haven't done more yet than the little experimenting that one late evening. I haven't looked at the Leo DOM at all. So, thanks for the work and news, am looking forward to giving it a shot one day .. and hoping that the notebook dreams are / come true. I also took a quick look of the new javascript notebook stuff in ipython svn, but didn't get too much of a hold of it yet -- interesting to see also. BTW I'm mostly using Python now for game development, game logic, animation code etc. in general in our own projects (at Playsign, ex-Kyperjokki), but also with a special interest in AI / behaviours that also do for some customers (bots etc) .. don't know yet if / how 'executable notebooks' come to play there. > Edward ~Toni From vivainio at gmail.com Wed Feb 6 04:32:12 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 6 Feb 2008 11:32:12 +0200 Subject: [IPython-dev] Leo's ipython plugin is complete In-Reply-To: <47A9793B.8000104@an.org> References: <252850.95677.qm@web57504.mail.re1.yahoo.com> <47A9793B.8000104@an.org> Message-ID: <46cb515a0802060132m7b9098feg7f14073e16007b8b@mail.gmail.com> On Feb 6, 2008 11:09 AM, Toni Alatalo wrote: > So, thanks for the work and news, am looking forward to giving it a shot > one day .. and hoping that the notebook dreams are / come true. I also I am quite confident that the dream will come true, I can't foresee any technical obstacles that we can't get over in quick/realistic timeframe. Incidentally, I will probably also explore the old SoC nbshell project as "prior art" for my thesis project (dippatyo) and see how well we can reach those goals with Leo-IPython workbook. My preliminary concept brief for the project is up at http://vvtools.googlecode.com/svn/trunk/doc/manticore.leo (Yes, it's of course a leo document) ;-) -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Fri Feb 8 11:16:25 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 8 Feb 2008 09:16:25 -0700 Subject: [IPython-dev] IPython1 API design question Message-ID: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> Hi, I have been working on IPython1 a lot this week and have run into an API design dilemma. Because the IPython1 API will be the core aPI that all of us develop to in the near future, I wanted to get some opinions about the issue. Here we go... The IPython1 API has methods: push(self, **namespace) # push namespace of key, values pairs into the users namespace pull(self, *keys) # get python objects from the users namespace by keys These are used in the following way: push(a=10, b=30) pull('a', 'b') The key point is that the usage of *keys and **namespace makes the methods very easy to use - you can basically get around creating tuples and dicts by hand. But, here is the problem we are running into. In various parts of the API, we need to add additional arguments to these methods. The two that have come up are: block -> should the call block or not? targets -> when you are working with multiple ipython core instances, which should you push/pull to. Our current API handles these in the following way: push(self, block, targets, **ns) pull(self, block, targets, *keys) And would be used in this way: push(True, [0,1,2,3], a=10, b=30) pull(False, 'all', 'b', 'c') Here are the problems I see with this: 1. We can't introduce new keyword arguments that have defaults 2. All new arguments have to be positional and thus completely change the API in a backwards incompatible manner. 3. The design violates my aesthetic sense which says the most important arguments (keys, ns) should come first, less important ones later. 4. We have to have additional API methods to handle default cases, thus we have pushAll/pullAll where targets is set to 'all' The alternate API would look some thing like this: push(self, ns, block=True, targets='all') pull(self, keys, block=True, targets='all') And be used: push(dict(a=10,b=30), block=False) # targets = 'all' pull(('a','b')) # block=True, tagets='all' the main disadvantage is that a user has to manually build the dict and tuple. For pull, we could do a test to see if keys is a string and build the tuple for then, but for the dict, they really have to build it using dict or {} So, which of these approaches seems better, both from the development and user's perspective? Thoughts? Brian From vivainio at gmail.com Fri Feb 8 13:09:36 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 8 Feb 2008 20:09:36 +0200 Subject: [IPython-dev] Launchpad? Message-ID: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> There was some talk on Leo mailing list about Launchpad, https://launchpad.net/ What do you guys think? Could Launchpad be the platform where we could be driving IPython forward? It would imply using bzr instead of hg, but I'm not sure the benefits of hg over bzr (performance comes to mind first) overweigh the advantages of using launchpad (one of the advantages being that it's ready now). Perhaps we should see how IPython svn can be exported to launchpad bzr, and see what kind of performance we can get out of it (it certainly can't be worse than svn). The usage patterns of hg and bzr are similar enough, so I don't care either way. It's certainly worth a thought. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Fri Feb 8 14:00:26 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 8 Feb 2008 12:00:26 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> Message-ID: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> Disclaimer: I have little experience with distributed version control and thus am not qualified to comment on the relative strengths and weaknesses of bsr and hg. WIth that said, Launchpad is extremely impressive. Not only is it ready today, it would free all of us up from having to play part time sys-admin with trac/hg/etc (a role that I am sure Fernando would be glad to give up). Even if we did have hg+trac fully setup today, the feature set, usability, maintainability of launchpad+bzr is still far better. As we move to a distributed model, it would still be useful to have a place for everyone to publish their branches - launchpad makes that easy. It would be a huge pain if everyone had to setup their own web server to share their hg repos with everyone else. Summary - hg would have to be _way_ better than bzr for us not to consider using launchpad+bzr. Brian On Feb 8, 2008 11:09 AM, Ville M. Vainio wrote: > There was some talk on Leo mailing list about Launchpad, https://launchpad.net/ > > What do you guys think? Could Launchpad be the platform where we could > be driving IPython forward? It would imply using bzr instead of hg, > but I'm not sure the benefits of hg over bzr (performance comes to > mind first) overweigh the advantages of using launchpad (one of the > advantages being that it's ready now). > > Perhaps we should see how IPython svn can be exported to launchpad > bzr, and see what kind of performance we can get out of it (it > certainly can't be worse than svn). The usage patterns of hg and bzr > are similar enough, so I don't care either way. > > It's certainly worth a thought. > > -- > Ville M. Vainio - vivainio.googlepages.com > blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From fperez.net at gmail.com Fri Feb 8 14:24:57 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 8 Feb 2008 12:24:57 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> Message-ID: On Feb 8, 2008 12:00 PM, Brian Granger wrote: > Disclaimer: I have little experience with distributed version control > and thus am not qualified to comment on the relative strengths and > weaknesses of bsr and hg. > > WIth that said, Launchpad is extremely impressive. Not only is it > ready today, it would free all of us up from having to play part time > sys-admin with trac/hg/etc (a role that I am sure Fernando would be > glad to give up). Even if we did have hg+trac fully setup today, the > feature set, usability, maintainability of launchpad+bzr is still far > better. > > As we move to a distributed model, it would still be useful to have a > place for everyone to publish their branches - launchpad makes that > easy. It would be a huge pain if everyone had to setup their own web > server to share their hg repos with everyone else. > > Summary - hg would have to be _way_ better than bzr for us not to > consider using launchpad+bzr. I'm open to anything, but I'm basically near-offline for 2 weeks now. I still have net access but zero time, sorry. I'll try to keep an eye on the discussion and feedback from others, and in a couple of weeks we can make a decision, if that's OK with everyone. Cheers, f From vivainio at gmail.com Fri Feb 8 14:28:08 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 8 Feb 2008 21:28:08 +0200 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> Message-ID: <46cb515a0802081128oa3e42al7ee3b0b2e3ce7edc@mail.gmail.com> On Feb 8, 2008 9:24 PM, Fernando Perez wrote: > I'll try to keep an eye on the discussion and feedback from others, > and in a couple of weeks we can make a decision, if that's OK with > everyone. Yeah, that's ok. At this point, I'm +1 for Launchpad and bzr. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Fri Feb 8 14:29:44 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 8 Feb 2008 12:29:44 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> Message-ID: <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> > I'm open to anything, but I'm basically near-offline for 2 weeks now. > I still have net access but zero time, sorry. > > I'll try to keep an eye on the discussion and feedback from others, > and in a couple of weeks we can make a decision, if that's OK with > everyone. No problem, I don't think there is any need to make a decision on this immediately. I would love to hear feedback from people who have used launchpad+bzr. I might have some time myself over the next few weeks to play around with launchpad+bzr. Meanwhile svn is working just fine. Brian > Cheers, > > f > From vivainio at gmail.com Fri Feb 8 14:32:05 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 8 Feb 2008 21:32:05 +0200 Subject: [IPython-dev] Launchpad? In-Reply-To: <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> Message-ID: <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> On Feb 8, 2008 9:29 PM, Brian Granger wrote: > launchpad+bzr. I might have some time myself over the next few weeks > to play around with launchpad+bzr. Meanwhile svn is working just > fine. While you are at it, why not put up an ipython1 project and import our svn there? If we consider it a failure, we can always delete the project. If we consider it a success, we will also create ipython (trunk) project. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Fri Feb 8 14:56:01 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Fri, 8 Feb 2008 21:56:01 +0200 Subject: [IPython-dev] Launchpad? In-Reply-To: <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> Message-ID: <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> On Feb 8, 2008 9:51 PM, Brian Granger wrote: > I just tried to create a ipython lauchpad project, but launchpad > already has an ipython project listed. Does anyone know who created > that? It looks unused, but I can't seem to find any info about who > added it. It would be great to simply be able to use the existing > one. Perhaps it's the ipython used in ubuntu (ubuntu is a high profile launchpad user). Just create 'ipython1' for now, and we'll hash out the name of ipython trunk later on. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From fperez.net at gmail.com Fri Feb 8 15:00:20 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Fri, 8 Feb 2008 13:00:20 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> Message-ID: On Feb 8, 2008 12:56 PM, Ville M. Vainio wrote: > On Feb 8, 2008 9:51 PM, Brian Granger wrote: > > I just tried to create a ipython lauchpad project, but launchpad > > already has an ipython project listed. Does anyone know who created > > that? It looks unused, but I can't seem to find any info about who > > added it. It would be great to simply be able to use the existing > > one. > > Perhaps it's the ipython used in ubuntu (ubuntu is a high profile > launchpad user). Just create 'ipython1' for now, and we'll hash out > the name of ipython trunk later on. Well, it wasn't me: I'm "fperez" on launchpad, with this gmail address, and ipython doesn't appear assigned to me. Furthermore, they list the source distro as a CVS checkout, and we haven't used CVS in like 4-5 years, so this is some stale registration information. I'd say go ahead for the test with an 'ipython1' project, and if it looks good, I'll later contact the admins to get the ipython project name reassigned to us. Cheers, f From stefan at sun.ac.za Fri Feb 8 17:12:59 2008 From: stefan at sun.ac.za (Stefan van der Walt) Date: Sat, 9 Feb 2008 00:12:59 +0200 Subject: [IPython-dev] IPython1 API design question In-Reply-To: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> References: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> Message-ID: <20080208221259.GE23594@mentat.za.net> On Fri, Feb 08, 2008 at 09:16:25AM -0700, Brian Granger wrote: > The alternate API would look some thing like this: > > push(self, ns, block=True, targets='all') > pull(self, keys, block=True, targets='all') > > And be used: > > push(dict(a=10,b=30), block=False) # targets = 'all' > pull(('a','b')) # block=True, tagets='all' > > the main disadvantage is that a user has to manually build the dict > and tuple. For pull, we could do a test to see if keys is a string > and build the tuple for then, but for the dict, they really have to > build it using dict or {} > > So, which of these approaches seems better, both from the development > and user's perspective? Thoughts? For what it's worth, I like the idea of pushing namespaces. With the current system, you can do (as a random example) push(*_ip.user_ns) but then you have to make sure that the dict doesn't contain some special keywords. I imagine it would be better to shield the user from such unexpected side-effects. Regards St?fan From barrywark at gmail.com Fri Feb 8 17:25:05 2008 From: barrywark at gmail.com (Barry Wark) Date: Fri, 8 Feb 2008 14:25:05 -0800 Subject: [IPython-dev] IPython1 API design question In-Reply-To: <20080208221259.GE23594@mentat.za.net> References: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> <20080208221259.GE23594@mentat.za.net> Message-ID: What about two push/pull method pairs... def push(self, **namespace): self.pushEx(namespace, block=, targets='all') def pull(self, *keys): self.pullEx(keys, block= , targets='all') and pushEx(self, ns, block=True, target='all') pullEx(self, keys, block=True, target='all') Obviously I haven't put a lot of thought into the names, but the idea is to provide a convenience method for the "expected" use case and an extended method with more options. barry On Feb 8, 2008 2:12 PM, Stefan van der Walt wrote: > On Fri, Feb 08, 2008 at 09:16:25AM -0700, Brian Granger wrote: > > The alternate API would look some thing like this: > > > > push(self, ns, block=True, targets='all') > > pull(self, keys, block=True, targets='all') > > > > And be used: > > > > push(dict(a=10,b=30), block=False) # targets = 'all' > > pull(('a','b')) # block=True, tagets='all' > > > > the main disadvantage is that a user has to manually build the dict > > and tuple. For pull, we could do a test to see if keys is a string > > and build the tuple for then, but for the dict, they really have to > > build it using dict or {} > > > > So, which of these approaches seems better, both from the development > > and user's perspective? Thoughts? > > For what it's worth, I like the idea of pushing namespaces. With the > current system, you can do (as a random example) > > push(*_ip.user_ns) > > but then you have to make sure that the dict doesn't contain some > special keywords. I imagine it would be better to shield the user > from such unexpected side-effects. > > Regards > St?fan > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Fri Feb 8 17:33:06 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Fri, 8 Feb 2008 15:33:06 -0700 Subject: [IPython-dev] IPython1 API design question In-Reply-To: References: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> <20080208221259.GE23594@mentat.za.net> Message-ID: <6ce0ac130802081433h566d6393p554fb167a89cb8ad@mail.gmail.com> > What about two push/pull method pairs... > > def push(self, **namespace): > self.pushEx(namespace, block=, targets='all') > def pull(self, *keys): > self.pullEx(keys, block= , targets='all') > > and > > pushEx(self, ns, block=True, target='all') > pullEx(self, keys, block=True, target='all') My current thinking is to basically do something like this. Most of the API (that developers see) would have the more verbose versions pushEx/pullEx, but the API that users see could be stateful and take **/* arguments. But one important point that I realized later is that most of the API cannot be stateful, so this solution only work in certain places where it is OK to hold state. Brian > Obviously I haven't put a lot of thought into the names, but the idea > is to provide a convenience method for the "expected" use case and an > extended method with more options. > > barry > > > On Feb 8, 2008 2:12 PM, Stefan van der Walt wrote: > > On Fri, Feb 08, 2008 at 09:16:25AM -0700, Brian Granger wrote: > > > The alternate API would look some thing like this: > > > > > > push(self, ns, block=True, targets='all') > > > pull(self, keys, block=True, targets='all') > > > > > > And be used: > > > > > > push(dict(a=10,b=30), block=False) # targets = 'all' > > > pull(('a','b')) # block=True, tagets='all' > > > > > > the main disadvantage is that a user has to manually build the dict > > > and tuple. For pull, we could do a test to see if keys is a string > > > and build the tuple for then, but for the dict, they really have to > > > build it using dict or {} > > > > > > So, which of these approaches seems better, both from the development > > > and user's perspective? Thoughts? > > > > For what it's worth, I like the idea of pushing namespaces. With the > > current system, you can do (as a random example) > > > > push(*_ip.user_ns) > > > > but then you have to make sure that the dict doesn't contain some > > special keywords. I imagine it would be better to shield the user > > from such unexpected side-effects. > > > > Regards > > St?fan > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From glenn at tarbox.org Fri Feb 8 18:52:07 2008 From: glenn at tarbox.org (Glenn Tarbox, PhD) Date: Fri, 8 Feb 2008 15:52:07 -0800 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> Message-ID: I've been a lurker for a little while now... have done some IPython hacking, but definitely "get it" On the subject of this thread, Launchpad / bzr. I was concerned when I saw IPython1 embrace hg. While git has its warts (the most glaring being its horrendous documentation), I'd make the unsupportable claim that its technically superior to the alternatives... But, technical issues aside... Git is gonna win. At the end of the day, git is going to have a much higher adoption rate and will be a better engine on which to progress... more tools, more conversion utilities, much bigger community, etc. Its momentum and superior technical core will help overcome its early documentation / porcelain problems. So, this thread discusses bzr/launchpad... my though: whats up with bzr? why did canonical go ahead and re-re-invent the wheel? we had hg, git was coming along... their time would have been much better spent building something on top of git. You already have a nice Trac site... I realize shedding the admin load would be nice... but I'm guessing that with a hosted service that there's work... and I'm also guessing that it works less well... always does. I cringe when I need to go to sourceforge... launchpad / canonical are beginning to show similar signs of commercial nastiness (closed source internal projects etc). I recommend staying away. -glenn -- Glenn H. Tarbox, PhD glenn at tarbox.org On 2/8/08, Fernando Perez wrote: > > On Feb 8, 2008 12:56 PM, Ville M. Vainio wrote: > > On Feb 8, 2008 9:51 PM, Brian Granger wrote: > > > I just tried to create a ipython lauchpad project, but launchpad > > > already has an ipython project listed. Does anyone know who created > > > that? It looks unused, but I can't seem to find any info about who > > > added it. It would be great to simply be able to use the existing > > > one. > > > > Perhaps it's the ipython used in ubuntu (ubuntu is a high profile > > launchpad user). Just create 'ipython1' for now, and we'll hash out > > the name of ipython trunk later on. > > > Well, it wasn't me: I'm "fperez" on launchpad, with this gmail > address, and ipython doesn't appear assigned to me. > > Furthermore, they list the source distro as a CVS checkout, and we > haven't used CVS in like 4-5 years, so this is some stale registration > information. > > I'd say go ahead for the test with an 'ipython1' project, and if it > looks good, I'll later contact the admins to get the ipython project > name reassigned to us. > > Cheers, > > > f > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ondrej at certik.cz Fri Feb 8 19:27:24 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 9 Feb 2008 01:27:24 +0100 Subject: [IPython-dev] older emails: display problems and progress bar Message-ID: <85b5c3130802081627wb18cbdfsfb7aec9c664953a0@mail.gmail.com> Hi, usually, things sort out by themselves in time, so: 1) this patch: http://lists.ipython.scipy.org/pipermail/ipython-dev/2008-January/003483.html We just included the progress bar in our FEM code here: http://code.google.com/p/sfepy/ So I am fine with not putting it to ipython. 2) display problems: http://lists.ipython.scipy.org/pipermail/ipython-dev/2008-January/003459.html http://lists.ipython.scipy.org/pipermail/ipython-dev/2008-January/003460.html In the meantime, I think we found a solution: http://code.google.com/p/sympy/issues/detail?id=568 simply we need to tweak sys.displayhook and friends. Ondrej From ondrej at certik.cz Fri Feb 8 19:28:10 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 9 Feb 2008 01:28:10 +0100 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> Message-ID: <85b5c3130802081628n5902b825wc9c755c8be01c180@mail.gmail.com> On Feb 9, 2008 12:52 AM, Glenn Tarbox, PhD wrote: > I've been a lurker for a little while now... have done some IPython hacking, > but definitely "get it" > > On the subject of this thread, Launchpad / bzr. I was concerned when I saw > IPython1 embrace hg. While git has its warts (the most glaring being its > horrendous documentation), I'd make the unsupportable claim that its > technically superior to the alternatives... > > But, technical issues aside... Git is gonna win. At the end of the day, > git is going to have a much higher adoption rate and will be a better engine > on which to progress... more tools, more conversion utilities, much bigger > community, etc. Its momentum and superior technical core will help overcome > its early documentation / porcelain problems. I completely agree, that one should either choose git, or the second most adopted VCS (which imho is hg?). But I think the people that actually do any coding should choose this, not me, who is just talking. Ondrej From vivainio at gmail.com Sat Feb 9 06:01:21 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sat, 9 Feb 2008 13:01:21 +0200 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <46cb515a0802081009n6edc8357w70e83c322d658124@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> Message-ID: <46cb515a0802090301k5472115ak98bf53a384df22f3@mail.gmail.com> On Feb 9, 2008 1:52 AM, Glenn Tarbox, PhD wrote: > You already have a nice Trac site... I realize shedding the admin load would > be nice... but I'm guessing that with a hosted service that there's work... > and I'm also guessing that it works less well... always does. I cringe > when I need to go to sourceforge... launchpad / canonical are beginning to > show similar signs of commercial nastiness (closed source internal projects > etc). The deal is that it's not uncommon that our svn is not working, due to server problems. And trac gets slow/broken sometimes. If we can have what we have now (plus a DVCS) on a well-maintained server, it's a net win. And if/when Git becomes the giant of DVCS's at some point, it's not the end of the world; we'll just import our bzr repo to git. We are not some clumsy megacorporation that can't make such a switch because of inertia. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From gael.varoquaux at normalesup.org Sat Feb 9 07:08:12 2008 From: gael.varoquaux at normalesup.org (Gael Varoquaux) Date: Sat, 9 Feb 2008 13:08:12 +0100 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> Message-ID: <20080209120812.GB14654@phare.normalesup.org> Hi, Pardon me, but you almost sound like a troll :->. Let me tell you why git is totally out of question. It works terribly poorly under windows. Up to recently it did not work at all under windows. Now it requires cygwin, and is terribly poorly integrated with the system. About Canonical reinventing the wheel with bzr. Well there might be more than one solution to one problem. Git, hg and bzr have all had their pros and their cons. The DVCS is a young field, and different players have tried different solutions. Currently the difference between hg and bzr is leveling off, as bzr is focusing on performance, and hg is focusing on merge. Anyhow, I don't want to go in another endless hg vs bzr discussion (I don't mention git for the reasons given above). Bzr works very well under windows, as one of its main developpers runs windows. I have seen hg successfully used under windows. I have used both, and I know both get the work done. I favor doing whatever gets the work done the easiest. It happens that launchpad is a very convenient (though not perfect) web platform. I use it for other projects, and I have been pleased not only by the current features, but also by the features they keep adding. These guys listen to their users. Moreover, I want to stress that a choice taken now can be revoked later. What matters is getting the work done! My 2 cents, Ga?l From ondrej at certik.cz Sat Feb 9 07:15:54 2008 From: ondrej at certik.cz (Ondrej Certik) Date: Sat, 9 Feb 2008 13:15:54 +0100 Subject: [IPython-dev] Launchpad? In-Reply-To: <20080209120812.GB14654@phare.normalesup.org> References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> <20080209120812.GB14654@phare.normalesup.org> Message-ID: <85b5c3130802090415x3b865b3ej540682a8a8e5114d@mail.gmail.com> On Feb 9, 2008 1:08 PM, Gael Varoquaux wrote: > Hi, > > Pardon me, but you almost sound like a troll :->. > > Let me tell you why git is totally out of question. It works terribly > poorly under windows. Up to recently it did not work at all under > windows. Now it requires cygwin, and is terribly poorly integrated with > the system. > > > About Canonical reinventing the wheel with bzr. Well there might be more > than one solution to one problem. Git, hg and bzr have all had their pros > and their cons. The DVCS is a young field, and different players have > tried different solutions. Currently the difference between hg and bzr is > leveling off, as bzr is focusing on performance, and hg is focusing on > merge. Anyhow, I don't want to go in another endless hg vs bzr discussion > (I don't mention git for the reasons given above). Bzr works very well > under windows, as one of its main developpers runs windows. I have > seen hg successfully used under windows. I have used both, and I know > both get the work done. > > I favor doing whatever gets the work done the easiest. It happens that > launchpad is a very convenient (though not perfect) web platform. I use > it for other projects, and I have been pleased not only by the current > features, but also by the features they keep adding. These guys listen to > their users. Moreover, I want to stress that a choice taken now can be > revoked later. What matters is getting the work done! > > My 2 cents, You are right. Ondrej From koepsell at gmail.com Sat Feb 9 13:51:23 2008 From: koepsell at gmail.com (killian koepsell) Date: Sat, 9 Feb 2008 10:51:23 -0800 Subject: [IPython-dev] IPython1 API design question In-Reply-To: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> References: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> Message-ID: On Feb 8, 2008 8:16 AM, Brian Granger wrote: > Hi, > > I have been working on IPython1 a lot this week and have run into an > API design dilemma. Because the IPython1 API will be the core aPI > that all of us develop to in the near future, I wanted to get some > opinions about the issue. Here we go... > > The IPython1 API has methods: > > push(self, **namespace) # push namespace of key, values > pairs into the users namespace > [...] > But, here is the problem we are running into. In various parts of the > API, we need to add additional arguments to these methods. The two > that have come up are: > > block -> should the call block or not? > targets -> when you are working with multiple ipython core instances, > which should you push/pull to. hi brian, what about an approach where you extract the reserved keywords manually? something like def push(self,**kargs): ns = kargs.copy() block = ns.pop('block', False) targest = ns.pop('targets', 'all') ... kilian From glenn at tarbox.org Sat Feb 9 13:58:08 2008 From: glenn at tarbox.org (Glenn Tarbox, PhD) Date: Sat, 9 Feb 2008 10:58:08 -0800 Subject: [IPython-dev] Launchpad? In-Reply-To: <85b5c3130802090415x3b865b3ej540682a8a8e5114d@mail.gmail.com> References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> <20080209120812.GB14654@phare.normalesup.org> <85b5c3130802090415x3b865b3ej540682a8a8e5114d@mail.gmail.com> Message-ID: All works for me. Didn't mean to be perceived as a troll... This was an "opinion" thread and I have one. Might even be wrong... who knows. And change can be made later... just easier to pick correctly and deliberately early on... and technical isn't the only thing that matters at some levels... I use bzr, svn, and git.... can't imagine that throwing hg into the mix will kill me. (more syntax... ugh :-) But IMHO (focus on O), this is a small group of developers with big plans... and IPython is magic and IPython1 can be truly revolutionary if successful and gets momentum... the key is momentum.... too much work, too little time. And I have no problem with Launchpad other than concerns about where they're headed (btw, I'm pretty heavily Ubuntu so I'm definitely no Canonical basher... far from it). But, its a commercial venture (hurray for success) and already making noise about being closed source on key activities (e.g. Lighthouse)... Unless I'm missing something, Ubuntu isn't a heavy Lauchpad user, they are Launchpad. So, of course, this is open source and nobody gets locked in... and change can always occur... but who has the time... so, I've put in my $.02... I'd like to participate regardless of which direction is chosen. I have a patch I'll put together which integrates twisted with IPython without polling... a slight hack starting with the ASPN code from way back. There might be a much better approach given IPython1's direction but the ASPN 1000hz polling loop didn't seem the way to go and I'm not sure I get where IPython1 is yet. I'll clean it up and send it in for opinions. -- -glenn Glenn H. Tarbox, PhD glenn at tarbox.org On 2/9/08, Ondrej Certik wrote: > > On Feb 9, 2008 1:08 PM, Gael Varoquaux > wrote: > > Hi, > > > > Pardon me, but you almost sound like a troll :->. > > > > Let me tell you why git is totally out of question. It works terribly > > poorly under windows. Up to recently it did not work at all under > > windows. Now it requires cygwin, and is terribly poorly integrated with > > the system. > > > > > > About Canonical reinventing the wheel with bzr. Well there might be more > > than one solution to one problem. Git, hg and bzr have all had their > pros > > and their cons. The DVCS is a young field, and different players have > > tried different solutions. Currently the difference between hg and bzr > is > > leveling off, as bzr is focusing on performance, and hg is focusing on > > merge. Anyhow, I don't want to go in another endless hg vs bzr > discussion > > (I don't mention git for the reasons given above). Bzr works very well > > under windows, as one of its main developpers runs windows. I have > > seen hg successfully used under windows. I have used both, and I know > > both get the work done. > > > > I favor doing whatever gets the work done the easiest. It happens that > > launchpad is a very convenient (though not perfect) web platform. I use > > it for other projects, and I have been pleased not only by the current > > features, but also by the features they keep adding. These guys listen > to > > their users. Moreover, I want to stress that a choice taken now can be > > revoked later. What matters is getting the work done! > > > > My 2 cents, > > > You are right. > > Ondrej > -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg.net at gmail.com Sat Feb 9 18:42:03 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 9 Feb 2008 16:42:03 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: <20080209120812.GB14654@phare.normalesup.org> References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081129r4682df8awa406793990857905@mail.gmail.com> <46cb515a0802081132v209035c5n9f06d5996a9f4d0b@mail.gmail.com> <6ce0ac130802081140w2bb5dc5cl5f7e4c51e9e5dab7@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> <20080209120812.GB14654@phare.normalesup.org> Message-ID: <6ce0ac130802091542q1a8f7568y2b2056fe10ad61f0@mail.gmail.com> > Let me tell you why git is totally out of question. It works terribly > poorly under windows. Up to recently it did not work at all under > windows. Now it requires cygwin, and is terribly poorly integrated with > the system. An important point is that Windows is a major platform for ipython development and users. So this is truly a show stopper for git+ipython. > About Canonical reinventing the wheel with bzr. Well there might be more > than one solution to one problem. Git, hg and bzr have all had their pros > and their cons. The DVCS is a young field, and different players have > tried different solutions. Currently the difference between hg and bzr is > leveling off, as bzr is focusing on performance, and hg is focusing on > merge. Anyhow, I don't want to go in another endless hg vs bzr discussion > (I don't mention git for the reasons given above). Bzr works very well > under windows, as one of its main developpers runs windows. I have > seen hg successfully used under windows. I have used both, and I know > both get the work done. This is very helpful feedback. I have only used hg and only a little at that. My impression is that hg+bzr are quickly converging in terms of performance and features. But it is useful to hear you have had success with both of them. > I favor doing whatever gets the work done the easiest. It happens that > launchpad is a very convenient (though not perfect) web platform. I use > it for other projects, and I have been pleased not only by the current > features, but also by the features they keep adding. These guys listen to > their users. Moreover, I want to stress that a choice taken now can be > revoked later. What matters is getting the work done! For me, I think launchpad is the most attractive from the getting work done perspective. Given the small size of our dev team, we really need somethiing that we don't have to think about and that "Just Works." > My 2 cents, > > Ga?l > > _______________________________________________ > IPython-dev mailing list > IPython-dev at scipy.org > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > From ellisonbg.net at gmail.com Sat Feb 9 18:57:28 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 9 Feb 2008 16:57:28 -0700 Subject: [IPython-dev] IPython1 API design question In-Reply-To: References: <6ce0ac130802080816u2333ba60p37ab8e8abe6b6a4f@mail.gmail.com> Message-ID: <6ce0ac130802091557r22822c8dm4d053c2871bcc009@mail.gmail.com> > what about an approach where you extract the reserved keywords manually? > something like We seriously considered this option, but rejected it for two reasons 1. IPython1 is heavily based on interfaces (as in zope.interface). Using this trick sort of makes all your interfaces to vague to be useful. 2. In most of the cases we were using **kwargs, there is nothing to prevent the user from trying to use keyword arguments for their own purposes. That is we could never know if block/targets should be interpreted as internal ipython keyword arguments or ones that should simply be passed on. This would lead to confusing wierd bugs. Brian > def push(self,**kargs): > ns = kargs.copy() > block = ns.pop('block', False) > targest = ns.pop('targets', 'all') > ... > > kilian > From glenn at tarbox.org Sat Feb 9 19:51:28 2008 From: glenn at tarbox.org (Glenn Tarbox, PhD) Date: Sat, 9 Feb 2008 16:51:28 -0800 Subject: [IPython-dev] Launchpad? In-Reply-To: <6ce0ac130802091551t48d5cdbax4df2082bb6f27591@mail.gmail.com> References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <46cb515a0802081142w2d6ebd85x2910ab69b00e5d2@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> <20080209120812.GB14654@phare.normalesup.org> <85b5c3130802090415x3b865b3ej540682a8a8e5114d@mail.gmail.com> <6ce0ac130802091551t48d5cdbax4df2082bb6f27591@mail.gmail.com> Message-ID: On 2/9/08, Brian Granger wrote: > I have a patch I'll put together which integrates twisted with IPython > > without polling... a slight hack starting with the ASPN code from way > back. > > There might be a much better approach given IPython1's direction but the > > ASPN 1000hz polling loop didn't seem the way to go and I'm not sure I > get > > where IPython1 is yet. I'll clean it up and send it in for opinions. > > > I have done some work on this front this week and finally have a nice > solution. There are two pieces: > > 1. I have created a simple class that runs the twisted reactor in the > thread. I think I took the other route... spawned the gtk / IPython thread off and kept the main thread for twisted def run(self): self.IP.mainloop() self.quitting = True self.IP.kill() def mainloop(self): # self.reactor.callLater(self.TIMEOUT, self.on_timer) def mainLoopThreadDeath(r): print "mainLoopThreadDeath: ", str(r) def spawnMainloopThread(): d=threads.deferToThread(self.run) d.addBoth(mainLoopThreadDeath) reactor.callWhenRunning(spawnMainloopThread) self.reactor.run() print "mainloop ending...." 2. The various parts of the twisted API can be called from ipython > using a function that recently made it into the twisted trunk, called > blockingCallFromThread. This function is not in twisted 2.5.0, so I > have put it into ipython1 until a new version of twisted is cut. Interesting, I was under the impression (perhaps mistakenly) that reactor.callFromThread() was blocking... actually, I think I've passed return values through it... hmmm... need to go back and check on that. This new arrangement is allowing us to develop very robust twisted > based clients that work in regular python/ipython sessions. This is a > massive improvement over anything else we have had before. > > Here are the relevant classes in ipython1: > > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/twistedutil.py > > These things are used here: > > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/multiengineclient.py > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/client.py > > I should have this branch ready to merge into ipython1's trunk in a few > days. > > Brian > > > -- > > > > > -glenn > > > > Glenn H. Tarbox, PhD > > glenn at tarbox.org > > > > > > > > On 2/9/08, Ondrej Certik wrote: > > > On Feb 9, 2008 1:08 PM, Gael Varoquaux > > wrote: > > > > Hi, > > > > > > > > Pardon me, but you almost sound like a troll :->. > > > > > > > > Let me tell you why git is totally out of question. It works > terribly > > > > poorly under windows. Up to recently it did not work at all under > > > > windows. Now it requires cygwin, and is terribly poorly integrated > with > > > > the system. > > > > > > > > > > > > About Canonical reinventing the wheel with bzr. Well there might be > more > > > > than one solution to one problem. Git, hg and bzr have all had their > > pros > > > > and their cons. The DVCS is a young field, and different players > have > > > > tried different solutions. Currently the difference between hg and > bzr > > is > > > > leveling off, as bzr is focusing on performance, and hg is focusing > on > > > > merge. Anyhow, I don't want to go in another endless hg vs bzr > > discussion > > > > (I don't mention git for the reasons given above). Bzr works very > well > > > > under windows, as one of its main developpers runs windows. I have > > > > seen hg successfully used under windows. I have used both, and I > know > > > > both get the work done. > > > > > > > > I favor doing whatever gets the work done the easiest. It happens > that > > > > launchpad is a very convenient (though not perfect) web platform. I > use > > > > it for other projects, and I have been pleased not only by the > current > > > > features, but also by the features they keep adding. These guys > listen > > to > > > > their users. Moreover, I want to stress that a choice taken now can > be > > > > revoked later. What matters is getting the work done! > > > > > > > > My 2 cents, > > > > > > > > > You are right. > > > > > > Ondrej > > > > > > > > > > > > > > > > _______________________________________________ > > IPython-dev mailing list > > IPython-dev at scipy.org > > http://lists.ipython.scipy.org/mailman/listinfo/ipython-dev > > > > > -- -glenn (**** Note: New Number ***) Glenn H. Tarbox, PhD **** 206-494-0819 *** => New Number glenn at tarbox.org -------------- next part -------------- An HTML attachment was scrubbed... URL: From ellisonbg.net at gmail.com Sat Feb 9 20:02:20 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 9 Feb 2008 18:02:20 -0700 Subject: [IPython-dev] Launchpad? In-Reply-To: References: <6ce0ac130802081100i6ab211efq54a80c60a87b60c0@mail.gmail.com> <6ce0ac130802081151s3c70b3ebn770f3fc503e4e1cf@mail.gmail.com> <46cb515a0802081156v355989c8m6e5d82f479f2bb93@mail.gmail.com> <20080209120812.GB14654@phare.normalesup.org> <85b5c3130802090415x3b865b3ej540682a8a8e5114d@mail.gmail.com> <6ce0ac130802091551t48d5cdbax4df2082bb6f27591@mail.gmail.com> Message-ID: <6ce0ac130802091702q3fe66a5ax332bcacd18cbd601@mail.gmail.com> > Interesting, I was under the impression (perhaps mistakenly) that > reactor.callFromThread() was blocking... actually, I think I've passed > return values through it... hmmm... need to go back and check on that. We are using blockCallFromThread, which is very different from callFromThread. But, blockingCallFromThread does block. It basically lets you call a twisted using/deferred returning function and block for the result. This allows you to run twisted in a thread, but create blocking interfaces to twisted using things that are in turn usable from ipython. Brian From glenn at tarbox.org Sat Feb 9 20:22:57 2008 From: glenn at tarbox.org (Glenn Tarbox, PhD) Date: Sat, 9 Feb 2008 17:22:57 -0800 Subject: [IPython-dev] do-over: Re: Launchpad? Message-ID: wierd, hit the wrong key in gmail and away we go.... do-over... the actual "intended" message is below On 2/9/08, Brian Granger wrote: > I have a patch I'll put together which integrates twisted with IPython > > without polling... a slight hack starting with the ASPN code from way > back. > > There might be a much better approach given IPython1's direction but the > > ASPN 1000hz polling loop didn't seem the way to go and I'm not sure I > get > > where IPython1 is yet. I'll clean it up and send it in for opinions. > > > I have done some work on this front this week and finally have a nice > solution. There are two pieces: > > 1. I have created a simple class that runs the twisted reactor in the > thread. I took the other route... spawned the gtk / IPython thread off and kept the main thread for twisted. Of course, its because I was starting with IPython and the ASPN code (included) 2. The various parts of the twisted API can be called from ipython > using a function that recently made it into the twisted trunk, called > blockingCallFromThread. This function is not in twisted 2.5.0, so I > have put it into ipython1 until a new version of twisted is cut. Interesting, I was under the impression (perhaps mistakenly) that reactor.callFromThread() was blocking... actually, I think I've passed return values through it... hmmm... need to go back and check on that. Brian then writes before I can resend this message ""We are using blockCallFromThread, which is very different from callFromThread. But, blockingCallFromThread does block. It basically lets you call a twisted using/deferred returning function and block for the result. This allows you to run twisted in a thread, but create blocking interfaces to twisted using things that are in turn usable from ipython."" OK, so now I need to read the code. Might even to do some thinking... I've included what I did cuz its trivial and only required a couplea lines to Shell... and a slight mod to the ASPN code (although I don't fully understand why a bunch of the ASPN stuff was necessary... didn't spend the time). So, I'll do so. I was inverting the approach and intending to only call into twisted through the proper protections from the gui... I think you're taking a more general approach which I'll need to noodle on a bit. - Show quoted text - This new arrangement is allowing us to develop very robust twisted > based clients that work in regular python/ipython sessions. This is a > massive improvement over anything else we have had before. > > Here are the relevant classes in ipython1: > > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/twistedutil.py > > These things are used here: > > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/multiengineclient.py > > http://ipython.scipy.org/ipython/ipython/browser/ipython1/branches/ipython1-client-r3021/ipython1/kernel/client.py > > I should have this branch ready to merge into ipython1's trunk in a few > days. > > Brian > > > -- > > > > > -glenn > > > > Glenn H. Tarbox, PhD > > glenn at tarbox.org > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: IPython.diff Type: text/x-patch Size: 1722 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: twistedIron.py Type: text/x-python Size: 3404 bytes Desc: not available URL: From ellisonbg.net at gmail.com Mon Feb 11 12:04:52 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Mon, 11 Feb 2008 10:04:52 -0700 Subject: [IPython-dev] IPython launchpad project is up Message-ID: <6ce0ac130802110904n77e985eap29337789676883fc@mail.gmail.com> Hi, For those interested, we now have an official ipython launchpad project site. Please feel free to use this to play around with launchpad: https://launchpad.net/ipython IMPORTANT: We have not made a formal decision to use launchpad+bzr for ipython development. A few of us are simply evaluating it for now. But, if you want to help evaluate it, please join us in trying it out. Brian From fperez.net at gmail.com Mon Feb 11 12:10:18 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 11 Feb 2008 10:10:18 -0700 Subject: [IPython-dev] IPython launchpad project is up In-Reply-To: <6ce0ac130802110904n77e985eap29337789676883fc@mail.gmail.com> References: <6ce0ac130802110904n77e985eap29337789676883fc@mail.gmail.com> Message-ID: On Feb 11, 2008 10:04 AM, Brian Granger wrote: > Hi, > > For those interested, we now have an official ipython launchpad > project site. Please feel free to use this to play around with > launchpad: > > https://launchpad.net/ipython Excellent, thanks! Cheers f From fperez.net at gmail.com Mon Feb 11 12:12:11 2008 From: fperez.net at gmail.com (Fernando Perez) Date: Mon, 11 Feb 2008 10:12:11 -0700 Subject: [IPython-dev] Mostly Offline for ~10 days. Message-ID: Hi all, just a quick heads-up that I'll be mostly offline for the next 10 days or so. If anything related to the project is urgently needed, Brian also has SSH access to the server, or contact me at Fernando.Perez at colorado.edu, since I'm unlikely to monitor this address much until I settle back down. Cheers, f From jorgen.stenarson at bostream.nu Wed Feb 13 14:57:23 2008 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Wed, 13 Feb 2008 20:57:23 +0100 Subject: [IPython-dev] Ticket #228 Message-ID: <47B34BA3.90001@bostream.nu> The ticket #228 has maintainer as reporter I'm not sure who that is but I'm hoping I can get some more information on the problem from him on list. In particular exactly how ipython was started, which ssh was used, is it running in a console window or somewhere else. Is python executing on the remote machine? My first guess is that if python is running on another through a SSH connection then pyreadline just won't work because it relies on getting a handle to the console that you are typing into and I would be surprised if that is transported over SSH. It would be good to figure out a way to detect if python was started this way so we can disable the call to pyreadline. Looking at the crash_report it looks like pyreadline_codepage is None. This may be the clue we need to decide that ipython was started through ssh. However it may be too late since pyreadline has already been imported and installed by the time we get this crash. Since by default pyreadline_codepage comes from sys.stdout.encoding we would just have to check that sys.stdout.encoding is not None before importing pyreadline. /J?rgen From vivainio at gmail.com Wed Feb 13 15:51:56 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Wed, 13 Feb 2008 22:51:56 +0200 Subject: [IPython-dev] Custom sentinels Message-ID: <46cb515a0802131251o40a98a9cyd93f21c867894c4b@mail.gmail.com> This is of interest to both leo-editor and ipython-dev, I think... I'm trying to use @others to run many ipython scripts in child nodes. However, there is a problem. _ip.runlines() expects "normal" IPython input, which means all top-level blocks need to be terminated with 2 empty lines (for IPython to know they are to be executed). e.g. if 1: print "hello" How can I cause those 2 empty lines to appear between every collected (tangled?) child node, when I'm using g.getscript()? If that is not easily doable, I can hack it into _ip.runlines() as well, i.e. have the whitespace handling in runlines() to be this: - First, remove all empty lines - Then, before every line with indent level 0 that is preceded by a line with higher indent level, add 2 empty lines. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From edreamleo at gmail.com Wed Feb 13 18:08:44 2008 From: edreamleo at gmail.com (Edward K. Ream) Date: Wed, 13 Feb 2008 17:08:44 -0600 Subject: [IPython-dev] Custom sentinels In-Reply-To: <46cb515a0802131251o40a98a9cyd93f21c867894c4b@mail.gmail.com> References: <46cb515a0802131251o40a98a9cyd93f21c867894c4b@mail.gmail.com> Message-ID: > How can I cause those 2 empty lines to appear between every collected (tangled?) child node, when I'm using g.getScript()? It's easily done, with some minor changes to Leo's core. g.getScript computes the script as follows: script = at.writeFromString( p.copy(),s, forcePythonSentinels=forcePythonSentinels, useSentinels=useSentinels) So we'll just add an isIPython keyword argument to both g.getScript and at.writeFromString. at.writeFromString (or more likely, one of its helpers) will then insert the extra lines as needed. As you can see, at.writeFromString already has several options that were put there to support scripting. Adding one more will be harmless. BTW, all these hacks could be done in a Leo plugin, because plugins can override any method in Leo's core. But there is no reason to do that--it's easier just make the changes directly. Edward -------------------------------------------------------------------- Edward K. Ream email: edreamleo at gmail.com Leo: http://webpages.charter.net/edreamleo/front.html -------------------------------------------------------------------- -------------- next part -------------- An HTML attachment was scrubbed... URL: From vivainio at gmail.com Sat Feb 16 19:14:43 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Sun, 17 Feb 2008 02:14:43 +0200 Subject: [IPython-dev] IPython + bzr Message-ID: <46cb515a0802161614q7b31da95h63916d0d2bb04871@mail.gmail.com> I just added IPython to launchpad bzr. The svn import is "processing", but it seems to take forever (due to some problems?). However, I now merely added the current IPython trunk w/o history, and it might be interesting experiment to make IPython 0.8.3 the first release based on bzr. I don't think having the history in bzr is all that important (at this point), it's accessible in svn anyway. If things get botched up, we just add all the code to svn normally. The branch is at https://code.launchpad.net/~ipython/ipython/stable-dev To be able to push there, you need to set up some ssh stuff; for windows, see this video: http://showmedo.com/videos/video?name=1510070&fromSeriesID=151 I have also created the IPython team, so if you want to be able to push there please register a launchpad account and join the team. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Sun Feb 17 17:03:01 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Mon, 18 Feb 2008 00:03:01 +0200 Subject: [IPython-dev] Fwd: [Question #24416]: Please import IPython svn repo to bzr In-Reply-To: <8771831223698744947@unknownmsgid> References: <20080212213442.12268.28424.lpquestions@gandwana.ubuntu.com> <8771831223698744947@unknownmsgid> Message-ID: <46cb515a0802171403n2da1466i849cb04410e63d28@mail.gmail.com> This is why IPython svn import to launchpad is failing... ---------- Forwarded message ---------- From: Michael Hudson Date: Feb 17, 2008 11:54 PM Subject: Re: [Question #24416]: Please import IPython svn repo to bzr To: vivainio at gmail.com Your question #24416 on Launchpad Bazaar Integration changed: https://answers.edge.launchpad.net/launchpad-bazaar/+question/24416 Status: Open => Answered Michael Hudson proposed the following answer: The import process is a bit naive in it's use of the network, and for an initial import requires that a great deal of connections succeed without any issues, and it seems that scipy.org's connection to the network is just a bit too flaky for this to succeed. If you can get a repository dump, we can do the import from that and then just track the incremental updates from the real server (which is much more likely to succeed). Otherwise, we can just keep trying and hope we get a success eventually... -- If this answers your question, please go to the following page to let us know that it is solved: https://answers.edge.launchpad.net/launchpad-bazaar/+question/24416/+confirm?answer_id=2 If you still need help, you can reply to this email or go to the following page to enter your feedback: https://answers.edge.launchpad.net/launchpad-bazaar/+question/24416 You received this question notification because you are a direct subscriber of the question. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Tue Feb 19 12:46:18 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Tue, 19 Feb 2008 19:46:18 +0200 Subject: [IPython-dev] About IPython and bzr Message-ID: <46cb515a0802190946j176ba5aeh5200a989eeef7722@mail.gmail.com> My recently-from-scratch installed IPython stable-dev (what used to be called trunk/) is now very easy to check out from launchpad: bzr branch lp:ipython One note though: NEVER CHECK IN SYMLINKS. I can't stress this enough. Windows bzr does not support symlinks (even in a half-assed way like svn), at least not yet, and I don't know whether such a checkin can be reverted. So essentially, if there is even one checkin with symlink, I would need to delete the branch and check it in again from local copy. -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Wed Feb 20 17:23:22 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 21 Feb 2008 00:23:22 +0200 Subject: [IPython-dev] Ticket #228 In-Reply-To: <47B34BA3.90001@bostream.nu> References: <47B34BA3.90001@bostream.nu> Message-ID: <46cb515a0802201423i8773813mb47156cc92285ca9@mail.gmail.com> On Wed, Feb 13, 2008 at 9:57 PM, J?rgen Stenarson wrote: > Looking at the crash_report it looks like pyreadline_codepage is None. > This may be the clue we need to decide that ipython was started through > ssh. However it may be too late since pyreadline has already been > imported and installed by the time we get this crash. Since by default > pyreadline_codepage comes from sys.stdout.encoding we would just have to > check that sys.stdout.encoding is not None before importing pyreadline. Why not just set encoding to ascii if it appears to be None? Either way, we must prevent this crash. BTW, how about setting pyreadline up at Launchpad as well? -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From ellisonbg.net at gmail.com Wed Feb 20 17:52:47 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Wed, 20 Feb 2008 15:52:47 -0700 Subject: [IPython-dev] IPython1 versioning conventions Message-ID: <6ce0ac130802201452u6b13cccex383ff60d5dbe0565@mail.gmail.com> Hi, I am getting ready to release a new version of IPython1 and I have a question about how we are versioning it. The previous version of IPython1 was listed as 0.9alpha3. I am not sure this makes sense and it is probably a good time to decide upon a versioning scheme for IPython1 before everyone jump into the code. So... How should we version IPython1? Should it have versions that: 1) Are somehow related to those of trunk IPython? (the current scheme suggests something like that). 2) Are completely independent? My initial thought is that it might be nice to pick version numbers that give us a little more room before hitting 1.0. We could jump back and call the next release of ipython1 0.1. Otherwise I fear we will quickly see version numbers like 0.9.4.35. I think our current thought is that when IPython1 reaches version 1.0, it would become a full replacement for IPython trunk (which would itself disappear). But that is still a ways off. Thoughts? Brian From vivainio at gmail.com Wed Feb 20 18:13:43 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 21 Feb 2008 01:13:43 +0200 Subject: [IPython-dev] IPython1 versioning conventions In-Reply-To: <6ce0ac130802201452u6b13cccex383ff60d5dbe0565@mail.gmail.com> References: <6ce0ac130802201452u6b13cccex383ff60d5dbe0565@mail.gmail.com> Message-ID: <46cb515a0802201513vb1d87c1nc17b9914bc4e2620@mail.gmail.com> On Thu, Feb 21, 2008 at 12:52 AM, Brian Granger wrote: > How should we version IPython1? Should it have versions that: > > 1) Are somehow related to those of trunk IPython? (the current scheme > suggests something like that). > 2) Are completely independent? +1 for completely independent, the relationship hardly exist at this point (it has not started to swallow the trunk code yet). > I think our current thought is that when IPython1 reaches version 1.0, > it would become a full replacement for IPython trunk (which would > itself disappear). But that is still a ways off. Yes, it is. Try something like 0.3, and we can start upping it when it's starting to close in at feature parity with trunk. How about creating that ipython1 project in launchpad now? I don't think we should have those branches in IPython project, it gets kinda messy as the code bases are not related (yet). -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From vivainio at gmail.com Thu Feb 21 03:01:03 2008 From: vivainio at gmail.com (Ville M. Vainio) Date: Thu, 21 Feb 2008 10:01:03 +0200 Subject: [IPython-dev] IPython1 versioning conventions In-Reply-To: <6ce0ac130802201554oc22ded3le78dddb9fe35f84d@mail.gmail.com> References: <6ce0ac130802201452u6b13cccex383ff60d5dbe0565@mail.gmail.com> <46cb515a0802201513vb1d87c1nc17b9914bc4e2620@mail.gmail.com> <6ce0ac130802201554oc22ded3le78dddb9fe35f84d@mail.gmail.com> Message-ID: <46cb515a0802210001o5bd3cbf4hd1dfb4cb9908e4eb@mail.gmail.com> Fwd to list. On Thu, Feb 21, 2008 at 1:54 AM, Brian Granger wrote: > On Wed, Feb 20, 2008 at 4:13 PM, Ville M. Vainio wrote: > > On Thu, Feb 21, 2008 at 12:52 AM, Brian Granger wrote: > > > > > How should we version IPython1? Should it have versions that: > > > > > > 1) Are somehow related to those of trunk IPython? (the current scheme > > > suggests something like that). > > > 2) Are completely independent? > > > > +1 for completely independent, the relationship hardly exist at this > > point (it has not started to swallow the trunk code yet). > > OK, I think this is a good idea. > > > > > > > I think our current thought is that when IPython1 reaches version 1.0, > > > it would become a full replacement for IPython trunk (which would > > > itself disappear). But that is still a ways off. > > > > Yes, it is. Try something like 0.3, and we can start upping it when > > it's starting to close in at feature parity with trunk. > > This sounds good to me. 0.1/0.2 would suggest that IPython1 was brand > new, which isn't true. > > > > How about creating that ipython1 project in launchpad now? I don't > > think we should have those branches in IPython project, it gets kinda > > messy as the code bases are not related (yet). > > What branches. You probably mean the old saw/chainsaw branches. > Those are long dead, and for a while now we have been working from a > top-level ipython1 repo that has its own branches/tags: > > http://ipython.scipy.org/ipython/ipython/browser/ipython1 > > But, I am getting close to creating an ipython1 lp branch. My plans > are as follows: > > * Finish updating the documentation to reflect the recent changes I > have been making > * Merge back into the ipython1 trunk. > * update the version of ipython1 to 0.3 > * Create a launchpad branch for ipython1 > * Make an announcement to ipython-dev about this. > > In other news, I have a good start of setting up a really nice way of > using RST files for documentation. I am almost done converting the > ipython1 docs to this format. This should make things much easier for > all of us. > More details to follow. Excellent! I have been waiting for this, and can start fixing the docs for ipython-stable as well... -- Ville M. Vainio - vivainio.googlepages.com blog=360.yahoo.com/villevainio - g[mail | talk]='vivainio' From steve at shrogers.com Thu Feb 21 08:10:49 2008 From: steve at shrogers.com (Steven H. Rogers) Date: Thu, 21 Feb 2008 06:10:49 -0700 Subject: [IPython-dev] IPython1 versioning conventions In-Reply-To: <46cb515a0802210001o5bd3cbf4hd1dfb4cb9908e4eb@mail.gmail.com> References: <6ce0ac130802201452u6b13cccex383ff60d5dbe0565@mail.gmail.com> <46cb515a0802201513vb1d87c1nc17b9914bc4e2620@mail.gmail.com> <6ce0ac130802201554oc22ded3le78dddb9fe35f84d@mail.gmail.com> <46cb515a0802210001o5bd3cbf4hd1dfb4cb9908e4eb@mail.gmail.com> Message-ID: <47BD7859.3030005@shrogers.com> Ville M. Vainio wrote: >> > Yes, it is. Try something like 0.3, and we can start upping it when >> > it's starting to close in at feature parity with trunk. >> >> +1 for independent versioning and 0.3 is probably a good place to start, maybe reaching 0.9 with IPython feature parity. # Steve From jorgen.stenarson at bostream.nu Thu Feb 21 12:53:26 2008 From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=) Date: Thu, 21 Feb 2008 18:53:26 +0100 Subject: [IPython-dev] Ticket #228 In-Reply-To: <46cb515a0802201423i8773813mb47156cc92285ca9@mail.gmail.com> References: <47B34BA3.90001@bostream.nu> <46cb515a0802201423i8773813mb47156cc92285ca9@mail.gmail.com> Message-ID: <47BDBA96.6070109@bostream.nu> Ville M. Vainio skrev: > On Wed, Feb 13, 2008 at 9:57 PM, J?rgen Stenarson > wrote: > >> Looking at the crash_report it looks like pyreadline_codepage is None. >> This may be the clue we need to decide that ipython was started through >> ssh. However it may be too late since pyreadline has already been >> imported and installed by the time we get this crash. Since by default >> pyreadline_codepage comes from sys.stdout.encoding we would just have to >> check that sys.stdout.encoding is not None before importing pyreadline. > > Why not just set encoding to ascii if it appears to be None? Either > way, we must prevent this crash. > There is a fix in trunk following this approach. However I'm not sure it fixes the whole problem. Since I don't have access to a windows ssh server with python and pyreadline installed I'm unable to test if it works. Any report on if it works/fails is appreciated. /J?rgen From ellisonbg.net at gmail.com Sat Feb 23 23:39:29 2008 From: ellisonbg.net at gmail.com (Brian Granger) Date: Sat, 23 Feb 2008 21:39:29 -0700 Subject: [IPython-dev] Help testing new version of ipython1 Message-ID: <6ce0ac130802232039o5ace05f4tcfa230d868f7e05@mail.gmail.com> Hi all, Over the last few weeks I have been beating on ipython1 to get it ready for everyone to join the fun. I just merged my branch with the ipython1 trunk: svn co http://ipython.scipy.org/svn/ipython/ipython1/trunk ipython1 Here are some of the highlights: * Completely reorganized documentation. See docs and run gen_html.py to create html docs from rst. * Lots or cleaning up and reorganization * Changed mostly everything to the official ipython1 coding convention of lowercase_with_underscores * Created a development guidelines doc for ipython1 development (see in docs/development.txt). * Massive refactoring of the parallel computing parts of ipython1. * Much better tests. Run "trial ipython1" to see if they pass :-) * Fully up-to-date docs of how to use ipython1 for parallel computing. I would love help testing this new version before I make it public: * Download install and run "trial ipython1" to test it * If you are using ipython1 for parallel computing, check out the new docs and examples and they try it out for yourself. * Look over the new developer guidelines in docs/development.txt Here is my plan: * Wait a few days for people to try this out and test it. * Meanwhile I am going to put together a more public announcement that includes a detailed list of API changes (there were lots). * Upload the new html docs to the website. * Release ipython1 0.3 and then move development of it over to launchpad (on a trial basis). Starting next Friday, a number of us (including Fernando) will be at a sprint, so my goal is to have all of this done by then. At that point, we should be ready for everyone to get organized and begin hacking on ipython1 with a furry. Cheers, Brian