From peter.skomoroch at gmail.com  Tue Jun  2 15:58:37 2009
From: peter.skomoroch at gmail.com (Peter Skomoroch)
Date: Tue, 2 Jun 2009 15:58:37 -0400
Subject: [IPython-dev] Ubuntu 9.04 Jaunty on EC2 w/openmpi,
	parallel ipython errors
Message-ID: <e4fc0d2a0906021258p664fe3cbq601d2c006de69110@mail.gmail.com>

When I try to run the parallel MPI example from the docs, I get:

ValueError: not a furl or a file containing a furl: /root/.ipython/security/
ipcontroller-engine.furl

This is on Ubuntu 9.04 Jaunty, using the following Amazon EC2 AMI with
openmpi:

http://developer.amazonwebservices.com/connect/entry.jspa?externalID=1952&categoryID=101

I'm able to run standard C MPI jobs without any issues,

$ ec2-run-instances -k gsg-keypair ami-bf5eb9d6

ssh in and run the following installs...

apt-get -y update
apt-get -y upgrade
apt-get -y install libopenmpi1 openmpi-bin openmpi-common libopenmpi-dev
apt-get -y install python-setuptools python-docutils
apt-get -y install python-support python-distutils-extra
apt-get -y install python-dev python-numpy python-numpy-ext python-scipy
cython
apt-get -y install ipython
easy_install mpi4py

ipython (to initialize the env, then ctl-D to exit)

mpirun -n 2 ipengine --mpi=mpi4py

/var/lib/python-support/python2.6/IPython/Magic.py:38: DeprecationWarning:
the sets module is deprecated
  from sets import Set
/usr/lib/python2.6/dist-packages/foolscap/vocab.py:2: DeprecationWarning:
the sha module is deprecated; use the hashlib module instead
  import sha
/var/lib/python-support/python2.6/IPython/Magic.py:38: DeprecationWarning:
the sets module is deprecated
  from sets import Set
/usr/lib/python2.6/dist-packages/foolscap/vocab.py:2: DeprecationWarning:
the sha module is deprecated; use the hashlib module instead
  import sha
2009-05-29 16:23:04+0000 [-] Log opened.
2009-05-29 16:23:04+0000 [-] MPI started with rank = 0 and size = 2
2009-05-29 16:23:04+0000 [-] Using furl file:
/root/.ipython/security/ipcontroller-engine.furl
2009-05-29 16:23:04+0000 [-] Traceback (most recent call last):
2009-05-29 16:23:04+0000 [-]   File "/usr/bin/ipengine", line 19, in
<module>
2009-05-29 16:23:04+0000 [-]     ipengine.main()
2009-05-29 16:23:04+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/scripts/ipengine.py", line
168, in main
2009-05-29 16:23:04+0000 [-]     start_engine()
2009-05-29 16:23:04+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/scripts/ipengine.py", line
109, in start_engine
2009-05-29 16:23:04+0000 [-]     d =
engine_connector.connect_to_controller(engine_service, furl_file)
2009-05-29 16:23:04+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/engineconnector.py", line
65, in connect_to_controller
2009-05-29 16:23:04+0000 [-]     self.furl = find_furl(furl_or_file)
2009-05-29 16:23:04+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/fcutil.py", line 56, in
find_furl
2009-05-29 16:23:04+0000 [-]     raise ValueError("not a furl or a file
containing a furl: %s" % furl_or_file)
2009-05-29 16:23:04+0000 [-] ValueError: not a furl or a file containing a
furl: /root/.ipython/security/ipcontroller-engine.furl
2009-05-29 16:23:04+0000 [-] Log opened.
2009-05-29 16:23:04+0000 [-] MPI started with rank = 1 and size = 2
2009-05-29 16:23:05+0000 [-] Using furl file:
/root/.ipython/security/ipcontroller-engine.furl
2009-05-29 16:23:05+0000 [-] Traceback (most recent call last):
2009-05-29 16:23:05+0000 [-]   File "/usr/bin/ipengine", line 19, in
<module>
2009-05-29 16:23:05+0000 [-]     ipengine.main()
2009-05-29 16:23:05+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/scripts/ipengine.py", line
168, in main
2009-05-29 16:23:05+0000 [-]     start_engine()
2009-05-29 16:23:05+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/scripts/ipengine.py", line
109, in start_engine
2009-05-29 16:23:05+0000 [-]     d =
engine_connector.connect_to_controller(engine_service, furl_file)
2009-05-29 16:23:05+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/engineconnector.py", line
65, in connect_to_controller
2009-05-29 16:23:05+0000 [-]     self.furl = find_furl(furl_or_file)
2009-05-29 16:23:05+0000 [-]   File
"/var/lib/python-support/python2.6/IPython/kernel/fcutil.py", line 56, in
find_furl
2009-05-29 16:23:05+0000 [-]     raise ValueError("not a furl or a file
containing a furl: %s" % furl_or_file)
2009-05-29 16:23:05+0000 [-] ValueError: not a furl or a file containing a
furl: /root/.ipython/security/ipcontroller-engine.furl


-- 
Peter N. Skomoroch
617.285.8348
http://www.datawrangling.com
http://delicious.com/pskomoroch
http://twitter.com/peteskomoroch
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090602/d6b99af4/attachment.html>

From ondrej at certik.cz  Tue Jun 23 00:21:40 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 22 Jun 2009 22:21:40 -0600
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
Message-ID: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>

Hi,

with the attached patch you can now do:

----
a = {3: 4}

def something():
    b = 45
    import IPython
    IPython.set_trace()

something()
----

and after you run it:

-----------
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: a
Out[1]: {3: 4}

In [2]: b
Out[2]: 45
-------


e.g. you can access locals() and globals() immediately. I use this
very often lately, as it is very handy to debug some broken place in
the code. Another usage is that I need to implement some difficult
operation with numpy arrays, dicts, etc. usually some oneliner, but I
am not sure about the API, so I just call IPython.set_trace(), play
with the things to get what I want, then just copy & paste the
oneliner to the code.

More info in the docstring:

In [2]: IPython.set_trace?
Type:		function
Base Class:	<type 'function'>
String Form:	<function set_trace at 0x7f31ca8b3140>
Namespace:	Interactive
File:		/home/ondrej/usr/lib/python/IPython/__init__.py
Definition:	IPython.set_trace()
Docstring:
    Starts the IPython shell inplace.

    Example:

    >> import IPython
    >> IPython.set_trace()

    The user namespace is set to the union of globals() and locals(), e.g.
    calling set_trace() is exactly equivalent to:

        import IPython
        IPython.Shell.IPShell(user_ns=dict(globals(), **locals())).mainloop()

    but the actual implementation of set_trace() needs to use the inspect
    module, because it needs to access the parents frame.



Let me know if the implementation is ok, or if some more work is needed.

Thanks,
Ondrej
-------------- next part --------------
A non-text attachment was scrubbed...
Name: set_trace.patch
Type: text/x-diff
Size: 1026 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090622/acf9a7ac/attachment.patch>

From gokhansever at gmail.com  Tue Jun 23 01:08:44 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 00:08:44 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
Message-ID: <49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>

On Mon, Jun 22, 2009 at 11:21 PM, Ondrej Certik <ondrej at certik.cz> wrote:

> Hi,
>
> with the attached patch you can now do:
>
> ----
> a = {3: 4}
>
> def something():
>    b = 45
>    import IPython
>    IPython.set_trace()
>
> something()
> ----
>
> and after you run it:
>
> -----------
> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: a
> Out[1]: {3: 4}
>
> In [2]: b
> Out[2]: 45
> -------
>
>
> e.g. you can access locals() and globals() immediately. I use this
> very often lately, as it is very handy to debug some broken place in
> the code. Another usage is that I need to implement some difficult
> operation with numpy arrays, dicts, etc. usually some oneliner, but I
> am not sure about the API, so I just call IPython.set_trace(), play
> with the things to get what I want, then just copy & paste the
> oneliner to the code.
>
> More info in the docstring:
>
> In [2]: IPython.set_trace?
> Type:           function
> Base Class:     <type 'function'>
> String Form:    <function set_trace at 0x7f31ca8b3140>
> Namespace:      Interactive
> File:           /home/ondrej/usr/lib/python/IPython/__init__.py
> Definition:     IPython.set_trace()
> Docstring:
>    Starts the IPython shell inplace.
>
>    Example:
>
>    >> import IPython
>    >> IPython.set_trace()
>
>    The user namespace is set to the union of globals() and locals(), e.g.
>    calling set_trace() is exactly equivalent to:
>
>        import IPython
>        IPython.Shell.IPShell(user_ns=dict(globals(),
> **locals())).mainloop()
>
>    but the actual implementation of set_trace() needs to use the inspect
>    module, because it needs to access the parents frame.
>
>
>
> Let me know if the implementation is ok, or if some more work is needed.
>
> Thanks,
> Ondrej
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>
>
Ondrej,

I am testing the patch. See my comments below:

[gsever at ccn ~]$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> b=25
>>> import numpy as np
>>> arr = np.arange(10)
>>> import IPython
>>> IPython.set_trace()
 Logging to /home/gsever/.ipython/2009-06-23.py

Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.bzr.r1174 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: whos   ###<--- Nothing is listed :(
Interactive namespace is empty.

In [2]: b
Out[2]: 25

In [4]: arr
Out[4]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])

In [5]: exit()    ###<--- it exits but act as if it in the IPython session
:(
Do you really want to exit ([y]/n)? y
>>> data[0]

### and can't completely exit, stays in a never-ending loop :(

Other than these complaints, a very nice addition indeed :)

Could you please tell whether IPython instantiation could be expanded to a
pylab enabled session? (A previously asked question, but just wondering...)

G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/36ade372/attachment.html>

From ondrej at certik.cz  Tue Jun 23 01:18:56 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 22 Jun 2009 23:18:56 -0600
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
Message-ID: <85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>

On Mon, Jun 22, 2009 at 11:08 PM, G?khan SEVER<gokhansever at gmail.com> wrote:
> On Mon, Jun 22, 2009 at 11:21 PM, Ondrej Certik <ondrej at certik.cz> wrote:
>>
>> Hi,
>>
>> with the attached patch you can now do:
>>
>> ----
>> a = {3: 4}
>>
>> def something():
>> ? ?b = 45
>> ? ?import IPython
>> ? ?IPython.set_trace()
>>
>> something()
>> ----
>>
>> and after you run it:
>>
>> -----------
>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
>> ? ? ? ? ? -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help ? ? ?-> Python's own help system.
>> object? ? -> Details about 'object'. ?object also works, ?? prints more.
>>
>> In [1]: a
>> Out[1]: {3: 4}
>>
>> In [2]: b
>> Out[2]: 45
>> -------
>>
>>
>> e.g. you can access locals() and globals() immediately. I use this
>> very often lately, as it is very handy to debug some broken place in
>> the code. Another usage is that I need to implement some difficult
>> operation with numpy arrays, dicts, etc. usually some oneliner, but I
>> am not sure about the API, so I just call IPython.set_trace(), play
>> with the things to get what I want, then just copy & paste the
>> oneliner to the code.
>>
>> More info in the docstring:
>>
>> In [2]: IPython.set_trace?
>> Type: ? ? ? ? ? function
>> Base Class: ? ? <type 'function'>
>> String Form: ? ?<function set_trace at 0x7f31ca8b3140>
>> Namespace: ? ? ?Interactive
>> File: ? ? ? ? ? /home/ondrej/usr/lib/python/IPython/__init__.py
>> Definition: ? ? IPython.set_trace()
>> Docstring:
>> ? ?Starts the IPython shell inplace.
>>
>> ? ?Example:
>>
>> ? ?>> import IPython
>> ? ?>> IPython.set_trace()
>>
>> ? ?The user namespace is set to the union of globals() and locals(), e.g.
>> ? ?calling set_trace() is exactly equivalent to:
>>
>> ? ? ? ?import IPython
>> ? ? ? ?IPython.Shell.IPShell(user_ns=dict(globals(),
>> **locals())).mainloop()
>>
>> ? ?but the actual implementation of set_trace() needs to use the inspect
>> ? ?module, because it needs to access the parents frame.
>>
>>
>>
>> Let me know if the implementation is ok, or if some more work is needed.
>>
>> Thanks,
>> Ondrej
>>
>> _______________________________________________
>> IPython-dev mailing list
>> IPython-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/ipython-dev
>>
>
> Ondrej,
>
> I am testing the patch. See my comments below:
>
> [gsever at ccn ~]$ python
> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
> [GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> b=25
>>>> import numpy as np
>>>> arr = np.arange(10)
>>>> import IPython
>>>> IPython.set_trace()
> ?Logging to /home/gsever/.ipython/2009-06-23.py
>
> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.10.bzr.r1174 -- An enhanced Interactive Python.
> ????????? -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help????? -> Python's own help system.
> object??? -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: whos?? ###<--- Nothing is listed :(
> Interactive namespace is empty.
>
> In [2]: b
> Out[2]: 25
>
> In [4]: arr
> Out[4]: array([0, 1, 2, 3, 4, 5, 6, 7, 8, 9])
>
> In [5]: exit()??? ###<--- it exits but act as if it in the IPython session
> :(
> Do you really want to exit ([y]/n)? y
>>>> data[0]
>
> ### and can't completely exit, stays in a never-ending loop :(

I don't exactly understand what is wrong with your session? It seems
to me it does exactly what I want --- it starts an ipython session
inplace and allows you to easily manipulate your namespace. As to
exiting, this works for me flawlessly:

$ python
>>> a = 43
>>> import IPython
>>> IPython.set_trace()
Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: a
Out[1]: 43

In [2]:
>>>


Which python version and which system are you using? I am on ubuntu
jaunty, the latest ipython and the default python2.6.

>
> Other than these complaints, a very nice addition indeed :)

Thanks. :)

>
> Could you please tell whether IPython instantiation could be expanded to a
> pylab enabled session? (A previously asked question, but just wondering...)

Right --- I don't know. As I said, try to see how the --pylab command
is passed to ipython and then trigger it in the code.

Ondrej


From gokhansever at gmail.com  Tue Jun 23 01:32:31 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 00:32:31 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
Message-ID: <49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>

>
> I don't exactly understand what is wrong with your session? It seems
> to me it does exactly what I want --- it starts an ipython session
> inplace and allows you to easily manipulate your namespace. As to
> exiting, this works for me flawlessly:
>
> $ python
> >>> a = 43
> >>> import IPython
> >>> IPython.set_trace()
> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
> ?         -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help      -> Python's own help system.
> object?   -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: a
> Out[1]: 43
>
> In [2]:
> >>>
>

Could you do a whos when you are in the IPython? It says to me the
interactive namespace is empty, which is not right in this case. Later, when
you are out of the IP try up arrow keys, whether you are moving through your
existing IPython shell or Python. Mine acts as if I were still in IPython...


>
>
> Which python version and which system are you using? I am on ubuntu
> jaunty, the latest ipython and the default python2.6.


Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
IPython 0.10.bzr.r1174 (pulled latest trunk just a few days ago)
Linux ccn 2.6.27.19-170.2.35.fc10.i686.PAE (Fedora 10)



>
>
> >
> > Other than these complaints, a very nice addition indeed :)
>
> Thanks. :)
>
> >
> > Could you please tell whether IPython instantiation could be expanded to
> a
> > pylab enabled session? (A previously asked question, but just
> wondering...)
>
> Right --- I don't know. As I said, try to see how the --pylab command
> is passed to ipython and then trigger it in the code.
>

Still trying to figure this out. I wish I knew how to easily debug python
scripts. Is there a way to setup on a specific line? I am so used to use
this method while coding for small microprocessors. There should definetely
be more advanced feature for Python.

Thanks

gs
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/6a7c0a78/attachment.html>

From ondrej at certik.cz  Tue Jun 23 01:52:06 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 22 Jun 2009 23:52:06 -0600
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
Message-ID: <85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>

On Mon, Jun 22, 2009 at 11:32 PM, G?khan SEVER<gokhansever at gmail.com> wrote:
>> I don't exactly understand what is wrong with your session? It seems
>> to me it does exactly what I want --- it starts an ipython session
>> inplace and allows you to easily manipulate your namespace. As to
>> exiting, this works for me flawlessly:
>>
>> $ python
>> >>> a = 43
>> >>> import IPython
>> >>> IPython.set_trace()
>> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
>> ? ? ? ? ? -> Introduction and overview of IPython's features.
>> %quickref -> Quick reference.
>> help ? ? ?-> Python's own help system.
>> object? ? -> Details about 'object'. ?object also works, ?? prints more.
>>
>> In [1]: a
>> Out[1]: 43
>>
>> In [2]:
>> >>>
>
> Could you do a whos when you are in the IPython? It says to me the
> interactive namespace is empty, which is not right in this case. Later, when

Right, it's empty in my case too. But otherwise things work.

> you are out of the IP try up arrow keys, whether you are moving through your
> existing IPython shell or Python. Mine acts as if I were still in IPython...


Right, the up/down arrows cycle through the commands that were in
ipython. It seems that ipython somehow rewrote the python's history.
But you are *not* running in ipython after exiting it.

>
>>
>> Which python version and which system are you using? I am on ubuntu
>> jaunty, the latest ipython and the default python2.6.
>
> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
> IPython 0.10.bzr.r1174 (pulled latest trunk just a few days ago)
> Linux ccn 2.6.27.19-170.2.35.fc10.i686.PAE (Fedora 10)
>
>
>>
>> >
>> > Other than these complaints, a very nice addition indeed :)
>>
>> Thanks. :)
>>
>> >
>> > Could you please tell whether IPython instantiation could be expanded to
>> > a
>> > pylab enabled session? (A previously asked question, but just
>> > wondering...)
>>
>> Right --- I don't know. As I said, try to see how the --pylab command
>> is passed to ipython and then trigger it in the code.
>
> Still trying to figure this out. I wish I knew how to easily debug python
> scripts. Is there a way to setup on a specific line? I am so used to use
> this method while coding for small microprocessors. There should definetely
> be more advanced feature for Python.

You can use "pdb", or winpdb. There are lots of other options too, see
this thread on the sage-devel:

http://groups.google.com/group/sage-devel/browse_thread/thread/d7fbcf164eaedbc5

Ondrej


From gokhansever at gmail.com  Tue Jun 23 01:58:16 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 00:58:16 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
	<85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
Message-ID: <49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>

>
> Right, the up/down arrows cycle through the commands that were in
> ipython. It seems that ipython somehow rewrote the python's history.
> But you are *not* running in ipython after exiting it.



Check out this:


[gsever at ccn python-repo]$ python
Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
[GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import IPython
>>> IPython.set_trace()
 Logging to /home/gsever/.ipython/2009-06-23.py

Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
Type "copyright", "credits" or "license" for more information.

IPython 0.10.bzr.r1174 -- An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object'. ?object also works, ?? prints more.

In [1]: exit()
Do you really want to exit ([y]/n)? y
>>> arr = arange(10)
---------------------------------------------------------------------------
NameError                                     Python 2.5.2: /usr/bin/python
                                                   Tue Jun 23 00:54:55 2009
A problem occured executing Python code.  Here is the sequence of function
calls leading up to the error, with the most recent (innermost) call last.

/home/gsever/Desktop/python-repo/ipython/IPython/FakeModule.pyc in
<module>()
----> 1
      2
      3
      4
      5
      6
      7
      8
      9
     10
     11
     12
     13
     14
     15
     16
     17
     18
     19
     20
     21
     22
     23
     24
     25
     26
     27
     28
     29
     30
     31

NameError: name 'arange' is not defined

**********************************************************************

Oops, IPython crashed. We do our best to make it stable, but...

A crash report was automatically generated with the following information:
  - A verbatim copy of the crash traceback.
  - A copy of your input history during this session.
  - Data on your current IPython configuration.

It was left in the file named:
    '/home/gsever/.ipython/IPython_crash_report.txt'
If you can email this file to the developers, the information in it will
help
them in understanding and correcting the problem.

You can mail it to: Ville Vainio at vivainio at gmail.com
with the subject 'IPython Crash Report'.

If you want to do it now, the following command will work (under Unix):
mail -s 'IPython Crash Report' vivainio at gmail.com <
/home/gsever/.ipython/IPython_crash_report.txt

To ensure accurate tracking of this issue, please file a report about it at:
https://bugs.launchpad.net/ipython/+filebug

Press enter to exit:


This error message tells me that somewhere in the background Ipython is
still active, can't completely kill its instance after exiting the session.


>
> You can use "pdb", or winpdb. There are lots of other options too, see
> this thread on the sage-devel:
>
>
> http://groups.google.com/group/sage-devel/browse_thread/thread/d7fbcf164eaedbc5
>
> Ondrej


Thanks for the exact link ;)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/078f1c62/attachment.html>

From ondrej at certik.cz  Tue Jun 23 02:17:14 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 23 Jun 2009 00:17:14 -0600
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
	<85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
	<49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>
Message-ID: <85b5c3130906222317w2b09acd1x68cab54ba4c7d225@mail.gmail.com>

On Mon, Jun 22, 2009 at 11:58 PM, G?khan SEVER<gokhansever at gmail.com> wrote:
>> Right, the up/down arrows cycle through the commands that were in
>> ipython. It seems that ipython somehow rewrote the python's history.
>> But you are *not* running in ipython after exiting it.
>
> Check out this:
>
>
> [gsever at ccn python-repo]$ python
> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
> [GCC 4.3.2 20080917 (Red Hat 4.3.2-4)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import IPython
>>>> IPython.set_trace()
> ?Logging to /home/gsever/.ipython/2009-06-23.py
>
> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.10.bzr.r1174 -- An enhanced Interactive Python.
> ????????? -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help????? -> Python's own help system.
> object??? -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: exit()
> Do you really want to exit ([y]/n)? y
>>>> arr = arange(10)
> ---------------------------------------------------------------------------
> NameError???????????????????????????????????? Python 2.5.2: /usr/bin/python
> ?????????????????????????????????????????????????? Tue Jun 23 00:54:55 2009
> A problem occured executing Python code.? Here is the sequence of function
> calls leading up to the error, with the most recent (innermost) call last.
>
> /home/gsever/Desktop/python-repo/ipython/IPython/FakeModule.pyc in
> <module>()
> ----> 1
> ????? 2
> ????? 3
> ????? 4
> ????? 5
> ????? 6
> ????? 7
> ????? 8
> ????? 9
> ???? 10
> ???? 11
> ???? 12
> ???? 13
> ???? 14
> ???? 15
> ???? 16
> ???? 17
> ???? 18
> ???? 19
> ???? 20
> ???? 21
> ???? 22
> ???? 23
> ???? 24
> ???? 25
> ???? 26
> ???? 27
> ???? 28
> ???? 29
> ???? 30
> ???? 31
>
> NameError: name 'arange' is not defined
>
> **********************************************************************
>
> Oops, IPython crashed. We do our best to make it stable, but...
>
> A crash report was automatically generated with the following information:
> ? - A verbatim copy of the crash traceback.
> ? - A copy of your input history during this session.
> ? - Data on your current IPython configuration.
>
> It was left in the file named:
> ??? '/home/gsever/.ipython/IPython_crash_report.txt'
> If you can email this file to the developers, the information in it will
> help
> them in understanding and correcting the problem.
>
> You can mail it to: Ville Vainio at vivainio at gmail.com
> with the subject 'IPython Crash Report'.
>
> If you want to do it now, the following command will work (under Unix):
> mail -s 'IPython Crash Report' vivainio at gmail.com <
> /home/gsever/.ipython/IPython_crash_report.txt
>
> To ensure accurate tracking of this issue, please file a report about it at:
> https://bugs.launchpad.net/ipython/+filebug
>
> Press enter to exit:
>
>
> This error message tells me that somewhere in the background Ipython is
> still active, can't completely kill its instance after exiting the session.


Great catch! You are absolutely right, ipython seems to be still
active. Yes, so this approach has it's limitations, that it will alter
the environment.

Never the less, it actually doesn't hurt me at all, because I use
set_trace() to debug some particular part of the code and once I am
done, I don't care if the program crashes or what, since I want to
exit it anyway.

So this should be mentioned in the docstring too. I'll wait for
ipython developers if they agree with the patch and if so, I'll update
the docstring with this information.

Ondrej


From gokhansever at gmail.com  Tue Jun 23 02:32:50 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 01:32:50 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <85b5c3130906222317w2b09acd1x68cab54ba4c7d225@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
	<85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
	<49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>
	<85b5c3130906222317w2b09acd1x68cab54ba4c7d225@mail.gmail.com>
Message-ID: <49d6b3500906222332kbdb61efxa7b0d427d9858cad@mail.gmail.com>

>
> Great catch! You are absolutely right, ipython seems to be still
> active. Yes, so this approach has it's limitations, that it will alter
> the environment.
>
> Never the less, it actually doesn't hurt me at all, because I use
> set_trace() to debug some particular part of the code and once I am
> done, I don't care if the program crashes or what, since I want to
> exit it anyway.
>

You are welcome :) To tell the truth I don't care too much about this
either, only for the sake of the integrity of the work.


>
> So this should be mentioned in the docstring too. I'll wait for
> ipython developers if they agree with the patch and if so, I'll update
> the docstring with this information.
>
> Ondrej
>

I might have devised a way to launch a pylab session from Python following
your footsteps. What I did was in the Shell.py
replaced the below line as is shown (My matplotlibrc file uses QT4 backend
currently but it works with WX too, so Mayavi will not complain :)

    #user_opts = set([s.replace('-','') for s in argv[:3]])
    user_opts = set(['pylab'])

Then replaced __init.py__ set_trace()

    #Shell.IPShell(user_ns=dict(globals, **locals)).mainloop()
    Shell.start(user_ns=dict(globals, **locals)).mainloop()

now IPython.set_trace() works fine with pylab too :), however this also
shows the similar strange behaviours as Shell.IPShell case...

Hope someone from the lead IPy dev team response this topic and the patch
will be neatly added into the code-base.

G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/059b018b/attachment.html>

From gokhansever at gmail.com  Tue Jun 23 12:16:06 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 11:16:06 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <49d6b3500906222332kbdb61efxa7b0d427d9858cad@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
	<85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
	<49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>
	<85b5c3130906222317w2b09acd1x68cab54ba4c7d225@mail.gmail.com>
	<49d6b3500906222332kbdb61efxa7b0d427d9858cad@mail.gmail.com>
Message-ID: <49d6b3500906230916v480591e5h275f2fa6639a9818@mail.gmail.com>

 Note: Cross-posting on IPy devel and Matplotlib-users list.

Hello,

My 8 hours sleep have helped me solve the riddle of IPython pylab entry from
a regular Python script. Although it is funky and experimental, it works
just as I have been seeking it to work for months :)

Here take a look at this piece of code:

(Again, the following lines must be modified in /IPython/Shell.py to start
with pylab each time a request is made for IPython instantiation. To me a
line can be added into ipy_user_config such as always_pylab =True )

     #user_opts = set([s.replace('-','') for s in argv[:3]])     user_opts =
set(['pylab'])


#!/usr/bin/env python

#import matplotlib.pyplot as plt
#import numpy as np
import IPython

ipy = IPython.Shell.start(user_ns=locals())

a = arange(10)           ###<-- I am in IPython -pylab at this point no need
for np.arange
plot(a)                        ###<-- or plt.plot(a)

ipy.mainloop()

Later, when I drop into the IPy shell I can go ahead and manipulate my
figure appropriately. Now whos() works also :)


G?khan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/c266f5ad/attachment.html>

From gokhansever at gmail.com  Tue Jun 23 12:21:27 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Tue, 23 Jun 2009 11:21:27 -0500
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <49d6b3500906230916v480591e5h275f2fa6639a9818@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
	<49d6b3500906222208o133ab0fcne7cf6cc8ff648162@mail.gmail.com>
	<85b5c3130906222218u155a0583h726e1491dccb7a48@mail.gmail.com>
	<49d6b3500906222232p530b52b3v3d4a0546aa196ed5@mail.gmail.com>
	<85b5c3130906222252u21d31f73i87c6d111f2046018@mail.gmail.com>
	<49d6b3500906222258o19e530ecmf4009145d2b02876@mail.gmail.com>
	<85b5c3130906222317w2b09acd1x68cab54ba4c7d225@mail.gmail.com>
	<49d6b3500906222332kbdb61efxa7b0d427d9858cad@mail.gmail.com>
	<49d6b3500906230916v480591e5h275f2fa6639a9818@mail.gmail.com>
Message-ID: <49d6b3500906230921k2667a2d7l1bd80b2812deffe0@mail.gmail.com>

On Tue, Jun 23, 2009 at 11:16 AM, G?khan SEVER <gokhansever at gmail.com>wrote:

> Note: Cross-posting on IPy devel and Matplotlib-users list.
>
> Hello,
>
> My 8 hours sleep have helped me solve the riddle of IPython pylab entry
> from a regular Python script. Although it is funky and experimental, it
> works just as I have been seeking it to work for months :)
>
> Here take a look at this piece of code:
>
> (Again, the following lines must be modified in /IPython/Shell.py to start
> with pylab each time a request is made for IPython instantiation. To me a
> line can be added into ipy_user_config such as always_pylab =True )
>
>      #user_opts = set([s.replace('-','') for s in argv[:3]])     user_opts
> = set(['pylab'])
>
>
> #!/usr/bin/env python
>
> #import matplotlib.pyplot as plt
> #import numpy as np
> import IPython
>
> ipy = IPython.Shell.start(user_ns=locals())
>
> a = arange(10)           ###<-- I am in IPython -pylab at this point no
> need for np.arange
> plot(a)                        ###<-- or plt.plot(a)
>
> ipy.mainloop()
>
> Later, when I drop into the IPy shell I can go ahead and manipulate my
> figure appropriately. Now whos() works also :)
>
>
> G?khan


Ondrej,

Now I can completely exit too :)

In [1]: whos
Variable   Type                    Data/Info
--------------------------------------------
a          ndarray                 10: 10 elems, type `int32`, 40 bytes
ipy        IPShellMatplotlibQt4    <IPShellMatplotlibQt4(Thread-1, started)>

In [2]: exit()
Do you really want to exit ([y]/n)? y
*Closing threads... Done.*
[gsever at ccn src]$
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20090623/fa0df6ba/attachment.html>

From gokhansever at gmail.com  Mon Jun 29 12:25:45 2009
From: gokhansever at gmail.com (=?UTF-8?Q?G=C3=B6khan_SEVER?=)
Date: Mon, 29 Jun 2009 11:25:45 -0500
Subject: [IPython-dev] [IPython-user] IPython tab-completion behaviour
	in Linux
In-Reply-To: <20090629140406.GA18206@melfpelt.swarpa.net>
References: <49d6b3500906282243p76a65231q5cdb13102a7be6c@mail.gmail.com>
	<20090629140406.GA18206@melfpelt.swarpa.net>
Message-ID: <49d6b3500906290925l342ea11w432c4b76bed79627@mail.gmail.com>

On Mon, Jun 29, 2009 at 9:04 AM, Jeff Kaufman<jeff at alum.swarthmore.edu> wrote:
> G??khan SEVER wrote:
>>
>> See the examples below:
>>
>> [gsever at ccn ~]$ ipython
>> ?Logging to /home/gsever/.ipython/2009-06-29.py
>>
>> Python 2.5.2 (r252:60911, Sep 30 2008, 15:41:38)
>> Type "copyright", "credits" or "license" for more information.
>>
>> IPython 0.10.bzr.r1174 -- An enhanced Interactive Python.
>> ...
>>
>> In [1]: "".
>> Display all 104 possibilities? (y or n)
>>
>> In [1]: "".s
>> .sage ? ? ? ? ? ? .serverauth.5593 ?.subversion
>> .scribus ? ? ? ? ?.ssh
>>
>> In [1]: [].
>> Display all 104 possibilities? (y or n)
>>
>> In [1]: [].sort()
>>
>> In [2]: [].sort?
>> Type: ? ? ? ? list
>> Base Class: ? <type 'list'>
>> String Form: ?[]
>> Namespace: ? ?Interactive
>> Length: ? ? ? ? ? ? ? 0
>> Docstring:
>> ? ? list() -> new list
>> ? ? list(sequence) -> new list initialized from sequence's items
>>
>
> Note that some of this does work for strings:
>
> ?In [1]: ''.join?
> Type: ? ? ? ? ? str
> Base Class: ? ? <type 'str'>
> String Form:
> Namespace: ? ? ?Interactive
> Length: ? ? ? ? 0
> Docstring:
> ? ?str(object) -> string
>
> ? ?Return a nice string representation of the object.
> ? ?If the argument is a string, the return value is the same object.
>
> ?In [2]: "".join?
> Type: ? ? ? ? ? str
> Base Class: ? ? <type 'str'>
> String Form:
> Namespace: ? ? ?Interactive
> Length: ? ? ? ? 0
> Docstring:
> ? ?str(object) -> string
>
> ? ?Return a nice string representation of the object.
> ? ?If the argument is a string, the return value is the same object.
>
> And the issues are not limited to empty strings:
>
> ?In [3]: "test".<tab>
> .svn/entries ? ?.svn/prop-base ?.svn/text-base
> .svn/format ? ? .svn/props ? ? ?.svn/tmp
>
> ?In [3]: "test".svn/
>
> (In this case, .svn is the only dotfile in the curdir)
>
> Spaces change the matter:
>
> ?In [4]: "this is a test".<tab>
>
> Does nothing.
>
> The problem has something to do with the interaction of tab completion
> and quote marks, and the current behavior doesn't make much sense.
>
> Jeff
>

I am pinging the devel list, just in case someone might say something
about this issue. If not, I will post a bug at
https://bugs.launchpad.net/ipython/

>From my IPy shell, as also Jeff noted, empty or non-empty objects
cannot list their associated methods and attributes without assigning
them to names.

Some additional information, I use:

Linux ccn 2.6.27.19-170.2.35.fc10.i686.PAE (Fedora 10)
Python 2.5.2
IPython 0.10.bzr.r1174
GNU bash, version 3.2.39

In [1]: [1,2,3].
Display all 104 possibilities? (y or n)

In [1]: {1:'a', 2:'b'}.
Display all 104 possibilities? (y or n)

In [2]: a = {1:'a', 2:'b'}

In [3]: a.
a.__class__         a.__le__            a.get
a.__cmp__           a.__len__           a.has_key
a.__contains__      a.__lt__            a.items
a.__delattr__       a.__ne__            a.iteritems
a.__delitem__       a.__new__           a.iterkeys
a.__doc__           a.__reduce__        a.itervalues
a.__eq__            a.__reduce_ex__     a.keys
a.__ge__            a.__repr__          a.pop
a.__getattribute__  a.__setattr__       a.popitem
a.__getitem__       a.__setitem__       a.setdefault
a.__gt__            a.__str__           a.update
a.__hash__          a.clear             a.values
a.__init__          a.copy
a.__iter__          a.fromkeys


From ondrej at certik.cz  Mon Jun 29 22:41:13 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 29 Jun 2009 20:41:13 -0600
Subject: [IPython-dev] [patch] IPython.set_trace() implemented
In-Reply-To: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
References: <85b5c3130906222121x1c07a770te417eb9dcaa529b1@mail.gmail.com>
Message-ID: <85b5c3130906291941p57b97f28s8569346056ccf9fc@mail.gmail.com>

On Mon, Jun 22, 2009 at 10:21 PM, Ondrej Certik<ondrej at certik.cz> wrote:
> Hi,
>
> with the attached patch you can now do:
>
> ----
> a = {3: 4}
>
> def something():
> ? ?b = 45
> ? ?import IPython
> ? ?IPython.set_trace()
>
> something()
> ----
>
> and after you run it:
>
> -----------
> Python 2.6.2 (release26-maint, Apr 19 2009, 01:58:18)
> Type "copyright", "credits" or "license" for more information.
>
> IPython 0.10.bzr.r1163 -- An enhanced Interactive Python.
> ? ? ? ? ? -> Introduction and overview of IPython's features.
> %quickref -> Quick reference.
> help ? ? ?-> Python's own help system.
> object? ? -> Details about 'object'. ?object also works, ?? prints more.
>
> In [1]: a
> Out[1]: {3: 4}
>
> In [2]: b
> Out[2]: 45
> -------
>
>
> e.g. you can access locals() and globals() immediately. I use this
> very often lately, as it is very handy to debug some broken place in
> the code. Another usage is that I need to implement some difficult
> operation with numpy arrays, dicts, etc. usually some oneliner, but I
> am not sure about the API, so I just call IPython.set_trace(), play
> with the things to get what I want, then just copy & paste the
> oneliner to the code.
>
> More info in the docstring:
>
> In [2]: IPython.set_trace?
> Type: ? ? ? ? ? function
> Base Class: ? ? <type 'function'>
> String Form: ? ?<function set_trace at 0x7f31ca8b3140>
> Namespace: ? ? ?Interactive
> File: ? ? ? ? ? /home/ondrej/usr/lib/python/IPython/__init__.py
> Definition: ? ? IPython.set_trace()
> Docstring:
> ? ?Starts the IPython shell inplace.
>
> ? ?Example:
>
> ? ?>> import IPython
> ? ?>> IPython.set_trace()
>
> ? ?The user namespace is set to the union of globals() and locals(), e.g.
> ? ?calling set_trace() is exactly equivalent to:
>
> ? ? ? ?import IPython
> ? ? ? ?IPython.Shell.IPShell(user_ns=dict(globals(), **locals())).mainloop()
>
> ? ?but the actual implementation of set_trace() needs to use the inspect
> ? ?module, because it needs to access the parents frame.
>
>
>
> Let me know if the implementation is ok, or if some more work is needed.


Any comments to that from ipython developers?

Ondrej


From ellisonbg.net at gmail.com  Tue Jun 30 16:20:41 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 13:20:41 -0700
Subject: [IPython-dev] IPython module and package reorganization
Message-ID: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>

Hi all,

I am about to begin a significant reorganization of nearly all of
IPython's packages and modules.  This is in preparation for the work
on the IPython core that I will be dong this summer.  Here is a
document describing what I propose to do:

http://bazaar.launchpad.net/~ipython-dev/ipython/module-reorg/annotate/head%3A/docs/source/development/reorg.txt

I highly recommend looking at this as it will affect everyone.

The main thing I need feedback on at this point are the top-level
module names we will use.  Please have a look and let me know what you
think.

Cheers,

Brian


From jorgen.stenarson at bostream.nu  Tue Jun 30 17:24:54 2009
From: jorgen.stenarson at bostream.nu (=?ISO-8859-1?Q?J=F6rgen_Stenarson?=)
Date: Tue, 30 Jun 2009 23:24:54 +0200
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
Message-ID: <4A4A82A6.7030206@bostream.nu>

Brian Granger skrev:
> Hi all,
> 
> I am about to begin a significant reorganization of nearly all of
> IPython's packages and modules.  This is in preparation for the work
> on the IPython core that I will be dong this summer.  Here is a
> document describing what I propose to do:
> 
> http://bazaar.launchpad.net/~ipython-dev/ipython/module-reorg/annotate/head%3A/docs/source/development/reorg.txt
> 
> I highly recommend looking at this as it will affect everyone.
> 
> The main thing I need feedback on at this point are the top-level
> module names we will use.  Please have a look and let me know what you
> think.
> 
Hi,

I'm happy to hear you have the opportunity to focus and do this work. I 
believe it will make it a lot easier to get new people involved if it is 
easier to understand how everything works.

Near the end of the document you say everrything will follow pep-8 but 
in all the renames the toplevel still has uppercase letters in it i.e. 
IPython. Is this an oversight or do you feel a change in top level name 
will cause too muh inconvenience? Personally I think this is the right 
opportunity to change it.

How are the changes to the testing framework going? Currently the tests 
don't work at all on windows in trunk. There is a WindowsError and no 
tests are processed.

/J?rgen



From gael.varoquaux at normalesup.org  Tue Jun 30 17:30:13 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 30 Jun 2009 23:30:13 +0200
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
Message-ID: <20090630213013.GA30980@phare.normalesup.org>

On Tue, Jun 30, 2009 at 01:20:41PM -0700, Brian Granger wrote:
> Hi all,

> I am about to begin a significant reorganization of nearly all of
> IPython's packages and modules.  This is in preparation for the work
> on the IPython core that I will be dong this summer.  Here is a
> document describing what I propose to do:

> http://bazaar.launchpad.net/~ipython-dev/ipython/module-reorg/annotate/head%3A/docs/source/development/reorg.txt

> I highly recommend looking at this as it will affect everyone.

Quick comments: I prefer tools, or utils, rather than python.

Appart from this small remark, it all ooks good to me. I am excited about
this.

Ga?l


From ellisonbg.net at gmail.com  Tue Jun 30 17:42:55 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 14:42:55 -0700
Subject: [IPython-dev] Configuration, hooks and plugins
Message-ID: <6ce0ac130906301442o1eaffb10qda43ba45a7efabc9@mail.gmail.com>

Hi,

One of the things I am going to do in refactoring the IPython core
this summer is to consolidate the current configuration and hooks
system.

As I see it, here is our current situation.  We have 3 different
configuration systems:

1.  The basic config stuff in ipythonrc.  This uses key/value pairs in the form:

key value

2.  The more dynamic stuff in ipy_user_conf.py

This allows the user to dynamically configure ipython after it is
running by calling ipapi, loading extensions and setting hooks.

3.  IPython.kernel uses .ini files for configuring the parallel computing stuff.

This we current have:  3 config systems and hooks.  I propose that we
move to 1 config system and a proper plugin system (maybe just a
different name for a hook).

Config System
===========

By a "config system" I mean key, value pairs that determine how
IPython classes are created and behave.  But, it is a bit more
complicated than this.  First, some types of keys/values should be
changed only when IPython is starting up, other types of keys can be
changed at any time while IPython is running (such as the traceback
type, color mode, etc).  Also, some types of key/values can be set at
the command line as well.  Thus, here are some characteristics that
our config system needs to have:

* KISS.  A pure python config system that can be implemented in a few
hours, not weeks or months.  If we use an external dependency it must
be trivial to package and pure python (no traits!)

* An IPython module/class/function needs to be able to say "here is a
configuration key, here is it's default value, it will appear is this
config file, here is how it should show up at the command line, it
should/shouldn't be changed dynamically"

* When objects are created, they need to be able to get the current
global value of the configuration key.  This raises the question of
how configuration information gets to IPython objects.

* We need a way of handling things that can be set at startup, but
that can also be changed at runtime (often through magics).  We might
way to keep these things separate though.  We could say that "config"
means things that are set when an object is created and can't be
changed from then on.  To handle things that can be changed at
runtime, we could have a completely different api that ties into the
magic system.

Plugins
----------

Currently we use "hooks" to allow people to customize how IPython
behaves in non-trivial ways.  Typically, hooks involve implementing
some complex behavior by in a python function.  The current hooks
system is quite messy and grew into existence partly because IPython
simply didn't use good object oriented design.  I propose that we move
to a plugin system, where a plugin is a python class that adheres to a
very narrow and API and implements a single behavior.  We should have
a very simple API for registering plugins at startup and runtime, and
we probably want to create a .ipython/plugins directory where users
can put plugins.  Many of the things in Extensions will probably end
up as plugins.  I have fewer thoughts on how to implement plugins, but
I know that we won't be able to use any of the fancy ways of doing
plugins like zope or setuptools.

At this point, I am looking for feedback, ideas and discussions about
these things.

Cheers,

Brian


From ellisonbg.net at gmail.com  Tue Jun 30 17:44:45 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 14:44:45 -0700
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <20090630213013.GA30980@phare.normalesup.org>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<20090630213013.GA30980@phare.normalesup.org>
Message-ID: <6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>

> Quick comments: I prefer tools, or utils, rather than python.

I wrote that part a while back and I agree with you.  Do you like
tools or utils better?

> Appart from this small remark, it all ooks good to me. I am excited about
> this.

Me too?

What about "sandbox"?  Do you think that is a good name?

Cheers,

Brian


From ellisonbg.net at gmail.com  Tue Jun 30 17:49:40 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 14:49:40 -0700
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <4A4A82A6.7030206@bostream.nu>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<4A4A82A6.7030206@bostream.nu>
Message-ID: <6ce0ac130906301449g52a4289ejc53085bf3e0c82a1@mail.gmail.com>

> I'm happy to hear you have the opportunity to focus and do this work. I
> believe it will make it a lot easier to get new people involved if it is
> easier to understand how everything works.

Yes, one of my main goals is to make IPython "hackable"

> Near the end of the document you say everrything will follow pep-8 but in
> all the renames the toplevel still has uppercase letters in it i.e. IPython.
> Is this an oversight or do you feel a change in top level name will cause
> too muh inconvenience? Personally I think this is the right opportunity to
> change it.

Yes, we have talked about this in the past.  You are right that maybe
now is the time to make this change.  One subtlety is filesystems that
are case insensitive (OS X).  On such systems, "IPython" (the old
name) and "ipython" are indistiguishable, so if someone has both
installed, they could get wierd things.  I guess we would just have to
make it clear to folks that as of such and such release, you *have* to
delete the older versions.

I would love to be able to type "ipython" rather than "IPython" all
the time so I am for this change.  What do others think?  Are there
problems with this that I am missing?

> How are the changes to the testing framework going? Currently the tests
> don't work at all on windows in trunk. There is a WindowsError and no tests
> are processed.

Ohh, I will look at that.  Last I checked it did work.  It is probably
related to your not having a dependency installed though.  Do you have
any of:

nose
zope.interface
twisted

installed?

Cheers,

Brian

> /J?rgen
>
>


From gael.varoquaux at normalesup.org  Tue Jun 30 17:57:39 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 30 Jun 2009 23:57:39 +0200
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<20090630213013.GA30980@phare.normalesup.org>
	<6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>
Message-ID: <20090630215739.GB30980@phare.normalesup.org>

On Tue, Jun 30, 2009 at 02:44:45PM -0700, Brian Granger wrote:
> > Quick comments: I prefer tools, or utils, rather than python.

> I wrote that part a while back and I agree with you.  Do you like
> tools or utils better?

I am ok with both.

> > Appart from this small remark, it all ooks good to me. I am excited about
> > this.

> Me too?
        ^
      Hum, how should I interprete that :)


> What about "sandbox"?  Do you think that is a good name?

I am fine with it. I can't think of a better one. Maybe 'fix'. But I
don't have much of an opinion.

Ga?l


From ellisonbg.net at gmail.com  Tue Jun 30 18:00:53 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 15:00:53 -0700
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <20090630215739.GB30980@phare.normalesup.org>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<20090630213013.GA30980@phare.normalesup.org>
	<6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>
	<20090630215739.GB30980@phare.normalesup.org>
Message-ID: <6ce0ac130906301500pf8524ecy5b3231095e823513@mail.gmail.com>

>> I wrote that part a while back and I agree with you. ?Do you like
>> tools or utils better?
>
> I am ok with both.
>
>> > Appart from this small remark, it all ooks good to me. I am excited about
>> > this.
>
>> Me too?
> ? ? ? ?^
> ? ? ?Hum, how should I interprete that :)

Me too = I am excited about all this too.

>
>> What about "sandbox"? ?Do you think that is a good name?
>
> I am fine with it. I can't think of a better one. Maybe 'fix'. But I
> don't have much of an opinion.

OK, great.

Brian

> Ga?l
>


From gael.varoquaux at normalesup.org  Tue Jun 30 18:02:21 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 1 Jul 2009 00:02:21 +0200
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <6ce0ac130906301500pf8524ecy5b3231095e823513@mail.gmail.com>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<20090630213013.GA30980@phare.normalesup.org>
	<6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>
	<20090630215739.GB30980@phare.normalesup.org>
	<6ce0ac130906301500pf8524ecy5b3231095e823513@mail.gmail.com>
Message-ID: <20090630220221.GC30980@phare.normalesup.org>

On Tue, Jun 30, 2009 at 03:00:53PM -0700, Brian Granger wrote:
> >> I wrote that part a while back and I agree with you. ?Do you like
> >> tools or utils better?

> > I am ok with both.

> >> > Appart from this small remark, it all ooks good to me. I am excited about
> >> > this.

> >> Me too?
> > ? ? ? ?^
> > ? ? ?Hum, how should I interprete that :)

> Me too = I am excited about all this too.

I was making fun of your question mark, which was probably a typo.

Ga?l


From gael.varoquaux at normalesup.org  Tue Jun 30 18:12:03 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Wed, 1 Jul 2009 00:12:03 +0200
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <6ce0ac130906301449g52a4289ejc53085bf3e0c82a1@mail.gmail.com>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<4A4A82A6.7030206@bostream.nu>
	<6ce0ac130906301449g52a4289ejc53085bf3e0c82a1@mail.gmail.com>
Message-ID: <20090630221203.GD30980@phare.normalesup.org>

On Tue, Jun 30, 2009 at 02:49:40PM -0700, Brian Granger wrote:
> I would love to be able to type "ipython" rather than "IPython" all
> the time so I am for this change.  What do others think?  

+1

Ga?l


From ellisonbg.net at gmail.com  Tue Jun 30 18:14:29 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 15:14:29 -0700
Subject: [IPython-dev] IPython module and package reorganization
In-Reply-To: <20090630220221.GC30980@phare.normalesup.org>
References: <6ce0ac130906301320x7f0d624dgc7cf911ae1924c0b@mail.gmail.com>
	<20090630213013.GA30980@phare.normalesup.org>
	<6ce0ac130906301444n3128d566u6746eed2b8b4d8d9@mail.gmail.com>
	<20090630215739.GB30980@phare.normalesup.org>
	<6ce0ac130906301500pf8524ecy5b3231095e823513@mail.gmail.com>
	<20090630220221.GC30980@phare.normalesup.org>
Message-ID: <6ce0ac130906301514u17ac0e44yabc0658e8cca8687@mail.gmail.com>

On Tue, Jun 30, 2009 at 3:02 PM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Tue, Jun 30, 2009 at 03:00:53PM -0700, Brian Granger wrote:
>> >> I wrote that part a while back and I agree with you. ?Do you like
>> >> tools or utils better?
>
>> > I am ok with both.
>
>> >> > Appart from this small remark, it all ooks good to me. I am excited about
>> >> > this.
>
>> >> Me too?
>> > ? ? ? ?^
>> > ? ? ?Hum, how should I interprete that :)
>
>> Me too = I am excited about all this too.

Ha, ha, I guess I was not feeling to confident...

Brian

> I was making fun of your question mark, which was probably a typo.
>
> Ga?l
>


From robert.kern at gmail.com  Tue Jun 30 18:38:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 30 Jun 2009 17:38:19 -0500
Subject: [IPython-dev] Configuration, hooks and plugins
In-Reply-To: <6ce0ac130906301442o1eaffb10qda43ba45a7efabc9@mail.gmail.com>
References: <6ce0ac130906301442o1eaffb10qda43ba45a7efabc9@mail.gmail.com>
Message-ID: <h2e44s$aat$1@ger.gmane.org>

On 2009-06-30 16:42, Brian Granger wrote:
> Hi,
>
> One of the things I am going to do in refactoring the IPython core
> this summer is to consolidate the current configuration and hooks
> system.
>
> As I see it, here is our current situation.  We have 3 different
> configuration systems:
>
> 1.  The basic config stuff in ipythonrc.  This uses key/value pairs in the form:
>
> key value
>
> 2.  The more dynamic stuff in ipy_user_conf.py
>
> This allows the user to dynamically configure ipython after it is
> running by calling ipapi, loading extensions and setting hooks.
>
> 3.  IPython.kernel uses .ini files for configuring the parallel computing stuff.
>
> This we current have:  3 config systems and hooks.  I propose that we
> move to 1 config system and a proper plugin system (maybe just a
> different name for a hook).
>
> Config System
> ===========
>
> By a "config system" I mean key, value pairs that determine how
> IPython classes are created and behave.  But, it is a bit more
> complicated than this.  First, some types of keys/values should be
> changed only when IPython is starting up, other types of keys can be
> changed at any time while IPython is running (such as the traceback
> type, color mode, etc).  Also, some types of key/values can be set at
> the command line as well.  Thus, here are some characteristics that
> our config system needs to have:
>
> * KISS.  A pure python config system that can be implemented in a few
> hours, not weeks or months.  If we use an external dependency it must
> be trivial to package and pure python (no traits!)
>
> * An IPython module/class/function needs to be able to say "here is a
> configuration key, here is it's default value, it will appear is this
> config file, here is how it should show up at the command line, it
> should/shouldn't be changed dynamically"
>
> * When objects are created, they need to be able to get the current
> global value of the configuration key.  This raises the question of
> how configuration information gets to IPython objects.
>
> * We need a way of handling things that can be set at startup, but
> that can also be changed at runtime (often through magics).  We might
> way to keep these things separate though.  We could say that "config"
> means things that are set when an object is created and can't be
> changed from then on.  To handle things that can be changed at
> runtime, we could have a completely different api that ties into the
> magic system.
>
> Plugins
> ----------
>
> Currently we use "hooks" to allow people to customize how IPython
> behaves in non-trivial ways.  Typically, hooks involve implementing
> some complex behavior by in a python function.  The current hooks
> system is quite messy and grew into existence partly because IPython
> simply didn't use good object oriented design.  I propose that we move
> to a plugin system, where a plugin is a python class that adheres to a
> very narrow and API and implements a single behavior.  We should have
> a very simple API for registering plugins at startup and runtime, and
> we probably want to create a .ipython/plugins directory where users
> can put plugins.  Many of the things in Extensions will probably end
> up as plugins.  I have fewer thoughts on how to implement plugins, but
> I know that we won't be able to use any of the fancy ways of doing
> plugins like zope or setuptools.

I like Mercurial's approach. Plugins are just Python packages/modules exposing a 
particular API. Having those packages installed *does not* enable them. To 
enable them, you list the ones you want in the configuration file. Truth be 
told, that's fairly similar to the current situation: you just import the 
appropriate things in your ipy_user_conf.py and maybe activate them, except that 
there is no standardized plugin API.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco



From ellisonbg.net at gmail.com  Tue Jun 30 18:44:26 2009
From: ellisonbg.net at gmail.com (Brian Granger)
Date: Tue, 30 Jun 2009 15:44:26 -0700
Subject: [IPython-dev] Configuration, hooks and plugins
In-Reply-To: <h2e44s$aat$1@ger.gmane.org>
References: <6ce0ac130906301442o1eaffb10qda43ba45a7efabc9@mail.gmail.com>
	<h2e44s$aat$1@ger.gmane.org>
Message-ID: <6ce0ac130906301544p4d9f9e51p75260cf95cceff25@mail.gmail.com>

> I like Mercurial's approach. Plugins are just Python packages/modules
> exposing a
> particular API. Having those packages installed *does not* enable them. To
> enable them, you list the ones you want in the configuration file. Truth be
> told, that's fairly similar to the current situation: you just import the
> appropriate things in your ipy_user_conf.py and maybe activate them, except
> that
> there is no standardized plugin API.

I too like this approach and a definitely think installing and
enabling should be two different steps.

How does Mercurial declare/discover plugins?  Does it use setuptools?
I will look at this.  Because IPython plugin's are often extremely
lightweight, it would be nice if users could simply drop them into
.ipython/plugins for installation.

Cheers,

Brian

> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless enigma
>   that is made terrible by our own mad attempt to interpret it as though it
> had
>   an underlying truth."
>    -- Umberto Eco
>
> _______________________________________________
> IPython-dev mailing list
> IPython-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/ipython-dev
>


From robert.kern at gmail.com  Tue Jun 30 18:54:42 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 30 Jun 2009 17:54:42 -0500
Subject: [IPython-dev] Configuration, hooks and plugins
In-Reply-To: <6ce0ac130906301544p4d9f9e51p75260cf95cceff25@mail.gmail.com>
References: <6ce0ac130906301442o1eaffb10qda43ba45a7efabc9@mail.gmail.com>	<h2e44s$aat$1@ger.gmane.org>
	<6ce0ac130906301544p4d9f9e51p75260cf95cceff25@mail.gmail.com>
Message-ID: <h2e53k$cn7$1@ger.gmane.org>

On 2009-06-30 17:44, Brian Granger wrote:
>> I like Mercurial's approach. Plugins are just Python packages/modules
>> exposing a
>> particular API. Having those packages installed *does not* enable them. To
>> enable them, you list the ones you want in the configuration file. Truth be
>> told, that's fairly similar to the current situation: you just import the
>> appropriate things in your ipy_user_conf.py and maybe activate them, except
>> that
>> there is no standardized plugin API.
>
> I too like this approach and a definitely think installing and
> enabling should be two different steps.
>
> How does Mercurial declare/discover plugins?  Does it use setuptools?

No. It reads the configuration files (~/.hgrc and the current repo's .hg/hgrc, 
if any) for a section that looks like this:

[extensions]
alias=
mq=
fetch=
bisect=
transplant=
hgk=
extdiff=
record=
purge=
#hgext.forest=/Users/rkern/hg/hgforest/forest.py
#hgext.qct=/Users/rkern/hg/qct/plugins/qct.py
graphlog=

The "foo=" lines are those plugins which are distributed with Mercurial itself, 
so there is a standard place to look for them. You can specify a particular path 
if it is not one of the standard plugins like the ones that I have commented out.

> I will look at this.  Because IPython plugin's are often extremely
> lightweight, it would be nice if users could simply drop them into
> .ipython/plugins for installation.

We already add ~/.ipython to the sys.path. If you want to to stop doing that, 
and only add ~/.ipython/plugins to the sys.path, that might be cleaner.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco