From mantegazza at ill.fr  Thu Dec  1 02:10:38 2005
From: mantegazza at ill.fr (=?iso-8859-15?q?Fr=E9d=E9ric_Mantegazza?=)
Date: Thu, 1 Dec 2005 08:10:38 +0100
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <438DEA06.3010106@colorado.edu>
References: <200511291321.57840.mantegazza@ill.fr>
	<200511301257.33338.mantegazza@ill.fr> <438DEA06.3010106@colorado.edu>
Message-ID: <200512010810.38874.mantegazza@ill.fr>

Le Mercredi 30 Novembre 2005 19:05, Fernando Perez a ?crit :

> > It works, but it displays 'None' before PyMAD. It is because
> > time.sleep() returns None, I presume. How can I make it return an empty
> > string, instead ?
>
> Wrap it:
>
> def mysleep(t):
>    time.sleep(t)
>    return ''

I tied that, but IPython can't find my function. Where do I have to put it 
to be reachable by the prompt at runtime?

I found another solution :

    str(time.sleep(0.1))[1:0]

-- 
   Fr?d?ric



From stefan at sun.ac.za  Thu Dec  1 07:13:02 2005
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Thu, 1 Dec 2005 14:13:02 +0200
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <438DEA06.3010106@colorado.edu>
References: <200511291321.57840.mantegazza@ill.fr>
	<438CB553.9060405@colorado.edu> <200511301257.33338.mantegazza@ill.fr>
	<438DEA06.3010106@colorado.edu>
Message-ID: <20051201121302.GG24577@alpha>

While looking at your problem I found another:

$ ipython -prompt_in1 | some_nonexisting_program

*boom*

In the manpage, it describes the input option as

-prompt_in1|pi1 <string>

which I find rather confusing.  Like

-no|banner

which clearly means -[no]banner, the options above looked more like

-promp_in1 or -prompt_pi1 to me.

After figuring that I had to use the -pi1 option, I tried

$ ipython -pi1 \$\{ }

*crash*

OK, so looks like I'm not going to be able to test this, but try
something like

argv = ["-pi1", "${mysleep = lambda t: return str(time.sleep(t))*0; mysleep(0.2)}PyMAD>>> ",
        "-po", " ",
        "-profile", "pymad"]

This will define mysleep every time the prompt comes up -- but since
you want a delay anyway, this won't matter :)

Cheers
St?fan

On Wed, Nov 30, 2005 at 11:05:58AM -0700, Fernando Perez wrote:
> Fr?d?ric Mantegazza wrote:
> > Le Mardi 29 Novembre 2005 21:08, Fernando Perez a ?crit :
> > 
> > 
> >>The prompt system is fully dynamic, so the easiest way is to insert a
> >>time.sleep() call into your prompt :)
> > 
> > 
> > You mean like this :
> > 
> > argv = ["-pi1", "${time.sleep(0.2)}PyMAD>>> ",
> >         "-po", " ",
> >         "-profile", "pymad"]
> > ipshell = IPShellEmbed(argv)
> > 
> > It works, but it displays 'None' before PyMAD. It is because time.sleep() 
> > returns None, I presume. How can I make it return an empty string, 
> > instead ?
> 
> Wrap it:
> 
> def mysleep(t):
>    time.sleep(t)
>    return ''
> 
> Cheers,
> 
> f



From Fernando.Perez at colorado.edu  Thu Dec  1 12:01:05 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Dec 2005 10:01:05 -0700
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <200512010810.38874.mantegazza@ill.fr>
References: <200511291321.57840.mantegazza@ill.fr>
	<200511301257.33338.mantegazza@ill.fr> <438DEA06.3010106@colorado.edu>
	<200512010810.38874.mantegazza@ill.fr>
Message-ID: <438F2C51.6000205@colorado.edu>

Fr?d?ric Mantegazza wrote:
> Le Mercredi 30 Novembre 2005 19:05, Fernando Perez a ?crit :
> 
> 
>>>It works, but it displays 'None' before PyMAD. It is because
>>>time.sleep() returns None, I presume. How can I make it return an empty
>>>string, instead ?
>>
>>Wrap it:
>>
>>def mysleep(t):
>>   time.sleep(t)
>>   return ''
> 
> 
> I tied that, but IPython can't find my function. Where do I have to put it 
> to be reachable by the prompt at runtime?

The prompt system executes in the global environment of the user.  The 
simplest way to define extra things is to put them in a pure python file and 
call that from your ipythonrc.  This should clarify it:

planck[~/.ipython]> cat ipythonrc-myprompt
prompt_in1 '${prompt1()} \nIn [\#]: '

execfile myprompt.py
planck[~/.ipython]> cat myprompt.py
def prompt1():
     return '<Hi Fred>'
planck[~/.ipython]> ipython -p myprompt
Python 2.3.4 (#1, Feb  2 2005, 12:11:53)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.16.svn -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

IPython profile: myprompt

<Hi Fred>
In [1]:
Do you really want to exit ([y]/n)?
planck[~/.ipython]>


> I found another solution :
> 
>     str(time.sleep(0.1))[1:0]

That will do too.

Cheers,

f



From Fernando.Perez at colorado.edu  Thu Dec  1 12:11:46 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Dec 2005 10:11:46 -0700
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <20051201121302.GG24577@alpha>
References: <200511291321.57840.mantegazza@ill.fr>
	<438CB553.9060405@colorado.edu> <200511301257.33338.mantegazza@ill.fr>
	<438DEA06.3010106@colorado.edu> <20051201121302.GG24577@alpha>
Message-ID: <438F2ED2.2020600@colorado.edu>

Stefan van der Walt wrote:
> While looking at your problem I found another:
> 
> $ ipython -prompt_in1 | some_nonexisting_program
> 
> *boom*
> 
> In the manpage, it describes the input option as
> 
> -prompt_in1|pi1 <string>
> 
> which I find rather confusing.  Like
> 
> -no|banner
> 
> which clearly means -[no]banner, the options above looked more like
> 
> -promp_in1 or -prompt_pi1 to me.

You are right, this is confusing, I should have worded it differently.  The 
second form is meant to be a shorhand for the first, so:

-foo|f -> use '-foo' or '-f'

and I should have written:

-[no]bar -> both '-bar' and '-nobar' are valid, the second negating the first.

I'll fix the manpage and docs accordingly, thanks for pointing this out.

> After figuring that I had to use the -pi1 option, I tried
> 
> $ ipython -pi1 \$\{ }
> 
> *crash*

I agree that it shouldn't crash, but you also shouldn't feed it unbalanced 
parens :)  This will work:

planck[~/.ipython]> ipython -pi1 'Hi Stefan>' -nobanner

Hi Stefan>

Cheers,

f



From stefan at sun.ac.za  Thu Dec  1 15:12:02 2005
From: stefan at sun.ac.za (Stefan van der Walt)
Date: Thu, 1 Dec 2005 22:12:02 +0200
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <438F2ED2.2020600@colorado.edu>
References: <200511291321.57840.mantegazza@ill.fr>
	<438CB553.9060405@colorado.edu> <200511301257.33338.mantegazza@ill.fr>
	<438DEA06.3010106@colorado.edu> <20051201121302.GG24577@alpha>
	<438F2ED2.2020600@colorado.edu>
Message-ID: <20051201201202.GO24577@alpha>

On Thu, Dec 01, 2005 at 10:11:46AM -0700, Fernando Perez wrote:
> >$ ipython -pi1 \$\{ }
> >
> >*crash*
> 
> I agree that it shouldn't crash, but you also shouldn't feed it unbalanced 
> parens :)  This will work:

Heh, you're right :)  I was trying to be nice, and executed

$ ipython -pi1 \$\{ \}

So, at first I didn't realise that I was being nasty ;)

Cheers
St?fan



From mantegazza at ill.fr  Fri Dec  2 03:51:15 2005
From: mantegazza at ill.fr (=?iso-8859-15?q?Fr=E9d=E9ric_Mantegazza?=)
Date: Fri, 2 Dec 2005 09:51:15 +0100
Subject: [IPython-dev] IPython prompt delay
In-Reply-To: <438F2C51.6000205@colorado.edu>
References: <200511291321.57840.mantegazza@ill.fr>
	<200512010810.38874.mantegazza@ill.fr> <438F2C51.6000205@colorado.edu>
Message-ID: <200512020951.16876.mantegazza@ill.fr>

Le Jeudi 1 D?cembre 2005 18:01, Fernando Perez a ?crit :
> Fr?d?ric Mantegazza wrote:

> The prompt system executes in the global environment of the user.  The
> simplest way to define extra things is to put them in a pure python file
> and call that from your ipythonrc.  This should clarify it:

Ok, I understand. Thanks.

-- 
   Fr?d?ric



From Fernando.Perez at colorado.edu  Sat Dec  3 14:11:57 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sat, 03 Dec 2005 12:11:57 -0700
Subject: [IPython-dev] Debugger patch
In-Reply-To: <4391D12E.7010103@vdesmedt.com>
References: <4389A377.1010608@vdesmedt.com> <438CB7F2.3080202@colorado.edu>
	<4391D12E.7010103@vdesmedt.com>
Message-ID: <4391EDFD.9000202@colorado.edu>

Hi Vivian,

Vivian De Smedt wrote:
> Hi Fernando,
> 
> I'm in the process a cutting my patch to the IPython code to improve the 
> Python debugger version of the IPython to make it more IPythonic ;-).

Excellent, many thanks for this.  I am in the process of looking over it.  I 
certainly find the final functionality to be a VAST improvement over the 
crippled pdb we get by default, so this will most defintely go in.  I just 
need to look in more detail at the implementation to try to make the 
integration as smooth as possible with the rest of the codebase.

In the meantime, I just wanted to offer you a little bit of practical feedback 
regarding patches, to make the process easier in the future:

- Your email didn't make it to the -dev list because it was ~800kb, which is 
too big for the list.  The reason for this size is that you sent the full 
IPython directory as a zip file, this is unnecessary.  Instead, you can simply 
do in your local copy of IPython

svn diff > vivian_pdb.diff

and attach (zipped, if you want) the .diff file.  That contains all the 
information necessary to apply your changes.  I've attached it here for reference.

- You don't need to mark your changes in the source (# << vivian).  The diff 
shows precisely what has changed, and that way there is no need to later go 
back and remove those marks manually (there is no point in cluttering the code 
in the long run with such markers).

> *The new features*
> 
> The change cover better support for:
> 
>     * the print of the current point of execution
>           o it is now printed the way IPython print traceback
>     * the "where" command
>           o each stack is printed the way IPython print the traceback frames
>           o the IPython stacks level are not printed anymore in the
>             result making the result clearer
>     * the "list" command
>           o the code is printed the way IPython print the traceback
>           o the break point number is printed instead of a simple B
>             (that is nice since it help you to know which break point
>             you should clear or disable)
>           o the disabled breakpoints appear in a different color that
>             the active breakpoints
>     * the code completion works when IPython run in debugger mode
>       (that's nice since it help you to explore more easily the content
>       of the variable to discover why the Exception was raised or why
>       the code is wrong)
> 
> To make it clearer you will find commented screen shot of before and 
> after the change together with the script I have used to produce them.

The functionality is excellent, and many thanks for the clear screenshots with 
your comments on them.  It makes for a really easy way to see what the 
improvements are.  I'll probably reuse them on the website to highlight the 
feature, if you don't mind.

> *Integration*
> 
> In a previous mail we spoke about sending to you this patch into smaller 
> part to help you understanding my changes, be sure that there have no 
> bad impact on existing features and ease the integration into your new code.
> 
> As I said previously only four files where modified three are only 
> slightly modified and one (Debugger.py) widely.
> Actually Debugger.py contains the definition of Debugger.Pdb which is a 
> derivation of pdb.Pdb to override some functions and modify this way 
> some behaviors of Pdb. The patch do not modify existing overriding but 
> add some to customize more behaviors
> 
> The modifications of the other files concerns
> 
>     * using Debugger.Pdb instead of pdb.Pdb,
>     * passing some color information to Debugger.Pdb to let it print the
>       colors the way the user want,
>     * passing some information about the stack levels the debugger
>       should use for the where command
>     * the code completion
> 
> In order to do that I have to choose which are the first feature to 
> integrate into IPython it is a difficult choice. Do you think something 
> about it. What is the part that interest you the most what feature 
> should cover the first patch tell me what you think.

Don't worry, what you sent me is enough.  I have a couple of free hours today 
before I have to work on my floors again, so I'll work on the patch right 
away.  I've kept you 'on hold' long enough that a bit of quick response will 
hopefully be a welcome change :)

Thanks again for these improvements.  I'll get working on the integration, and 
will let you know either when I commit, or if I need to discuss further work 
with you.

Cheers,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: pdb.diff
Type: text/x-patch
Size: 14833 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20051203/bc242742/attachment.bin>

From vivainio at gmail.com  Thu Dec 22 11:23:53 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 08:23:53 -0800
Subject: [IPython-dev] Distribution via "Easy Install"?
Message-ID: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>

Anyone looked into distributing IPython via easy-install mechanism yet?

It would totally rock to be able to just get the latest version by doing

easy-install ipython

even on a win32 box!

http://peak.telecommunity.com/DevCenter/EasyInstall

It would also ease making ipython a requirement for other packages and
having it automatically installed during the installation process for
other package.

--
Ville Vainio    http://tinyurl.com/2prnb



From Fernando.Perez at colorado.edu  Thu Dec 22 11:42:10 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 09:42:10 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
Message-ID: <43AAD762.8030808@colorado.edu>

Ville Vainio wrote:
> Anyone looked into distributing IPython via easy-install mechanism yet?
> 
> It would totally rock to be able to just get the latest version by doing
> 
> easy-install ipython
> 
> even on a win32 box!
> 
> http://peak.telecommunity.com/DevCenter/EasyInstall
> 
> It would also ease making ipython a requirement for other packages and
> having it automatically installed during the installation process for
> other package.

I haven't, but I'd certainly be willing to make whatever changes (if any) are 
needed to play nice with the eggs system.  If anyone knows what needs to be 
done right away, by all means let me know.

Cheers,

f



From vivainio at gmail.com  Thu Dec 22 12:30:56 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 19:30:56 +0200
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <43AAD762.8030808@colorado.edu>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
Message-ID: <46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>

On 12/22/05, Fernando Perez <Fernando.Perez at colorado.edu> wrote:

> I haven't, but I'd certainly be willing to make whatever changes (if any) are
> needed to play nice with the eggs system.  If anyone knows what needs to be
> done right away, by all means let me know.

>From http://peak.telecommunity.com/DevCenter/PythonEggs

"""
Edit the target package's setup.py and add "from setuptools import
setup" such that it replaces the existing import of the setup
function. Then run setup.py bdist_egg.
"""

What's frightening is that it seemed to work. :-D

Or more correctly, "python setup.py install" installed ipython as an
egg in site-packages, and running "ipython" launched the new ipython.

/usr/bin/ipython contents are:

---------- cut -------------------
#!/usr/bin/python
# EASY-INSTALL-SCRIPT: 'ipython==0.6.16.svn','ipython'
__requires__ = 'ipython==0.6.16.svn'
import pkg_resources
pkg_resources.run_script('ipython==0.6.16.svn', 'ipython')
-------------------------------------

If this turns out to be all we need... "wow" is all I can say.

Of course you need setuptools to do this, but it's trivial:

http://peak.telecommunity.com/DevCenter/EasyInstall#installing-easy-install

--
Ville Vainio    http://tinyurl.com/2prnb



From vivainio at gmail.com  Thu Dec 22 12:48:14 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 19:48:14 +0200
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
Message-ID: <46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>

On 12/22/05, Ville Vainio <vivainio at gmail.com> wrote:

> If this turns out to be all we need... "wow" is all I can say.

Well, of course it won't be.

in ipmaker.py:

    # we need the directory where IPython itself is installed
    import IPython
    IPython_dir = os.path.dirname(IPython.__file__)
    del IPython

is going to be an immediate problem, because the files won't really be
there (they are in the egg). The dir is used to exec the scripts as
specified in ipythonrc...

Also the windows post install script is a problem, the files it refers
to are not visible either.

--
Ville Vainio    http://tinyurl.com/2prnb



From Fernando.Perez at colorado.edu  Thu Dec 22 12:50:21 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 10:50:21 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
Message-ID: <43AAE75D.4010402@colorado.edu>

Ville Vainio wrote:
> On 12/22/05, Fernando Perez <Fernando.Perez at colorado.edu> wrote:
> 
> 
>>I haven't, but I'd certainly be willing to make whatever changes (if any) are
>>needed to play nice with the eggs system.  If anyone knows what needs to be
>>done right away, by all means let me know.
> 
> 
>>From http://peak.telecommunity.com/DevCenter/PythonEggs
> 
> """
> Edit the target package's setup.py and add "from setuptools import
> setup" such that it replaces the existing import of the setup
> function. Then run setup.py bdist_egg.
> """
> 
> What's frightening is that it seemed to work. :-D
> 
> Or more correctly, "python setup.py install" installed ipython as an
> egg in site-packages, and running "ipython" launched the new ipython.
> 
> /usr/bin/ipython contents are:
> 
> ---------- cut -------------------
> #!/usr/bin/python
> # EASY-INSTALL-SCRIPT: 'ipython==0.6.16.svn','ipython'
> __requires__ = 'ipython==0.6.16.svn'
> import pkg_resources
> pkg_resources.run_script('ipython==0.6.16.svn', 'ipython')
> -------------------------------------

Note that all those better be wrapped in try/except blocks: while I'm all for 
_allowing_ egg support, I am _absolutely_ against requiring it.  The 
__requires__ var is nice and good, since it's just some private data, but the 
imports would be protected by try blocks.

Cheers,

f



From Fernando.Perez at colorado.edu  Thu Dec 22 12:52:24 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 10:52:24 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
	<46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
Message-ID: <43AAE7D8.40508@colorado.edu>

Ville Vainio wrote:
> On 12/22/05, Ville Vainio <vivainio at gmail.com> wrote:
> 
> 
>>If this turns out to be all we need... "wow" is all I can say.
> 
> 
> Well, of course it won't be.
> 
> in ipmaker.py:
> 
>     # we need the directory where IPython itself is installed
>     import IPython
>     IPython_dir = os.path.dirname(IPython.__file__)
>     del IPython
> 
> is going to be an immediate problem, because the files won't really be
> there (they are in the egg). The dir is used to exec the scripts as
> specified in ipythonrc...
> 
> Also the windows post install script is a problem, the files it refers
> to are not visible either.

Mmh.  My last encounter with eggs wasn't the most pleasant, but I preferred to 
reserve that comment.  I am sure that in the long run eggs will be a good 
cross-platform solution, and I'm all for including in the next release support 
for them.  Let the patches roll :)

Cheers,

f



From vivainio at gmail.com  Thu Dec 22 13:05:57 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 20:05:57 +0200
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <43AAE7D8.40508@colorado.edu>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
	<46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
	<43AAE7D8.40508@colorado.edu>
Message-ID: <46cb515a0512221005r24bf39fehb6fe26408a458a87@mail.gmail.com>

Relevant links for my previous mail, i.e. the way to get around trying
to access data files:

http://peak.telecommunity.com/DevCenter/setuptools#accessing-data-files-at-runtime

http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources

(BTW, I'm not on crack, just looked into Easy Install today and got a
sting of enthusiasm, learning as I go. Hence the spamming of the
mailing list).

It seems like proper egg-support would indicate an easy-install
requirement for ipython. I don't really see this as a problem, now
that something like TurboGears requires it as well. I.e. it's
mainstream and stable enough.

I guess I could look into the required modifications one of these
days, if the easy-install dependency is ok.



From Fernando.Perez at colorado.edu  Thu Dec 22 13:12:11 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 11:12:11 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512221005r24bf39fehb6fe26408a458a87@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
	<46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
	<43AAE7D8.40508@colorado.edu>
	<46cb515a0512221005r24bf39fehb6fe26408a458a87@mail.gmail.com>
Message-ID: <43AAEC7B.1050901@colorado.edu>

Ville Vainio wrote:
> Relevant links for my previous mail, i.e. the way to get around trying
> to access data files:
> 
> http://peak.telecommunity.com/DevCenter/setuptools#accessing-data-files-at-runtime
> 
> http://peak.telecommunity.com/DevCenter/PythonEggs#accessing-package-resources
> 
> (BTW, I'm not on crack, just looked into Easy Install today and got a
> sting of enthusiasm, learning as I go. Hence the spamming of the
> mailing list).
> 
> It seems like proper egg-support would indicate an easy-install
> requirement for ipython. I don't really see this as a problem, now
> that something like TurboGears requires it as well. I.e. it's
> mainstream and stable enough.
> 
> I guess I could look into the required modifications one of these
> days, if the easy-install dependency is ok.

As I said, the easy-install OPTIONAL dependency, for those wanting egg 
support, is absolutely OK.  But for now (and probably always), we won't make 
it mandatory: I had a hideous experience with it a few months ago, and I'm not 
yet convinced that the technology has matured (especially in regards to 
unix-based, non-root installs being properly handled) to the point of depening 
on it.

Robert Kern is quite the expert on eggs, if he's around and not traveling, a 
bit of wisdom from him on the current situation would be very welcome at this 
point.  From what he's told me, things have improved quite a bit since my own 
clash with it a while back.

So just to be clear: optional dependencies and egg supoort, I'm all for. 
Mandatory, not for now.

Cheers,

f



From vivainio at gmail.com  Thu Dec 22 13:55:58 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 20:55:58 +0200
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <43AAEC7B.1050901@colorado.edu>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
	<46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
	<43AAE7D8.40508@colorado.edu>
	<46cb515a0512221005r24bf39fehb6fe26408a458a87@mail.gmail.com>
	<43AAEC7B.1050901@colorado.edu>
Message-ID: <46cb515a0512221055q70f3a823rdafc283bfa0cde95@mail.gmail.com>

> So just to be clear: optional dependencies and egg supoort, I'm all for.
> Mandatory, not for now.

I hear you. Perhaps we could just have an optional & experimental
setup.py (or a branch in current setup.py) for building an egg-version
of ipython for easy-install goodness.

The problem in ipmaker.py that I mentioned is rather small; one can
always copy pysh.py to somewhere in pythonpath. $HOME/.ipython should
also be in the search path for

    # Execute the files the user wants in ipythonrc
    for file in IP_rc.execfile:
        try:
            file = filefind(file,sys.path+[IPython_dir])
        except IOError:
            warn(itpl('File $file not found. Skipping it.'))
        else:
            IP.safe_execfile(os.path.expanduser(file),IP.user_ns)

in addition to the now-bogus IPython_dir.

The egg would probably be a bit more minimal than the current ipython
"full distribution", at least initially, but many users probably won't
care, just wanting to get a working ipython environment quickly.

Gotta explore this a bit more, but it seems to be good stuff all in all.

--
Ville Vainio    http://tinyurl.com/2prnb



From vivainio at gmail.com  Thu Dec 22 14:20:54 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Thu, 22 Dec 2005 21:20:54 +0200
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512221055q70f3a823rdafc283bfa0cde95@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<43AAD762.8030808@colorado.edu>
	<46cb515a0512220930j22afa0fax804ed324364a9526@mail.gmail.com>
	<46cb515a0512220948i4391d2adx30bd62bdd62d01ae@mail.gmail.com>
	<43AAE7D8.40508@colorado.edu>
	<46cb515a0512221005r24bf39fehb6fe26408a458a87@mail.gmail.com>
	<43AAEC7B.1050901@colorado.edu>
	<46cb515a0512221055q70f3a823rdafc283bfa0cde95@mail.gmail.com>
Message-ID: <46cb515a0512221120s3a91610ard2af640bb9196339@mail.gmail.com>

Ok, here's how it's made to work easily, if not as neatly (instead of
single .egg file in site-packages we get a .egg directory there, with
all the ipython files blown open). User won't see the difference,
though.

In setup.py, change the setup command to be used and add zip_safe =
false to setup options:

#from distutils.core import setup
from setuptools import setup

..

setup(name             = name,
      version          = version,
      description      = description,
      long_description = long_description,
      author           = authors['Fernando'][0],
      author_email     = authors['Fernando'][1],
      url              = url,
      license          = license,
      platforms        = platforms,
      keywords         = keywords,
      zip_safe         = False,

#    ^^^^^^^^^^^^^^ here!

      packages         = ['IPython', 'IPython.Extensions'],
      scripts          = scriptfiles,


And then, to create the egg do

> python setup.py bdist_egg

the egg can be installed by

> sudo easy_install dist/ipython-0.6.16.svn-py2.4.egg


Try it out, it worked for me, even w/ "ipython -p pysh" and "rm -Rf ~/.ipython".



From robert.kern at gmail.com  Thu Dec 22 14:27:34 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 22 Dec 2005 14:27:34 -0500
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
Message-ID: <doeun5$p0l$1@sea.gmane.org>

Ville Vainio wrote:
> Anyone looked into distributing IPython via easy-install mechanism yet?

To consolidate replies to a few messages:

* ipython builds as an egg just fine out-of-box. It is not zip-safe since it
uses a few __file__ thingies, but that's okay. easy_install correctly unpacks
the egg. You can build ipython as an egg without changing the source:

  $ python -c "import setuptools; execfile('setup.py')" bdist_egg

Of course, easy_install will do this for you.

* I'm not entirely sure what needs to happen with the post-install script on
Windows. Possibly make it a module in the IPython package so that one could do

  In [1]: from IPython.win32_post_install import run
  In [2]: run()

manual.pdf might have to be made package_data in that case, though.

* The /usr/bin/ipython contents that Ville posted is automatically generated by
easy_install (the actual scripts live inside the egg; the scripts installed to
/usr/whatever/bin are bootstraps to load and run those scripts), so Fernando
doesn't need to worry about wrapping anything with try: except: blocks.

* I think the only thing that really needs to happen now is that the PyPI record
needs a link to the actual download location or the tarballs could be uploaded
to PyPI itself.

* However, setuptools and pkg_resources (the module that handles the resource
location and extraction from zipped eggs; included with setuptools) are fairly
stable and useful. easy_install is possibly still problematic for non-root UNIX
installs, but I think even that is probably okay now if you ignore the
ez_setup.py bootstrap. In any case, setuptools and pkg_resources can be used to
good effect without thrusting easy_install's problems on anyone. Using recent
(last week or so) setuptools, one can now install setuptoolized packages as
regular packages with the egg metadata alongside rather than full eggs:

  $ python setup.py install --single-version-externally-managed

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From Fernando.Perez at colorado.edu  Thu Dec 22 15:42:35 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 13:42:35 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <doeun5$p0l$1@sea.gmane.org>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<doeun5$p0l$1@sea.gmane.org>
Message-ID: <43AB0FBB.4050902@colorado.edu>

Hey Robert,

as usual, thanks for the useful, detailed reply.

Robert Kern wrote:
> Ville Vainio wrote:
> 
>>Anyone looked into distributing IPython via easy-install mechanism yet?
> 
> 
> To consolidate replies to a few messages:
> 
> * ipython builds as an egg just fine out-of-box. It is not zip-safe since it
> uses a few __file__ thingies, but that's okay. easy_install correctly unpacks
> the egg. You can build ipython as an egg without changing the source:
> 
>   $ python -c "import setuptools; execfile('setup.py')" bdist_egg
> 
> Of course, easy_install will do this for you.

This is for _building_ an egg, right?  Should I distribute the .egg in this 
manner as well at ipython/dist?  I'm perfectly happy installing setuptools and 
adding the above to my release script, but I don't know if that's the 
preferred mechanism.

> * I'm not entirely sure what needs to happen with the post-install script on
> Windows. Possibly make it a module in the IPython package so that one could do
> 
>   In [1]: from IPython.win32_post_install import run
>   In [2]: run()

This would be easy: I leave it out for cleanliness, but it's not really a big 
deal if there is a significant win.

> manual.pdf might have to be made package_data in that case, though.
> 
> * The /usr/bin/ipython contents that Ville posted is automatically generated by
> easy_install (the actual scripts live inside the egg; the scripts installed to
> /usr/whatever/bin are bootstraps to load and run those scripts), so Fernando
> doesn't need to worry about wrapping anything with try: except: blocks.

Do you mean that we shouldn't add 'import egg_stuff' _anywhere_ in the ipython 
code, or do you refer only to the scripts?  If the latter, which components do 
still require egg yolks in them?

> * I think the only thing that really needs to happen now is that the PyPI record
> needs a link to the actual download location or the tarballs could be uploaded
> to PyPI itself.

I can do that, as I already register.  Should this be a link to the actual raw 
.tar.gz?

> * However, setuptools and pkg_resources (the module that handles the resource
> location and extraction from zipped eggs; included with setuptools) are fairly
> stable and useful. easy_install is possibly still problematic for non-root UNIX
> installs, but I think even that is probably okay now if you ignore the
> ez_setup.py bootstrap. In any case, setuptools and pkg_resources can be used to
> good effect without thrusting easy_install's problems on anyone. Using recent
> (last week or so) setuptools, one can now install setuptoolized packages as
> regular packages with the egg metadata alongside rather than full eggs:
> 
>   $ python setup.py install --single-version-externally-managed


Thanks again.  With some guidance from those who know eggs better, I'll be 
happy to add the relevant support.

Cheers,

f



From robert.kern at gmail.com  Thu Dec 22 16:08:40 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 22 Dec 2005 16:08:40 -0500
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <43AB0FBB.4050902@colorado.edu>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<doeun5$p0l$1@sea.gmane.org> <43AB0FBB.4050902@colorado.edu>
Message-ID: <dof4kn$ejf$1@sea.gmane.org>

Fernando Perez wrote:
> Hey Robert,
> 
> as usual, thanks for the useful, detailed reply.

My pleasure.

> Robert Kern wrote:

>>* ipython builds as an egg just fine out-of-box. It is not zip-safe since it
>>uses a few __file__ thingies, but that's okay. easy_install correctly unpacks
>>the egg. You can build ipython as an egg without changing the source:
>>
>>  $ python -c "import setuptools; execfile('setup.py')" bdist_egg
>>
>>Of course, easy_install will do this for you.
> 
> This is for _building_ an egg, right? 

Correct.

> Should I distribute the .egg in this 
> manner as well at ipython/dist?  I'm perfectly happy installing setuptools and 
> adding the above to my release script, but I don't know if that's the 
> preferred mechanism.

Since ipython is pure Python, yes, providing an egg in addition to the source
tarball would be great. It won't have the documentation, though, unless if it is
included as package_data.

>>* The /usr/bin/ipython contents that Ville posted is automatically generated by
>>easy_install (the actual scripts live inside the egg; the scripts installed to
>>/usr/whatever/bin are bootstraps to load and run those scripts), so Fernando
>>doesn't need to worry about wrapping anything with try: except: blocks.
> 
> Do you mean that we shouldn't add 'import egg_stuff' _anywhere_ in the ipython 
> code, or do you refer only to the scripts?  If the latter, which components do 
> still require egg yolks in them?

The only places where you might want to use pkg_resources are the places where
you use __file__ and such. Doing that would make the egg zip-safe so it doesn't
have to be unpacked, but it's not necessary. Specifically, here are the things
that make the egg not zip-safe:

zip_safe flag not set; analyzing archive contents...
IPython.deep_reload: module references __path__
IPython.ipmaker: module references __file__
IPython.Magic: module MAY be using inspect.getabsfile
IPython.OInspect: module MAY be using inspect.getsource
IPython.OInspect: module MAY be using inspect.getabsfile
IPython.ultraTB: module MAY be using inspect.getinnerframes

So no, there are no *requirements* at this time to import any egg_stuff in the
ipython package or the setup.py. It would enable some minor conveniences, but at
the cost of a (buildtime or runtime) dependency on setuptools and Python 2.3.
It's probably not worthwhile at this time.

>>* I think the only thing that really needs to happen now is that the PyPI record
>>needs a link to the actual download location or the tarballs could be uploaded
>>to PyPI itself.
> 
> I can do that, as I already register.  Should this be a link to the actual raw 
> .tar.gz?

A link to http://ipython.scipy.org/dist/ is sufficient. easy_install can
screenscrape that page just fine. In fact, this works right now:

[src]$ easy_install -f http://ipython.scipy.org/dist ipython
Reading http://ipython.scipy.org/dist
Searching for ipython
Best match: ipython 0.6.15
Downloading http://ipython.scipy.org/dist/ipython-0.6.15.tar.gz
Processing ipython-0.6.15.tar.gz
Running ipython-0.6.15/setup.py -q bdist_egg --dist-dir
/tmp/easy_install-PMohs7/ipython-0.6.15/egg-dist-tmp-yqP99d
zip_safe flag not set; analyzing archive contents...
IPython.deep_reload: module references __path__
IPython.ipmaker: module references __file__
IPython.Magic: module MAY be using inspect.getabsfile
IPython.OInspect: module MAY be using inspect.getsource
IPython.OInspect: module MAY be using inspect.getabsfile
IPython.ultraTB: module MAY be using inspect.getinnerframes
Adding ipython 0.6.15 to easy-install.pth file
Installing ipython script to /usr/local/stow/local/bin
Installing pycolor script to /usr/local/stow/local/bin

Installed
/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/ipython-0.6.15-py2.4.egg
Processing dependencies for ipython
[src]$ ipython
Activating auto-logging.
Current session state plus future input saved to: /Users/kern/.ipython/ipython.log
Logging mode:  backup
*** Loaded SciPy
Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.15 -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
%magic  -> Information about IPython's 'magic' % functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

In [1]:

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From Fernando.Perez at colorado.edu  Thu Dec 22 19:41:05 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Dec 2005 17:41:05 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <dof4kn$ejf$1@sea.gmane.org>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<doeun5$p0l$1@sea.gmane.org> <43AB0FBB.4050902@colorado.edu>
	<dof4kn$ejf$1@sea.gmane.org>
Message-ID: <43AB47A1.2040101@colorado.edu>

Robert Kern wrote:
> Fernando Perez wrote:

>>Robert Kern wrote:
> 
> 

>>Should I distribute the .egg in this 
>>manner as well at ipython/dist?  I'm perfectly happy installing setuptools and 
>>adding the above to my release script, but I don't know if that's the 
>>preferred mechanism.
> 
> 
> Since ipython is pure Python, yes, providing an egg in addition to the source
> tarball would be great. It won't have the documentation, though, unless if it is
> included as package_data.

Mmh, this would force me to jump to py2.4, it seems:

abdul[distutils]> grep package_data /usr/lib/python2.3/distutils/*.py
abdul[distutils]> grep package_data /usr/lib/python2.4/distutils/*.py
/usr/lib/python2.4/distutils/dist.py:        self.package_data = {}

otherwise the package_data keyword will bork 2.2-2.3 installs.  I guess I can 
build the dict of options dynamically, and then call

setup(**args)

instead.  Ugly, but it will do the job.  Is it worth it?  What else does it 
buy us?

>>>* The /usr/bin/ipython contents that Ville posted is automatically generated by
>>>easy_install (the actual scripts live inside the egg; the scripts installed to
>>>/usr/whatever/bin are bootstraps to load and run those scripts), so Fernando
>>>doesn't need to worry about wrapping anything with try: except: blocks.
>>
>>Do you mean that we shouldn't add 'import egg_stuff' _anywhere_ in the ipython 
>>code, or do you refer only to the scripts?  If the latter, which components do 
>>still require egg yolks in them?
> 
> 
> The only places where you might want to use pkg_resources are the places where
> you use __file__ and such. Doing that would make the egg zip-safe so it doesn't
> have to be unpacked, but it's not necessary. Specifically, here are the things
> that make the egg not zip-safe:
> 
> zip_safe flag not set; analyzing archive contents...
> IPython.deep_reload: module references __path__
> IPython.ipmaker: module references __file__
> IPython.Magic: module MAY be using inspect.getabsfile
> IPython.OInspect: module MAY be using inspect.getsource
> IPython.OInspect: module MAY be using inspect.getabsfile
> IPython.ultraTB: module MAY be using inspect.getinnerframes
> 
> So no, there are no *requirements* at this time to import any egg_stuff in the
> ipython package or the setup.py. It would enable some minor conveniences, but at
> the cost of a (buildtime or runtime) dependency on setuptools and Python 2.3.
> It's probably not worthwhile at this time.
> 
> 
>>>* I think the only thing that really needs to happen now is that the PyPI record
>>>needs a link to the actual download location or the tarballs could be uploaded
>>>to PyPI itself.
>>
>>I can do that, as I already register.  Should this be a link to the actual raw 
>>.tar.gz?
> 
> 
> A link to http://ipython.scipy.org/dist/ is sufficient. easy_install can
> screenscrape that page just fine. In fact, this works right now:
> 
> [src]$ easy_install -f http://ipython.scipy.org/dist ipython

[...]

nice.  How is that link provided to setuptools?  Is it just a matter of 
passing in the download_url keyword to the normal setup() call, or does 
setuptools need the info elsewhere?

I guess what I'm asking is: given that easy_install in your example already 
works, how much exactly should I do to provide 'good citizen' egg support? 
Would the following be enough?

1. Adding download_url='http://ipython.scipy.org/dist' to setup.py.

2. Installing setuptools on my system, so that I can build an egg and put it 
up at /dist/ when new releases are made.

Cheers,

f



From robert.kern at gmail.com  Thu Dec 22 22:09:23 2005
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 22 Dec 2005 22:09:23 -0500
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <43AB47A1.2040101@colorado.edu>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<doeun5$p0l$1@sea.gmane.org> <43AB0FBB.4050902@colorado.edu>
	<dof4kn$ejf$1@sea.gmane.org> <43AB47A1.2040101@colorado.edu>
Message-ID: <dofpp3$9h6$1@sea.gmane.org>

Fernando Perez wrote:
> Robert Kern wrote:
> 
>>Fernando Perez wrote:
> 
>>>Robert Kern wrote:
> 
>>>Should I distribute the .egg in this 
>>>manner as well at ipython/dist?  I'm perfectly happy installing setuptools and 
>>>adding the above to my release script, but I don't know if that's the 
>>>preferred mechanism.
>>
>>Since ipython is pure Python, yes, providing an egg in addition to the source
>>tarball would be great. It won't have the documentation, though, unless if it is
>>included as package_data.
> 
> Mmh, this would force me to jump to py2.4, it seems:
> 
> abdul[distutils]> grep package_data /usr/lib/python2.3/distutils/*.py
> abdul[distutils]> grep package_data /usr/lib/python2.4/distutils/*.py
> /usr/lib/python2.4/distutils/dist.py:        self.package_data = {}
> 
> otherwise the package_data keyword will bork 2.2-2.3 installs.  I guess I can 
> build the dict of options dynamically, and then call
> 
> setup(**args)
> 
> instead.  Ugly, but it will do the job.  Is it worth it?  What else does it 
> buy us?

setuptools backports package_data to 2.3 as well. Since eggs and setuptools only
work with 2.3+, you would only be building eggs for 2.3 and 2.4 in any case.

Actually, all package_data is is a convenient shortcut for what you do with
UserConfig/ already. You could just do that if you really want to package the
documentation inside the egg.

>>A link to http://ipython.scipy.org/dist/ is sufficient. easy_install can
>>screenscrape that page just fine. In fact, this works right now:
>>
>>[src]$ easy_install -f http://ipython.scipy.org/dist ipython
> 
> [...]
> 
> nice.  How is that link provided to setuptools?  Is it just a matter of 
> passing in the download_url keyword to the normal setup() call, or does 
> setuptools need the info elsewhere?
> 
> I guess what I'm asking is: given that easy_install in your example already 
> works, how much exactly should I do to provide 'good citizen' egg support? 
> Would the following be enough?
> 
> 1. Adding download_url='http://ipython.scipy.org/dist' to setup.py.
> 
> 2. Installing setuptools on my system, so that I can build an egg and put it 
> up at /dist/ when new releases are made.

Yes. Anything else might be slightly more convenient, but probably not worth
your time.

-- 
Robert Kern
robert.kern at gmail.com

"In the fields of hell where the grass grows high
 Are the graves of dreams allowed to die."
  -- Richard Harter



From Fernando.Perez at colorado.edu  Sat Dec 24 19:59:21 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sat, 24 Dec 2005 17:59:21 -0700
Subject: [IPython-dev] Distribution via "Easy Install"?
In-Reply-To: <dofpp3$9h6$1@sea.gmane.org>
References: <46cb515a0512220823h421564c4sc8dda48fc05699b2@mail.gmail.com>
	<doeun5$p0l$1@sea.gmane.org> <43AB0FBB.4050902@colorado.edu>
	<dof4kn$ejf$1@sea.gmane.org> <43AB47A1.2040101@colorado.edu>
	<dofpp3$9h6$1@sea.gmane.org>
Message-ID: <43ADEEE9.3090508@colorado.edu>

Robert Kern wrote:

>>I guess what I'm asking is: given that easy_install in your example already 
>>works, how much exactly should I do to provide 'good citizen' egg support? 
>>Would the following be enough?
>>
>>1. Adding download_url='http://ipython.scipy.org/dist' to setup.py.
>>
>>2. Installing setuptools on my system, so that I can build an egg and put it 
>>up at /dist/ when new releases are made.
> 
> 
> Yes. Anything else might be slightly more convenient, but probably not worth
> your time.

OK, 1. has been committed along with the big pdb rework, and I'll do 2 on my 
box.  These obviously won't have any impact til .16 is out, since the 
register() call is only made at new release time.

Thanks anyway for the tips.  As setuptools mature and we move into the release 
phase for the chainsaw branch, I'll try to make sure that we are good players 
in that environment.

Best,

f



From Fernando.Perez at colorado.edu  Sat Dec 24 20:09:43 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sat, 24 Dec 2005 18:09:43 -0700
Subject: [IPython-dev] Debugger patch
In-Reply-To: <4391D12E.7010103@vdesmedt.com>
References: <4389A377.1010608@vdesmedt.com> <438CB7F2.3080202@colorado.edu>
	<4391D12E.7010103@vdesmedt.com>
Message-ID: <43ADF157.4050209@colorado.edu>

Hi Vivian,

Vivian De Smedt wrote:

> I'm in the process a cutting my patch to the IPython code to improve the 
> Python debugger version of the IPython to make it more IPythonic ;-).

> *The new features*
> 
> The change cover better support for:
> 
>     * the print of the current point of execution
>           o it is now printed the way IPython print traceback
>     * the "where" command
>           o each stack is printed the way IPython print the traceback frames
>           o the IPython stacks level are not printed anymore in the
>             result making the result clearer
>     * the "list" command
>           o the code is printed the way IPython print the traceback
>           o the break point number is printed instead of a simple B
>             (that is nice since it help you to know which break point
>             you should clear or disable)
>           o the disabled breakpoints appear in a different color that
>             the active breakpoints
>     * the code completion works when IPython run in debugger mode
>       (that's nice since it help you to explore more easily the content
>       of the variable to discover why the Exception was raised or why
>       the code is wrong)

OK, I finally found some time to finish up this work and committed it.  Many 
thanks again, I think these are really very nice improvements, you did an 
excellent job.  You may be interested in loooking at the diff of what I 
actually did at:

http://projects.scipy.org/ipython/ipython/changeset/951

You may find it useful in terms of understanding the organization of the 
codebase for future contributions to IPython (your patch was fine, I just 
reworked things somewhat to make them fit better in the rest, reusing more 
functionality where possible which was already there).

As I am trying to get closer to making a new release with all the accumulated 
contributions of the last few months, it would be great to have some users 
running off SVN to let me know of any problems I may have introduced.  In 
particular Vivian, please make sure that what I committed didn't break any of 
your intended functionality.

Again, many thanks for this contribution, and sorry for my being so delayed in 
  its integration.  It makes for a nice IPython Christmas present :)

Best to all, and happy holidays!

Cheers,

f



From Fernando.Perez at colorado.edu  Tue Dec 27 03:02:11 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Tue, 27 Dec 2005 01:02:11 -0700
Subject: [IPython-dev] smart indent,
	multiline statement history and	multiline statement edition
In-Reply-To: <434009FB.4020205@vdesmedt.com>
References: <434009FB.4020205@vdesmedt.com>
Message-ID: <43B0F503.4040607@colorado.edu>

Hi Vivian,

continuing to catch up with my gigantic backlog...

Vivian De Smedt wrote:

> In my quest to contribute I would like to improve IPython in two areas:
>  - copy and paste
>  - multiline edition
> 
> 1.) By improving IPython for copy and paste I mean try to find a way to 
> paste multiline statement when autoindent is on.
> What I had in mind is the following:
> 
> Since autoindent is pretty enough to propose you identation each time 
> you could need one or to say it differently when autoindent is on the 
> tab touch is unnecessary I propose a smartindent mode (which could just 
> the autoindent mode if you like it).
> 
> In that mode the autoindent proposition of leading space is forgotten if 
> the first character of a line is a space or a tabulation.
> 
> In such mode autoindentation is working more or less the same way as 
> before but if you copy a bunch of code from elsewhere it will be paste 
> with the original indentation.
> 
> Let me know what you think about this proposition, if you did try such 
> aproach in the paste and if so what was the techincal difficulties you 
> faced (I guess here the various implementation of readline will play 
> they role and I will have to test my patch against each of them before 
> submision).

OK, I couldn't quite convince myself whether this was easy, possible but hard, 
or plain impossible.  In trying to give you a decent answer, I ended up just 
implementing it, sorry :)  It's now in SVN.

Note there is still a caveat: you can now paste code, but it still looks wrong 
on screen and in the readline history.  This due to limitations imposed by 
readline, and I can't fix that.  But the pasted code is interpreted correctly, 
and the internal history (non-readline) is OK as well.

Please test it, and let me know of any problems.

> 2.) I would like to improve multiline support. I have various ideas some 
> modest some more ambitious.
> 
> The first is just about the history of the command. Now when you type a 
> multiline statement each line of the statement is a separate entry in 
> the command history.
> 
> def foo(bar):
>     bar += 3
>     return bar * 2
> 
> However the chance that you are interested by the line "    return bar * 
> 2" by itself is not big.
> 
> My proposition was that only the first "def foo(bar)" was visible in the 
> command history and that when you choose it (hit return) the next one 
> ("    bar += 3", "    return bar * 2") appear in the forward part of the 
> history such that you can quickly review a multiline statement.
> 
> Here is a small scenario illustration the usage of my proposition:
> 
> def foo(bar):
>     bar += 3
>     return bar * 2
> 
> [up-arrow]
> edit the first line of the multiline statement
> [return]
> [down-arrow]
> edit the second  line of the multiline statement
> [return]
> [down-arrow]
> edit the third line of the multiline statement
> [return]
> 
> Let me know if it is clear, if you like it, if you think it could be 
> usefull if you think it is feasible.

While in principle I like these ideas, I think that they are impossible to 
implement within the limitations of Python's readline API.  I am not sure if 
the full GNU readline has enough functionality for this to work, but python 
certainly doesn't expose enough of it.

But feel free to prove me wrong with a patch :)

Keep in mind that I am currently trying to flush out all the accumulated 
patches, questions and contributions to ipython for a .16 release, so that I 
don't ignore what you and many others have done over the past few months.  But 
all real new development is taking place in the chainsaw branch (not really 
functional for interactive work yet).  This will eventually provide an ipython 
'kernel' which can be easily embedded, for example, in a GUI or in a curses 
environment, where decent multi-line editing becomes much more natural.

Once I 'pay my debt' to the community by incorporating and resolving all the 
accumulated issues, I'll continue mostly working on moving the new system 
forward.  Still, patches to trunk will continue to be accepted (as long as 
they don't require too much reworking on my part) until the chainsaw branch 
becomes functional enough to fully deprecate the current trunk.

Best,

f



From Fernando.Perez at colorado.edu  Tue Dec 27 03:03:42 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Tue, 27 Dec 2005 01:03:42 -0700
Subject: [IPython-dev] smart indent,
	multiline statement history and	multiline statement edition
In-Reply-To: <200510031500.42197.hans_meine@gmx.net>
References: <434009FB.4020205@vdesmedt.com>
	<200510031500.42197.hans_meine@gmx.net>
Message-ID: <43B0F55E.5040606@colorado.edu>

Hans Meine wrote:
> On Sunday 02 October 2005 18:25, Vivian De Smedt wrote:
> 
>>Since autoindent is pretty enough to propose you identation each time
>>you could need one or to say it differently when autoindent is on the
>>tab touch is unnecessary I propose a smartindent mode (which could just
>>the autoindent mode if you like it).
>>
>>In that mode the autoindent proposition of leading space is forgotten if
>>the first character of a line is a space or a tabulation.
> 
> Nice idea indeed.
> 
> I wonder what we can do about another use case which I stumble over 
> repeatedly:
> 
> Usually, I like tabs in my code files.  However, when cut&pasting from 
> (X)Emacs into IPython, readline interprets the <tab>s as completion requests.  
> 
> AFAICS, a smart indent mode could help: It could simply interpret tabs at the 
> beginning of the line different from tabs after some non-whitespace 
> characters!?  (/me hopes that readline does not make this too difficult)
> 
> I am very much looking forward to such a thing indeed! :-)

Fixed in SVN, along with the improved cut-and-paste support.

Thanks for the suggestion.

Cheers,

f



From Fernando.Perez at colorado.edu  Wed Dec 28 18:25:26 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Wed, 28 Dec 2005 16:25:26 -0700
Subject: [IPython-dev] Preventing "Oops, IPython crashed" ?
In-Reply-To: <200508121033.07962.hans_meine@gmx.net>
References: <200505311445.23247.hans_meine@gmx.net>
	<200508051235.44378.hans_meine@gmx.net> <42FBABE5.20304@colorado.edu>
	<200508121033.07962.hans_meine@gmx.net>
Message-ID: <43B31EE6.3@colorado.edu>

Hans Meine wrote:
> On Thursday 11 August 2005 21:49, Fernando Perez wrote:
> 
>>So I'm afraid I don't really know how to solve this particular problem
>>cleanly.  Any ideas would be most welcome.  Specifically, a way to trap the
>>mpl exceptions only would do the trick, but all try/excepts I've tried to
>>wrap around the mpl threads have so far failed.
> 
> 
> Yes, I understood the problem is hard-to-impossible to solve cleanly, but what 
> I wanted to say is that it'd make sense to document the workaround
> 
>  sys.excepthook = __IPYTHON__.excepthook
> 
> or even make it a configurable option.  The problem I have is not with 
> matplotlib, but with Qt:  If you write an interactive Qt program (which is 
> most probable if you're using Qt in the first place), your code will most 
> probably be executed from the GUI thread and as such all small mistakes in 
> your own code will result in this lengthy default exc.handler output.
> 
> I hard-coded the above statement into IPShellQt.__init__ since IPShellQt is 
> much less usable otherwise.  So I propose a documented ipythonrc switch that 
> activates this, possibly only for IPShellQt if other toolkits work 
> differently.
> 
> I hope this made my intention more clear. :-)

OK, I just committed this.  Straight from the changelog:

	(InteractiveShell.__init__): change threaded shells to not use the
	ipython crash handler.  This was causing more problems than not,
	as exceptions in the main thread (GUI code, typically) would
	always show up as a 'crash', when they really weren't.

	The colors and exception mode commands (%colors/%xmode) have been
	synchronized to also take this into account, so users can get
	verbose exceptions for their threaded code as well.  I also added
	support for activating pdb inside this exception handler as well,
	so now GUI authors can use IPython's enhanced pdb at runtime.


So now you get pdb, pretty tracebacks, color highlighting, and even the 
verbose tracebacks (%xmode verbose) in your GUI code.

Let me know how this works.

Best,

f



From Fernando.Perez at colorado.edu  Thu Dec 29 13:31:56 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 29 Dec 2005 11:31:56 -0700
Subject: [IPython-dev] [PATCH] Further improvements of paging
	system	(Windows-specific)
In-Reply-To: <dihaoc$pcp$1@sea.gmane.org>
References: <dihaoc$pcp$1@sea.gmane.org>
Message-ID: <43B42B9C.8090500@colorado.edu>

Alexander Belchenko wrote:
>   Another improvements of paging system (on Windows):
>   - function get_console_size for Windows console
>     returns size of current console as (sizex,sizey)
>   - with function get_console_size() page_dumb can fit
>     each page exactly on one full screen.
> 
> Added module winconsole.py with special function get_console_size(). 
> This module needed only for Windows. Function get_console_size works 
> only if ctypes installed. Otherwise it returns default values for size 
> of console (80, 25).
> 
> In addition with my previous patch to page_dumb it makes dumb pager is 
> not so dumb ;-)

OK, sorry for the very long delay on this issue.  As you can tell from the 
recent string of emails, I'm catching up :)

Many thanks for your patch, I just committed it.  Please test from SVN and let 
me know how it works (as well as other win32 users).

Best,

f



From Fernando.Perez at colorado.edu  Thu Dec 29 13:46:35 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 29 Dec 2005 11:46:35 -0700
Subject: [IPython-dev] Windows share patch
In-Reply-To: <43691BB5.5040500@bostream.nu>
References: <43691BB5.5040500@bostream.nu>
Message-ID: <43B42F0B.8070401@colorado.edu>

J?rgen Stenarson wrote:
> Hi,
> 
> The patch still works as expected for me. I have not run into any 
> problems yet.

Committed, with minor mods.  Please test and let me know how things go.

Many thanks for the work!

Cheers,

f



From Fernando.Perez at colorado.edu  Thu Dec 29 13:53:41 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 29 Dec 2005 11:53:41 -0700
Subject: [IPython-dev] magic docstrings
In-Reply-To: <20051103140222.GA4659@sun.ac.za>
References: <20051103140222.GA4659@sun.ac.za>
Message-ID: <43B430B5.6060002@colorado.edu>

Stefan van der Walt wrote:
> I compiled the iPython docs earlier today, and found some problems in
> the generated magic.tex.
> 
> One problem is that LaTeX # character is not escaped, which can be
> easily fixed:

Thanks!  I am finally catching up ...

I committed your patch more or less as-is.  Later we can discuss more 
permanent, long-term solutions, but this already helps.  I'd argue that 
ipython is already better documented than many free software projects, so I 
don't feel too bad on this front :)

Thanks again, and sorry for the gigantic delay.  Ben will have told you why :)

Best,

f



From Fernando.Perez at colorado.edu  Thu Dec 29 22:30:47 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 29 Dec 2005 20:30:47 -0700
Subject: [IPython-dev] IPython 0.7.0.rc1, and development...
Message-ID: <43B4A9E7.9090201@colorado.edu>

Hi everyone,

in addition to wishing all of you a Happy New Year, I'd like to update you 
with some development news, and a request for testing.

As you may have noticed, in recent days I've been trying to catch up with my 
gigantic backlog.  By way of apology, a large home remodeling effort has taken 
far, far more time than I initially estimated, which is why for most of this 
semester I hardly did anything with ipython.  While we're by no means 
finished, at least I now have a normal desk, a computer to work on, and some 
semblance of a (half) normal place.  So I am doing my best to catch up with you.

Status report
-------------

I went through all of my accumulated ipython-related mail, as well as the bug 
tracker, and took care of every single bug report, feature request and patch 
submission.  The actual list of improvements is quite significant, to a large 
extent thanks to all those of you who sent in ideas backed with patches. 
Many, many thanks.

Here's a very brief list of highlights of the new things since the last 
official release (0.6.15):

   - Wildcard patterns in searches, supported by the %psearch magic. 
Extremely useful, thanks to J?rgen Stenarson.

   - Major improvements to the pdb mode.  It now has tab-completion, syntax 
highlighting and better stack handling.  Thanks to Vivian De Smedt for this 
work (double-points given that pdb has a well-deserved reputation for being 
very unpleasant to work with).

   - Support for input with empty lines.  If you have auto-indent on, this 
means that you need to either hit enter _twice_, or add/remove a space to your 
last blank line, to indicate you're done entering input.  These changes also 
allow us to provide copy/paste of code with blank lines.

   - Support for pasting multiline input even with autoindent on.  The code 
will look wrong on screen, but it will be stored and executed correctly 
internally.

   - TAB on an otherwise empty line actually inserts a tab.  Convenient for 
indenting (for those who don't use autoindent).

   - Significant improvements for all multithreaded versions of ipython.  Now, 
if your threaded code raises exceptions, instead of seeing a crash report, a 
normal (colored, verbose, etc.) exception is printed.  Additionally, if you 
have pdb on, it will activate in your threaded code.  Very nice for 
interactively debugging GUI programs.

   - Many fixes to embedded ipython, including proper handling of globals and 
tab completion.

   - New -t and -o options to %logstart, to respectively put timestamps in 
your logs, and to also log all output (tagged as #[Out]#).  The default log 
name is now ipython_log.py, to better reflect that logs remain valid Python 
source.

   - Lightweight persistence mechanism via %store.  IPython had always had 
%save, to write out a group of input lines directly to a file.  Now, its 
%store companion stores persistently (associated with your profile, and 
auto-loaded at startup) not just source, but any python variable which can be 
pickled.  Thanks to Matt Wilkie for the request, and ville for the patches.

   - New guarantee that, if you disable autocalling, ipython will never call 
getattr() on your objects.  This solves problems with code that has 
side-effects on attribute access.  Note that TAB-completion inevitably does 
call getattr(), so not all forms of side-effects can be eliminated.

   - Unicode support for prompts.

   - Improvements to path handling under win32.  Thanks to Ville and Jorgen 
for the patches.

   - Improvements to pager under win32.  Contributed by Alexander Belchenko.

   - Demo class for interactive demos using ipython.

   - %pycat magic for showing syntax-highlighted python sources

   - support for download_url in setup.py, so PyPI (and setuptools) work 
transparently with ipython.

   - New exit/quit magics to exit, conditionally asking (%Exit/%Quit don't)

   - Automatically reopen the editor if your file has a syntax error in it 
(when using the %edit system).

   - A large amount of internal reorganization and cleanup, to allow the code 
to be more readily moved over to the chainsaw branch (see below).

   - Many other small fixes and enhancements.  The changelog has full details.


All of these changes are significant enough, that the new release will be 
numbered 0.7.0.  I would greatly appreciate testing of the release candidate I 
just uploaded here:

http://ipython.scipy.org/dist/testing

With all these changes, I'm sure I've introduced some problems.  Please let me 
know (on-list) so I can fix them asap.


Next: playing with the chainsaw
-------------------------------

Now, as I've mentioned before, the long-term development branch for ipython is 
chainsaw:

http://projects.scipy.org/ipython/ipython/browser/ipython/branches/chainsaw/

Brian Granger has been busy at work here, building from the ground up the 
infrastructure for what will be IPython 1.0.  Instead of trying to refactor 
the current mess and stalling him while I work, we've been working on a good 
design, and we will port over all the good pieces from the current codebase. 
Much of my recent cleanup work has gone in that direction, so in a sense 0.7.0 
is the beginning of that branch, though I don't expect it to be fully 
functional for a while yet.  In the meantime, I'll continue to accept patches 
for trunk, as well as fixing bugs.

I have very high expectations for what we'll be able to do with this new code. 
  These two presentations highlight some of what we have in mind:

http://ipython.scipy.org/misc/ipython-notebooks-scipy05.pdf
http://ipython.scipy.org/misc/scipy05-parallel.pdf

All the work done this summer by Robert, Tzanko and Toni will be used by this 
new infrastructure, as well as providing a system to do a number of new and 
interesting things.  Anyone willing to work on this (experience with parallel 
computing, twisted, xml or wxpython a bonus) is welcome to join in.


So I think that the future looks bright for ipython: the current system has 
new and important functionality, we're making good progress on the foundations 
for the new one, and I'm coding again :)  Many thanks to all who have helped 
so far, and Happy New Year to everybody!

Best,

f



From vivainio at gmail.com  Fri Dec 30 03:01:22 2005
From: vivainio at gmail.com (Ville Vainio)
Date: Fri, 30 Dec 2005 10:01:22 +0200
Subject: [IPython-dev] System commands fail (path not assigned)
Message-ID: <46cb515a0512300001i2bec66c4x75ac63d0989e44fa@mail.gmail.com>

There's a bug in current svn.

Did svn up, setup.py install, started ipython -p pysh, entered system
command ls, got

[opt\environmentswitch]|1> ls
---------------------------------------------------------------------------
exceptions.UnboundLocalError                         Traceback (most recent call
 last)

c:\python24\Lib\site-packages\IPython\iplib.py in <lambda>(cmd)
    518         self.system = lambda cmd: shell(self.var_expand(cmd),
    519                                         header='IPython system call: ',
--> 520                                         verbose=self.rc.system_verbose)
    521         # These are for getoutput and getoutputerror:
    522         self.getoutput = lambda cmd: \

c:\python24\Lib\site-packages\IPython\genutils.py in shell(cmd, verbose, debug,
header)
    353                 os.chdir(path)
    354         else:
--> 355             shell_ori('"pushd %s&&"'%path+cmd,verbose,debug,header)
    356
    357     shell.__doc__ = shell_ori.__doc__

UnboundLocalError: local variable 'path' referenced before assignment

--
Ville Vainio    http://tinyurl.com/2prnb
  Can't you see the world is burning
  Can't you feel its Fire burning
  Don't you know we all are burning
  The Fire of Life       -- R.N.Taylor / Changes



From Fernando.Perez at colorado.edu  Fri Dec 30 10:45:55 2005
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Fri, 30 Dec 2005 08:45:55 -0700
Subject: [IPython-dev] System commands fail (path not assigned)
In-Reply-To: <46cb515a0512300001i2bec66c4x75ac63d0989e44fa@mail.gmail.com>
References: <46cb515a0512300001i2bec66c4x75ac63d0989e44fa@mail.gmail.com>
Message-ID: <43B55633.7060808@colorado.edu>

Ville Vainio wrote:
> There's a bug in current svn.
> 
> Did svn up, setup.py install, started ipython -p pysh, entered system
> command ls, got
> 
> [opt\environmentswitch]|1> ls
> ---------------------------------------------------------------------------
> exceptions.UnboundLocalError                         Traceback (most recent call
>  last)
> 
> c:\python24\Lib\site-packages\IPython\iplib.py in <lambda>(cmd)
>     518         self.system = lambda cmd: shell(self.var_expand(cmd),
>     519                                         header='IPython system call: ',
> --> 520                                         verbose=self.rc.system_verbose)
>     521         # These are for getoutput and getoutputerror:
>     522         self.getoutput = lambda cmd: \
> 
> c:\python24\Lib\site-packages\IPython\genutils.py in shell(cmd, verbose, debug,
> header)
>     353                 os.chdir(path)
>     354         else:
> --> 355             shell_ori('"pushd %s&&"'%path+cmd,verbose,debug,header)
>     356
>     357     shell.__doc__ = shell_ori.__doc__
> 
> UnboundLocalError: local variable 'path' referenced before assignment

OOPS, sorry.  The bug is win32-only, I mis-applied Jorgen's patch for network 
shares when I refactored it.  Thanks for reporting this.  I've just updated 
svn and released an rc2 with the fix.  If you want to apply it manually, it's 
trivial:

Index: IPython/genutils.py
===================================================================
--- IPython/genutils.py (revision 979)
+++ IPython/genutils.py (working copy)
@@ -352,7 +352,7 @@
              finally:
                  os.chdir(path)
          else:
-            shell_ori('"pushd %s&&"'%path+cmd,verbose,debug,header)
+            shell_ori(cmd,verbose,debug,header)

      shell.__doc__ = shell_ori.__doc__


Thanks for the report.

Cheers,

f