From Fernando.Perez at colorado.edu  Thu Jul  1 02:47:50 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Jul 2004 00:47:50 -0600
Subject: [IPython-dev] tab completion w/ spaces in filename - still broken
In-Reply-To: <40E32AB2.60806@colorado.edu>
References: <40E19655.1020301@kolumbus.fi> <40E1A7C3.8070803@colorado.edu>
	<40E1C00A.5050806@kolumbus.fi> <40E1DBAB.3090902@colorado.edu>
	<40E1EE84.2050209@kolumbus.fi> <40E2382A.70204@colorado.edu>
	<40E30C35.10104@kolumbus.fi> <40E32142.1000805@colorado.edu>
	<40E32921.3080000@kolumbus.fi> <40E32AB2.60806@colorado.edu>
Message-ID: <40E3B396.5040502@colorado.edu>

Ok, here's the best I could do with this as a patch against CVS.  I played for 
a while with readline, and it _almost_ works, but it's ugly.  Here's the behavior:

maqroll[~/test]|1> ls div*
div.c  div.f  div.py
maqroll[~/test]|2> cat div <TAB HERE>
div.c   div.f   div.py
maqroll[~/test]|2> cat div. <TAB HERE>
div.c   div.f   div.py
maqroll[~/test]|2> ls aa*
aa b c  aa b d
maqroll[~/test]|3> cat aa <TAB HERE>
aa\ b\ c  aa\ b\ d
maqroll[~/test]|3> cat aa\ b\ <TAB HERE>
c  d

As you can see, when the filenames have spaces in them, the printed 
completions are only the ends instead of the full names.  This is a bit 
disconcerting, but I failed to fix it and I can't really spend much more time 
on this.

At least it _does_ provide the correct completions, so while I'm not quite 
100% happy, for now I may leave it at that unless you can finish fixing it.

It does need a bit of cleaning (easy), and I'd move the regex to an init-time 
instance var to avoid the extra dict lookups of the compile on every complete 
call.  But that's a minor optimization for later.  It would be great if you 
can hammer the last bit on this one (or provide an alternative which works 
100% correctly).

Note that I also didn't test every combination of funny filenames/spaces 
possible, so I may have missed a corner case.

Regards,

f
-------------- next part --------------
An embedded and charset-unspecified text was scrubbed...
Name: space_completers.diff
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20040701/9ad16914/attachment.ksh>

From mwh at python.net  Thu Jul  1 10:47:35 2004
From: mwh at python.net (Michael Hudson)
Date: Thu, 01 Jul 2004 15:47:35 +0100
Subject: [IPython-dev] Re: tab completion w/ spaces in filename - still
	broken
References: <40E19655.1020301@kolumbus.fi> <40E1A7C3.8070803@colorado.edu>
	<40E1C00A.5050806@kolumbus.fi> <40E1DBAB.3090902@colorado.edu>
	<40E1EE84.2050209@kolumbus.fi> <40E2382A.70204@colorado.edu>
	<40E30C35.10104@kolumbus.fi> <40E32142.1000805@colorado.edu>
	<40E32921.3080000@kolumbus.fi>
Message-ID: <2mhdsr3htk.fsf@starship.python.net>

Ville Vainio <vivainio at kolumbus.fi> writes:

> I've never really used readline,

You lucky, lucky man.

Cheers,
mwh

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                                and quoted by Aahz on comp.lang.python



From jack at xiph.org  Thu Jul  1 10:56:54 2004
From: jack at xiph.org (Jack Moffitt)
Date: Thu, 1 Jul 2004 08:56:54 -0600
Subject: [IPython-dev] [jdc@uwo.ca: Bug#256781: ipython: funny behaviour of
	timeit.timeit and timeit.repeat]
Message-ID: <20040701145654.GO6897@i.cantcode.com>

The following bug came into Debian a few days back, and I'd though I'd
forward it here to see if you had any ideas on what it was.  The user
made a second followup which you can find here:

http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=256781

jack.

----- Forwarded message from Dan Christensen <jdc at uwo.ca> -----

Subject: Bug#256781: ipython: funny behaviour of timeit.timeit and
	timeit.repeat
From: Dan Christensen <jdc at uwo.ca>
To: Debian Bug Tracking System <submit at bugs.debian.org>
Date: Mon, 28 Jun 2004 21:48:50 -0400
Resent-Sender: Debian BTS <debbugs at bugs.debian.org>

Package: ipython
Version: 0.6.0-1
Severity: normal

[I manually changed the input prompts to >>> and removed the
output labels.]

On a 2.0GHz P4 laptop (speedstep disabled) running mostly testing:

Python 2.3.4 (#2, May 29 2004, 03:31:27) 
Type "copyright", "credits" or "license" for more information.

IPython 0.6.0 -- An enhanced Interactive Python.

>>> from timeit import *
>>> from time import *
>>> t=Timer("pass","pass",timer=time)
>>> t.timeit(1000000)
0.15831089019775391
>>> t.repeat(1,1000000)
[0.079621076583862305]

Note the factor of two difference!  It's consistent:

>>> t.timeit(1000000)
0.14923691749572754
>>> t.repeat(1,1000000)
[0.080674886703491211]
>>> for i in range(3):
...     t.timeit(1000000)
0.15053701400756836
0.15190696716308594
0.15034103393554688
>>> t.repeat(3,1000000)
[0.080902099609375,
 0.081040859222412109,
 0.080629110336303711]

And it happens with timer=clock too:

>>> t2=Timer("pass","pass",timer=clock)
>>> t2.timeit(1000000)
0.11000000000000032
>>> t2.repeat(3,1000000)
[0.079999999999999627, 0.080000000000000071, 0.090000000000000302]

It also happens with statements other than "pass".  The ratio is
different then, and sometimes the repeat version takes longer
(in one case, consistently 20% longer).

This doesn't happen when the above is run in plain python.

And it doesn't happen using ipython on a dual Xeon machine I have
access to, which also runs Debian testing.

Any ideas what's up?  I've looked at timeit.py and can't see how this
is possible...  Let me know if I can help debug it.

Dan


-- System Information:
Debian Release: testing/unstable
  APT prefers testing
  APT policy: (990, 'testing'), (500, 'unstable'), (1, 'experimental')
Architecture: i386 (i686)
Kernel: Linux 2.4.26
Locale: LANG=C, LC_CTYPE=C

Versions of packages ipython depends on:
ii  python                        2.3.4-1    An interactive high-level object-o

-- no debconf information

----- End forwarded message -----



From jack at xiph.org  Thu Jul  1 10:59:23 2004
From: jack at xiph.org (Jack Moffitt)
Date: Thu, 1 Jul 2004 08:59:23 -0600
Subject: [IPython-dev] "@hist -n" raises error
Message-ID: <20040701145923.GQ6897@i.cantcode.com>

Another Debian bug:
http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=252525

Entering

  @hist -n

without a following integer in the IPython shell raises a ValueError,
while documentation states this should be possible.

jack.



From Fernando.Perez at colorado.edu  Thu Jul  1 11:53:00 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Jul 2004 09:53:00 -0600
Subject: [IPython-dev] "@hist -n" raises error
In-Reply-To: <20040701145923.GQ6897@i.cantcode.com>
References: <20040701145923.GQ6897@i.cantcode.com>
Message-ID: <40E4335C.9030907@colorado.edu>

Jack Moffitt wrote:
> Another Debian bug:
> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=252525
> 
> Entering
> 
>   @hist -n
> 
> without a following integer in the IPython shell raises a ValueError,
> while documentation states this should be possible.

Known, fixed in current CVS.

Cheers,

f



From Fernando.Perez at colorado.edu  Thu Jul  1 11:57:53 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Jul 2004 09:57:53 -0600
Subject: [IPython-dev] [jdc@uwo.ca: Bug#256781: ipython: funny behaviour
	of	timeit.timeit and timeit.repeat]
In-Reply-To: <20040701145654.GO6897@i.cantcode.com>
References: <20040701145654.GO6897@i.cantcode.com>
Message-ID: <40E43481.30605@colorado.edu>

Jack Moffitt wrote:
> The following bug came into Debian a few days back, and I'd though I'd
> forward it here to see if you had any ideas on what it was.  The user
> made a second followup which you can find here:

This is really, really weird.  I can't think of anything in ipython which 
could cause this, and since your user seems to be narrowing it down to a pure 
python problem, I'll wait for now.  Keep me posted if you guys conclude that 
it's really an ipython-caused issue (I hope not :)

> [I manually changed the input prompts to >>> and removed the
> output labels.]

You can run ipython in 'classic' mode to get these prompts if you want:

[~]> ipython -classic
Python 2.2.3 (#1, Oct 15 2003, 23:33:35)
Type "copyright", "credits" or "license" for more information.

IPython 0.6.1.cvs -- An enhanced Interactive Python.
?       -> Introduction to IPython's features.
@magic  -> Information about IPython's 'magic' @ functions.
help    -> Python's own help system.
object? -> Details about 'object'. ?object also works, ?? prints more.

Launching IPython in classic mode. No config file read.
 >>> 9
9
 >>>


Best,

f



From vivainio at kolumbus.fi  Thu Jul  1 14:08:36 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Thu, 01 Jul 2004 21:08:36 +0300
Subject: [IPython-dev] readline suckage
In-Reply-To: <40E44BAB.1090400@colorado.edu>
References: <40E44A78.7070205@kolumbus.fi> <40E44BAB.1090400@colorado.edu>
Message-ID: <40E45324.80009@kolumbus.fi>

Fernando Perez wrote:

> Did you have a go at that bitchy readline thing with whitespaces?  It 
> pissed me off last night...

No, I figured the fix you implemented is good enough for now, and if the 
readline really expects the completions given by "complete()" to 
actually begin with the "text" argument (as the docs state), there is 
very little we can do about it for now. Perhaps the fix should be 
focused on the python implementation of readline, which is needed on 
windows anyway (and windows tend to suffer from space-ridden filenames 
more than *n*x anyway...).

However, gnu readline does support spaces in filenames (as evidenced by 
bash & friends). So perhaps the fix should be made to the python 
readline wrapper?



From Fernando.Perez at colorado.edu  Thu Jul  1 15:12:29 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 01 Jul 2004 13:12:29 -0600
Subject: [IPython-dev] readline suckage
In-Reply-To: <40E45324.80009@kolumbus.fi>
References: <40E44A78.7070205@kolumbus.fi> <40E44BAB.1090400@colorado.edu>
	<40E45324.80009@kolumbus.fi>
Message-ID: <40E4621D.2090402@colorado.edu>

Ville Vainio wrote:

> No, I figured the fix you implemented is good enough for now, and if the 
> readline really expects the completions given by "complete()" to 
> actually begin with the "text" argument (as the docs state), there is 
> very little we can do about it for now. Perhaps the fix should be 
> focused on the python implementation of readline, which is needed on 
> windows anyway (and windows tend to suffer from space-ridden filenames 
> more than *n*x anyway...).

Ok, so it stays for now.  People shouldn't use spaces in their filenames 
anyway, we might as well ipython as an educational tool :)

> However, gnu readline does support spaces in filenames (as evidenced by 
> bash & friends). So perhaps the fix should be made to the python 
> readline wrapper?

I think that the problem is that readline.py only wraps part of the full gnu 
readline.  If you look at the GNU readline docs:

http://cnswww.cns.cwru.edu/php/chet/readline/rltop.html

you'll see that it's huge.  Python's readline module doesn't really expose too 
much:

planck[~/test]|6> import readline
planck[~/test]|7> readline.
readline.add_history                 readline.parse_and_bind
readline.get_begidx                  readline.read_history_file
readline.get_completer               readline.read_init_file
readline.get_completer_delims        readline.redisplay
readline.get_current_history_length  readline.set_completer
readline.get_endidx                  readline.set_completer_delims
readline.get_history_item            readline.set_history_length
readline.get_history_length          readline.set_pre_input_hook
readline.get_line_buffer             readline.set_startup_hook
readline.insert_text                 readline.write_history_file

Enough for many purposes, but definitely not the whole monster.

The reason is probably the normal historical one: they wrapped as much as was 
needed for the more common usage cases, and it stopped there.  I'm sure one 
could dig into the C wrapper and finish up the job.  However, I will most 
definitely NOT spend my time on that, given that the cost/benefit ratio feels 
to me very much divergent.

Best,

f



From vivainio at kolumbus.fi  Fri Jul  2 05:49:52 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Fri, 02 Jul 2004 12:49:52 +0300
Subject: [IPython-dev] Re: bookmarks, episode 4 (patch)
In-Reply-To: <40E507E2.40802@colorado.edu>
References: <40E44A78.7070205@kolumbus.fi> <40E507E2.40802@colorado.edu>
Message-ID: <40E52FC0.4060703@kolumbus.fi>

Fernando Perez wrote:

> That's all I can think of now.  I also committed the spaces in 
> filenames stuff, after cleaning it up a bit.  It's ugly, but at least 
> it does provide the needed functionality.  So it stays.

Yep, I didn't have any gripes with the "feel" of the functionality.

> If you have any experience with gettext and can help with (a), 
> awesome.  If not, I'll try to work on it this weekend.  I'll probably 
> put out a release candidate and let it simmer for a full week: I'm 
> gone on a

Could the bug be circumvented by doing "tr = _" immediately after 
l.install(), and just using 'tr' instead of _ for translating stuff?



From Fernando.Perez at colorado.edu  Fri Jul  2 09:01:59 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Fri, 02 Jul 2004 07:01:59 -0600
Subject: [IPython-dev] Re: bookmarks, episode 4 (patch)
In-Reply-To: <40E52FC0.4060703@kolumbus.fi>
References: <40E44A78.7070205@kolumbus.fi> <40E507E2.40802@colorado.edu>
	<40E52FC0.4060703@kolumbus.fi>
Message-ID: <40E55CC7.7000503@colorado.edu>

Ville Vainio wrote:

>>If you have any experience with gettext and can help with (a), 
>>awesome.  If not, I'll try to work on it this weekend.  I'll probably 
>>put out a release candidate and let it simmer for a full week: I'm 
>>gone on a
> 
> 
> Could the bug be circumvented by doing "tr = _" immediately after 
> l.install(), and just using 'tr' instead of _ for translating stuff?

Do you mean circumvented by the user?  That's definitely an option, but I'd 
like to really fix it in the core.  Especially because they would have to do

l.install();tr=_

so that it all happens before ipython has a chance to mangle the _ variable. 
Plus, it makes testing of real code fragments which rely on _ tricky in 
ipython.  So I think it's worth fixing it for real.

Cheers,

f



From vivainio at kolumbus.fi  Fri Jul  2 09:48:33 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Fri, 02 Jul 2004 16:48:33 +0300
Subject: [IPython-dev] underscore and gettext interaction
In-Reply-To: <40E55CC7.7000503@colorado.edu>
References: <40E44A78.7070205@kolumbus.fi> <40E507E2.40802@colorado.edu>
	<40E52FC0.4060703@kolumbus.fi> <40E55CC7.7000503@colorado.edu>
Message-ID: <40E567B1.2060402@kolumbus.fi>

Fernando Perez wrote:

> Do you mean circumvented by the user?  That's definitely an option, 
> but I'd like to really fix it in the core.  Especially because they 
> would have to do
>
> l.install();tr=_
>
> so that it all happens before ipython has a chance to mangle the _ 
> variable. Plus, it makes testing of real code fragments which rely on 
> _ tricky in ipython.  So I think it's worth fixing it for real.

Apparently normal python interactive interpreter stores "_" in 
__builtin__, which should achieve what you want...



From Fernando.Perez at colorado.edu  Fri Jul  2 12:02:33 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Fri, 02 Jul 2004 10:02:33 -0600
Subject: [IPython-dev] underscore and gettext interaction
In-Reply-To: <40E567B1.2060402@kolumbus.fi>
References: <40E44A78.7070205@kolumbus.fi> <40E507E2.40802@colorado.edu>
	<40E52FC0.4060703@kolumbus.fi> <40E55CC7.7000503@colorado.edu>
	<40E567B1.2060402@kolumbus.fi>
Message-ID: <40E58719.7050802@colorado.edu>

Ville Vainio wrote:
> Fernando Perez wrote:
> 
> 
>>Do you mean circumvented by the user?  That's definitely an option, 
>>but I'd like to really fix it in the core.  Especially because they 
>>would have to do
>>
>>l.install();tr=_
>>
>>so that it all happens before ipython has a chance to mangle the _ 
>>variable. Plus, it makes testing of real code fragments which rely on 
>>_ tricky in ipython.  So I think it's worth fixing it for real.
> 
> 
> Apparently normal python interactive interpreter stores "_" in 
> __builtin__, which should achieve what you want...

Thanks for the hint, that put me on the right track.  In fact, ipython is now 
_better_ than standard python in its handling of '_' for interactive work: 
real python eventually clobbers gettext, but only on a second call (because 
gettext also puts '_' in __builtin__).  IPython will detect usage of '_' in 
__builtin__ and stay away always.

It took some playing to get it to work, but it's now in CVS.

I think we're getting ready to cut an RC for 0.6.1 this weekend.  Anyone with 
a remaining issue please say so now.

Best,

f



From vivainio at kolumbus.fi  Sat Jul  3 04:35:24 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Sat, 03 Jul 2004 11:35:24 +0300
Subject: [IPython-dev] 'pysh' script in the 'scripts' dir
Message-ID: <40E66FCC.3000502@kolumbus.fi>

I think it might be handy to have a script in the scripts directory that 
runs

ipython -p pysh $1 $2 $3 $4 $5

People are going to want to have one anyway.



From Fernando.Perez at colorado.edu  Sat Jul  3 16:14:15 2004
From: Fernando.Perez at colorado.edu (Fernando.Perez at colorado.edu)
Date: Sat,  3 Jul 2004 14:14:15 -0600
Subject: [IPython-dev] 'pysh' script in the 'scripts' dir
In-Reply-To: <40E66FCC.3000502@kolumbus.fi>
References: <40E66FCC.3000502@kolumbus.fi>
Message-ID: <1088885655.40e71397dab26@webmail.colorado.edu>

Quoting Ville Vainio <vivainio at kolumbus.fi>:

> I think it might be handy to have a script in the scripts directory that
> runs
>
> ipython -p pysh $1 $2 $3 $4 $5
>
> People are going to want to have one anyway.

Not for now. Anything going into scripts requires proper full manpages, written
in troff, to satisfy Debian packaging policies.  Whoever wants this is capable
of writing a one-line alias (I have aliased pysh to 'ipython --nobanner -p
pysh').

Cheers,

f



From Fernando.Perez at colorado.edu  Sun Jul 18 22:52:31 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Sun, 18 Jul 2004 20:52:31 -0600
Subject: [IPython-dev] IPython 0.6.1 -- RC1 is out
Message-ID: <40FB376F.6090501@colorado.edu>

Hi everyone,

I've put out a release candidate for ipython 0.6.1 in

http://ipython.scipy.org/dist/testing

This includes pysh, along with a bunch of recent fixes.  If all goes well and 
nobody reports major breakage, I'll cut 0.6.1 in a week or so.  I'd very much 
appreciate some feedback, since I plan on announcing this release on c.l.py 
and the scipy lists (it's been a long time since I've announced anything 
ipython-related outside of our internal mailing list).

Regards,

f



From vivainio at kolumbus.fi  Mon Jul 19 15:43:00 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Mon, 19 Jul 2004 22:43:00 +0300
Subject: [IPython-dev] Another filename completion defect on win32 - parens
Message-ID: <40FC2444.5050202@kolumbus.fi>

It sucks to be the bearer of bad win32 news again ;-).

In the latest release candidate, completion of spaces works ok. 
Completion of parens, OTOH, doesn't.

@ursa[tmp]|261> mkdir "(123)foobar"
@ursa[tmp]|262> ls
(123)foobar\  (hi)ho\
@ursa[tmp]|263> cd (123)foobar\
------------------------------------------------------------
   File "<console>", line 1
     cd (123)foobar\
                  ^
SyntaxError: invalid syntax

@ursa[tmp]|264> mkdir foobar
@ursa[tmp]|265> cd foobar/

Note how the suggested completion for (123)foobar ends with the dreaded 
backslash. That should be forward slash, which works ok for plain 
"foobar" without parens, as can be seen from the example. cd'ing to the 
directory works with backslashes, and doesn't work without:

@ursa[tmp]|272> cd (123)foobar
------------------------------------------------------------
   File "<console>", line 1
     cd (123)foobar
                  ^
SyntaxError: invalid syntax

@ursa[tmp]|273> cd \(123\)foobar
@ursa[(123)foobar]|274>

It works with quotes too: cd "(123)foobar" is ok.

Typing "cd \(12" and pressing tabs results in a horror of chtulhuan 
proportions:

cd \(/(123/

Interestingly, the following (quotes do not start the file name) works 
without a hitch:

@ursa[tmp]|281> mkdir barbar(123)
@ursa[tmp]|282> cd barbar(123)/
@ursa[barbar(123)]|283>

The completion esacping hack that was done to spaces should probably be 
done to parens too.



From Fernando.Perez at colorado.edu  Mon Jul 19 17:33:06 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Mon, 19 Jul 2004 15:33:06 -0600
Subject: [IPython-dev] Another filename completion defect on win32 - parens
In-Reply-To: <40FC2444.5050202@kolumbus.fi>
References: <40FC2444.5050202@kolumbus.fi>
Message-ID: <40FC3E12.1090305@colorado.edu>

Ville Vainio wrote:
> It sucks to be the bearer of bad win32 news again ;-).
> 
> In the latest release candidate, completion of spaces works ok. 
> Completion of parens, OTOH, doesn't.

[snip]

> The completion esacping hack that was done to spaces should probably be 
> done to parens too.

Good god!  I'm honestly not sure I have the stomach to write all the necessary 
escaping code for correctly handling '()[] ' characters uniformly in 
filenames, with completions.  I'll gladly accept a patch.

I know this reply sucks, but I'm too swamped now with work and other things, 
and I'd like to at least put officially 0.6.1 out, even if the tab-completion 
is not perfect.  To be honest, filenames with such characters are a pain to 
handle even in unix with 'good' shells (bash/tcsh), since you can't for 
example use highlight/middle-button-paste on them.  You must either wrap them 
in "" or manually escape all those special chars.  That's why I just think 
they don't belong in filenames to begin with :)

If you think you have the patience to code this robustly for all the legal 
characters which require escaping, I'll obviously accept a patch.  But right 
now it's a bit more than I'm willing to tackle myself: the effort/benefit 
ratio just isn't there.  I have very little time for ipython, and I really 
want to write the multithreaded version so people can use ipython with 
matplotlib and other WxPython/GTK apps.

It might not be a bad idea to file your last email on the bug tracker so we 
don't lose track of this in the future, if neither of us gets to do it now.  I 
_do_ think it's important, I just can't do it now myself.

Cheers,

f



From vivainio at kolumbus.fi  Tue Jul 20 03:37:02 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Tue, 20 Jul 2004 10:37:02 +0300
Subject: [IPython-dev] Another filename completion defect on win32 - parens
In-Reply-To: <40FC3E12.1090305@colorado.edu>
References: <40FC2444.5050202@kolumbus.fi> <40FC3E12.1090305@colorado.edu>
Message-ID: <40FCCB9E.5040106@kolumbus.fi>

Fernando Perez wrote:

>> The completion esacping hack that was done to spaces should probably 
>> be done to parens too.
>
>
> Good god!  I'm honestly not sure I have the stomach to write all the 
> necessary escaping code for correctly handling '()[] ' characters 
> uniformly in filenames, with completions.  I'll gladly accept a patch.
>
> I know this reply sucks, but I'm too swamped now with work and other 
> things, and I'd like to at least put officially 0.6.1 out, even if the 
> tab-completion is not perfect.  To be honest, filenames with such

I'll look into fixing it at some point. If it's not ready this week, it 
will probably make sense to just put out 0.6.1 and fix this later on.

> It might not be a bad idea to file your last email on the bug tracker 
> so we don't lose track of this in the future, if neither of us gets to 
> do it now.  I _do_ think it's important, I just can't do it now myself.

I'll file it if we can't fix it now :-).



From vivainio at kolumbus.fi  Tue Jul 20 03:54:57 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Tue, 20 Jul 2004 10:54:57 +0300
Subject: [IPython-dev] Another filename completion defect on win32 - parens
In-Reply-To: <40FCCB9E.5040106@kolumbus.fi>
References: <40FC2444.5050202@kolumbus.fi> <40FC3E12.1090305@colorado.edu>
	<40FCCB9E.5040106@kolumbus.fi>
Message-ID: <40FCCFD1.4090703@kolumbus.fi>

Ville Vainio wrote:

>> It might not be a bad idea to file your last email on the bug tracker 
>> so we don't lose track of this in the future, if neither of us gets 
>> to do it now.  I _do_ think it's important, I just can't do it now 
>> myself.
>
>
> I'll file it if we can't fix it now :-).

Just so you know, it seems to be broken on Linux as well, albeit 
differently. Actually, this is a good thing - I won't need to be in 
windos to fix this ;-).



From vivainio at kolumbus.fi  Tue Jul 20 06:21:53 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Tue, 20 Jul 2004 13:21:53 +0300
Subject: [IPython-dev] Patch for paren completion glitch
Message-ID: <40FCF241.2050003@kolumbus.fi>

I managed to make "(foo)bar" completion work for filenames, without 
breaking completion for space characters (or anything else I could try 
out quickly). It seems to work on Linux at  least - haven't tried it on 
Windows yet, I thought it might be a good idea to show the patch for 
some quick feedback before that.

As you can guess it's brutally hackish, but I figure that's the way it 
goes with readline ;-).

It should be trivial to add the functionality for other delimiter chars. 
I used shlex.split directly, it should be changef to shlex_split from 
magic for bacwards compatibility but I thought I'll do it later on...
-------------- next part --------------
A non-text attachment was scrubbed...
Name: special_char_completion.diff
Type: text/x-patch
Size: 3895 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20040720/473eef05/attachment.bin>

From vivainio at kolumbus.fi  Tue Jul 20 13:11:45 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Tue, 20 Jul 2004 20:11:45 +0300
Subject: [IPython-dev] paren completion works ok win32 too
Message-ID: <40FD5251.1070006@kolumbus.fi>

I'm just confirming that the paren completion patch I sent today works 
ok on windows too, so it should be committable after fixing the shlex 
part to use shlex_split.

Windows install script should perhaps also install a "pysh" shortcut in 
the start menu that does:

I:\opt\Python23\python.exe "C:\Program Files\IPython\IPython_shell.py" 
-p pysh



From Fernando.Perez at colorado.edu  Tue Jul 20 14:20:56 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Tue, 20 Jul 2004 12:20:56 -0600
Subject: [IPython-dev] paren completion works ok win32 too
In-Reply-To: <40FD5251.1070006@kolumbus.fi>
References: <40FD5251.1070006@kolumbus.fi>
Message-ID: <40FD6288.704@colorado.edu>

Ville Vainio wrote:
> I'm just confirming that the paren completion patch I sent today works 
> ok on windows too, so it should be committable after fixing the shlex 
> part to use shlex_split.

If you have py2.2, could you please test this?  Or even hard-wire shlex_split 
to be the one used in your tests?  Since I need to support py22, this is 
necessary.

> Windows install script should perhaps also install a "pysh" shortcut in 
> the start menu that does:
> 
> I:\opt\Python23\python.exe "C:\Program Files\IPython\IPython_shell.py" 
> -p pysh

Noted.  I'll see if there's an easy way to fish out the python.exe path, I'm 
sure not everybody has it in I:\opt :)

I won't get to any of this till the weekend, I'm very busy.  So feel free to 
refine it and test it further.

Best, and thanks for the work,

f



From vivainio at kolumbus.fi  Tue Jul 20 14:58:14 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Tue, 20 Jul 2004 21:58:14 +0300
Subject: [IPython-dev] paren completion works ok win32 too
In-Reply-To: <40FD6288.704@colorado.edu>
References: <40FD5251.1070006@kolumbus.fi> <40FD6288.704@colorado.edu>
Message-ID: <40FD6B46.8020503@kolumbus.fi>

Fernando Perez wrote:

> If you have py2.2, could you please test this?  Or even hard-wire 
> shlex_split to be the one used in your tests?  Since I need to support 
> py22, this is necessary.


Ok, I'm hard_wiring shlex_split.


> Noted.  I'll see if there's an easy way to fish out the python.exe 
> path, I'm sure not everybody has it in I:\opt :)

Do whatever you are doing to install the normal ipython shortcut, i.e. 
in ipython_win_post_install.py:

    make_shortcut('IPython.lnk',sys.executable, '"%s"' % ip_filename,
                  my_documents_dir,
                  'IPython - Enhanced python command line interpreter',
                  (pycon_icon,0))

(i.e. sys.executable is the path to python).

> I won't get to any of this till the weekend, I'm very busy.  So feel 
> free to refine it and test it further.


Ack.



From Fernando.Perez at colorado.edu  Thu Jul 22 02:42:10 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Jul 2004 00:42:10 -0600
Subject: [IPython-dev] Patch for paren completion glitch
In-Reply-To: <40FCF241.2050003@kolumbus.fi>
References: <40FCF241.2050003@kolumbus.fi>
Message-ID: <40FF61C2.5090301@colorado.edu>

Ville Vainio wrote:
> I managed to make "(foo)bar" completion work for filenames, without 
> breaking completion for space characters (or anything else I could try 
> out quickly). It seems to work on Linux at  least - haven't tried it on 
> Windows yet, I thought it might be a good idea to show the patch for 
> some quick feedback before that.
> 
> As you can guess it's brutally hackish, but I figure that's the way it 
> goes with readline ;-).
> 
> It should be trivial to add the functionality for other delimiter chars. 
> I used shlex.split directly, it should be changef to shlex_split from 
> magic for bacwards compatibility but I thought I'll do it later on...

It's pretty ugly, but I doubt much better is achievable given the problem at 
hand.  I am, however, a bit concerned about performance given that this stuff 
gets called for _every_ filename in the completions match, which can be a lot 
in a big directory.  People expect tab-completion to be near-instantaneous, 
and I'd like to keep it that way.  In particular, I think that

def unprotect_filename(s):
     chs = []
     in_escape = False
     for ch in s:
         if in_escape:
             chs.append(ch)
             in_escape = False
             continue
         if ch == '\\':
             in_escape = True
             continue
         chs.append(ch)

     return "".join(chs)

is essentially:

# Alternative unprotect_filename
# About 5 times faster than the original
unprotect_filename2 = lambda s:s.replace('\\','')

Am I right?  If that's the case, it can (and should) be explicitly inlined, 
since function call overhead in python is violent.  Even as a function, the 
second form is about 5 times faster for short strings, which is significant. 
Once inlined, the payoff will be even bigger.

I'm attaching a file which tests that indeed these two return identical 
results for a bunch of random tests.  I also checked the protect_filename, and 
could manage very minor improvements by using a string instead of a list for 
the 'in' check: checking 'char in string' is faster than 'char in list_of_chars'.

You can use the tester at the end for other checks like the one I suggest below.

It would be worth also checking if this:

+            lsplit = shlex.split(lbuf[:self.readline.get_endidx()])[-1]

is faster when done with a regexp instead of shlex.split (the latter is HUGE, 
so I expect it to be pretty slow).

Here:
+                matches = [text0 + protect_filename(f[len(lsplit):]) for f in m0]

the len(lsplit) should be kept in a local outside, since python does not lift 
constants out of loops or listcomps (the python compiler is absolutely 
primitive in the optimizations it attempts).

I agree that it's necessary to do this correctly because people do have 
filenames with these chars in them. But since this is smack in the middle of 
the interactive loop, I really want to be sure that the code is as absolutely 
tight as possible.  Also keep in mind that there may be users out there with 
hardware much slower than yours, so coding for absolute efficiency is 
important, even if it seems to run fine on good hardware.

I'm sure we'll converge on a nice solution shortly.  Just go over every line 
with a maniac eye for optimization fine-tunings.  I've always tried to write 
ipython that way, in the code paths which lie in the middle of the interactive 
loop.  I think the fact that even with all the stuff that goes on it still 
feels reasonably responsive is a testament to the effort being worth it (and 
obviously to the quality of python's implementation).

Thanks for the work!

Best,

f
-------------- next part --------------
A non-text attachment was scrubbed...
Name: comp_strfuns.py
Type: text/x-python
Size: 2238 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20040722/1f0f61fb/attachment.py>

From Fernando.Perez at colorado.edu  Thu Jul 22 10:44:23 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Thu, 22 Jul 2004 08:44:23 -0600
Subject: [IPython-dev] Patch for paren completion glitch
In-Reply-To: <40FF61C2.5090301@colorado.edu>
References: <40FCF241.2050003@kolumbus.fi> <40FF61C2.5090301@colorado.edu>
Message-ID: <40FFD2C7.2060804@colorado.edu>

Fernando Perez wrote:

> and I'd like to keep it that way.  In particular, I think that
> 
> def unprotect_filename(s):
>      chs = []
>      in_escape = False
>      for ch in s:
>          if in_escape:
>              chs.append(ch)
>              in_escape = False
>              continue
>          if ch == '\\':
>              in_escape = True
>              continue
>          chs.append(ch)
> 
>      return "".join(chs)
> 
> is essentially:
> 
> # Alternative unprotect_filename
> # About 5 times faster than the original
> unprotect_filename2 = lambda s:s.replace('\\','')
> 
> Am I right?  If that's the case, it can (and should) be explicitly inlined, 

Ok, I just realized that they are NOT the same when the input has several 
contiguous '\' in it.  Is this kind of filename realistic though?  Are there 
any valid filenames where the two alternatives differ?

cheers,

f



From vivainio at kolumbus.fi  Sun Jul 25 04:52:20 2004
From: vivainio at kolumbus.fi (Ville Vainio)
Date: Sun, 25 Jul 2004 11:52:20 +0300
Subject: [IPython-user] Re: [IPython-dev] Patch for paren completion glitch
In-Reply-To: <4101B6E5.6050609@colorado.edu>
References: <40FCF241.2050003@kolumbus.fi> <40FF61C2.5090301@colorado.edu>
	<4100B635.2010607@kolumbus.fi> <41016260.1080904@colorado.edu>
	<4101B2C1.8000907@kolumbus.fi> <4101B6E5.6050609@colorado.edu>
Message-ID: <410374C4.9090601@kolumbus.fi>

Fernando Perez wrote:

> Cool.  I'll wait for your patch.  So the escapes string is ' ()[]{}', 
> right?

Right. The patch is attached.

Note that shlex.split raises an exception when there is unescaping \ or 
pairless " - currently, this results in no completions.

Interesting feature:

ville at axp[~/tmp]|8> ls
(foo)bar  a b c  a c d  foo(bar)
ville at axp[~/tmp]|9> cat "a b"\ c

(I pressed tab after "a b"). So quotes work ok for completion, they just 
need to be paired.


-------------- next part --------------
A non-text attachment was scrubbed...
Name: ipython_paren_completion_patch.diff
Type: text/x-patch
Size: 6307 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/ipython-dev/attachments/20040725/9b05a9d1/attachment.bin>

From Fernando.Perez at colorado.edu  Mon Jul 26 15:35:14 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Mon, 26 Jul 2004 13:35:14 -0600
Subject: [IPython-user] Re: [IPython-dev] Patch for paren completion glitch
In-Reply-To: <410374C4.9090601@kolumbus.fi>
References: <40FCF241.2050003@kolumbus.fi> <40FF61C2.5090301@colorado.edu>
	<4100B635.2010607@kolumbus.fi> <41016260.1080904@colorado.edu>
	<4101B2C1.8000907@kolumbus.fi> <4101B6E5.6050609@colorado.edu>
	<410374C4.9090601@kolumbus.fi>
Message-ID: <41055CF2.2080005@colorado.edu>

Ville Vainio wrote:
> Fernando Perez wrote:
> 
> 
>>Cool.  I'll wait for your patch.  So the escapes string is ' ()[]{}', 
>>right?
> 
> 
> Right. The patch is attached.

Thanks, applied.  I'll be releasing 0.6.1 probably a bit later today or tonight.

Many thanks for all your excellent work!

Best,

f



From Fernando.Perez at colorado.edu  Mon Jul 26 20:39:51 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Mon, 26 Jul 2004 18:39:51 -0600
Subject: [IPython-dev] IPython 0.6.1 is officially out
Message-ID: <4105A457.2070606@colorado.edu>

Hi all,

I've just uplodaded officially IPython 0.6.1.  Many thanks to all who 
contributed comments, bug reports, ideas and patches.  I'd like in particular 
to thank Ville Vainio, who helped a lot with many of the features for pysh, 
and was willing to put code in front of his ideas.

As always, a big Thank You goes to Enthought and the Scipy crowd for hosting 
ipython and all its attending support services (bug tracker, mailing lists, 
website and downloads, etc).

The download location, as usual, is:

http://ipython.scipy.org/dist

A detailed NEWS file can be found here: http://ipython.scipy.org/NEWS, so I 
won't repeat it.  I will only mention the highlights of this released compared 
to 0.6.0:

* BACKWARDS-INCOMPATIBLE CHANGE:  Users will need to update their ipythonrc 
files and replace '%n' with '\D' in their prompt_in2 settings everywhere. 
Sorry, but there's otherwise no clean way to get all prompts to properly 
align.  The ipythonrc shipped with IPython has been updated.

* 'pysh' profile, which allows you to use ipython as a system shell.  This 
includes mechanisms for easily capturing shell output into python strings and 
lists, and for expanding python variables back to the shell.  It is started, 
like all profiles, with 'ipython -p pysh'.  The following is a brief example 
of the possibilities:

planck[~/test]|3> $$a=ls *.py
planck[~/test]|4> type(a)
               <4> <type 'list'>
planck[~/test]|5> for f in a:
               |.>     if f.startswith('e'):
               |.>         wc -l $f
               |.>
113 error.py
9 err.py
2 exit2.py
10 exit.py

You can get the necessary profile into your ~/.ipython directory by running 
'ipython -upgrade', or by copying it from the IPython/UserConfig directory 
(ipythonrc-pysh).  Note that running -upgrade will rename your existing config 
files to prevent clobbering them with new ones.

This feature had been long requested by many users, and it's at last 
officially part of ipython.

* Improved the @alias mechanism.  It is now based on a fast, lightweight 
dictionary implementation, which was a requirement for making the pysh 
functionality possible.  A new pair of magics, @rehash and @rehashx, allow you 
to load ALL of your $PATH into ipython as aliases at runtime.

* New plot2 function added to the Gnuplot support module, to plot dictionaries 
and lists/tuples of arrays.  Also added automatic EPS generation to hardcopy().

* History is now profile-specific.

* New @bookmark magic to keep a list of directory bookmarks for quick navigation.

* New mechanism for profile-specific persistent data storage.  Currently only 
the new @bookmark system uses it, but it can be extended to hold arbitrary 
picklable data in the future.

* New @system_verbose magic to view all system calls made by ipython.

* For Windows users:  all this functionality now works under Windows, but some 
external libraries are required.  Details here: 
http://ipython.scipy.org/doc/manual/node2.html#sub:Under-Windows

* Fix bugs with '_' conflicting with the gettext library.

* Many, many other bugfixes and minor enhancements.  See the NEWS file linked 
above for the full details.

Enjoy, and please report any problems.

Best,

Fernando Perez.



From Fernando.Perez at colorado.edu  Tue Jul 27 18:28:08 2004
From: Fernando.Perez at colorado.edu (Fernando Perez)
Date: Tue, 27 Jul 2004 16:28:08 -0600
Subject: [IPython-dev] Updated rpms for python 2.3
Message-ID: <4106D6F8.1050708@colorado.edu>

Hi all,

the rpms I uploaded yesterday tagged as py23 were inadvertedly built with 
py22, so things were going to the wrong directory.  I just uploaded corrected 
ones (no code changes at all).  Thanks to Stephen Walton for reporting the 
problem.

Note that the actual version on the site is 0.6.2.  I discovered a small 
problem with the 'cd' command yesterday right after releasing 0.6.1, so I did 
a silent update to 0.6.2 after fixing the bug.

Best,

f