From dagss at student.matnat.uio.no  Sat Aug  1 05:57:15 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 1 Aug 2009 11:57:15 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
Message-ID: <a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>

I am going to play the devil's advocate here -- I'm not into this in order
to make myself enemies, I just have some sincere questions.

Joe Harrington wrote:
> I define success as popular adoption in preference to commercial
> packages.  I believe in vote-with-your-feet: this goal will not be
> reached until all aspects of the package and its presentation to the
> world exceed those of our commercial competition.  Scipy is now a
> grass roots effort, but that takes it only so far.  Other projects,
> such as OpenOffice and Sage, don't follow this model and do produce
> quality products that compete with commercial offerings, at least on
> open-source platforms.  Before we can even hope for that, we have to
> do the following:
>

<snip>

> - Public communication
>   - A real marketing plan
>   - Executing on that plan
>   - Web site geared toward multiple audiences, run by experts at that
>     kind of communication
>   - More webinars, conference booths, training, aimed at all levels
>   - Demos, testimonials, topical forums, all showcased

A thing OpenOffice.org and Sage both have is a very clear sense of
direction and a clearly stated goal.

SciPy might also have that for all I know, but I must admit I haven't
understood what it is in the past year following the SciPy and NumPy
lists, and reading the SciPy site. But I have seen email threads asking
what the SciPy goal is, without any clear resolution (?).

The website says this: "SciPy is open-source software for mathematics,
science, and engineering."

Which of course says nothing at all. Someone asked me what SciPy is the
other day, and while I more or less "know" when I'd try to look in SciPy
for an algorithm (instead of going to, say, R, or netlib.org, or
whatever), I was more or less forced to say that it is a "dumping ground
for various algorithms people have found useful, with the link being them
being either written in Python or wrapped for Python".

That's probably an unfair description -- the point is: If one needs to
formulate a two- or three-liner about SciPy, what would it be? Is it a
goal to reimplement stuff in SciPy that's (for instance) already thriving
in the open source R community, or is that not a goal? And so on.

You might feel this is going off-topic, but I somehow feel that a very
clear sense of direction is paramount when talking of these issues -- just
look at the Sage project.

Dag Sverre



From josef.pktd at gmail.com  Sat Aug  1 08:47:36 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 1 Aug 2009 08:47:36 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
Message-ID: <1cd32cbb0908010547y17385fb6sa790fddf73a37ba9@mail.gmail.com>

On Sat, Aug 1, 2009 at 5:57 AM, Dag Sverre
Seljebotn<dagss at student.matnat.uio.no> wrote:
> I am going to play the devil's advocate here -- I'm not into this in order
> to make myself enemies, I just have some sincere questions.
>
> Joe Harrington wrote:
>> I define success as popular adoption in preference to commercial
>> packages. ?I believe in vote-with-your-feet: this goal will not be
>> reached until all aspects of the package and its presentation to the
>> world exceed those of our commercial competition. ?Scipy is now a
>> grass roots effort, but that takes it only so far. ?Other projects,
>> such as OpenOffice and Sage, don't follow this model and do produce
>> quality products that compete with commercial offerings, at least on
>> open-source platforms. ?Before we can even hope for that, we have to
>> do the following:
>>
>
> <snip>
>
>> - Public communication
>> ? - A real marketing plan
>> ? - Executing on that plan
>> ? - Web site geared toward multiple audiences, run by experts at that
>> ? ? kind of communication
>> ? - More webinars, conference booths, training, aimed at all levels
>> ? - Demos, testimonials, topical forums, all showcased
>
> A thing OpenOffice.org and Sage both have is a very clear sense of
> direction and a clearly stated goal.
>
> SciPy might also have that for all I know, but I must admit I haven't
> understood what it is in the past year following the SciPy and NumPy
> lists, and reading the SciPy site. But I have seen email threads asking
> what the SciPy goal is, without any clear resolution (?).
>
> The website says this: "SciPy is open-source software for mathematics,
> science, and engineering."
>
> Which of course says nothing at all. Someone asked me what SciPy is the
> other day, and while I more or less "know" when I'd try to look in SciPy
> for an algorithm (instead of going to, say, R, or netlib.org, or
> whatever), I was more or less forced to say that it is a "dumping ground
> for various algorithms people have found useful, with the link being them
> being either written in Python or wrapped for Python".

I think scipy is a pretty much the same as a collection of matlab tool
boxes, either with more enhanced basic numerical algorithms (linalg,
special, optimize, interpolate, sparse, fft, spatial) or toolboxes
with wider applicability (stats including cluster, odr and maxentropy,
signal, ndimage+stsci?). This misses weave.

Which algorithms are actually included and some of the structure still
reflects the "dumping ground
 for various algorithms people have found useful". And some parts
don't look very used.

There is still a lot of cleaning and testing to do, but the
description as analogy to matlab toolboxes is pretty accurate, if a
description by analogy is allowed. E.g. to understand more of
scipy.signal, I started to read the help for matlabs signal toolbox.

That's my impression of scipy after working my way through some parts
of it in the last year.

>
> That's probably an unfair description -- the point is: If one needs to
> formulate a two- or three-liner about SciPy, what would it be? Is it a
> goal to reimplement stuff in SciPy that's (for instance) already thriving
> in the open source R community, or is that not a goal? And so on.

For stats, I consider matlab and maybe gauss for econometrics as
benchmark, not the coverage of a specialized language/package like R,
but I'm no statistician and I don't know anyone personally that uses
R.

Josef

>
> You might feel this is going off-topic, but I somehow feel that a very
> clear sense of direction is paramount when talking of these issues -- just
> look at the Sage project.
>
> Dag Sverre
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From josef.pktd at gmail.com  Sat Aug  1 11:29:59 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 1 Aug 2009 11:29:59 -0400
Subject: [SciPy-dev] nbinom.ppf
In-Reply-To: <3d375d730907280956m5425d997u5b37d8a355047439@mail.gmail.com>
References: <199F6194-4953-41F2-9F47-13CC4A5AD936@gmail.com>
	<3d375d730907271255i32283ae9j624e2a57b52fdac4@mail.gmail.com>
	<1cd32cbb0907271329q33bec67dqc120575454dbf3dc@mail.gmail.com>
	<3d375d730907271358s361cd41cke0804f1b80332c93@mail.gmail.com>
	<64EF39E2-8AF5-47C0-84D8-ADB4E063A211@gmail.com>
	<1cd32cbb0907271610w6986b256me9730f2ee3c436c5@mail.gmail.com>
	<3d375d730907271621m4aabe914l1536fd6a49da0ee5@mail.gmail.com>
	<1cd32cbb0907271811x411a32dbs4f11bf2f220f6781@mail.gmail.com>
	<3d375d730907280956m5425d997u5b37d8a355047439@mail.gmail.com>
Message-ID: <1cd32cbb0908010829pb812229hb6d0203d6d2cb386@mail.gmail.com>

On Tue, Jul 28, 2009 at 12:56 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Mon, Jul 27, 2009 at 20:11, <josef.pktd at gmail.com> wrote:
>> That's better. It took me a while to understand the logic behind the
>> way the ceiling error is corrected. The same pattern is also followed
>> by the other discrete distributions that define a _ppf method. It is
>> cleaner then the epsilon correction, but takes longer to figure out
>> what it does.
>>
>> To understand the logic more easily and to be DRY, it would be better
>> to replace the duplication of the _cdf method directly with a call to
>> self._cdf.
>> For example, in changeset 4673, Robert, you changed the _cdf method to
>> use betainc instead of nbdtr, but not the _ppf method. Without the
>> code duplication, partial corrections could be more easily avoided.
>>
>> Is there a reason not to call self._cdf instead?
>
> Nope. Go ahead.
>
> --
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


for the record: dlaplace and boltzman also fail the roundtrip test

boltzmann (1.3999999999999999, 19)
[ 1.  2.  3.  3.]
[ 0.  1.  2.  3.]
False True False [ 0.  3.]
dlaplace (0.80000000000000004,)
[-5. -4. -3. -2. -1.  1.  1.  2.  3.  4.  5.]
[-5. -4. -3. -2. -1.  0.  1.  2.  3.  4.  5.]

Josef


From jh at physics.ucf.edu  Sat Aug  1 12:20:17 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sat, 01 Aug 2009 12:20:17 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	(dagss@student.matnat.uio.no)
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
Message-ID: <wl8d47f4ibi.fsf@glup.physics.ucf.edu>

Dag wrote:
> I have seen email threads asking
> what the SciPy goal is, without any clear resolution (?).

How's this for a goal/mission statement (for SciPy, IDL, and Matlab):

(The toolstack) is a professional-quality numerical computation and
visualization environment that supports convenient handling of
numerical arrays, provides a rich set of basic tools and algorithms
for science and engineering, and supports a variety of both general
and discipline-specific application software.  It is easy for
numerically savvy teens to learn, but rich enough to support the most
complex of professional applications.  It can be run both
non-interactively and interactively, with the latter featuring both
GUI and rich command-line interfaces.  It comes with full
documentation, is easy to install and run on all popular platforms,
has a strong online user community spanning all disciplines, and has
commercial support and consulting.

For SciPy, I'd replace the part after the last comma with "is free and
open-source, supports cloud computing, and has options for commercial
user support and consulting."  One could add to the list of general
features, such as symbolic manipulation, parallel processing, etc.,
but it's already getting long.

For SciPy, some of this, of course, is not yet true, which is the
point of the current thread.

Another way of looking at it:

For me, SciPy is a replacement for IDL that improves on it in some
areas.  No more, but no less.  That doesn't say what it *is*, since it
just begs the question, "what is IDL", but it does identify the space
I'd like to see SciPy occupy.  It occupies most of the space IDL
occupied for me now, except for a few crucial areas.  The main one is
that enough of my colleagues use it that I can exchange codes with
them.  A code written in an interpreted language that your colleague
does not use is not useful to them.  If it's not useful to them, then
the interest in your contribution is that much smaller.  So, my goal
is to make SciPy (the toolstack, not the package) *to them* be what
IDL is to them today.  That is a lot more than what IDL is to me,
since I have more of a knack for computers than most of my colleagues.
They need a one-touch install, hold-your-hand docs, GUIs, and so
forth.  They are also less interested in the linguistic improvements
of Python over IDL.  Or, they are until they really get coding, which
is long after they make the decision to give it a spin.  This is a
good thing in a way, since it means that once they try it, they
*really* like it.  Most current SciPy users, I think, are savvy enough
about computers that we can work around the shortcomings, but the next
round of adopters will always be less savvy than the last, on the
whole, hence the need for better and lower-level docs, professional
packaging on all platforms, etc.

--jh--


From cournape at gmail.com  Sat Aug  1 12:21:10 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 2 Aug 2009 01:21:10 +0900
Subject: [SciPy-dev] [SciPy-User] SciPy Foundation
In-Reply-To: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
Message-ID: <5b8d13220908010921m49b218f2ga13b5b75a2aebaff@mail.gmail.com>

Hi Joe,

On Sat, Aug 1, 2009 at 2:06 AM, Joe Harrington<jh at physics.ucf.edu> wrote:

>
> I define success as popular adoption in preference to commercial
> packages. ?I believe in vote-with-your-feet: this goal will not be
> reached until all aspects of the package and its presentation to the
> world exceed those of our commercial competition. ?Scipy is now a
> grass roots effort, but that takes it only so far. ?Other projects,
> such as OpenOffice and Sage, don't follow this model and do produce
> quality products that compete with commercial offerings, at least on
> open-source platforms.

I am not sure openoffice is a good example, but I share the sentiment
that something is missing in the organization of the community.

I think it is very important to keep in mind that in any open source
project, telling people what to do does not work well. Not everybody
will share the same goals, are interested in scipy in the same way,
etc...  So any structure should help people doing what they want for
scipy's sake, but above all, should not alienate anyone who would have
worked on scipy otherwise. It may just be rhetoric, but saying that
"it would be nice for scipy to have this goal" instead of "we should
do this" matters IMHO.

Some of the things I am missing:
 - no quantifiable feedback from users: if we want to work on a set of
features, we cannot prioritize.  Likewise, we have very little
statistics on usage, platforms, etc... OTOH, this is often hard to
obtain for open source projects.
 - a scipy foundation: several times already, I have been asked
privately to do add some feature to scipy, generally things which
takes a few hours max, in exchange for some money. It is too much of a
hassle to set up things to get money for a few hours work, and
frankly, for a few hours, I would prefer to ask people to give money
to a scipy foundation instead. Something like the R foundation
(http://www.r-project.org/foundation/main.html). A foundation with a
legal status would make the situation much easier w.r.t donations I
believe. It should not be that hard to set up.
 - website: I think the root of the problem is lack of a dedicated
person for it, a person with design skills ideally, to design a
coherent graphic "chart" (not sure about the exact English word),
etc... I don't know how to get volunteers for this: it seems like many
projects manage to have such volunteers.

About the more particular points you raised:

> - Packaging
> ?- Personal Package Archive or equivalent for every release of every
> ? ?OS for the full toolstack (There are tools that do this but we
> ? ?don't use them. ?NSF requires Metronome - http://nmi.cs.wisc.edu/
> ? ?- for funding most development grants, so right now we're not even
> ? ?on NSF's radar.)
> ?- Track record of having the whole toolstack installation "just
> ? ?work" in a few command lines or clicks for *everyone*
> ?- Regular, scheduled releases of numpy and scipy
> ?- Coordinated releases of numpy, scipy, and stable scikits into PPA system

The problem of packaging is that it is hard to do well, but has no
technically challenging part in it. And it usually does not fall into
the "scratching ones' itch", because once you know how to build the
software, you are done and usually want to start using the damn thing.
Worse, it needs to be done every-time (every release). So this is
fundamentally different than doc: having done a great packaging work
for version N is useless after N+1 is out. It does not make sense to
pay someone to do it once.

Having some infrastructure would help: for example, something which
automatically builds packages on a set of supported platforms. It has
to be 100 % automatic, so that pushing one button get you the sources,
build the package, install it, and test it. This costs money and time
to set up.

> - Public communication
> ?- A real marketing plan
> ?- Executing on that plan
> ?- Web site geared toward multiple audiences, run by experts at that
> ? ?kind of communication
> ?- More webinars, conference booths, training, aimed at all levels
> ?- Demos, testimonials, topical forums, all showcased

Concerning communication with users, I think that the mailing lists do
not work well. It is ok for development, but it kinda sucks for
helping average users. Since I have been working on the dark side for
numpy/scipy- windows, I have been regularly using stackoverflow to ask
for some obscure windows stuff. stackoverflow is a a mix between a FAQ
and wikipedia. It works extremely well, and the user experience is way
above anything I have seen in this vein. Something like this to use
for scipy/numpy would be extremely useful I believe. It is vastly
superior to ML or wiki for focused problems ("how to do this in
matlab", "how to install on this linux distribution", etc...).

As an example of usage, R has recently used the main website so that
the most upvoted N R questions would be answered by R core developers
(during a R conference I believe). This all feels much better than ML
to me (again, as far as average user usage is concerned, not for
developer communication).

One website to handle all the user community, no need for complicated
forum rules and all (everything works with search and tags).
Stackoverflow works without any fixed hierarchy for many times more
participants that we will ever have, and much broader topics than us.

They will have soon a dedicated solution for custom websites using the
same stack - maybe something can be worked on as a open source
project.

David


From jh at physics.ucf.edu  Sat Aug  1 13:10:45 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sat, 01 Aug 2009 13:10:45 -0400
Subject: [SciPy-dev] [SciPy-User] SciPy Foundation
In-Reply-To: <5b8d13220908010921m49b218f2ga13b5b75a2aebaff@mail.gmail.com>
	(message from David Cournapeau on Sun, 2 Aug 2009 01:21:10 +0900)
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<5b8d13220908010921m49b218f2ga13b5b75a2aebaff@mail.gmail.com>
Message-ID: <wl8ab2j4fze.fsf@glup.physics.ucf.edu>

[Replying only on scipy-dev, per the original post.]

David wrote:

> I think it is very important to keep in mind that in any open source
> project, telling people what to do does not work well. Not everybody
> will share the same goals, are interested in scipy in the same way,
> etc...  So any structure should help people doing what they want for
> scipy's sake, but above all, should not alienate anyone who would
> have worked on scipy otherwise. It may just be rhetoric, but saying
> that "it would be nice for scipy to have this goal" instead of "we
> should do this" matters IMHO.

I think (hope!) that everyone understands that anything posted here is
a personal opinion and that none of us feels we are in a position to
give orders.  Nobody is boss or supervisor to the whole list.  When I
write, "We need...," of course I am writing "It is my opinion that we
need," etc., but that gets tedious both to write and to read.  Visions
should be bold.

That said, there do need to be goals, standards, etc.  Those do
translate into telling people what to do.  I think the key point is
that it must be the community, not any individual, that does the
telling.  For example, we are engaged in a discussion of a plan I
floated.  The list I posted is "my plan", but already we've added code
to the funding umbrella and no doubt there will be more changes (I
fully expected Robert Kern to flip out about my suggestion to remove
functions from numpy...maybe he didn't read that far...I expect to
lose that one.:-).  I think that once it's the community's plan, we
can say no to contributions that don't fit, that conflict with others,
that are too slow or insufficient, and so on, because we will have the
critical mass to replace those contributions with ones the community
thinks are better.  We see this already with the vigilant rejection of
change requests to the numpy API and the review comment system on the
doc wiki.  We can and have to say no occasionally, to maintain our
direction and our standards.  We just have to be careful about it and
make sure it is based on established community goals and norms, not
one person's random opinion.

More on some of your other points later...

--jh--


From tgrav at mac.com  Sat Aug  1 13:11:09 2009
From: tgrav at mac.com (Tommy Grav)
Date: Sat, 01 Aug 2009 13:11:09 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8d47f4ibi.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
Message-ID: <2FEC9DDC-1748-4C87-B5A4-6A8224E25D14@mac.com>


On Aug 1, 2009, at 12:20 PM, Joe Harrington wrote:

> Dag wrote:
>> I have seen email threads asking
>> what the SciPy goal is, without any clear resolution (?).
>
> For me, SciPy is a replacement for IDL that improves on it in some
> areas.  No more, but no less.

I have been using python, numpy and matplotlib for a few years as part
of my astronomy research. While I find numpy and matplotlib extremely
useful, scipy just don't seem to help me much. I think the problem is  
that
it is very unfocused. To me scipy is not a replacement of IDL, it is a  
python
implementation of Numerical Recipes, but it because of its lack of focus
it has become very chaotic. So far I have only found use for the  
integrate.leastsq
and spatial.KDTree packages from scipy. Packages like pyfits, pyraf,  
AstLib, etc.
take care of the more astronomy related problems. So I would  
personally like to
see scipy become a package that binds the numpy package to the more  
field
specific packages, by providing numerical methods that are broadly  
applicable
in many fields (i.e. least square minimization, KDTree implementation,  
Runga-Kutta
and other type of integration schemes, differential equation solvers  
and so on).

Making scipy into a tool for science and engineering is in my opinion  
a to broad a
goal. Making into a set of tools that are useable in many fields and  
thus supporting
development of field specific packages is in again my opinion the way  
to go. It narrows
the focus and makes the project more self contained.

Cheers
Tommy Grav
+ 
----------------------------------------------------------------------------+
Associate Researcher @ Dept. of Physics and Astronomy
Johns Hopkins University
+ 
----------------------------------------------------------------------------+
tgrav at pha.jhu.edu
(410) 516-7683
http://web.mac.com/tgrav/Astronomy/Welcome.html
+ 
----------------------------------------------------------------------------+



From dagss at student.matnat.uio.no  Sat Aug  1 13:49:21 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Sat, 1 Aug 2009 19:49:21 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8d47f4ibi.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
Message-ID: <704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>

Joe Harrington wrote:
> Dag wrote:
>> I have seen email threads asking
>> what the SciPy goal is, without any clear resolution (?).
>
> For me, SciPy is a replacement for IDL that improves on it in some
> areas.  No more, but no less.  That doesn't say what it *is*, since it
> just begs the question, "what is IDL", but it does identify the space
> I'd like to see SciPy occupy.  It occupies most of the space IDL
> occupied for me now, except for a few crucial areas.  The main one is
> that enough of my colleagues use it that I can exchange codes with
> them.  A code written in an interpreted language that your colleague
> does not use is not useful to them.  If it's not useful to them, then
> the interest in your contribution is that much smaller.  So, my goal
> is to make SciPy (the toolstack, not the package) *to them* be what
> IDL is to them today.  That is a lot more than what IDL is to me,
> since I have more of a knack for computers than most of my colleagues.
> They need a one-touch install, hold-your-hand docs, GUIs, and so
> forth.  They are also less interested in the linguistic improvements
> of Python over IDL.  Or, they are until they really get coding, which
> is long after they make the decision to give it a spin.  This is a
> good thing in a way, since it means that once they try it, they
> *really* like it.  Most current SciPy users, I think, are savvy enough
> about computers that we can work around the shortcomings, but the next
> round of adopters will always be less savvy than the last, on the
> whole, hence the need for better and lower-level docs, professional
> packaging on all platforms, etc.

I really, really want what you seem to want too. BUT, I'll continue my
criticism, in the hope that something may come out of it.

What you mention above seem to be A LOT of work (in particular
"professional packaging on all platforms"), and as others have mentioned
partly in conflict with the way people tend to view SciPy currently, and
so on.

As you say it is indeed the whole stack that is important. Still, part of
what you write seems to be an effort to do what many are doing already:
- EPD
- Sage (currently maths focused, but it does bundle SciPy and integrating
it better would )
- SPD (Sage without some of the math libs)
- Python(x,y)

These all bundle SciPy, but also sets up the whole stack, and can focus on
the whole picture.

Are you saying that you just want to do it better than these, through a
foundation? Wouldn't it be better to direct any funding through one of
these existing candidates?

This post I've written on the Sage list is very related and is about SciPy
vs. Sage:
http://groups.google.com/group/sage-devel/msg/78e2a2032042d35b

The parent thread is a bit long but lots of related material in there:
http://groups.google.com/group/sage-devel/browse_thread/thread/bef2010f45984730/78e2a2032042d35b?#78e2a2032042d35b

Dag Sverre



From josef.pktd at gmail.com  Sat Aug  1 16:10:06 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 1 Aug 2009 16:10:06 -0400
Subject: [SciPy-dev] nbinom.ppf
In-Reply-To: <1cd32cbb0908010829pb812229hb6d0203d6d2cb386@mail.gmail.com>
References: <199F6194-4953-41F2-9F47-13CC4A5AD936@gmail.com>
	<3d375d730907271255i32283ae9j624e2a57b52fdac4@mail.gmail.com>
	<1cd32cbb0907271329q33bec67dqc120575454dbf3dc@mail.gmail.com>
	<3d375d730907271358s361cd41cke0804f1b80332c93@mail.gmail.com>
	<64EF39E2-8AF5-47C0-84D8-ADB4E063A211@gmail.com>
	<1cd32cbb0907271610w6986b256me9730f2ee3c436c5@mail.gmail.com>
	<3d375d730907271621m4aabe914l1536fd6a49da0ee5@mail.gmail.com>
	<1cd32cbb0907271811x411a32dbs4f11bf2f220f6781@mail.gmail.com>
	<3d375d730907280956m5425d997u5b37d8a355047439@mail.gmail.com>
	<1cd32cbb0908010829pb812229hb6d0203d6d2cb386@mail.gmail.com>
Message-ID: <1cd32cbb0908011310t39fec09dg2c3037db02dbda72@mail.gmail.com>

On Sat, Aug 1, 2009 at 11:29 AM, <josef.pktd at gmail.com> wrote:
> On Tue, Jul 28, 2009 at 12:56 PM, Robert Kern<robert.kern at gmail.com> wrote:
>> On Mon, Jul 27, 2009 at 20:11, <josef.pktd at gmail.com> wrote:
>>> That's better. It took me a while to understand the logic behind the
>>> way the ceiling error is corrected. The same pattern is also followed
>>> by the other discrete distributions that define a _ppf method. It is
>>> cleaner then the epsilon correction, but takes longer to figure out
>>> what it does.
>>>
>>> To understand the logic more easily and to be DRY, it would be better
>>> to replace the duplication of the _cdf method directly with a call to
>>> self._cdf.
>>> For example, in changeset 4673, Robert, you changed the _cdf method to
>>> use betainc instead of nbdtr, but not the _ppf method. Without the
>>> code duplication, partial corrections could be more easily avoided.
>>>
>>> Is there a reason not to call self._cdf instead?
>>
>> Nope. Go ahead.
>>
>> --
>> Robert Kern
>>
>> "I have come to believe that the whole world is an enigma, a harmless
>> enigma that is made terrible by our own mad attempt to interpret it as
>> though it had an underlying truth."
>> ?-- Umberto Eco
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
> for the record: dlaplace and boltzman also fail the roundtrip test
>
> boltzmann (1.3999999999999999, 19)
> [ 1. ?2. ?3. ?3.]
> [ 0. ?1. ?2. ?3.]
> False True False [ 0. ?3.]
> dlaplace (0.80000000000000004,)
> [-5. -4. -3. -2. -1. ?1. ?1. ?2. ?3. ?4. ?5.]
> [-5. -4. -3. -2. -1. ?0. ?1. ?2. ?3. ?4. ?5.]
>
> Josef
>

and planck

>>> stats.planck.ppf(stats.planck.cdf(np.arange(10),.51),.51)
array([  1.,   1.,   2.,   3.,   4.,   5.,   7.,   7.,   8.,  10.])

fixed in 5889

Josef


From d_l_goldsmith at yahoo.com  Sat Aug  1 17:56:52 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 1 Aug 2009 14:56:52 -0700 (PDT)
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <2FEC9DDC-1748-4C87-B5A4-6A8224E25D14@mac.com>
Message-ID: <642650.96553.qm@web52109.mail.re2.yahoo.com>


--- On Sat, 8/1/09, Tommy Grav <tgrav at mac.com> wrote:

> Making scipy into a tool for science and engineering is in
> my opinion? 
> a to broad a
> goal. Making into a set of tools that are useable in many
> fields and? 
> thus supporting
> development of field specific packages is in again my
> opinion the way? 
> to go. 

Please clarify what you see as the difference between these two - to me, on the surface of it, your goal statement is no more "focused" nor "self-contained" than Joe's.  Perhaps if you clarify what you see as the differences, we all may discover that your vision and Joe's actually aren't that far apart.

DG

> It narrows
> the focus and makes the project more self contained.
> 
> Cheers
> Tommy Grav
> + 
> ----------------------------------------------------------------------------+
> Associate Researcher @ Dept. of Physics and Astronomy
> Johns Hopkins University
> + 
> ----------------------------------------------------------------------------+
> tgrav at pha.jhu.edu
> (410) 516-7683
> http://web.mac.com/tgrav/Astronomy/Welcome.html
> + 
> ----------------------------------------------------------------------------+
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From gael.varoquaux at normalesup.org  Sat Aug  1 18:52:16 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 2 Aug 2009 00:52:16 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
Message-ID: <20090801225216.GB31380@phare.normalesup.org>

On Sat, Aug 01, 2009 at 07:49:21PM +0200, Dag Sverre Seljebotn wrote:
> As you say it is indeed the whole stack that is important. Still, part of
> what you write seems to be an effort to do what many are doing already:
> - EPD
> - Sage (currently maths focused, but it does bundle SciPy and integrating
> it better would )
> - SPD (Sage without some of the math libs)
> - Python(x,y)

> These all bundle SciPy, but also sets up the whole stack, and can focus on
> the whole picture.

> Are you saying that you just want to do it better than these, through a
> foundation? Wouldn't it be better to direct any funding through one of
> these existing candidates?

> This post I've written on the Sage list is very related and is about SciPy
> vs. Sage:
> http://groups.google.com/group/sage-devel/msg/78e2a2032042d35b

I am jumping in this discussion (something that I have been trying to
avoid, because such discussions are very hard to drive to a useful
point). I'll try to write a clear e-mail, to the point, however, as the
previous discussion you are pointing to does not reflect my needs.

On the various usecases and users
===================================

I think that the discussion on the Sage mailing list, and a few points of
the last e-mails I have seen on this mailing list, miss a very important
point for many users of the scipy stack that I see around me:

We want a tool, or a set of tools, to build our own entry points. We want
more than an IDE like Matlab, Mathematica. We want to be able to use the
tools separated, to do data mining on servers log, to build custom
applications for eg medical image analysis, or to control a physics
experiment (there are a lot of talks at the scipy conference this year on
this). Most of the scipy users are "even more applied than applied math"
(golly, this sounds almost dirty ;> ).

Building a reusable stack is why we need tools to be broken up separating
features. Scipy as a community and an umbrella project may benefit from
an IDE, like matlab, or a web interface like the amazing one Sage has,
but we don't want to bundle these features with the core numerical tools
of scipy.

Now this might actually concern only a fractions of users. Many users
(including me) mostly use the scipy tool stack as a matlab/mathematica
replacement. However, these users are not the main code contributors. If
somebody develops an algorithm he wants to ship or to share, chances are
he wants it not to be bound to a heavy platform, but more to a light core
(hey, numpy is even shipped by default on macOSX and many linux
distributions nowadays).


An integrated environment as an entry point
==============================================

Besides building a good set of tools and their documentation, we need to
address two separate issues to make life easier for users: building an
integrated environment (what I call an entry point) and building
distributions. It is tempting to do both at the same time, however, I
think that if we collapse the two problems, we are going in the wrong
directions: I want to be able to reuse the underlying technology of the
integrated environment, for instance to build an astronomic-specific IDE,
and I want to be able to contribute modules to it even if those modules
are not distributed together.

Like many people, my working environment is IPython. It suits my needs,
and I get scientific results using it. However, I can see that it is not
the best solution to guide a beginner. Inspired by matlab, IDL or
mathematica, we have been dreaming of having an IDE for a long while.
Last year, Enthought has payed me to start work on making IPython
GUI-friendly to plug one of the missing bricks to assembling the tool
stack in an IDE. I have been unable to work on this for a year, as it is
not a priority for my research, but the effort lives on in the IPython
repository, and it would be great to see IDE build upon it, and improve
it.

An IDE for easy scientific development with Python would bring together
tools such as a shell, easy access to documentation, and an editor
(reinventing any one of these components might not be necessary). There
is EPDLab, which is being developed in the ETS repository. I love the
technology stack that it is built upon (ETS provides good tools for
building GUIs, and IPython provides an very handy and powerful command
line), and I am thus full of hope for EPDLab. I can see however that
people might be afraid of using it, let alone contributing to it, as it
bares strong Enthought branding. This is a pity, because in this case we
have the chance of having a compagny's interest lying in the same
direction than the community.

For a web environment, the Sage notebook is amazing. Unfortunately last
time I looked, it was GPL licensed, which renders it improper for my use,
as the tools we use at the lab must be BSD, in order to be able to build
(eventually) medical imaging products from them one day.

But, from a more pragmatic point of view the simplest thing to do to make
it easier for a beginner to get started, would be to improve the
documentation on the web. I am not thinking of the specific packages
documentation, but more describing how things fit together: giving the
workflow, and pointing to the various main packages used for different
things. We already have a lot of material on the webpages, but this
material is not as 'sexy' as it could be, and not as to-the-point as
possible. Sure, this is a lot of work too.

Building standard distributions
=================================

I am a huge fan of distributions. Every large applied lab I know ends up
building a distribution mechanism. Without standard distributions, we
cannot reuse each-other's effort to distribute, but also we have huge
friction on reusing each-other's tools: installing on your computer may
be easy, but if you have to worry whether your non-technical users will
succeed in installing a tool, you start wondering whether you want to
rely on the tool, or whether you are going to reimplement it.

However, the other side of the problem is that distributions could end up
developing tools that make use of the tight integration that they provide
to solve numerical or usability problems quicker, while locking the users
in the distribution. If I want to integrate an algorithm developed by
another lab in a medical imaging platform, I cannot afford to drag in
Sage, just like I cannot afford R, or Maltab, as they are too big
dependencies. An IDE that works only on a distribution is not one that I
will rely on for teaching). This is why I believe that every single piece
of code in a distribution should be usable outside of this distribution
(and I applaud the SPD effort started by Ondrej and the SAGE guys).

Concrete suggestions to ease the progress
==========================================

Of course providing a consistent environment is a hard problem, but
hey, this is a problem many of us face. I believe that we are making
progress with many encouraging projects such as Sage, EPD, Python(x,y),
or SPD. Establishing scientific environments in Python is an ambitious
project; there will not be a one-size-fits-all solution and having many
different approaches is healthy, as long as we keep it friendly and learn
from all the efforts. I strongly believe that we will be getting more and
more satisfactory solutions in the next years.

Specifically, I would love to see an official umbrella project for
BSD-licensed tools for building scientific projects with Python. As the
"scipy" name is well branded (through the website, and the conference),
we could call this the 'scipy project'. I would personally like to limit
wheel reinvention and have preferred solutions for the various bricks (I
am thinking of the unfortunate Chaco versus Matplotlib situation, where I
have to depend on both libraries that complement each other). 

Back to the scipy foundation idea
==================================

The idea of the scipy foundation is an idea that has been floating around
for a while. If it is manned by a variety of people who express the wills
and needs of users and developers of the scipy ecosystem, it could be a
great thing. But I see two road blocks: first, as Robert points out,
telling somebody what to do will not achieve anything. I am already way
too busy scratching my own itches. Second, who will find the time to take
care of this?

And now, I have to catch up on sleep.

Ga?l


From cournape at gmail.com  Sat Aug  1 20:39:48 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 2 Aug 2009 09:39:48 +0900
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <20090801225216.GB31380@phare.normalesup.org>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
Message-ID: <5b8d13220908011739w7bf2eaf2tf5e652ca5a76bc7@mail.gmail.com>

On Sun, Aug 2, 2009 at 7:52 AM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
>
> Back to the scipy foundation idea
> ==================================
>
> The idea of the scipy foundation is an idea that has been floating around
> for a while. If it is manned by a variety of people who express the wills
> and needs of users and developers of the scipy ecosystem, it could be a
> great thing. But I see two road blocks: first, as Robert points out,
> telling somebody what to do will not achieve anything.

To have a foundation, by itself, has no consequence on telling people
what to do. It is just a way to have a single point of entry for
people who want to interact with the community, and to have the legal
right to collect money.

> I am already way
> too busy scratching my own itches. Second, who will find the time to take
> care of this?

There is an inherent amount of bureaucracy involved with those things,
but it does not have to always be done by the same people, and
rotation works better than for code I think.

David


From tgrav at mac.com  Sun Aug  2 08:32:56 2009
From: tgrav at mac.com (Tommy Grav)
Date: Sun, 02 Aug 2009 08:32:56 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <642650.96553.qm@web52109.mail.re2.yahoo.com>
References: <642650.96553.qm@web52109.mail.re2.yahoo.com>
Message-ID: <51FCDB99-5D9C-42D8-9864-658755C01C98@mac.com>


On Aug 1, 2009, at 5:56 PM, David Goldsmith wrote:

>
> --- On Sat, 8/1/09, Tommy Grav <tgrav at mac.com> wrote:
>
>> Making scipy into a tool for science and engineering is in my opinion
>> a to broad a goal. Making into a set of tools that are useable in  
>> many
>> fields and thus supporting development of field specific packages  
>> is in again my
>> opinion the way to go.
>
> Please clarify what you see as the difference between these two - to  
> me, on the surface
> of it, your goal statement is no more "focused" nor "self-contained"  
> than Joe's.  Perhaps
> if you clarify what you see as the differences, we all may discover  
> that your vision and
> Joe's actually aren't that far apart.

I don't think that Joe and I are that far apart either. My point (very  
badly formulated) was
that trying to make scipy be a replacement for IDL or matlab is in my  
opinion not the right
goal. IDL in particular has a lot of field specific code available in  
it. I would like to see a
structure where scipy provides the underlaying code needed by many  
fields (like the
Numerical Recipes codes) but stay away from providing field specific  
code. Also scipy
should not venture into GUI or provide an interactive environment like  
IDL (there are
other packages that provide this).

Just my opinion
    Tommy Grav


From d_l_goldsmith at yahoo.com  Sun Aug  2 14:58:11 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 2 Aug 2009 11:58:11 -0700 (PDT)
Subject: [SciPy-dev] SciPy Foundation
Message-ID: <274366.95245.qm@web52106.mail.re2.yahoo.com>


--- On Sun, 8/2/09, Tommy Grav <tgrav at mac.com> wrote:

> I don't think that Joe and I are that far apart either. My
> point (very? 
> badly formulated) was
> that trying to make scipy be a replacement for IDL or
> matlab is in my? 
> opinion not the right
> goal. IDL in particular has a lot of field specific code
> available in? 
> it. I would like to see a
> structure where scipy provides the underlaying code needed
> by many? 
> fields (like the
> Numerical Recipes codes) but stay away from providing field
> specific? 
> code. Also scipy
> should not venture into GUI or provide an interactive
> environment like? 
> IDL (there are
> other packages that provide this).
> 
> Just my opinion
> ? ? Tommy Grav

OK, that helps. :-)

Fine goal (between the two, I choose to remain neutral for now), but one comment: you say avoid a GUI, but the kind of "tool set" you describe would greatly benefit from (dare I say require) some sort of UI that makes it "easy" for the uninitiated (at the very least) to find the specific resources they need; IMO, for example, the UI LAPACK provides for this is a good example of how *not* to do it.

DG
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From nmb at wartburg.edu  Sun Aug  2 16:44:49 2009
From: nmb at wartburg.edu (Neil Martinsen-Burrell)
Date: Sun, 02 Aug 2009 15:44:49 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <274366.95245.qm@web52106.mail.re2.yahoo.com>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
Message-ID: <4A75FAC1.7070600@wartburg.edu>

On 08/02/2009 01:58 PM, David Goldsmith wrote:

>> I don't think that Joe and I are that far apart either. My point
>> (very badly formulated) was that trying to make scipy be a
>> replacement for IDL or matlab is in my opinion not the right goal.
>> IDL in particular has a lot of field specific code available in it.
>> I would like to see a structure where scipy provides the
>> underlaying code needed by many fields (like the Numerical Recipes
>> codes) but stay away from providing field specific code. Also
>> scipy should not venture into GUI or provide an interactive
>> environment like IDL (there are other packages that provide this).
>>
>> Just my opinion Tommy Grav
>
> OK, that helps. :-)
>

> Fine goal (between the two, I choose to remain neutral for now), but
> one comment: you say avoid a GUI, but the kind of "tool set" you
> describe would greatly benefit from (dare I say require) some sort of
> UI that makes it "easy" for the uninitiated (at the very least) to
> find the specific resources they need; IMO, for example, the UI
> LAPACK provides for this is a good example of how *not* to do it.

This may be an instrumentality on the way to the "Goal of Scipy" 
(whatever that is) but I wanted to mention here the importance of 
reaching students with SciPy.  Software vendors know this: if a student 
learns about a certain type of computing using your software, then they 
are likely to continue using your software throughout their career. 
Matlab has been stupendously good at this sort of marketing in 
engineering schools, where learning Matlab is seen by some as a 
*required* part of the curriculum, due to its industry dominance.

Apropos of David's point about the relevance of a GUI, I think that in 
addition to the packaging, documentation and communication aspects of 
Joe's plan, an easy-to-install environment for interactive computation 
is important for teaching students with SciPy.  When I taught an 
undergraduate class on Markov chains using numpy and scipy, it was hard 
for students to install scipy.  Once they had it installed, they were 
able to be moderately productive in IDLE, but they missed some of the 
features of IPython (command completion, saved inputs and output).  An 
interactive Python environment that allowed access to documentation, an 
editor and a rich interpreter would have made the uptake much easier for 
students.

In the past, Alan has spoken strongly about the importance of the matrix 
class for teaching linear algebra and I want to echo his message about 
the importance of pedagogical usability for the continued adoption of 
the SciPy stack.  Students who start using software in their classes 
will continue using that software throughout their careers, particularly 
so for something such as SciPy which has some significant advantages 
over its better-known competitors.  I think that there is a tendency for 
active researchers to underestimate the importance of 
undergraduate-level learning and I hope that in this discussion, we will 
keep in mind the singular importance of that young audience.

-Neil


From vanforeest at gmail.com  Sun Aug  2 16:55:46 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Sun, 2 Aug 2009 22:55:46 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A75FAC1.7070600@wartburg.edu>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
	<4A75FAC1.7070600@wartburg.edu>
Message-ID: <fa510ff80908021355k6f92ddfbma7d3e1d6ca26169b@mail.gmail.com>

Hi,

my 2 cts: I completely agree. I try to "force" python upon my
students, and advice them to use python xy. In my opinion this package
is certainly a step in the right direction as it makes using
python/numpy/scipy very easy. Hopefully I am helping raising zealots
(of the right type :-) ) with this.

bye

NIcky

2009/8/2 Neil Martinsen-Burrell <nmb at wartburg.edu>:
> On 08/02/2009 01:58 PM, David Goldsmith wrote:
>
>>> I don't think that Joe and I are that far apart either. My point
>>> (very badly formulated) was that trying to make scipy be a
>>> replacement for IDL or matlab is in my opinion not the right goal.
>>> IDL in particular has a lot of field specific code available in it.
>>> I would like to see a structure where scipy provides the
>>> underlaying code needed by many fields (like the Numerical Recipes
>>> codes) but stay away from providing field specific code. Also
>>> scipy should not venture into GUI or provide an interactive
>>> environment like IDL (there are other packages that provide this).
>>>
>>> Just my opinion Tommy Grav
>>
>> OK, that helps. :-)
>>
>
>> Fine goal (between the two, I choose to remain neutral for now), but
>> one comment: you say avoid a GUI, but the kind of "tool set" you
>> describe would greatly benefit from (dare I say require) some sort of
>> UI that makes it "easy" for the uninitiated (at the very least) to
>> find the specific resources they need; IMO, for example, the UI
>> LAPACK provides for this is a good example of how *not* to do it.
>
> This may be an instrumentality on the way to the "Goal of Scipy"
> (whatever that is) but I wanted to mention here the importance of
> reaching students with SciPy. ?Software vendors know this: if a student
> learns about a certain type of computing using your software, then they
> are likely to continue using your software throughout their career.
> Matlab has been stupendously good at this sort of marketing in
> engineering schools, where learning Matlab is seen by some as a
> *required* part of the curriculum, due to its industry dominance.
>
> Apropos of David's point about the relevance of a GUI, I think that in
> addition to the packaging, documentation and communication aspects of
> Joe's plan, an easy-to-install environment for interactive computation
> is important for teaching students with SciPy. ?When I taught an
> undergraduate class on Markov chains using numpy and scipy, it was hard
> for students to install scipy. ?Once they had it installed, they were
> able to be moderately productive in IDLE, but they missed some of the
> features of IPython (command completion, saved inputs and output). ?An
> interactive Python environment that allowed access to documentation, an
> editor and a rich interpreter would have made the uptake much easier for
> students.
>
> In the past, Alan has spoken strongly about the importance of the matrix
> class for teaching linear algebra and I want to echo his message about
> the importance of pedagogical usability for the continued adoption of
> the SciPy stack. ?Students who start using software in their classes
> will continue using that software throughout their careers, particularly
> so for something such as SciPy which has some significant advantages
> over its better-known competitors. ?I think that there is a tendency for
> active researchers to underestimate the importance of
> undergraduate-level learning and I hope that in this discussion, we will
> keep in mind the singular importance of that young audience.
>
> -Neil
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From alan at ajackson.org  Sun Aug  2 17:03:44 2009
From: alan at ajackson.org (alan at ajackson.org)
Date: Sun, 2 Aug 2009 16:03:44 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A75FAC1.7070600@wartburg.edu>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
	<4A75FAC1.7070600@wartburg.edu>
Message-ID: <20090802160344.11aa8999@ajackson.org>

---------------8< snip -------------------
>This may be an instrumentality on the way to the "Goal of Scipy" 
>(whatever that is) but I wanted to mention here the importance of 
>reaching students with SciPy.  Software vendors know this: if a student 
>learns about a certain type of computing using your software, then they 
>are likely to continue using your software throughout their career. 
>Matlab has been stupendously good at this sort of marketing in 
>engineering schools, where learning Matlab is seen by some as a 
>*required* part of the curriculum, due to its industry dominance.
>
>
---------------8< snip -------------------
>-Neil

I'd like to echo these comments. I have been working for several years now to
get people in my company to use python, numpy, scipy, etc, and have made
progress, but the biggest battle I fight is with the Matlab people. Pretty
nearly every person we hire just out of school, and every summer intern comes
to us as a Matlab user, due to the excellent job Matlab has done with cheap
academic licenses. Anything we can do do get professors to start using the
python suite so that their students will learn and use it will pay great
dividends in the future.

- Alan

-- 
-----------------------------------------------------------------------
| Alan K. Jackson            | To see a World in a Grain of Sand      |
| alan at ajackson.org          | And a Heaven in a Wild Flower,         |
| www.ajackson.org           | Hold Infinity in the palm of your hand |
| Houston, Texas             | And Eternity in an hour. - Blake       |
-----------------------------------------------------------------------


From gael.varoquaux at normalesup.org  Sun Aug  2 17:09:42 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 2 Aug 2009 23:09:42 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A75FAC1.7070600@wartburg.edu>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
	<4A75FAC1.7070600@wartburg.edu>
Message-ID: <20090802210942.GC25001@phare.normalesup.org>

On Sun, Aug 02, 2009 at 03:44:49PM -0500, Neil Martinsen-Burrell wrote:
>  I think that there is a tendency for active researchers to
>  underestimate the importance of undergraduate-level learning and I
>  hope that in this discussion, we will keep in mind the singular
>  importance of that young audience.

That's all good and nice. I agree with you it is important, and I am very
happy to hear people talking about this, because it makes me hope that we
will be getting more help to do this.

If I work my ass off on an IDE, or more simply a GUI frontend, it won't
help me get more work done, which means shooting papers out, to be
cynical, and, in a few years, I will most likely not be doing any
scientific Python anymore. On the other hand, if I work on something that
is useful for my day to day work, I get some traction at the lab, and my
sleepless face is more easily forgiven. If I build an IDE that is of no
use to our work, nobody cares, and for a good reason.

This is not to say that we shouldn't be working on the IDE, I believe
that I am one of the people that have actually written code to do this,
but there is a lot of work to be done here, and working on making sure
that we have a shell to do this, and interactive plotting, and good
documentation is part of this work, and can be reused for direct research
interests. Writing docs is also something that can help a lot, does not
require extensive technical knowledge and takes a lot of time.

Actually, I must point out that I am quite unhappy, because I am very
tired, I have spent the week end fixing bugs on various open source
projects (nipy and mayavi) and answering complicated users questions. I
find that to be told that we are underestimating the importance of ease
of use and ease of learning is unfair. This simply takes a lot of time
and some of us are working on it.

Ga?l


From tgrav at mac.com  Sun Aug  2 17:22:54 2009
From: tgrav at mac.com (Tommy Grav)
Date: Sun, 02 Aug 2009 17:22:54 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A75FAC1.7070600@wartburg.edu>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
	<4A75FAC1.7070600@wartburg.edu>
Message-ID: <6F225505-2C39-4B32-AE49-6108FCCB2301@mac.com>


On Aug 2, 2009, at 4:44 PM, Neil Martinsen-Burrell wrote:

> This may be an instrumentality on the way to the "Goal of Scipy"
> (whatever that is) but I wanted to mention here the importance of
> reaching students with SciPy.  Software vendors know this: if a  
> student
> learns about a certain type of computing using your software, then  
> they
> are likely to continue using your software throughout their career.
> Matlab has been stupendously good at this sort of marketing in
> engineering schools, where learning Matlab is seen by some as a
> *required* part of the curriculum, due to its industry dominance.
>
> Apropos of David's point about the relevance of a GUI, I think that in
> addition to the packaging, documentation and communication aspects of
> Joe's plan, an easy-to-install environment for interactive computation
> is important for teaching students with SciPy.  When I taught an
> undergraduate class on Markov chains using numpy and scipy, it was  
> hard
> for students to install scipy.  Once they had it installed, they were
> able to be moderately productive in IDLE, but they missed some of the
> features of IPython (command completion, saved inputs and output).  An
> interactive Python environment that allowed access to documentation,  
> an
> editor and a rich interpreter would have made the uptake much easier  
> for
> students.

I agree with what you are saying, but I don't think scipy is the right  
package
for this. The scipy package should in my opinion be like numpy, a self
contained package of methods that are frequently used in science and
engineering. In a sense it should provide the applied math. Then one can
have separate packages providing interpreters ala matlab and IDL that  
sits
on top of the scipy package and other more field specific packages. I  
think
that in thinking of scipy as a replacement for IDL and Matlab the  
project becomes
to broad reaching and it gets harder to get everyone to pull in  
approximately
the same direction.

> In the past, Alan has spoken strongly about the importance of the  
> matrix
> class for teaching linear algebra and I want to echo his message about
> the importance of pedagogical usability for the continued adoption of
> the SciPy stack.  Students who start using software in their classes
> will continue using that software throughout their careers,  
> particularly
> so for something such as SciPy which has some significant advantages
> over its better-known competitors.  I think that there is a tendency  
> for
> active researchers to underestimate the importance of
> undergraduate-level learning and I hope that in this discussion, we  
> will
> keep in mind the singular importance of that young audience.

I agree again, but I also think that students should learn how to code  
in Python,
not in Sage/Python(x,y)/Scipy. The more of the core language the  
student learns
the more powerful all the tools become.

Tommy



From d_l_goldsmith at yahoo.com  Sun Aug  2 18:00:44 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 2 Aug 2009 15:00:44 -0700 (PDT)
Subject: [SciPy-dev] Difficulty creating an example for illustrating
	memmap.flush()
Message-ID: <928017.59766.qm@web52111.mail.re2.yahoo.com>


Hi, folks.  So, I'm trying to devise an example to illustrate memmap.flush().  It didn't work the way I was intuitively expecting (i.e., it didn't appear to do anything at all) so I went looking for a "ready-made" example and found:

http://www.slideshare.net/enthought/python-for-scientific-computing-webinar-may-22-2009

which has an example on Slide 25.  However, here's what I get when I try to duplicate the example:

Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
win32
>>> import numpy as np
>>> np.version.version
'1.3.0rc2'
>>> q = np.memmap('new_file.dat',mode='w+',shape=(2,5))
>>> q
memmap([[0, 0, 0, 0, 0],
       [0, 0, 0, 0, 0]], dtype=uint8)
>>> # Print out underlying file contents
... # Note: not using iPython, so have to use os.system
...
>>> import os
>>> os.system('type new_file.dat')
          0
>>> # Note: already a little different than Webinar Expl.
...
>>> # Next write ascii value for 'A' (65) into q
...
>>> q[:] = ord('A')
>>> q
memmap([[65, 65, 65, 65, 65],
       [65, 65, 65, 65, 65]], dtype=uint8)
>>> # Do I need to call flush before file is written to?
...
>>> os.system('type new_file.dat')
AAAAAAAAAA0
>>> # No! Does flushing change anything?
...
>>> q.flush()
>>> os.system('type new_file.dat')
AAAAAAAAAA0
>>> # No! Is it because I printed q before checking it on disc?
...
>>> # Start afresh, but don't print memmap before checking it
...
>>> r = np.memmap('new_file2.dat',mode='w+',shape=(2,5))
>>> os.system('type new_file2.dat') # "reproducibility check"
          0
>>> r[:] = ord('A')
>>> os.system('type new_file2.dat') # Checking file on disc immediately
AAAAAAAAAA0
>>> # File is updated without calling flush, indeed without any
... # intervening access to the memmap at all!

What gives?  Is this a bug?  If not, can someone please furnish me with an example that clearly an explicitly illustrates the function (and necessity) of memmap.flush()?

Thanks!

DG


      


From d_l_goldsmith at yahoo.com  Sun Aug  2 18:22:56 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 2 Aug 2009 15:22:56 -0700 (PDT)
Subject: [SciPy-dev] SciPy Foundation
Message-ID: <89797.28963.qm@web52109.mail.re2.yahoo.com>


Not wishing to turn this into a mutual admiration society, but "+10" vis-a-vis everything Neil said! ;-)

DG

PS: I do feel, however, that a UI as rich as Neil implies, though such should be wholly supported both materially and "in spirit" by whatever entity takes responsibility for SciPy, such a UI should be "separable" from the SciPy "core," so that the latter is deliverable both with a "rich" UI and a "serviceable" UI.

--- On Sun, 8/2/09, Neil Martinsen-Burrell <nmb at wartburg.edu> wrote:

> From: Neil Martinsen-Burrell <nmb at wartburg.edu>
> Subject: Re: [SciPy-dev] SciPy Foundation
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Sunday, August 2, 2009, 1:44 PM
> On 08/02/2009 01:58 PM, David
> Goldsmith wrote:
> 
> >> I don't think that Joe and I are that far apart
> either. My point
> >> (very badly formulated) was that trying to make
> scipy be a
> >> replacement for IDL or matlab is in my opinion not
> the right goal.
> >> IDL in particular has a lot of field specific code
> available in it.
> >> I would like to see a structure where scipy
> provides the
> >> underlaying code needed by many fields (like the
> Numerical Recipes
> >> codes) but stay away from providing field specific
> code. Also
> >> scipy should not venture into GUI or provide an
> interactive
> >> environment like IDL (there are other packages
> that provide this).
> >>
> >> Just my opinion Tommy Grav
> >
> > OK, that helps. :-)
> >
> 
> > Fine goal (between the two, I choose to remain neutral
> for now), but
> > one comment: you say avoid a GUI, but the kind of
> "tool set" you
> > describe would greatly benefit from (dare I say
> require) some sort of
> > UI that makes it "easy" for the uninitiated (at the
> very least) to
> > find the specific resources they need; IMO, for
> example, the UI
> > LAPACK provides for this is a good example of how
> *not* to do it.
> 
> This may be an instrumentality on the way to the "Goal of
> Scipy" 
> (whatever that is) but I wanted to mention here the
> importance of 
> reaching students with SciPy.? Software vendors know
> this: if a student 
> learns about a certain type of computing using your
> software, then they 
> are likely to continue using your software throughout their
> career. 
> Matlab has been stupendously good at this sort of marketing
> in 
> engineering schools, where learning Matlab is seen by some
> as a 
> *required* part of the curriculum, due to its industry
> dominance.
> 
> Apropos of David's point about the relevance of a GUI, I
> think that in 
> addition to the packaging, documentation and communication
> aspects of 
> Joe's plan, an easy-to-install environment for interactive
> computation 
> is important for teaching students with SciPy.? When I
> taught an 
> undergraduate class on Markov chains using numpy and scipy,
> it was hard 
> for students to install scipy.? Once they had it
> installed, they were 
> able to be moderately productive in IDLE, but they missed
> some of the 
> features of IPython (command completion, saved inputs and
> output).? An 
> interactive Python environment that allowed access to
> documentation, an 
> editor and a rich interpreter would have made the uptake
> much easier for 
> students.
> 
> In the past, Alan has spoken strongly about the importance
> of the matrix 
> class for teaching linear algebra and I want to echo his
> message about 
> the importance of pedagogical usability for the continued
> adoption of 
> the SciPy stack.? Students who start using software in
> their classes 
> will continue using that software throughout their careers,
> particularly 
> so for something such as SciPy which has some significant
> advantages 
> over its better-known competitors.? I think that there
> is a tendency for 
> active researchers to underestimate the importance of 
> undergraduate-level learning and I hope that in this
> discussion, we will 
> keep in mind the singular importance of that young
> audience.
> 
> -Neil
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From 00ai99 at gmail.com  Sun Aug  2 19:56:21 2009
From: 00ai99 at gmail.com (David Gowers)
Date: Mon, 3 Aug 2009 09:26:21 +0930
Subject: [SciPy-dev] Difficulty creating an example for illustrating
	memmap.flush()
In-Reply-To: <928017.59766.qm@web52111.mail.re2.yahoo.com>
References: <928017.59766.qm@web52111.mail.re2.yahoo.com>
Message-ID: <23f4e3390908021656q426d4820va1fce9e688ef8930@mail.gmail.com>

On Mon, Aug 3, 2009 at 7:30 AM, David Goldsmith <d_l_goldsmith at yahoo.com>wrote:

>
> Hi, folks.  So, I'm trying to devise an example to illustrate
> memmap.flush().  It didn't work the way I was intuitively expecting (i.e.,
> it didn't appear to do anything at all) so I went looking for a "ready-made"
> example and found:
>
>
> http://www.slideshare.net/enthought/python-for-scientific-computing-webinar-may-22-2009
>
> which has an example on Slide 25.  However, here's what I get when I try to
> duplicate the example:
>
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC v.1310 32 bit
> win32
> >>> import numpy as np
> >>> np.version.version
> '1.3.0rc2'
> >>> q = np.memmap('new_file.dat',mode='w+',shape=(2,5))
> >>> q
> memmap([[0, 0, 0, 0, 0],
>       [0, 0, 0, 0, 0]], dtype=uint8)
> >>> # Print out underlying file contents
> ... # Note: not using iPython, so have to use os.system
> ...
> >>> import os
> >>> os.system('type new_file.dat')
>          0
> >>> # Note: already a little different than Webinar Expl.
> ...
> >>> # Next write ascii value for 'A' (65) into q
> ...
> >>> q[:] = ord('A')
> >>> q
> memmap([[65, 65, 65, 65, 65],
>       [65, 65, 65, 65, 65]], dtype=uint8)
> >>> # Do I need to call flush before file is written to?
> ...
> >>> os.system('type new_file.dat')
> AAAAAAAAAA0
> >>> # No! Does flushing change anything?
> ...
> >>> q.flush()
> >>> os.system('type new_file.dat')
> AAAAAAAAAA0
> >>> # No! Is it because I printed q before checking it on disc?
> ...
> >>> # Start afresh, but don't print memmap before checking it
> ...
> >>> r = np.memmap('new_file2.dat',mode='w+',shape=(2,5))
> >>> os.system('type new_file2.dat') # "reproducibility check"
>          0
> >>> r[:] = ord('A')
> >>> os.system('type new_file2.dat') # Checking file on disc immediately
> AAAAAAAAAA0
> >>> # File is updated without calling flush, indeed without any
> ... # intervening access to the memmap at all!
>
> What gives?  Is this a bug?  If not, can someone please furnish me with an
> example that clearly an explicitly illustrates the function (and necessity)
> of memmap.flush()?
>
Sorry, I can't. I can say that I encountered this when I first discovered
np.memmap and began using it for a lot of things. At one point, I didn't
seem to need to flush; In other cases I did.

I wonder whether it has to do with the underlying i/o buffering (eg. if your
array is smaller than the buffer, all changes are written immediately)

David
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090803/6d56027d/attachment.html>

From d_l_goldsmith at yahoo.com  Sun Aug  2 23:21:25 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 2 Aug 2009 20:21:25 -0700 (PDT)
Subject: [SciPy-dev] Difficulty creating an example for illustrating
	memmap.flush()
Message-ID: <723404.56571.qm@web52102.mail.re2.yahoo.com>


Hi, David.  Thanks for your (validating) reply. :-)  Well, there's something more going on than just the size of the mmap or the size of the change to the mmap: I experimented with adding a shape (10,) complex128 array to shape (10, 10), (10, 10, 10), (10, 10, 10, 10), (10, 10, 10, 10, 10) mmaps (and their corresponding "flat" mmaps, shape (100,), etc.), both just in the last dimension using [9, 9, 9, 9, :], e.g., and everywhere using [:,:,:,:,:] - which worked! - and the file on disc was updated every time without ever having to call flush, not once!  So, hopefully, someone who knows what's going on will chime in...

DG

--- On Sun, 8/2/09, David Gowers <00ai99 at gmail.com> wrote:

> From: David Gowers <00ai99 at gmail.com>
> Subject: Re: [SciPy-dev] Difficulty creating an example for illustrating memmap.flush()
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Sunday, August 2, 2009, 4:56 PM
> 
> 
> On Mon, Aug 3, 2009 at 7:30 AM,
> David Goldsmith <d_l_goldsmith at yahoo.com>
> wrote:
> 
> 
> 
> Hi, folks. ?So, I'm trying to devise an example to
> illustrate memmap.flush(). ?It didn't work the way I
> was intuitively expecting (i.e., it didn't appear to do
> anything at all) so I went looking for a
> "ready-made" example and found:
> 
> 
> 
> 
> http://www.slideshare.net/enthought/python-for-scientific-computing-webinar-may-22-2009
> 
> 
> 
> which has an example on Slide 25. ?However, here's
> what I get when I try to duplicate the example:
> 
> 
> 
> Python 2.5.2 (r252:60911, Feb 21 2008, 13:11:45) [MSC
> v.1310 32 bit
> 
> win32
> 
> >>> import numpy as np
> 
> >>> np.version.version
> 
> '1.3.0rc2'
> 
> >>> q =
> np.memmap('new_file.dat',mode='w+',shape=(2,5))
> 
> >>> q
> 
> memmap([[0, 0, 0, 0, 0],
> 
>  ? ? ? [0, 0, 0, 0, 0]], dtype=uint8)
> 
> >>> # Print out underlying file contents
> 
> ... # Note: not using iPython, so have to use os.system
> 
> ...
> 
> >>> import os
> 
> >>> os.system('type new_file.dat')
> 
>  ? ? ? ? ?0
> 
> >>> # Note: already a little different than
> Webinar Expl.
> 
> ...
> 
> >>> # Next write ascii value for 'A' (65)
> into q
> 
> ...
> 
> >>> q[:] = ord('A')
> 
> >>> q
> 
> memmap([[65, 65, 65, 65, 65],
> 
>  ? ? ? [65, 65, 65, 65, 65]], dtype=uint8)
> 
> >>> # Do I need to call flush before file is
> written to?
> 
> ...
> 
> >>> os.system('type new_file.dat')
> 
> AAAAAAAAAA0
> 
> >>> # No! Does flushing change anything?
> 
> ...
> 
> >>> q.flush()
> 
> >>> os.system('type new_file.dat')
> 
> AAAAAAAAAA0
> 
> >>> # No! Is it because I printed q before
> checking it on disc?
> 
> ...
> 
> >>> # Start afresh, but don't print memmap
> before checking it
> 
> ...
> 
> >>> r =
> np.memmap('new_file2.dat',mode='w+',shape=(2,5))
> 
> >>> os.system('type new_file2.dat') #
> "reproducibility check"
> 
>  ? ? ? ? ?0
> 
> >>> r[:] = ord('A')
> 
> >>> os.system('type new_file2.dat') #
> Checking file on disc immediately
> 
> AAAAAAAAAA0
> 
> >>> # File is updated without calling flush,
> indeed without any
> 
> ... # intervening access to the memmap at all!
> 
> 
> 
> What gives? ?Is this a bug? ?If not, can someone please
> furnish me with an example that clearly an explicitly
> illustrates the function (and necessity) of memmap.flush()?
> Sorry, I can't. I can say that I
> encountered this when I first discovered np.memmap and began
> using it for a lot of things. At one point, I didn't
> seem to need to flush; In other cases I did.
> 
> 
> I wonder whether it has to do with the underlying i/o
> buffering (eg. if your array is smaller than the buffer, all
> changes are written immediately)
> 
> David
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From charlesr.harris at gmail.com  Sun Aug  2 23:48:21 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sun, 2 Aug 2009 21:48:21 -0600
Subject: [SciPy-dev] Difficulty creating an example for illustrating
	memmap.flush()
In-Reply-To: <723404.56571.qm@web52102.mail.re2.yahoo.com>
References: <723404.56571.qm@web52102.mail.re2.yahoo.com>
Message-ID: <e06186140908022048r212c9ea9x64097775116ce44c@mail.gmail.com>

On Sun, Aug 2, 2009 at 9:21 PM, David Goldsmith <d_l_goldsmith at yahoo.com>wrote:

>
> Hi, David.  Thanks for your (validating) reply. :-)  Well, there's
> something more going on than just the size of the mmap or the size of the
> change to the mmap: I experimented with adding a shape (10,) complex128
> array to shape (10, 10), (10, 10, 10), (10, 10, 10, 10), (10, 10, 10, 10,
> 10) mmaps (and their corresponding "flat" mmaps, shape (100,), etc.), both
> just in the last dimension using [9, 9, 9, 9, :], e.g., and everywhere using
> [:,:,:,:,:] - which worked! - and the file on disc was updated every time
> without ever having to call flush, not once!  So, hopefully, someone who
> knows what's going on will chime in...
>

The data in memmapped files is swapped in and out of memory by the operating
system, so what happens to be in memory at any given time is OS dependent
and also changes with demands for memory from other processes and such. It's
a bunch a behind the scenes machinery and not predictable. Even a flush can
be delayed. My guess is that you need to flush the data if you want it to go
to disk, which might be the case if you have a program that needs to push
out data and you want to minimise losses from power outages and such.
Otherwise the data could just sit in memory and never get written out until
the file is closed.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090802/47746e81/attachment.html>

From d_l_goldsmith at yahoo.com  Mon Aug  3 00:04:11 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 2 Aug 2009 21:04:11 -0700 (PDT)
Subject: [SciPy-dev] Difficulty creating an example for illustrating
	memmap.flush()
Message-ID: <68517.77484.qm@web52108.mail.re2.yahoo.com>


OK, thanks, Charles, that explains a lot (which I'll be sure to put in the docstring - don't worry, not all of it, just the "highpoints"), but: if anyone knows a way to actually _prevent_ assignment from writing to the file - and forcing flush to do so right away - so that it may be illustrated with an example, that would still be appreciated (of course, I could "fudge" and call flush and then show the file contents, pretending that it hadn't already happened, but that would be dishonest.)

DG

--- On Sun, 8/2/09, Charles R Harris <charlesr.harris at gmail.com> wrote:

> From: Charles R Harris <charlesr.harris at gmail.com>
> Subject: Re: [SciPy-dev] Difficulty creating an example for illustrating memmap.flush()
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Sunday, August 2, 2009, 8:48 PM
> 
> 
> On Sun, Aug 2, 2009 at 9:21 PM,
> David Goldsmith <d_l_goldsmith at yahoo.com>
> wrote:
> 
> 
> 
> Hi, David. ?Thanks for your (validating) reply. :-)
> ?Well, there's something more going on than just the
> size of the mmap or the size of the change to the mmap: I
> experimented with adding a shape (10,) complex128 array to
> shape (10, 10), (10, 10, 10), (10, 10, 10, 10), (10, 10, 10,
> 10, 10) mmaps (and their corresponding "flat"
> mmaps, shape (100,), etc.), both just in the last dimension
> using [9, 9, 9, 9, :], e.g., and everywhere using
> [:,:,:,:,:] - which worked! - and the file on disc was
> updated every time without ever having to call flush, not
> once! ?So, hopefully, someone who knows what's going on
> will chime in...
> 
> 
> 
> The data in memmapped files is swapped in and out of memory
> by the operating system, so what happens to be in memory at
> any given time is OS dependent and also changes with demands
> for memory from other processes and such. It's a bunch a
> behind the scenes machinery and not predictable. Even a
> flush can be delayed. My guess is that you need to flush the
> data if you want it to go to disk, which might be the case
> if you have a program that needs to push out data and you
> want to minimise losses from power outages and such.
> Otherwise the data could just sit in memory and never get
> written out until the file is closed.
> 
> 
> Chuck
> 
> 
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From nmb at wartburg.edu  Mon Aug  3 10:29:35 2009
From: nmb at wartburg.edu (Neil Martinsen-Burrell)
Date: Mon, 03 Aug 2009 09:29:35 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <20090802210942.GC25001@phare.normalesup.org>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>	<4A75FAC1.7070600@wartburg.edu>
	<20090802210942.GC25001@phare.normalesup.org>
Message-ID: <4A76F44F.7080201@wartburg.edu>

On 08/02/2009 04:09 PM, Gael Varoquaux wrote:
> On Sun, Aug 02, 2009 at 03:44:49PM -0500, Neil Martinsen-Burrell wrote:
>>   I think that there is a tendency for active researchers to
>>   underestimate the importance of undergraduate-level learning and I
>>   hope that in this discussion, we will keep in mind the singular
>>   importance of that young audience.
>
> That's all good and nice. I agree with you it is important, and I am very
> happy to hear people talking about this, because it makes me hope that we
> will be getting more help to do this.

As I have time to spare apart from the teaching and researching duties 
that I need to do to keep *my* job, I am glad to volunteer my time for 
this effort.  I have some things in mind for making Scipy accessible as 
a module within a Numerical Analysis or Scientific Computing course that 
I hope to work on within this calendar year.

> If I work my ass off on an IDE, or more simply a GUI frontend, it won't
> help me get more work done, which means shooting papers out, to be
> cynical, and, in a few years, I will most likely not be doing any
> scientific Python anymore. On the other hand, if I work on something that
> is useful for my day to day work, I get some traction at the lab, and my
> sleepless face is more easily forgiven. If I build an IDE that is of no
> use to our work, nobody cares, and for a good reason.
>
> This is not to say that we shouldn't be working on the IDE, I believe
> that I am one of the people that have actually written code to do this,
> but there is a lot of work to be done here, and working on making sure
> that we have a shell to do this, and interactive plotting, and good
> documentation is part of this work, and can be reused for direct research
> interests. Writing docs is also something that can help a lot, does not
> require extensive technical knowledge and takes a lot of time.

Indeed, you have highlighted one of the difficulties in depending on 
active domain scientists to create software projects: scratching one's 
itch is not selfish, but necessary for their career. As Joe mentioned 
about the Doc marathon funded through some of his grants, as his 
granting situation gets tighter, the funding that he is able to devote 
to SciPy development is drying up.  I think that this is a persuasive 
argument for the establishment of a SciPy Foundation which can provide 
the organizational structure to pay willing developers for some of the 
code which they develop. In doing so, we provide an alternative system 
of rewards (however small) from the scientific career track.

> Actually, I must point out that I am quite unhappy, because I am very
> tired, I have spent the week end fixing bugs on various open source
> projects (nipy and mayavi) and answering complicated users questions. I
> find that to be told that we are underestimating the importance of ease
> of use and ease of learning is unfair. This simply takes a lot of time
> and some of us are working on it.

I certainly appreciate your work fixing bugs on open-source scientific 
projects.  Thank you.  It was not my intent to say that anyone is 
"underestimating the importance of ease of use and ease of learning". 
My intent was to highlight an audience for Scipy that has significant 
importance for future uptake.

-Neil


From prabhu at aero.iitb.ac.in  Mon Aug  3 13:20:28 2009
From: prabhu at aero.iitb.ac.in (Prabhu Ramachandran)
Date: Mon, 03 Aug 2009 22:50:28 +0530
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
Message-ID: <4A771C5C.4060702@aero.iitb.ac.in>

On 07/31/09 22:36, Joe Harrington wrote:
> About sixteen months ago, I launched the SciPy Documentation Project
> and its Marathon.  Dozens pitched in and now numpy docs are rapidly
> approaching a professional level.  The "pink wave" ("Needs Review"
> status) is at 56% today!  There is consensus among doc writers that
> much of the rest can be labeled in the "unimportant" category, so
> we're close to starting the review push (hold your fire, there is a
> web site mod to be done first).
> 
> We're also nearing the end of the summer, and it's time to look ahead.
> The path for docs is clear, but the path for SciPy is not.  I think
> our weakest area right now is organization of the project.  There is
> no consensus-based plan for improvement of the whole toward a stated
> goal, no centralized coordination of work, and no funded work focused
> on many of our weaknesses, notwithstanding my doc effort and what
> Enthought does for code.

Thank you for your efforts!

I believe I will be able to help this effort in various ways over the 
next few years from India as part of a large government grant.  I do not 
have the time to discuss it here at the moment but I will be at SciPy09 
and would love to discuss it there in person.  I will also be talking 
briefly about our overall goals there.  Specifically see:

  http://conference.scipy.org/abstract?id=13

regards,
prabhu


From d_l_goldsmith at yahoo.com  Mon Aug  3 13:33:27 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Mon, 3 Aug 2009 10:33:27 -0700 (PDT)
Subject: [SciPy-dev] SciPy Foundation
Message-ID: <191449.13197.qm@web52108.mail.re2.yahoo.com>


Thanks, Prabhu, this looks very promising!  I look forward to your talk!

DG

--- On Mon, 8/3/09, Prabhu Ramachandran <prabhu at aero.iitb.ac.in> wrote:

> From: Prabhu Ramachandran <prabhu at aero.iitb.ac.in>
> Subject: Re: [SciPy-dev] SciPy Foundation
> To: jh at physics.ucf.edu, scipy-dev at scipy.org
> Cc: "Scipy Users" <scipy-user at scipy.org>, "Astronomy Python" <astropy at scipy.org>, numpy-discussion at scipy.org
> Date: Monday, August 3, 2009, 10:20 AM
> On 07/31/09 22:36, Joe Harrington
> wrote:
> > About sixteen months ago, I launched the SciPy
> Documentation Project
> > and its Marathon.? Dozens pitched in and now
> numpy docs are rapidly
> > approaching a professional level.? The "pink
> wave" ("Needs Review"
> > status) is at 56% today!? There is consensus
> among doc writers that
> > much of the rest can be labeled in the "unimportant"
> category, so
> > we're close to starting the review push (hold your
> fire, there is a
> > web site mod to be done first).
> > 
> > We're also nearing the end of the summer, and it's
> time to look ahead.
> > The path for docs is clear, but the path for SciPy is
> not.? I think
> > our weakest area right now is organization of the
> project.? There is
> > no consensus-based plan for improvement of the whole
> toward a stated
> > goal, no centralized coordination of work, and no
> funded work focused
> > on many of our weaknesses, notwithstanding my doc
> effort and what
> > Enthought does for code.
> 
> Thank you for your efforts!
> 
> I believe I will be able to help this effort in various
> ways over the 
> next few years from India as part of a large government
> grant.? I do not 
> have the time to discuss it here at the moment but I will
> be at SciPy09 
> and would love to discuss it there in person.? I will
> also be talking 
> briefly about our overall goals there.? Specifically
> see:
> 
> ? http://conference.scipy.org/abstract?id=13
> 
> regards,
> prabhu
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From charlesr.harris at gmail.com  Mon Aug  3 13:44:38 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Mon, 3 Aug 2009 11:44:38 -0600
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A76F44F.7080201@wartburg.edu>
References: <274366.95245.qm@web52106.mail.re2.yahoo.com>
	<4A75FAC1.7070600@wartburg.edu>
	<20090802210942.GC25001@phare.normalesup.org>
	<4A76F44F.7080201@wartburg.edu>
Message-ID: <e06186140908031044ya190594v7147ead325bb491c@mail.gmail.com>

On Mon, Aug 3, 2009 at 8:29 AM, Neil Martinsen-Burrell <nmb at wartburg.edu>wrote:

> On 08/02/2009 04:09 PM, Gael Varoquaux wrote:
> > On Sun, Aug 02, 2009 at 03:44:49PM -0500, Neil Martinsen-Burrell wrote:
> >>   I think that there is a tendency for active researchers to
> >>   underestimate the importance of undergraduate-level learning and I
> >>   hope that in this discussion, we will keep in mind the singular
> >>   importance of that young audience.
> >
> > That's all good and nice. I agree with you it is important, and I am very
> > happy to hear people talking about this, because it makes me hope that we
> > will be getting more help to do this.
>
> As I have time to spare apart from the teaching and researching duties
> that I need to do to keep *my* job, I am glad to volunteer my time for
> this effort.  I have some things in mind for making Scipy accessible as
> a module within a Numerical Analysis or Scientific Computing course that
> I hope to work on within this calendar year.
>
> > If I work my ass off on an IDE, or more simply a GUI frontend, it won't
> > help me get more work done, which means shooting papers out, to be
> > cynical, and, in a few years, I will most likely not be doing any
> > scientific Python anymore. On the other hand, if I work on something that
> > is useful for my day to day work, I get some traction at the lab, and my
> > sleepless face is more easily forgiven. If I build an IDE that is of no
> > use to our work, nobody cares, and for a good reason.
> >
> > This is not to say that we shouldn't be working on the IDE, I believe
> > that I am one of the people that have actually written code to do this,
> > but there is a lot of work to be done here, and working on making sure
> > that we have a shell to do this, and interactive plotting, and good
> > documentation is part of this work, and can be reused for direct research
> > interests. Writing docs is also something that can help a lot, does not
> > require extensive technical knowledge and takes a lot of time.
>
> Indeed, you have highlighted one of the difficulties in depending on
> active domain scientists to create software projects: scratching one's
> itch is not selfish, but necessary for their career.


Linus on selfish apropos Microsoft contributing driver code to linux:

I agree that it's driven by selfish reasons, but that's how all open source
code gets written! We all "scratch our own itches". It's why I started
Linux, it's why I started git, and it's why I am still involved. It's the
reason for everybody to end up in open source, to some degree.

So complaining about the fact that Microsoft picked a selfish area to work
on is just silly. Of course they picked an area that helps them. That's the
point of open source - the ability to make the code better for your
particular needs, whoever the 'your' in question happens to be.

Does anybody complain when hardware companies write drivers for the hardware
they produce? No. That would be crazy. Does anybody complain when IBM funds
all the POWER development, and works on enterprise features because they
sell into the enterprise? No. That would be insane.

So the people who complain about Microsoft writing drivers for their own
virtualization model should take a long look in the mirror and ask
themselves why they are being so hypocritical.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090803/1fb89453/attachment.html>

From ondrej at certik.cz  Mon Aug  3 17:32:31 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 3 Aug 2009 15:32:31 -0600
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <20090801225216.GB31380@phare.normalesup.org>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
Message-ID: <85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>

On Sat, Aug 1, 2009 at 4:52 PM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
[...]
> For a web environment, the Sage notebook is amazing. Unfortunately last
> time I looked, it was GPL licensed, which renders it improper for my use,
> as the tools we use at the lab must be BSD, in order to be able to build
> (eventually) medical imaging products from them one day.

Actually, in this thread:

http://groups.google.com/group/sage-devel/browse_thread/thread/65ca1e0489a0a980/

most (if not all) contributors to the Sage notebook agreed to release
their code as BSD.

The same about William being positive to license the build system as
BSD too. So we can get lots of done by working on these things
together with Sage.

Ondrej


From gael.varoquaux at normalesup.org  Mon Aug  3 17:36:49 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 3 Aug 2009 23:36:49 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
Message-ID: <20090803213649.GI32408@phare.normalesup.org>

On Mon, Aug 03, 2009 at 03:32:31PM -0600, Ondrej Certik wrote:
> On Sat, Aug 1, 2009 at 4:52 PM, Gael
> Varoquaux<gael.varoquaux at normalesup.org> wrote:
> [...]
> > For a web environment, the Sage notebook is amazing. Unfortunately last
> > time I looked, it was GPL licensed, which renders it improper for my use,
> > as the tools we use at the lab must be BSD, in order to be able to build
> > (eventually) medical imaging products from them one day.

> Actually, in this thread:

> http://groups.google.com/group/sage-devel/browse_thread/thread/65ca1e0489a0a980/

> most (if not all) contributors to the Sage notebook agreed to release
> their code as BSD.

> The same about William being positive to license the build system as
> BSD too. So we can get lots of done by working on these things
> together with Sage.

I can see that a lot of good things are coming out of Sage (the current
Cython development frenzy was clearly helped by the needs of Sage). It is
really nice to see our community (I am talking in the sens of a
scientific Python community, agnostic of tools and distribution) growing.

Cheers to these guys, that notebook is really amazing!

Ga?l


From ondrej at certik.cz  Mon Aug  3 17:54:08 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Mon, 3 Aug 2009 15:54:08 -0600
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <20090803213649.GI32408@phare.normalesup.org>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
	<20090803213649.GI32408@phare.normalesup.org>
Message-ID: <85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>

On Mon, Aug 3, 2009 at 3:36 PM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Mon, Aug 03, 2009 at 03:32:31PM -0600, Ondrej Certik wrote:
>> On Sat, Aug 1, 2009 at 4:52 PM, Gael
>> Varoquaux<gael.varoquaux at normalesup.org> wrote:
>> [...]
>> > For a web environment, the Sage notebook is amazing. Unfortunately last
>> > time I looked, it was GPL licensed, which renders it improper for my use,
>> > as the tools we use at the lab must be BSD, in order to be able to build
>> > (eventually) medical imaging products from them one day.
>
>> Actually, in this thread:
>
>> http://groups.google.com/group/sage-devel/browse_thread/thread/65ca1e0489a0a980/
>
>> most (if not all) contributors to the Sage notebook agreed to release
>> their code as BSD.
>
>> The same about William being positive to license the build system as
>> BSD too. So we can get lots of done by working on these things
>> together with Sage.
>
> I can see that a lot of good things are coming out of Sage (the current
> Cython development frenzy was clearly helped by the needs of Sage). It is
> really nice to see our community (I am talking in the sens of a
> scientific Python community, agnostic of tools and distribution) growing.
>
> Cheers to these guys, that notebook is really amazing!

Yep. And Cython is BSD like (resp Apache) license too, so I think that
for these basic tools that everyone needs (cython/notebook/build
infrustructure) Sage is not against BSD at all.

Ondrej


From sebastian.walter at gmail.com  Tue Aug  4 04:00:26 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Tue, 4 Aug 2009 10:00:26 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
	<20090803213649.GI32408@phare.normalesup.org>
	<85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>
Message-ID: <ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>

2 cents from an outsider who thought about contributing to
scipy/scikits (but didn't (yet)):

I think it is a good idea to make scipy easy to use for beginners.
However, after reading this thread, I have the impression that it is
not the goal to provide state of the art algorithms but rather making
Scipy as popular as possible by putting money and effort into the
"marketing" of Scipy.
Don't get me wrong, I think there are some good reasons why a project
should thrive for a large user base. Some of the best projects are
popular.
Alas, correlation does not imply causality.

Me for instance, would rather like to see more efforts to get state of
the art algorithms to be implemented in Scipy because that's something
that would make a real difference in my research work. Of course,
targeting the "clueless Matlab" users is quite pointless if it is that
what you are after.
IMHO the way to go is to convince experts to implement their research
prototypes as part of scipy.
Then you really get some "killer applications". I could name a few
people who are coding some cool state of the art algorithms but waste
so much time because they started coding directly in C++. In the
meantime, they could have implemented the algorithms in Python _and_
in C++. If scipy had something really good that Matlab etc. do not
have: guess what ppl would do....

What would you need to get experts contribute to scipy instead of
hacking their prototype in Matlab or C++?
I can't speak for everyone, so I'll just say what I think (and feel):
I would instantly start "contributing research prototypes" to scipy if
scipy offered:
1) an easy, modular and flexible build system (fortran, c, c++, D,
swig, boost:python, cython,...)
2) very low entry barrier for possible contributors:
  a simple checkout, then  ./manage.py startapp  mycoolmodule
  and everything is ready to go ( "Start coding in 5 minutes!")
3) a distributed version control system (e.g. git). SVN really scares me off...
4) standardized unit tests
5) automated documentation generation

Then I could simply
1) fork the master branch
2) ./manage.py startapp mycoolmodule
3) adjust config files that were written in ./scipy/mycoolmodule/config.py
4) start coding
5) share the experimental code with collaborators or interested users
who are not afraid to use experimental code
6) eventually, when the project has matured, hope that it gets
included in the master branch


hope that made sense,
Sebastian






On Mon, Aug 3, 2009 at 11:54 PM, Ondrej Certik<ondrej at certik.cz> wrote:
> On Mon, Aug 3, 2009 at 3:36 PM, Gael
> Varoquaux<gael.varoquaux at normalesup.org> wrote:
>> On Mon, Aug 03, 2009 at 03:32:31PM -0600, Ondrej Certik wrote:
>>> On Sat, Aug 1, 2009 at 4:52 PM, Gael
>>> Varoquaux<gael.varoquaux at normalesup.org> wrote:
>>> [...]
>>> > For a web environment, the Sage notebook is amazing. Unfortunately last
>>> > time I looked, it was GPL licensed, which renders it improper for my use,
>>> > as the tools we use at the lab must be BSD, in order to be able to build
>>> > (eventually) medical imaging products from them one day.
>>
>>> Actually, in this thread:
>>
>>> http://groups.google.com/group/sage-devel/browse_thread/thread/65ca1e0489a0a980/
>>
>>> most (if not all) contributors to the Sage notebook agreed to release
>>> their code as BSD.
>>
>>> The same about William being positive to license the build system as
>>> BSD too. So we can get lots of done by working on these things
>>> together with Sage.
>>
>> I can see that a lot of good things are coming out of Sage (the current
>> Cython development frenzy was clearly helped by the needs of Sage). It is
>> really nice to see our community (I am talking in the sens of a
>> scientific Python community, agnostic of tools and distribution) growing.
>>
>> Cheers to these guys, that notebook is really amazing!
>
> Yep. And Cython is BSD like (resp Apache) license too, so I think that
> for these basic tools that everyone needs (cython/notebook/build
> infrustructure) Sage is not against BSD at all.
>
> Ondrej
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From gael.varoquaux at normalesup.org  Tue Aug  4 04:18:25 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 4 Aug 2009 10:18:25 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
	<20090803213649.GI32408@phare.normalesup.org>
	<85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>
	<ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>
Message-ID: <20090804081825.GC17519@phare.normalesup.org>

On Tue, Aug 04, 2009 at 10:00:26AM +0200, Sebastian Walter wrote:
> Me for instance, would rather like to see more efforts to get state of
> the art algorithms to be implemented in Scipy because that's something
> that would make a real difference in my research work.

On this side, we are hiring a talented engineer to work on machine
learning in scipy, via the scikit learn. We already have the algorithm,
it is a question of QAing them, integrating them in the scikit,
writing docs and making releases.

Ga?l


From david at ar.media.kyoto-u.ac.jp  Tue Aug  4 04:35:02 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Tue, 04 Aug 2009 17:35:02 +0900
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>	<20090801225216.GB31380@phare.normalesup.org>	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>	<20090803213649.GI32408@phare.normalesup.org>	<85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>
	<ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>
Message-ID: <4A77F2B6.8040108@ar.media.kyoto-u.ac.jp>

Sebastian Walter wrote:
> 2 cents from an outsider who thought about contributing to
> scipy/scikits (but didn't (yet)):
>
> I think it is a good idea to make scipy easy to use for beginners.
> However, after reading this thread, I have the impression that it is
> not the goal to provide state of the art algorithms but rather making
> Scipy as popular as possible by putting money and effort into the
> "marketing" of Scipy.
> Don't get me wrong, I think there are some good reasons why a project
> should thrive for a large user base. Some of the best projects are
> popular.
> Alas, correlation does not imply causality.
>
> Me for instance, would rather like to see more efforts to get state of
> the art algorithms to be implemented in Scipy because that's something
> that would make a real difference in my research work. Of course,
> targeting the "clueless Matlab" users is quite pointless if it is that
> what you are after.
>   

One point which has not been mentioned concerning matlab-like
environment - maybe it is obvious and everyone implicitly acknowledges
it, but Mathworks is a 30 years old company, with > 1000 people today.

Building something like matlab, with a good GUI and top notch
documentation takes a huge amount of resources, of which the 'useful'
code is only a fraction. I of course don't know the details of matlab
implementation, but I know that for music oriented softwares (which need
good UI to sell well, and have non trivial computational requirements,
so the comparison is not totally stupid), the graphical code is 80 % of
the code. This ratio is consistent with the big open source audio
softwares as well (ardour, rosegarden). Worse, being cross platform
makes the problem much more difficult. For music softwares market, mac
os x is rarely ignored (~ 40-50% of the market I believe), so people
need to support two platforms, and that's really a lot of work. For
scientific software, I think you can go the non native route for the
graphical toolkit, though.

Also, very few open source software are successful as far as good GUI
are concerned (I don't want to enter into a debate here, but there are
good documents/studies on this topic). You need financial incentive for
this, so only projects backed up by big companies managed to pull it of.

IOW, I am pretty pessimistic about being a 'matlab' clone. We should
rather shoot for what makes numpy/scipy better (extensibility, cross
platform, actual language, etc...), because really, matlab will always
be a much better matlab than us. Price and licensing are not good enough
to justify migration - if what you want is a free matlab clone, why not
using octave or scilab anyway.

That does NOT mean that we should not aim at making the software more
accessible. I (and I guess other developers) are definitely interested
in a more product-like, integrated stack, to make the barrier of entry
lower. I for example am really tired of the installation problems
consistently reported. I feel like we cover mac os x and windows pretty
well now, but the linux situation is still dreadful. I have a few ideas
on how to improve the situation, but they all requires quite a bit of
work/infrastructure. I hope that soon, the scenario "I see this cool
python script on the internet, it requires this numpy/scipy thing, can I
try it in 2 minutes ?" will be a reality.

> Then you really get some "killer applications". I could name a few
> people who are coding some cool state of the art algorithms but waste
> so much time because they started coding directly in C++. In the
> meantime, they could have implemented the algorithms in Python _and_
> in C++. If scipy had something really good that Matlab etc. do not
> have: guess what ppl would do....
>   

Yes, there are a lot of people who still don't know that there are
languages outside Fortran, C and C++. In my field, I still see some
people who implement parsers in C...

> 1) an easy, modular and flexible build system (fortran, c, c++, D,
> swig, boost:python, cython,...)
>   

you mean like numscons :) ? Adding D support to numscons should be easy.
For example, I added initial cython support in a couple of minutes
during the cython talk at SciPy08, adding new languages is relatively
easy thanks to scons.

> 2) very low entry barrier for possible contributors:
>   a simple checkout, then  ./manage.py startapp  mycoolmodule
>   and everything is ready to go ( "Start coding in 5 minutes!")
>   

there are various pieces to enable this (in place build, develop command
of setuptools, virtualenv/pip/easy_install), but yes, the situation is
kind of messy. For scikits, that's not so difficult  - you should be
able to implement a trivial scikit by copying the scikits.example
package and starting from there.

One problem is that it is technically impossible to build in place and
test in one go because of a nose limitation ATM (for some reason, nose
fails to import a package if it is in the current directory).

> 3) a distributed version control system (e.g. git). SVN really scares me off...
>   

That's a sensitive issue, I think we should avoid starting this one here
:) Needless to say, you can use git-svn - several core developers use it
for numpy/scipy dev, and we distribute an official import:

http://projects.scipy.org/numpy/browse_git

At least I have not touched svn for numpy/scipy development for > 6
months now, except to check releases when I tag them.

> 4) standardized unit tests
>   

What do you mean exactly here ? We use nose for testing, what do you
consider "non standard".

> 5) automated documentation generation
>   

It is almost automated now - but an example for scikits is missing in
the example package :)

cheers,

David


From sebastian.walter at gmail.com  Tue Aug  4 05:25:55 2009
From: sebastian.walter at gmail.com (Sebastian Walter)
Date: Tue, 4 Aug 2009 11:25:55 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <4A77F2B6.8040108@ar.media.kyoto-u.ac.jp>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<a37fe44e2b2b8b3acff3631e015ffaf5.squirrel@webmail.uio.no>
	<wl8d47f4ibi.fsf@glup.physics.ucf.edu>
	<704e22078868036a820177ce53122d48.squirrel@webmail.uio.no>
	<20090801225216.GB31380@phare.normalesup.org>
	<85b5c3130908031432l6190c654t87dcb6c1ae44bb5e@mail.gmail.com>
	<20090803213649.GI32408@phare.normalesup.org>
	<85b5c3130908031454p5f75e950occc7a89553ace535@mail.gmail.com>
	<ec9f80fa0908040100n2b59f99am290f624f0a4ff32a@mail.gmail.com>
	<4A77F2B6.8040108@ar.media.kyoto-u.ac.jp>
Message-ID: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>

On Tue, Aug 4, 2009 at 10:35 AM, David
Cournapeau<david at ar.media.kyoto-u.ac.jp> wrote:
> Sebastian Walter wrote:
>> 2 cents from an outsider who thought about contributing to
>> scipy/scikits (but didn't (yet)):
>>
>> I think it is a good idea to make scipy easy to use for beginners.
>> However, after reading this thread, I have the impression that it is
>> not the goal to provide state of the art algorithms but rather making
>> Scipy as popular as possible by putting money and effort into the
>> "marketing" of Scipy.
>> Don't get me wrong, I think there are some good reasons why a project
>> should thrive for a large user base. Some of the best projects are
>> popular.
>> Alas, correlation does not imply causality.
>>
>> Me for instance, would rather like to see more efforts to get state of
>> the art algorithms to be implemented in Scipy because that's something
>> that would make a real difference in my research work. Of course,
>> targeting the "clueless Matlab" users is quite pointless if it is that
>> what you are after.
>>
>
> One point which has not been mentioned concerning matlab-like
> environment - maybe it is obvious and everyone implicitly acknowledges
> it, but Mathworks is a 30 years old company, with > 1000 people today.
>
> Building something like matlab, with a good GUI and top notch
> documentation takes a huge amount of resources, of which the 'useful'
> code is only a fraction. I of course don't know the details of matlab
> implementation, but I know that for music oriented softwares (which need
> good UI to sell well, and have non trivial computational requirements,
> so the comparison is not totally stupid), the graphical code is 80 % of
> the code. This ratio is consistent with the big open source audio
> softwares as well (ardour, rosegarden). Worse, being cross platform
> makes the problem much more difficult. For music softwares market, mac
> os x is rarely ignored (~ 40-50% of the market I believe), so people
> need to support two platforms, and that's really a lot of work. For
> scientific software, I think you can go the non native route for the
> graphical toolkit, though.
>
> Also, very few open source software are successful as far as good GUI
> are concerned (I don't want to enter into a debate here, but there are
> good documents/studies on this topic). You need financial incentive for
> this, so only projects backed up by big companies managed to pull it of.
>
> IOW, I am pretty pessimistic about being a 'matlab' clone. We should
> rather shoot for what makes numpy/scipy better (extensibility, cross
> platform, actual language, etc...), because really, matlab will always
> be a much better matlab than us. Price and licensing are not good enough
> to justify migration - if what you want is a free matlab clone, why not
> using octave or scilab anyway.
>
> That does NOT mean that we should not aim at making the software more
> accessible. I (and I guess other developers) are definitely interested
> in a more product-like, integrated stack, to make the barrier of entry
> lower. I for example am really tired of the installation problems
> consistently reported. I feel like we cover mac os x and windows pretty
> well now, but the linux situation is still dreadful. I have a few ideas
> on how to improve the situation, but they all requires quite a bit of
> work/infrastructure. I hope that soon, the scenario "I see this cool
> python script on the internet, it requires this numpy/scipy thing, can I
> try it in 2 minutes ?" will be a reality.
>
>> Then you really get some "killer applications". I could name a few
>> people who are coding some cool state of the art algorithms but waste
>> so much time because they started coding directly in C++. In the
>> meantime, they could have implemented the algorithms in Python _and_
>> in C++. If scipy had something really good that Matlab etc. do not
>> have: guess what ppl would do....
>>
>
> Yes, there are a lot of people who still don't know that there are
> languages outside Fortran, C and C++. In my field, I still see some
> people who implement parsers in C...
>
>> 1) an easy, modular and flexible build system (fortran, c, c++, D,
>> swig, boost:python, cython,...)
>>
>
> you mean like numscons :) ? Adding D support to numscons should be easy.
> For example, I added initial cython support in a couple of minutes
> during the cython talk at SciPy08, adding new languages is relatively
> easy thanks to scons.
>
>> 2) very low entry barrier for possible contributors:
>>   a simple checkout, then  ./manage.py startapp  mycoolmodule
>>   and everything is ready to go ( "Start coding in 5 minutes!")
>>
>
> there are various pieces to enable this (in place build, develop command
> of setuptools, virtualenv/pip/easy_install), but yes, the situation is
> kind of messy. For scikits, that's not so difficult  - you should be
> able to implement a trivial scikit by copying the scikits.example
> package and starting from there.
>
> One problem is that it is technically impossible to build in place and
> test in one go because of a nose limitation ATM (for some reason, nose
> fails to import a package if it is in the current directory).
>
>> 3) a distributed version control system (e.g. git). SVN really scares me off...
>>
>
> That's a sensitive issue, I think we should avoid starting this one here
> :) Needless to say, you can use git-svn - several core developers use it
> for numpy/scipy dev, and we distribute an official import:
>
> http://projects.scipy.org/numpy/browse_git
>
> At least I have not touched svn for numpy/scipy development for > 6
> months now, except to check releases when I tag them.
>
>> 4) standardized unit tests
>>
>
> What do you mean exactly here ? We use nose for testing, what do you
> consider "non standard".
>
>> 5) automated documentation generation
>>
>
> It is almost automated now - but an example for scikits is missing in
> the example package :)
>

Just enumerating what I think would be useful to attract high quality
contributors.  I'm aware that scipy has already  a lot of the features
(which is nice).
But it would be even nicer to have a really low entry barrier and have
a framework that guides you to write good (and documented) code with
extensive unit tests, just like the big web frameworks (Django, RoR,
...)
It has to be a win-win situation for both the community and the developer.


> cheers,
>
> David
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Tue Aug  4 13:17:31 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 4 Aug 2009 10:17:31 -0700 (PDT)
Subject: [SciPy-dev] Anyone up for a Skypecon?
Message-ID: <907186.73991.qm@web52103.mail.re2.yahoo.com>


Well, past two weeks, Summer Marathon Skypecon participation has been nil.  Jack Liddle just Skyped me asking me about this week - which I presume means he's interested and available, in principle at least - and I said "regular time, tomorrow (Wed.) 19:00 UTC," but if it would increase participation to hold it at a different time, just let me know...

DG


      


From d_l_goldsmith at yahoo.com  Tue Aug  4 14:53:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 4 Aug 2009 11:53:20 -0700 (PDT)
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>
Message-ID: <291714.68720.qm@web52104.mail.re2.yahoo.com>


At this point I think the question becomes: do we let the (clear) fact that there is not a single set of priorities for where SciPy should be headed (which I do not see as a bad thing at this stage) get in the way of the community moving on *some* proposal (e.g., Joe's, with mods) for *some* "not-for-profit entity" (e.g., a "SciPy Foundation," the original topic of this thread) that will function as an institutional resource for furthering whichever priorities for SciPy should bubble to the surface?  In other words, this thread is diverging (into territory necessary to discuss, yes), but can we at least agree (a semi-rhetorical question because I think the answer is clearly "yes") that something along the lines of a "SciPy Foundation" would be useful, certainly for helping us move SciPy where we want it to go, but perhaps also for helping us decide where as well?

DG

--- On Tue, 8/4/09, Sebastian Walter <sebastian.walter at gmail.com> wrote:

> From: Sebastian Walter <sebastian.walter at gmail.com>
> Subject: Re: [SciPy-dev] SciPy Foundation
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 4, 2009, 2:25 AM
> On Tue, Aug 4, 2009 at 10:35 AM,
> David
> Cournapeau<david at ar.media.kyoto-u.ac.jp>
> wrote:
> > Sebastian Walter wrote:
> >> 2 cents from an outsider who thought about
> contributing to
> >> scipy/scikits (but didn't (yet)):
> >>
> >> I think it is a good idea to make scipy easy to
> use for beginners.
> >> However, after reading this thread, I have the
> impression that it is
> >> not the goal to provide state of the art
> algorithms but rather making
> >> Scipy as popular as possible by putting money and
> effort into the
> >> "marketing" of Scipy.
> >> Don't get me wrong, I think there are some good
> reasons why a project
> >> should thrive for a large user base. Some of the
> best projects are
> >> popular.
> >> Alas, correlation does not imply causality.
> >>
> >> Me for instance, would rather like to see more
> efforts to get state of
> >> the art algorithms to be implemented in Scipy
> because that's something
> >> that would make a real difference in my research
> work. Of course,
> >> targeting the "clueless Matlab" users is quite
> pointless if it is that
> >> what you are after.
> >>
> >
> > One point which has not been mentioned concerning
> matlab-like
> > environment - maybe it is obvious and everyone
> implicitly acknowledges
> > it, but Mathworks is a 30 years old company, with >
> 1000 people today.
> >
> > Building something like matlab, with a good GUI and
> top notch
> > documentation takes a huge amount of resources, of
> which the 'useful'
> > code is only a fraction. I of course don't know the
> details of matlab
> > implementation, but I know that for music oriented
> softwares (which need
> > good UI to sell well, and have non trivial
> computational requirements,
> > so the comparison is not totally stupid), the
> graphical code is 80 % of
> > the code. This ratio is consistent with the big open
> source audio
> > softwares as well (ardour, rosegarden). Worse, being
> cross platform
> > makes the problem much more difficult. For music
> softwares market, mac
> > os x is rarely ignored (~ 40-50% of the market I
> believe), so people
> > need to support two platforms, and that's really a lot
> of work. For
> > scientific software, I think you can go the non native
> route for the
> > graphical toolkit, though.
> >
> > Also, very few open source software are successful as
> far as good GUI
> > are concerned (I don't want to enter into a debate
> here, but there are
> > good documents/studies on this topic). You need
> financial incentive for
> > this, so only projects backed up by big companies
> managed to pull it of.
> >
> > IOW, I am pretty pessimistic about being a 'matlab'
> clone. We should
> > rather shoot for what makes numpy/scipy better
> (extensibility, cross
> > platform, actual language, etc...), because really,
> matlab will always
> > be a much better matlab than us. Price and licensing
> are not good enough
> > to justify migration - if what you want is a free
> matlab clone, why not
> > using octave or scilab anyway.
> >
> > That does NOT mean that we should not aim at making
> the software more
> > accessible. I (and I guess other developers) are
> definitely interested
> > in a more product-like, integrated stack, to make the
> barrier of entry
> > lower. I for example am really tired of the
> installation problems
> > consistently reported. I feel like we cover mac os x
> and windows pretty
> > well now, but the linux situation is still dreadful. I
> have a few ideas
> > on how to improve the situation, but they all requires
> quite a bit of
> > work/infrastructure. I hope that soon, the scenario "I
> see this cool
> > python script on the internet, it requires this
> numpy/scipy thing, can I
> > try it in 2 minutes ?" will be a reality.
> >
> >> Then you really get some "killer applications". I
> could name a few
> >> people who are coding some cool state of the art
> algorithms but waste
> >> so much time because they started coding directly
> in C++. In the
> >> meantime, they could have implemented the
> algorithms in Python _and_
> >> in C++. If scipy had something really good that
> Matlab etc. do not
> >> have: guess what ppl would do....
> >>
> >
> > Yes, there are a lot of people who still don't know
> that there are
> > languages outside Fortran, C and C++. In my field, I
> still see some
> > people who implement parsers in C...
> >
> >> 1) an easy, modular and flexible build system
> (fortran, c, c++, D,
> >> swig, boost:python, cython,...)
> >>
> >
> > you mean like numscons :) ? Adding D support to
> numscons should be easy.
> > For example, I added initial cython support in a
> couple of minutes
> > during the cython talk at SciPy08, adding new
> languages is relatively
> > easy thanks to scons.
> >
> >> 2) very low entry barrier for possible
> contributors:
> >>???a simple checkout, then?
> ./manage.py startapp? mycoolmodule
> >>???and everything is ready to go (
> "Start coding in 5 minutes!")
> >>
> >
> > there are various pieces to enable this (in place
> build, develop command
> > of setuptools, virtualenv/pip/easy_install), but yes,
> the situation is
> > kind of messy. For scikits, that's not so
> difficult? - you should be
> > able to implement a trivial scikit by copying the
> scikits.example
> > package and starting from there.
> >
> > One problem is that it is technically impossible to
> build in place and
> > test in one go because of a nose limitation ATM (for
> some reason, nose
> > fails to import a package if it is in the current
> directory).
> >
> >> 3) a distributed version control system (e.g.
> git). SVN really scares me off...
> >>
> >
> > That's a sensitive issue, I think we should avoid
> starting this one here
> > :) Needless to say, you can use git-svn - several core
> developers use it
> > for numpy/scipy dev, and we distribute an official
> import:
> >
> > http://projects.scipy.org/numpy/browse_git
> >
> > At least I have not touched svn for numpy/scipy
> development for > 6
> > months now, except to check releases when I tag them.
> >
> >> 4) standardized unit tests
> >>
> >
> > What do you mean exactly here ? We use nose for
> testing, what do you
> > consider "non standard".
> >
> >> 5) automated documentation generation
> >>
> >
> > It is almost automated now - but an example for
> scikits is missing in
> > the example package :)
> >
> 
> Just enumerating what I think would be useful to attract
> high quality
> contributors.? I'm aware that scipy has already?
> a lot of the features
> (which is nice).
> But it would be even nicer to have a really low entry
> barrier and have
> a framework that guides you to write good (and documented)
> code with
> extensive unit tests, just like the big web frameworks
> (Django, RoR,
> ...)
> It has to be a win-win situation for both the community and
> the developer.
> 
> 
> > cheers,
> >
> > David
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From robert.kern at gmail.com  Tue Aug  4 15:37:01 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 4 Aug 2009 14:37:01 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <291714.68720.qm@web52104.mail.re2.yahoo.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com> 
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
Message-ID: <3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>

On Tue, Aug 4, 2009 at 13:53, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>
> At this point I think the question becomes: do we let the (clear) fact that there is not a single set of priorities for where SciPy should be headed (which I do not see as a bad thing at this stage) get in the way of the community moving on *some* proposal (e.g., Joe's, with mods) for *some* "not-for-profit entity" (e.g., a "SciPy Foundation," the original topic of this thread) that will function as an institutional resource for furthering whichever priorities for SciPy should bubble to the surface? ?In other words, this thread is diverging (into territory necessary to discuss, yes), but can we at least agree (a semi-rhetorical question because I think the answer is clearly "yes") that something along the lines of a "SciPy Foundation" would be useful, certainly for helping us move SciPy where we want it to go, but perhaps also for helping us decide where as well?

Perhaps a new name would be in order. I think a lot of the
disagreement in vision arises from the fact that a number of the very
good ideas about how to encourage the use of Python in the sciences,
which could be implemented by the people involved in
SciPy-the-project, are being conflated with scipy-the-package. Things
like IDEs and GUIs and applications do not fit into scipy-the-package
as it currently exists, and changing scipy-the-package such that they
do fit in deteriorates what scipy-the-package is good at now.

Personally, I see scipy-the-package as something very close in spirit
to what GSL is to C: a library of quality numerical algorithms useful
to science and engineering. scipy-the-package is not everything that
is required to advance Python's use in the sciences. It can't be. A
single Python package is the wrong technology for delivering all of
that functionality.

I think we need to step back and question the question itself. Perhaps
we should not be asking "where should scipy(-the-package) be heading?"
but "what do we need to do advance Python's use in the sciences?" I
don't think a Foundation helps the former much, but I do think the
latter would be an excellent mission for one. scipy-the-package is a
component of what the Foundation might work one, but I think it would
make a huge mistake if it fixated on scipy-the-package and assumed
that all of the work it does needs to be jammed into
scipy-the-package.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From gael.varoquaux at normalesup.org  Tue Aug  4 15:41:00 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 4 Aug 2009 21:41:00 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>
Message-ID: <20090804194100.GD11772@phare.normalesup.org>

I fully agree with your analysis Robert. 

I had this discussion with Eric, and he did mention that it would be
useful if the name was reminiscent of 'SciPy', because it is a higly
visible name.

Should we have a BOF on that at the SciPy conference? Mailing list
discussions tend to go in a circle.

On Tue, Aug 04, 2009 at 02:37:01PM -0500, Robert Kern wrote:
> Perhaps a new name would be in order. I think a lot of the
> disagreement in vision arises from the fact that a number of the very
> good ideas about how to encourage the use of Python in the sciences,
> which could be implemented by the people involved in
> SciPy-the-project, are being conflated with scipy-the-package. Things
> like IDEs and GUIs and applications do not fit into scipy-the-package
> as it currently exists, and changing scipy-the-package such that they
> do fit in deteriorates what scipy-the-package is good at now.

> Personally, I see scipy-the-package as something very close in spirit
> to what GSL is to C: a library of quality numerical algorithms useful
> to science and engineering. scipy-the-package is not everything that
> is required to advance Python's use in the sciences. It can't be. A
> single Python package is the wrong technology for delivering all of
> that functionality.

> I think we need to step back and question the question itself. Perhaps
> we should not be asking "where should scipy(-the-package) be heading?"
> but "what do we need to do advance Python's use in the sciences?" I
> don't think a Foundation helps the former much, but I do think the
> latter would be an excellent mission for one. scipy-the-package is a
> component of what the Foundation might work one, but I think it would
> make a huge mistake if it fixated on scipy-the-package and assumed
> that all of the work it does needs to be jammed into
> scipy-the-package.



From robert.kern at gmail.com  Tue Aug  4 15:45:32 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 4 Aug 2009 14:45:32 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <20090804194100.GD11772@phare.normalesup.org>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com> 
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com> 
	<20090804194100.GD11772@phare.normalesup.org>
Message-ID: <3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>

On Tue, Aug 4, 2009 at 14:41, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> I fully agree with your analysis Robert.
>
> I had this discussion with Eric, and he did mention that it would be
> useful if the name was reminiscent of 'SciPy', because it is a higly
> visible name.
>
> Should we have a BOF on that at the SciPy conference? Mailing list
> discussions tend to go in a circle.

We could get a bikeshed, some paint, and some brushes. Everyone who
wants to contribute an idea must paint it on the bikeshed.

I like it.

Anyways, it could probably even be called the SciPy Foundation as long
as the introductory material was very explicit about its relationship
to scipy-the-package and the founding members use language carefully.
Tricky, but doable.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Tue Aug  4 15:48:10 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 4 Aug 2009 14:48:10 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com> 
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com> 
	<20090804194100.GD11772@phare.normalesup.org>
	<3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>
Message-ID: <3d375d730908041248y1008c6a0y9bf904abe3c9cd8e@mail.gmail.com>

On Tue, Aug 4, 2009 at 14:45, Robert Kern<robert.kern at gmail.com> wrote:
> We could get a bikeshed, some paint, and some brushes. Everyone who
> wants to contribute an idea must paint it on the bikeshed.

In their preferred color, of course.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ondrej at certik.cz  Tue Aug  4 17:28:52 2009
From: ondrej at certik.cz (Ondrej Certik)
Date: Tue, 4 Aug 2009 15:28:52 -0600
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <3d375d730908041248y1008c6a0y9bf904abe3c9cd8e@mail.gmail.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>
	<20090804194100.GD11772@phare.normalesup.org>
	<3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>
	<3d375d730908041248y1008c6a0y9bf904abe3c9cd8e@mail.gmail.com>
Message-ID: <85b5c3130908041428k428cd249y97313b768b32ff94@mail.gmail.com>

On Tue, Aug 4, 2009 at 1:48 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Tue, Aug 4, 2009 at 14:45, Robert Kern<robert.kern at gmail.com> wrote:
>> We could get a bikeshed, some paint, and some brushes. Everyone who
>> wants to contribute an idea must paint it on the bikeshed.
>
> In their preferred color, of course.

Maybe everyone should bring a bike too.

Ondrej


From charlesr.harris at gmail.com  Tue Aug  4 17:47:21 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 4 Aug 2009 15:47:21 -0600
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <85b5c3130908041428k428cd249y97313b768b32ff94@mail.gmail.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>
	<20090804194100.GD11772@phare.normalesup.org>
	<3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>
	<3d375d730908041248y1008c6a0y9bf904abe3c9cd8e@mail.gmail.com>
	<85b5c3130908041428k428cd249y97313b768b32ff94@mail.gmail.com>
Message-ID: <e06186140908041447x2948cc7fu80efb2318d92019c@mail.gmail.com>

On Tue, Aug 4, 2009 at 3:28 PM, Ondrej Certik <ondrej at certik.cz> wrote:

> On Tue, Aug 4, 2009 at 1:48 PM, Robert Kern<robert.kern at gmail.com> wrote:
> > On Tue, Aug 4, 2009 at 14:45, Robert Kern<robert.kern at gmail.com> wrote:
> >> We could get a bikeshed, some paint, and some brushes. Everyone who
> >> wants to contribute an idea must paint it on the bikeshed.
> >
> > In their preferred color, of course.
>
> Maybe everyone should bring a bike too.
>

It would be nice if the hotels would offer rental bikes. As is, the bike
stores are far enough away that getting the bike and dropping it off on
departure, is too much hassle.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090804/c85b934e/attachment.html>

From gael.varoquaux at normalesup.org  Tue Aug  4 17:49:41 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Tue, 4 Aug 2009 23:49:41 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <e06186140908041447x2948cc7fu80efb2318d92019c@mail.gmail.com>
References: <ec9f80fa0908040225ue616d45p3a558e9c9097b9eb@mail.gmail.com>
	<291714.68720.qm@web52104.mail.re2.yahoo.com>
	<3d375d730908041237g35c415f8vf1506087f12197ba@mail.gmail.com>
	<20090804194100.GD11772@phare.normalesup.org>
	<3d375d730908041245w39ef56f2y5d394098b5e5932e@mail.gmail.com>
	<3d375d730908041248y1008c6a0y9bf904abe3c9cd8e@mail.gmail.com>
	<85b5c3130908041428k428cd249y97313b768b32ff94@mail.gmail.com>
	<e06186140908041447x2948cc7fu80efb2318d92019c@mail.gmail.com>
Message-ID: <20090804214941.GA23662@phare.normalesup.org>

On Tue, Aug 04, 2009 at 03:47:21PM -0600, Charles R Harris wrote:
>    On Tue, Aug 4, 2009 at 3:28 PM, Ondrej Certik <[1]ondrej at certik.cz> wrote:

>      On Tue, Aug 4, 2009 at 1:48 PM, Robert Kern<[2]robert.kern at gmail.com>
>      wrote:
>      > On Tue, Aug 4, 2009 at 14:45, Robert Kern<[3]robert.kern at gmail.com>
>      wrote:
>      >> We could get a bikeshed, some paint, and some brushes. Everyone who
>      >> wants to contribute an idea must paint it on the bikeshed.

>      > In their preferred color, of course.

>      Maybe everyone should bring a bike too.

>    It would be nice if the hotels would offer rental bikes. As is, the bike
>    stores are far enough away that getting the bike and dropping it off on
>    departure, is too much hassle.

The good news is that you'll find a bikeshed to shelter your bike when
you get there.

Actually, I suggest some people bring bikesheds too. As I am unsure of
my favorite color.

Ga?l


From d_l_goldsmith at yahoo.com  Wed Aug  5 00:52:24 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 4 Aug 2009 21:52:24 -0700 (PDT)
Subject: [SciPy-dev] Trying to finish off the docstring for i0
Message-ID: <645082.77656.qm@web52109.mail.re2.yahoo.com>


Hi!  Because i0 (modified Bessel func., first kind, order zero) is not your typical "run-of-the-mill" transcendental function, I was hoping to include a little info in its Notes section about the algorithm we use, convergence, etc.  I found the code in function_base.py; long story short, it led me to:

http://kobesearch.cpan.org/htdocs/Math-Cephe/Math/Cephes.html#i_i0_i_Modified_Bessel_function_of_o

which looks like what we're using, correct?

DG


      


From stefan at sun.ac.za  Wed Aug  5 01:11:32 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 5 Aug 2009 00:11:32 -0500
Subject: [SciPy-dev] Anyone up for a Skypecon?
In-Reply-To: <907186.73991.qm@web52103.mail.re2.yahoo.com>
References: <AcoVJ3licRRxJZ1kS7Cf3t6SDoKthA==>
	<907186.73991.qm@web52103.mail.re2.yahoo.com>
Message-ID: <9457e7c80908042211n634c8063w60d45c0eb2c06e51@mail.gmail.com>

Hi David

Apologies from my side, but I am currently traveling in your beautiful
country.  Looking forward to seeing all of you at SciPy'09!

Happy documenting! (1000 words for a high quality cotton T-shirt (TM)
-- bargain!).

Cheers
St?fan

2009/8/4 David Goldsmith <d_l_goldsmith at yahoo.com>:
>
> Well, past two weeks, Summer Marathon Skypecon participation has been nil. ?Jack Liddle just Skyped me asking me about this week - which I presume means he's interested and available, in principle at least - and I said "regular time, tomorrow (Wed.) 19:00 UTC," but if it would increase participation to hold it at a different time, just let me know...
>
> DG
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Wed Aug  5 13:33:48 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 10:33:48 -0700 (PDT)
Subject: [SciPy-dev] Two Marathon questions
Message-ID: <940468.47590.qm@web52104.mail.re2.yahoo.com>

0) Are there any Category Leaders who do *not* want help finishing their categories?

1) Is there anyone "in-the-know" who feels that reading "numpy-docs/reference/routines.ma.rst" (as it is now) is *insufficient* preparation for assisting w/ the Masked Array docstrings; or, put another way, feels that if one is not at least "well-practiced" using masked arrays, then one should not touch their docstrings?

But for the masked array categories and the uncategorized docstrings, we're pretty close!

DG


      


From d_l_goldsmith at yahoo.com  Wed Aug  5 14:03:43 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 11:03:43 -0700 (PDT)
Subject: [SciPy-dev] Hour 'til Skypecon.
Message-ID: <560636.82353.qm@web52105.mail.re2.yahoo.com>




      


From emmanuelle.gouillart at normalesup.org  Wed Aug  5 14:43:42 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Wed, 5 Aug 2009 20:43:42 +0200
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code in docstrings
Message-ID: <20090805184342.GA7664@phare.normalesup.org>

	Hello list,

	disclaimer: I don't have much hindsight about what I'm talking
about in the following, so I apologize if it doesn't make much sense...

	My question is: is there some way (apart from copy-and-paste :D)
to execute some of the code inside docstrings, in order e.g. to generate
pylab plots? This may be a useful feature for the documentation of scipy:
indeed, a plot may speak for itself better than long explanations about
the output of scipy's algorithms. Some docstrings already include calls
to plotting commands (one example:
http://docs.scipy.org/scipy/docs/scipy.stats.distributions.poisson/),
but of course, the plots are not created while viewing the help. 

	Maybe it's a stupid idea, but I'm thinking about a %demo magic
function in Ipython that would print the docstring of an object and
execute the code of the docstring (preferably in a separate namespace)
and, in particular, display pylab's windows. 

	Does such a feature already exist somewhere? If not, do you see
any interest in coding it? Matplotlib and Mayavi2 call special demo
functions and it would be possible to do the same for scipy, but on the
other side, using directly the docstrings for demos might be just as
well... 

	Regards,

	Emmanuelle

	 



From d_l_goldsmith at yahoo.com  Wed Aug  5 14:48:42 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 11:48:42 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
Message-ID: <819183.15509.qm@web52106.mail.re2.yahoo.com>

I guess I don't really understand this too well - is the below correct behavior, and if so, why?

>>> x = np.array([1, 2, 3])
>>> y = np.array([[4], [5], [6]])
>>> b = np.broadcast(x, y)
>>> b.nd # returns what I'd expect
2
>>> b = np.broadcast(x, y, x, y)
>>> b.nd # doesn't return what I'd expect
2
>>> del b # maybe problem is that I have to "clear" b first?
>>> # or maybe it's that all args have to be different?
...
>>> b = np.broadcast(x, y, x * y)
>>> b.nd
2
>>> z = x * y # grasping at straws now
>>> z
array([[ 4,  8, 12],
       [ 5, 10, 15],
       [ 6, 12, 18]])
>>> x
array([1, 2, 3])
>>> y
array([[4],
       [5],
       [6]])
>>> del b # trying everything to get intuitive behavior
>>> b = np.broadcast(x, y, z)
>>> b.nd
2

Huhn?

DG


      


From robert.kern at gmail.com  Wed Aug  5 14:51:25 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 13:51:25 -0500
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <819183.15509.qm@web52106.mail.re2.yahoo.com>
References: <819183.15509.qm@web52106.mail.re2.yahoo.com>
Message-ID: <3d375d730908051151g7b8a0d95t23f2a16309e31986@mail.gmail.com>

On Wed, Aug 5, 2009 at 13:48, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> I guess I don't really understand this too well - is the below correct behavior, and if so, why?
>
>>>> x = np.array([1, 2, 3])
>>>> y = np.array([[4], [5], [6]])
>>>> b = np.broadcast(x, y)
>>>> b.nd # returns what I'd expect
> 2
>>>> b = np.broadcast(x, y, x, y)
>>>> b.nd # doesn't return what I'd expect
> 2

Why don't you expect this? It's the correct answer. (x*y*x*y).shape == (3,3).

>>>> del b # maybe problem is that I have to "clear" b first?
>>>> # or maybe it's that all args have to be different?
> ...
>>>> b = np.broadcast(x, y, x * y)
>>>> b.nd
> 2
>>>> z = x * y # grasping at straws now
>>>> z
> array([[ 4, ?8, 12],
> ? ? ? [ 5, 10, 15],
> ? ? ? [ 6, 12, 18]])

Yup. z.shape == (3,3). Broadcasting that with a (3,1) array or a (3,)
array still gives a (3,3) array.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From pgmdevlist at gmail.com  Wed Aug  5 14:59:49 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Wed, 5 Aug 2009 14:59:49 -0400
Subject: [SciPy-dev] Two Marathon questions
In-Reply-To: <940468.47590.qm@web52104.mail.re2.yahoo.com>
References: <940468.47590.qm@web52104.mail.re2.yahoo.com>
Message-ID: <71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>


On Aug 5, 2009, at 1:33 PM, David Goldsmith wrote:

> 0) Are there any Category Leaders who do *not* want help finishing  
> their categories?
>
> 1) Is there anyone "in-the-know" who feels that reading "numpy-docs/ 
> reference/routines.ma.rst" (as it is now) is *insufficient*  
> preparation for assisting w/ the Masked Array docstrings; or, put  
> another way, feels that if one is not at least "well-practiced"  
> using masked arrays, then one should not touch their docstrings?

Mmh. Can't really tell. Experience with masked arrays is certainly a  
prerequisite, but one shouldn't need to be an expert.
Nevertheless, I'd be obliged if anybody willing to edit the MA  
docstrings could contact me beforehand w/ questions.



From d_l_goldsmith at yahoo.com  Wed Aug  5 15:19:53 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 12:19:53 -0700 (PDT)
Subject: [SciPy-dev] Skypecon aborted due to host's technical difficulties
Message-ID: <658720.89931.qm@web52112.mail.re2.yahoo.com>

Oooops: main computer stolen last week; thought replacement (my previous computer, borrowed back from my son) had built in mike, but doesn't appear to; only Ralf called in anyway, so we've opted to reschedule (or just cancel, picking-up again next week.)  If we have it at a different time this week, can more people "make it"?  If so, when's good for people?

DG


      


From ralf.gommers at googlemail.com  Wed Aug  5 15:28:38 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Wed, 5 Aug 2009 15:28:38 -0400
Subject: [SciPy-dev] Two Marathon questions
In-Reply-To: <71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
References: <940468.47590.qm@web52104.mail.re2.yahoo.com>
	<71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
Message-ID: <dde7764a0908051228t3f2a536bkcf98d5b8871fad59@mail.gmail.com>

On Wed, Aug 5, 2009 at 2:59 PM, Pierre GM <pgmdevlist at gmail.com> wrote:

>
> On Aug 5, 2009, at 1:33 PM, David Goldsmith wrote:
>
> > 0) Are there any Category Leaders who do *not* want help finishing
> > their categories?
> >
> > 1) Is there anyone "in-the-know" who feels that reading "numpy-docs/
> > reference/routines.ma.rst" (as it is now) is *insufficient*
> > preparation for assisting w/ the Masked Array docstrings; or, put
> > another way, feels that if one is not at least "well-practiced"
> > using masked arrays, then one should not touch their docstrings?
>
> Mmh. Can't really tell. Experience with masked arrays is certainly a
> prerequisite, but one shouldn't need to be an expert.
> Nevertheless, I'd be obliged if anybody willing to edit the MA
> docstrings could contact me beforehand w/ questions.
>

Pierre, I did a lot of the MA docstrings over the past few days. I would
appreciate it if you could look at some of those (see
http://docs.scipy.org/numpy/changes/ for recent changes) and let me know if
there's anything that you think should be done differently.

I also moved a lot of the private functions/classes to Unimportant status,
but I did fix the markup for a lot of them and added some Parameters
sections etc where I thought it was helpful. Again, please let me know if
you have any recommendations.

Cheers,
Ralf


> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090805/396a539d/attachment.html>

From d_l_goldsmith at yahoo.com  Wed Aug  5 15:38:19 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 12:38:19 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <3d375d730908051151g7b8a0d95t23f2a16309e31986@mail.gmail.com>
Message-ID: <120029.98483.qm@web52107.mail.re2.yahoo.com>

--- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:

> wrote:
> > I guess I don't really understand this too well - is
> the below correct behavior, and if so, why?
> >
> >>>> b = np.broadcast(x, y, x, y)
> >>>> b.nd # doesn't return what I'd expect
> > 2

Why isn't that 4?

> Why don't you expect this? It's the correct answer.
> (x*y*x*y).shape == (3,3).
> 
> >>>> del b # maybe problem is that I have to
> "clear" b first?
> >>>> # or maybe it's that all args have to be
> different?
> > ...
> >>>> b = np.broadcast(x, y, x * y)
> >>>> b.nd
> > 2

Why isn't that 3?

If x0, ..., xN are the arguments to `broadcast` and D = max(x0.nd, ..., xN.nd), is broadcast.nd necessarily <= D?  If so, then I think I'm on the road to understanding.

DG


      


From robert.kern at gmail.com  Wed Aug  5 15:41:51 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 14:41:51 -0500
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <120029.98483.qm@web52107.mail.re2.yahoo.com>
References: <3d375d730908051151g7b8a0d95t23f2a16309e31986@mail.gmail.com> 
	<120029.98483.qm@web52107.mail.re2.yahoo.com>
Message-ID: <3d375d730908051241g122ed99bn5c0b86c74690adfb@mail.gmail.com>

On Wed, Aug 5, 2009 at 14:38, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> --- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:
>
>> wrote:
>> > I guess I don't really understand this too well - is
>> the below correct behavior, and if so, why?
>> >
>> >>>> b = np.broadcast(x, y, x, y)
>> >>>> b.nd # doesn't return what I'd expect
>> > 2
>
> Why isn't that 4?

Why would it be 4?

>> Why don't you expect this? It's the correct answer.
>> (x*y*x*y).shape == (3,3).

This is the example you need to pay attention to.

>> >>>> del b # maybe problem is that I have to
>> "clear" b first?
>> >>>> # or maybe it's that all args have to be
>> different?
>> > ...
>> >>>> b = np.broadcast(x, y, x * y)
>> >>>> b.nd
>> > 2
>
> Why isn't that 3?
>
> If x0, ..., xN are the arguments to `broadcast` and D = max(x0.nd, ..., xN.nd), is broadcast.nd necessarily <= D? ?If so, then I think I'm on the road to understanding.

It is necessarily == D. Broadcasting is associative. The (x*y*z).shape
== (x*(y*z)).shape == ((x*y)*z).shape.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Wed Aug  5 15:46:27 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 12:46:27 -0700 (PDT)
Subject: [SciPy-dev] Two Marathon questions
In-Reply-To: <71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
Message-ID: <829204.30757.qm@web52105.mail.re2.yahoo.com>

Thanks for your reply, Pierre.

--- On Wed, 8/5/09, Pierre GM <pgmdevlist at gmail.com> wrote:

> On Aug 5, 2009, at 1:33 PM, David Goldsmith wrote:
> 
> > 1) Is there anyone "in-the-know" who feels that
> reading "numpy-docs/ 
> > reference/routines.ma.rst" (as it is now) is
> *insufficient*? 
> > preparation for assisting w/ the Masked Array
> docstrings; or, put? 
> > another way, feels that if one is not at least
> "well-practiced"? 
> > using masked arrays, then one should not touch their
> docstrings?
> 
> Mmh. Can't really tell. Experience with masked arrays is
> certainly a? 
> prerequisite, but one shouldn't need to be an expert.
> Nevertheless, I'd be obliged if anybody willing to edit the
> MA? 
> docstrings could contact me beforehand w/ questions.

Well, for example, I have experience *masking* arrays in IDL, but not in Numpy, and not using an object-oriented model.  Nevertheless, I'd be happy to ask you q's...

I'll finish the uncategorized docstrings I feel comfortable with (e.g., *not* distutils), and then help w/ any remaining MA's.

Others, of course, do as thou willst, modulo Pierre's request.

Thanks again!

DG 

> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Wed Aug  5 16:01:23 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 13:01:23 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <3d375d730908051241g122ed99bn5c0b86c74690adfb@mail.gmail.com>
Message-ID: <391244.80171.qm@web52111.mail.re2.yahoo.com>

--- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:

> It is necessarily == D. Broadcasting is associative. 

Ah, that's the key I didn't understand!  That helps (me) a lot; I'm going to make a "Note" of it in numpy.broadcast's docstring.

DG

> The
> (x*y*z).shape
> == (x*(y*z)).shape == ((x*y)*z).shape.
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From pav+sp at iki.fi  Wed Aug  5 16:32:13 2009
From: pav+sp at iki.fi (Pauli Virtanen)
Date: Wed, 5 Aug 2009 20:32:13 +0000 (UTC)
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code in docstrings
References: <20090805184342.GA7664@phare.normalesup.org>
Message-ID: <h5cq8d$au2$1@ger.gmane.org>

On 2009-08-05, Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org> wrote:
> 	Hello list,
>
> 	disclaimer: I don't have much hindsight about what I'm talking
> about in the following, so I apologize if it doesn't make much sense...
>
> 	My question is: is there some way (apart from copy-and-paste :D)
> to execute some of the code inside docstrings, in order e.g. to generate
> pylab plots? This may be a useful feature for the documentation of scipy:
> indeed, a plot may speak for itself better than long explanations about
> the output of scipy's algorithms. Some docstrings already include calls
> to plotting commands (one example:
> http://docs.scipy.org/scipy/docs/scipy.stats.distributions.poisson/),
> but of course, the plots are not created while viewing the help. 

The plots do appear in the final documentation, cf. 
http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.gamma.html

(The scipy.stats.distributions examples are not actually 
executable Python code, being more pseudo-codeish. This could and 
probably should be fixed, though.)

It's not really feasible to have them appear in the doc editor -- 
there's no reliable & easy way to sandbox Python code, and I'm 
not comfortable with having a way to run potentially untrusted 
code on the servers.

One thing that I'm not very happy with the Sphinx output is that 
copy & paste of the examples is quite difficult, since you get 
the >>> and ... prompts. This could be avoided with suitable HTML 
magick.

> 	Maybe it's a stupid idea, but I'm thinking about a %demo magic
> function in Ipython that would print the docstring of an object and
> execute the code of the docstring (preferably in a separate namespace)
> and, in particular, display pylab's windows. 
>
> 	Does such a feature already exist somewhere? If not, do you see
> any interest in coding it? Matplotlib and Mayavi2 call special demo
> functions and it would be possible to do the same for scipy, but on the
> other side, using directly the docstrings for demos might be just as
> well... 

I think such a demo function could be easy to implement: just 
pick up the doctest lines and run them. I think a IPython 
extension could easily be written for this: just check what's in 
the ipy_*.py files under IPython/Extensions and adapt one of 
them.

There's a ready-made implementation of the doctest pickup in 
plot_directive.py under numpy/doc/sphinxext.

-- 
Pauli Virtanen



From d_l_goldsmith at yahoo.com  Wed Aug  5 16:40:01 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 13:40:01 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <391244.80171.qm@web52111.mail.re2.yahoo.com>
Message-ID: <49739.57430.qm@web52105.mail.re2.yahoo.com>

--- On Wed, 8/5/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:

> > It is necessarily == D. Broadcasting is associative. 
> > The
> > (x*y*z).shape
> > == (x*(y*z)).shape == ((x*y)*z).shape.

Um:

>>> x = np.array((1, 2, 3))
>>> y = np.array([[4], [5], [6]])
>>> z = x * y
>>> x; y; z
array([1, 2, 3])
array([[4],
       [5],
       [6]])
array([[ 4,  8, 12],
       [ 5, 10, 15],
       [ 6, 12, 18]])
>>> B = np.broadcast
>>> X = B(x, y, z)
>>> Y = B(x, B(y, z))
>>> Z = B(B(x, y), z)
>>> X.numiter, Y.numiter, Z.numiter
(3, 2, 2)
>>> X.nd, Y.nd, Z.nd
(2, 1, 2)
>>> X.shape, Y.shape, Z.shape
((3, 3), (3,), (3, 3))

Am I doing something wrong?

DG


      


From emmanuelle.gouillart at normalesup.org  Wed Aug  5 16:49:41 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Wed, 5 Aug 2009 22:49:41 +0200
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code	in docstrings
In-Reply-To: <h5cq8d$au2$1@ger.gmane.org>
References: <20090805184342.GA7664@phare.normalesup.org>
	<h5cq8d$au2$1@ger.gmane.org>
Message-ID: <20090805204941.GA25169@phare.normalesup.org>

Hi Pauli,

thanks for your answer!

> The plots do appear in the final documentation, cf. 
> http://docs.scipy.org/doc/numpy/reference/generated/numpy.random.gamma.html

True! I hadn't noticed as I usually use only Ipython's help, but the html
pages look really nice with the plots.

> It's not really feasible to have them appear in the doc editor -- 
> there's no reliable & easy way to sandbox Python code, and I'm 
> not comfortable with having a way to run potentially untrusted 
> code on the servers.

Sure, I wasn't talking about having them appear in the doc editor. We can
afford doing some copy & paste while using the doc editor, I think...

> One thing that I'm not very happy with the Sphinx output is that 
> copy & paste of the examples is quite difficult, since you get 
> the >>> and ... prompts. This could be avoided with suitable HTML 
> magick.

Maybe the wonderful %doctest_mode magic command of Ipython should be
advertised somewhere... I use it all the time since I've discovered the
feature, it's awfully convenient :D.

> I think such a demo function could be easy to implement: just 
> pick up the doctest lines and run them. I think a IPython 
> extension could easily be written for this: just check what's in 
> the ipy_*.py files under IPython/Extensions and adapt one of 
> them.

> There's a ready-made implementation of the doctest pickup in 
> plot_directive.py under numpy/doc/sphinxext.

That's exactly the kind of hints I was looking for, many thanks! I'll
have a look at the files you mention to see how it could be done.

	Cheers,

	Emmanuelle


From robert.kern at gmail.com  Wed Aug  5 17:01:32 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 16:01:32 -0500
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <49739.57430.qm@web52105.mail.re2.yahoo.com>
References: <391244.80171.qm@web52111.mail.re2.yahoo.com>
	<49739.57430.qm@web52105.mail.re2.yahoo.com>
Message-ID: <3d375d730908051401t28ecc411jabc2ba93edd91887@mail.gmail.com>

On Wed, Aug 5, 2009 at 15:40, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> --- On Wed, 8/5/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:
>
>> > It is necessarily == D. Broadcasting is associative.
>> > The
>> > (x*y*z).shape
>> > == (x*(y*z)).shape == ((x*y)*z).shape.
>
> Um:
>
>>>> x = np.array((1, 2, 3))
>>>> y = np.array([[4], [5], [6]])
>>>> z = x * y
>>>> x; y; z
> array([1, 2, 3])
> array([[4],
> ? ? ? [5],
> ? ? ? [6]])
> array([[ 4, ?8, 12],
> ? ? ? [ 5, 10, 15],
> ? ? ? [ 6, 12, 18]])
>>>> B = np.broadcast
>>>> X = B(x, y, z)
>>>> Y = B(x, B(y, z))
>>>> Z = B(B(x, y), z)
>>>> X.numiter, Y.numiter, Z.numiter
> (3, 2, 2)
>>>> X.nd, Y.nd, Z.nd
> (2, 1, 2)
>>>> X.shape, Y.shape, Z.shape
> ((3, 3), (3,), (3, 3))
>
> Am I doing something wrong?

You are passing a broadcast iterator as an argument to broadcast. The
broadcast iterator iterates over the inputs in-parallel. This is
entirely different from actually operating on the inputs.

In [1]: x = array([1, 2, 3])

In [2]: y = array([[4], [5], [6]])

In [3]: z = x * y

In [4]: x*y*z
Out[4]:
array([[ 16,  64, 144],
       [ 25, 100, 225],
       [ 36, 144, 324]])

In [5]: (x*y)*z
Out[5]:
array([[ 16,  64, 144],
       [ 25, 100, 225],
       [ 36, 144, 324]])

In [6]: x*(y*z)
Out[6]:
array([[ 16,  64, 144],
       [ 25, 100, 225],
       [ 36, 144, 324]])

In [8]: array(list(broadcast(x,y)))
Out[8]:
array([[1, 4],
       [2, 4],
       [3, 4],
       [1, 5],
       [2, 5],
       [3, 5],
       [1, 6],
       [2, 6],
       [3, 6]])

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Wed Aug  5 18:15:43 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 15:15:43 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <3d375d730908051401t28ecc411jabc2ba93edd91887@mail.gmail.com>
Message-ID: <668591.72009.qm@web52106.mail.re2.yahoo.com>

Sorry to be "dense", but can someone please show me how to exhibit associativity with the broadcast object - it's that (not broadcasting as a process) I'm trying to understand (and illustrate).

DG

--- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] numpy.broadcast
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Wednesday, August 5, 2009, 2:01 PM
> On Wed, Aug 5, 2009 at 15:40, David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > --- On Wed, 8/5/09, David Goldsmith <d_l_goldsmith at yahoo.com>
> wrote:
> >
> >> > It is necessarily == D. Broadcasting is
> associative.
> >> > The
> >> > (x*y*z).shape
> >> > == (x*(y*z)).shape == ((x*y)*z).shape.
> >
> > Um:
> >
> >>>> x = np.array((1, 2, 3))
> >>>> y = np.array([[4], [5], [6]])
> >>>> z = x * y
> >>>> x; y; z
> > array([1, 2, 3])
> > array([[4],
> > ? ? ? [5],
> > ? ? ? [6]])
> > array([[ 4, ?8, 12],
> > ? ? ? [ 5, 10, 15],
> > ? ? ? [ 6, 12, 18]])
> >>>> B = np.broadcast
> >>>> X = B(x, y, z)
> >>>> Y = B(x, B(y, z))
> >>>> Z = B(B(x, y), z)
> >>>> X.numiter, Y.numiter, Z.numiter
> > (3, 2, 2)
> >>>> X.nd, Y.nd, Z.nd
> > (2, 1, 2)
> >>>> X.shape, Y.shape, Z.shape
> > ((3, 3), (3,), (3, 3))
> >
> > Am I doing something wrong?
> 
> You are passing a broadcast iterator as an argument to
> broadcast. The
> broadcast iterator iterates over the inputs in-parallel.
> This is
> entirely different from actually operating on the inputs.
> 
> In [1]: x = array([1, 2, 3])
> 
> In [2]: y = array([[4], [5], [6]])
> 
> In [3]: z = x * y
> 
> In [4]: x*y*z
> Out[4]:
> array([[ 16,? 64, 144],
> ? ? ???[ 25, 100, 225],
> ? ? ???[ 36, 144, 324]])
> 
> In [5]: (x*y)*z
> Out[5]:
> array([[ 16,? 64, 144],
> ? ? ???[ 25, 100, 225],
> ? ? ???[ 36, 144, 324]])
> 
> In [6]: x*(y*z)
> Out[6]:
> array([[ 16,? 64, 144],
> ? ? ???[ 25, 100, 225],
> ? ? ???[ 36, 144, 324]])
> 
> In [8]: array(list(broadcast(x,y)))
> Out[8]:
> array([[1, 4],
> ? ? ???[2, 4],
> ? ? ???[3, 4],
> ? ? ???[1, 5],
> ? ? ???[2, 5],
> ? ? ???[3, 5],
> ? ? ???[1, 6],
> ? ? ???[2, 6],
> ? ? ???[3, 6]])
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From robert.kern at gmail.com  Wed Aug  5 18:17:23 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 17:17:23 -0500
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <668591.72009.qm@web52106.mail.re2.yahoo.com>
References: <3d375d730908051401t28ecc411jabc2ba93edd91887@mail.gmail.com> 
	<668591.72009.qm@web52106.mail.re2.yahoo.com>
Message-ID: <3d375d730908051517k3b746ea7ra81a775690fa4528@mail.gmail.com>

On Wed, Aug 5, 2009 at 17:15, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> Sorry to be "dense", but can someone please show me how to exhibit associativity with the broadcast object - it's that (not broadcasting as a process) I'm trying to understand (and illustrate).

The broadcast *object* doesn't associate. *Broadcasting* associates.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Wed Aug  5 18:23:51 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 5 Aug 2009 15:23:51 -0700 (PDT)
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <3d375d730908051517k3b746ea7ra81a775690fa4528@mail.gmail.com>
Message-ID: <879886.78095.qm@web52110.mail.re2.yahoo.com>

--- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:

> The broadcast *object* doesn't associate. *Broadcasting*
> associates.

I understand that; I guess what I don't understand is the relationship between the object and the process.

DG

> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From fperez.net at gmail.com  Wed Aug  5 18:38:05 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 5 Aug 2009 15:38:05 -0700
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code in 	docstrings
In-Reply-To: <20090805204941.GA25169@phare.normalesup.org>
References: <20090805184342.GA7664@phare.normalesup.org>
	<h5cq8d$au2$1@ger.gmane.org> 
	<20090805204941.GA25169@phare.normalesup.org>
Message-ID: <db6b5ecc0908051538p183bf5edu786fdaeeeaa5817e@mail.gmail.com>

On Wed, Aug 5, 2009 at 1:49 PM, Emmanuelle
Gouillart<emmanuelle.gouillart at normalesup.org> wrote:
>> One thing that I'm not very happy with the Sphinx output is that
>> copy & paste of the examples is quite difficult, since you get
>> the >>> and ... prompts. This could be avoided with suitable HTML
>> magick.
>
> Maybe the wonderful %doctest_mode magic command of Ipython should be
> advertised somewhere... I use it all the time since I've discovered the
> feature, it's awfully convenient :D.

Thanks for answering for me :)

Regarding the advertising, I'll be the first to admit we could do
better.  But in our defense, it's at least documented:

http://ipython.scipy.org/doc/stable/html/interactive/tutorial.html?highlight=doctest_mode

http://ipython.scipy.org/doc/stable/html/overview.html?highlight=doctest_mode

http://ipython.scipy.org/doc/stable/html/development/overview.html?highlight=doctest_mode

http://ipython.scipy.org/doc/stable/html/api/generated/IPython.Magic.html?highlight=doctest_mode#IPython.Magic.Magic.magic_doctest_mode

Maybe ipython should open a twitter account with a tweet-tip
(tweetip?) of the day everyday :)

>> I think such a demo function could be easy to implement: just
>> pick up the doctest lines and run them. I think a IPython
>> extension could easily be written for this: just check what's in
>> the ipy_*.py files under IPython/Extensions and adapt one of
>> them.
>
>> There's a ready-made implementation of the doctest pickup in
>> plot_directive.py under numpy/doc/sphinxext.
>
> That's exactly the kind of hints I was looking for, many thanks! I'll
> have a look at the files you mention to see how it could be done.

And when you finish, send it our way.  Operators are standing by to
take your patch and your credit card number... :)

Take care,

f


From robert.kern at gmail.com  Wed Aug  5 18:40:07 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 17:40:07 -0500
Subject: [SciPy-dev] numpy.broadcast
In-Reply-To: <879886.78095.qm@web52110.mail.re2.yahoo.com>
References: <3d375d730908051517k3b746ea7ra81a775690fa4528@mail.gmail.com> 
	<879886.78095.qm@web52110.mail.re2.yahoo.com>
Message-ID: <3d375d730908051540t5decee87w6f17eccc23a69ae0@mail.gmail.com>

On Wed, Aug 5, 2009 at 17:23, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> --- On Wed, 8/5/09, Robert Kern <robert.kern at gmail.com> wrote:
>
>> The broadcast *object* doesn't associate. *Broadcasting*
>> associates.
>
> I understand that; I guess what I don't understand is the relationship between the object and the process.

The broadcast object is an iterator. It has attributes .nd, .shape,
and .size which tell you the number of dimensions, the shape, and the
number of elements of what the broadcasted forms of each the input
arrays would be. For N input arrays, iterating over the broadcast
object would yield you .size N-tuples with the elements from each
input array just as if you were to form the broadcasted arrays for
each of the inputs and then iterate over zip(broadcasted_x.flat,
broadcasted_y.flat, ...).

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Wed Aug  5 20:24:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 5 Aug 2009 19:24:19 -0500
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code in 	docstrings
In-Reply-To: <db6b5ecc0908051538p183bf5edu786fdaeeeaa5817e@mail.gmail.com>
References: <20090805184342.GA7664@phare.normalesup.org>
	<h5cq8d$au2$1@ger.gmane.org> 
	<20090805204941.GA25169@phare.normalesup.org>
	<db6b5ecc0908051538p183bf5edu786fdaeeeaa5817e@mail.gmail.com>
Message-ID: <3d375d730908051724n34bc785dwb8dbce35ca0d46e2@mail.gmail.com>

On Wed, Aug 5, 2009 at 17:38, Fernando Perez<fperez.net at gmail.com> wrote:
> On Wed, Aug 5, 2009 at 1:49 PM, Emmanuelle
> Gouillart<emmanuelle.gouillart at normalesup.org> wrote:

>>> I think such a demo function could be easy to implement: just
>>> pick up the doctest lines and run them. I think a IPython
>>> extension could easily be written for this: just check what's in
>>> the ipy_*.py files under IPython/Extensions and adapt one of
>>> them.
>>
>>> There's a ready-made implementation of the doctest pickup in
>>> plot_directive.py under numpy/doc/sphinxext.
>>
>> That's exactly the kind of hints I was looking for, many thanks! I'll
>> have a look at the files you mention to see how it could be done.
>
> And when you finish, send it our way. ?Operators are standing by to
> take your patch and your credit card number... :)

Something like this?

In [1]: %run_examples np.broadcast
Produce an object that mimics broadcasting.

    Parameters
    ----------
    in1, in2, ... : array_like
        Input parameters.

    Returns
    -------
    b : broadcast object
        Broadcast the input parameters against one another, and
        return an object that encapsulates the result.
        Amongst others, it has ``shape`` and ``nd`` properties, and
        may be used as an iterator.

    Examples
    --------
    Manually adding two vectors, using broadcasting:


>>> x = np.array([[1], [2], [3]])
Press <q> to quit, <Enter> to execute...

>>> y = np.array([4, 5, 6])
Press <q> to quit, <Enter> to execute...

>>> b = np.broadcast(x, y)
Press <q> to quit, <Enter> to execute...


>>> out = np.empty(b.shape)
Press <q> to quit, <Enter> to execute...

>>> out.flat = [u+v for (u,v) in b]
Press <q> to quit, <Enter> to execute...

>>> out
array([[ 5.,  6.,  7.],
       [ 6.,  7.,  8.],
       [ 7.,  8.,  9.]])

Press <q> to quit, <Enter> to execute...
 output:
array([[ 5.,  6.,  7.],
       [ 6.,  7.,  8.],
       [ 7.,  8.,  9.]])

    Compare against built-in broadcasting:


>>> x + y
array([[5, 6, 7],
       [6, 7, 8],
       [7, 8, 9]])

Press <q> to quit, <Enter> to execute...
 output:
array([[5, 6, 7],
       [6, 7, 8],
       [7, 8, 9]])

END OF DEMO
Use <demo_name>.reset() if you want to rerun it.


It uses IPython.demo. It's not especially pretty, but it's servicable.

Code is at the end of this file:

http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/kernmagic/file/c18f492e9688/kernmagic/mymagics.py

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From fperez.net at gmail.com  Wed Aug  5 20:35:54 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Wed, 5 Aug 2009 17:35:54 -0700
Subject: [SciPy-dev] On scipy/numpy documentation,
	and executing code in 	docstrings
In-Reply-To: <3d375d730908051724n34bc785dwb8dbce35ca0d46e2@mail.gmail.com>
References: <20090805184342.GA7664@phare.normalesup.org>
	<h5cq8d$au2$1@ger.gmane.org> 
	<20090805204941.GA25169@phare.normalesup.org>
	<db6b5ecc0908051538p183bf5edu786fdaeeeaa5817e@mail.gmail.com> 
	<3d375d730908051724n34bc785dwb8dbce35ca0d46e2@mail.gmail.com>
Message-ID: <db6b5ecc0908051735m21a9977ak2971d42b5a96881c@mail.gmail.com>

On Wed, Aug 5, 2009 at 5:24 PM, Robert Kern<robert.kern at gmail.com> wrote:
>> And when you finish, send it our way. ?Operators are standing by to
>> take your patch and your credit card number... :)
>
> Something like this?

[...]  # Robert, as usual, rocks.

> It uses IPython.demo. It's not especially pretty, but it's servicable.

Yes, with Tom's recent demo improvements, this is precisely what I had in mind.

> Code is at the end of this file:
>
> http://www.enthought.com/~rkern/cgi-bin/hgwebdir.cgi/kernmagic/file/c18f492e9688/kernmagic/mymagics.py

Fantastic, many thanks.  We can now properly track it:

https://bugs.launchpad.net/ipython/+bug/409633


Hopefully once we land Brian's refactoring, we can start merging
features again :)


Now, that credit card number...

f


From scott.sinclair.za at gmail.com  Thu Aug  6 02:19:50 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Thu, 6 Aug 2009 08:19:50 +0200
Subject: [SciPy-dev] Two Marathon questions
In-Reply-To: <71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
References: <940468.47590.qm@web52104.mail.re2.yahoo.com>
	<71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
Message-ID: <6a17e9ee0908052319s2794095bq2dbcf670670dd1de@mail.gmail.com>

> 2009/8/5 Pierre GM <pgmdevlist at gmail.com>:
>
> On Aug 5, 2009, at 1:33 PM, David Goldsmith wrote:
>
>> 1) Is there anyone "in-the-know" who feels that reading "numpy-docs/
>> reference/routines.ma.rst" (as it is now) is *insufficient*
>> preparation for assisting w/ the Masked Array docstrings; or, put
>> another way, feels that if one is not at least "well-practiced"
>> using masked arrays, then one should not touch their docstrings?
>
> Mmh. Can't really tell. Experience with masked arrays is certainly a
> prerequisite, but one shouldn't need to be an expert.
> Nevertheless, I'd be obliged if anybody willing to edit the MA
> docstrings could contact me beforehand w/ questions.

The best place to get an overview of masked arrays is Pierre's introduction here

http://docs.scipy.org/doc/numpy/reference/maskedarray.html

It's a little hard to locate in the doc-wiki, but can be found here

http://docs.scipy.org/numpy/docs/numpy-docs/reference/maskedarray.rst

Ralf (or others) feel free to work on the docstrings in Milestone
"Operations On Masks" if you finish the other stuff and I still
haven't gotten much further. You obviously have much more
enthusiasm/time :)

Something worth paying attention to, is making sure that the
MaskedArray method docstrings and the equivalent functions in the ma
module refer to each other and spell out any differences in behaviour.
Compare

http://docs.scipy.org/numpy/docs/numpy.ma.core.MaskedArray.set_fill_value/
http://docs.scipy.org/numpy/docs/numpy.ma.core.set_fill_value/

Cheers,
Scott


From ralf.gommers at googlemail.com  Thu Aug  6 11:37:57 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Thu, 6 Aug 2009 11:37:57 -0400
Subject: [SciPy-dev] Two Marathon questions
In-Reply-To: <6a17e9ee0908052319s2794095bq2dbcf670670dd1de@mail.gmail.com>
References: <940468.47590.qm@web52104.mail.re2.yahoo.com>
	<71D66DD2-F618-47E7-8F61-C2E37677EBA2@gmail.com>
	<6a17e9ee0908052319s2794095bq2dbcf670670dd1de@mail.gmail.com>
Message-ID: <dde7764a0908060837h1206de96i86270fef21a10b31@mail.gmail.com>

On Thu, Aug 6, 2009 at 2:19 AM, Scott Sinclair
<scott.sinclair.za at gmail.com>wrote:

> > 2009/8/5 Pierre GM <pgmdevlist at gmail.com>:
> >
> > On Aug 5, 2009, at 1:33 PM, David Goldsmith wrote:
> >
> >> 1) Is there anyone "in-the-know" who feels that reading "numpy-docs/
> >> reference/routines.ma.rst" (as it is now) is *insufficient*
> >> preparation for assisting w/ the Masked Array docstrings; or, put
> >> another way, feels that if one is not at least "well-practiced"
> >> using masked arrays, then one should not touch their docstrings?
> >
> > Mmh. Can't really tell. Experience with masked arrays is certainly a
> > prerequisite, but one shouldn't need to be an expert.
> > Nevertheless, I'd be obliged if anybody willing to edit the MA
> > docstrings could contact me beforehand w/ questions.
>
> The best place to get an overview of masked arrays is Pierre's introduction
> here
>
> http://docs.scipy.org/doc/numpy/reference/maskedarray.html
>
> It's a little hard to locate in the doc-wiki, but can be found here
>
> http://docs.scipy.org/numpy/docs/numpy-docs/reference/maskedarray.rst
>
> Ralf (or others) feel free to work on the docstrings in Milestone
> "Operations On Masks" if you finish the other stuff and I still
> haven't gotten much further. You obviously have much more
> enthusiasm/time :)
>

more enthusiasm than time. luckily time is something you can make ...

>
> Something worth paying attention to, is making sure that the
> MaskedArray method docstrings and the equivalent functions in the ma
> module refer to each other and spell out any differences in behaviour.
> Compare
>
> http://docs.scipy.org/numpy/docs/numpy.ma.core.MaskedArray.set_fill_value/
> http://docs.scipy.org/numpy/docs/numpy.ma.core.set_fill_value/


good catch. i'll fix that and check the other ones i did.

cheers,
ralf


>
>
> Cheers,
> Scott
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090806/586cc844/attachment.html>

From pgmdevlist at gmail.com  Thu Aug  6 16:34:03 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 6 Aug 2009 16:34:03 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it wise ?
Message-ID: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>

All,
Consider the poisson distribution in stats.distributions: it requires  
a mandatory argument, `mu`, as the mean/variance of the distribution.  
All is fine, but the `loc` parameter is still available, and that's my  
problem. When `loc` is not 0, the mean becomes `mu+loc`,  
`.cdf(range(loc))==0`, but the variance stays `mu`. That's a bit  
confusing.
I thought I could use `loc` as a way to control truncation, but that  
doesn't seem to work either: emulating zero-truncation by using  
`loc=1` gives a distribution with a mean `mu+1` when is should be `mu/ 
(1-exp(-mu))` (the exact expression for zero-truncation).
In short, I don't really see any advantage in having a location  
parameter for the Poisson distribution. AAMOF, for any discrete  
distribution. I suggest we would implement some mechanism to force loc  
to 0 while outputting a warning.
Any comment ?
P.



From vanforeest at gmail.com  Thu Aug  6 17:21:40 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Thu, 6 Aug 2009 23:21:40 +0200
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
Message-ID: <fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>

Hi,

I agree. Anything that makes the behavior of the distribution
functions more intuitive is helpful, at least to me.

BTW, I find the term loc already by itself very confusing---what does
it actually mean? For instance,
>>> Help on gamma_gen in module scipy.stats.distributions object
...

 |  cdf(self, x, *args, **kwds)
 |      Cumulative distribution function at x of the given RV.
 |
 |      Parameters
 |      ----------
 |      x : array-like
 |          quantiles
 |      arg1, arg2, arg3,... : array-like
 |          The shape parameter(s) for the distribution (see docstring of the
 |          instance object for more information)
 |      loc : array-like, optional
 |          location parameter (default=0)
 |      scale : array-like, optional
 |          scale parameter (default=1)

I am inclined to characterize the gamma distbution by means of n
(number of stages if one is used to the Erlang distribution) and the
rate parameter lambda, say, and I am clueless as to the meaning of
scale and location here. Actually, I am not alone in this: see for
instance:

http://www.johndcook.com/blog/2009/07/20/probability-distributions-scipy/

Of course, this is not to say that I am not happy with the
distribution package. It makes me a happier man every day :-)

Nicky




2009/8/6 Pierre GM <pgmdevlist at gmail.com>:
> All,
> Consider the poisson distribution in stats.distributions: it requires
> a mandatory argument, `mu`, as the mean/variance of the distribution.
> All is fine, but the `loc` parameter is still available, and that's my
> problem. When `loc` is not 0, the mean becomes `mu+loc`,
> `.cdf(range(loc))==0`, but the variance stays `mu`. That's a bit
> confusing.
> I thought I could use `loc` as a way to control truncation, but that
> doesn't seem to work either: emulating zero-truncation by using
> `loc=1` gives a distribution with a mean `mu+1` when is should be `mu/
> (1-exp(-mu))` (the exact expression for zero-truncation).
> In short, I don't really see any advantage in having a location
> parameter for the Poisson distribution. AAMOF, for any discrete
> distribution. I suggest we would implement some mechanism to force loc
> to 0 while outputting a warning.
> Any comment ?
> P.
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From robert.kern at gmail.com  Thu Aug  6 17:34:08 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 6 Aug 2009 16:34:08 -0500
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com> 
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
Message-ID: <3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>

On Thu, Aug 6, 2009 at 16:21, nicky van foreest<vanforeest at gmail.com> wrote:
> Hi,
>
> I agree. Anything that makes the behavior of the distribution
> functions more intuitive is helpful, at least to me.
>
> BTW, I find the term loc already by itself very confusing---what does
> it actually mean? For instance,
>>>> Help on gamma_gen in module scipy.stats.distributions object
> ...
>
> ?| ?cdf(self, x, *args, **kwds)
> ?| ? ? ?Cumulative distribution function at x of the given RV.
> ?|
> ?| ? ? ?Parameters
> ?| ? ? ?----------
> ?| ? ? ?x : array-like
> ?| ? ? ? ? ?quantiles
> ?| ? ? ?arg1, arg2, arg3,... : array-like
> ?| ? ? ? ? ?The shape parameter(s) for the distribution (see docstring of the
> ?| ? ? ? ? ?instance object for more information)
> ?| ? ? ?loc : array-like, optional
> ?| ? ? ? ? ?location parameter (default=0)
> ?| ? ? ?scale : array-like, optional
> ?| ? ? ? ? ?scale parameter (default=1)
>
> I am inclined to characterize the gamma distbution by means of n
> (number of stages if one is used to the Erlang distribution) and the
> rate parameter lambda, say, and I am clueless as to the meaning of
> scale and location here.

Every probability distribution can be generalized to accept a location
and scale parameter even if their standard treatments do not.

  pdf(x; loc,scale) -> pdf((x-loc)/scale)/scale

The other related functions transform in easily derivable ways. This
is covered at the top of the document scipy/stats/continuous.lyx in
the source distribution.

The reason we do this is partly generality and mostly convenience of
implementation; all of the distributions can share the shifting and
scaling code instead of implementing it separately. I once floated the
idea of removing this for the distributions whose standard definitions
do not include such parameters, specifically gamma. However, there was
an objection from someone who apparently has used a "shifted gamma"
distribution to model sunspot radii where loc>0, if I remember
correctly, so I dropped my proposal.

If you don't need to use them, don't. If you want to prevent
confusion, help port the LyX documentation into the main
documentation.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From pgmdevlist at gmail.com  Thu Aug  6 17:43:12 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 6 Aug 2009 17:43:12 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
Message-ID: <FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>


On Aug 6, 2009, at 5:34 PM, Robert Kern wrote:
>
> Every probability distribution can be generalized to accept a location
> and scale parameter even if their standard treatments do not.
>
>  pdf(x; loc,scale) -> pdf((x-loc)/scale)/scale

Agreed, as long as we are talking about *continuous* distributions.
The behavior is quite different for *discrete* distributions. Even if  
the scale is simply discarded already, using a location will probably  
NOT give the expected result





From robert.kern at gmail.com  Thu Aug  6 17:49:22 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 6 Aug 2009 16:49:22 -0500
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com> 
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com> 
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com> 
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
Message-ID: <3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>

On Thu, Aug 6, 2009 at 16:43, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 6, 2009, at 5:34 PM, Robert Kern wrote:
>>
>> Every probability distribution can be generalized to accept a location
>> and scale parameter even if their standard treatments do not.
>>
>> ?pdf(x; loc,scale) -> pdf((x-loc)/scale)/scale
>
> Agreed, as long as we are talking about *continuous* distributions.
> The behavior is quite different for *discrete* distributions. Even if
> the scale is simply discarded already, using a location will probably
> NOT give the expected result

It depends on what your expectations are. For the discrete
distributions, all the loc parameter means is this, as documented:

  pmf(x; loc) -> pmf(x-loc)

That's it. I don't know why you would expect anything else.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From pgmdevlist at gmail.com  Thu Aug  6 18:02:57 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 6 Aug 2009 18:02:57 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>
Message-ID: <9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>


On Aug 6, 2009, at 5:49 PM, Robert Kern wrote:

> On Thu, Aug 6, 2009 at 16:43, Pierre GM<pgmdevlist at gmail.com> wrote:
>> Even if
>> the scale is simply discarded already, using a location will probably
>> NOT give the expected result
>
> It depends on what your expectations are. For the discrete
> distributions, all the loc parameter means is this, as documented:
>
>  pmf(x; loc) -> pmf(x-loc)
>
> That's it. I don't know why you would expect anything else.

Because using a location parameter, you change the support domain.  
Back to the example of a Poisson distribution with loc=1, the support  
domain is now x>=1, which amounts to truncating the zeroes. The mean  
of a zero-truncated Poisson with parameter pr should be pr/(1-exp(- 
pr)), but we end up with pr+1. Not the expected result.
I think it's a source of confusion to keep a location parameter for  
discrete distributions. it'd be worth to implement method to allow  
truncation, but just a loc parameter doesn't do it.




From robert.kern at gmail.com  Thu Aug  6 18:11:04 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 6 Aug 2009 17:11:04 -0500
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com> 
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com> 
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com> 
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com> 
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
Message-ID: <3d375d730908061511r666d1646p825e07dcffa8043a@mail.gmail.com>

On Thu, Aug 6, 2009 at 17:02, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 6, 2009, at 5:49 PM, Robert Kern wrote:
>
>> On Thu, Aug 6, 2009 at 16:43, Pierre GM<pgmdevlist at gmail.com> wrote:
>>> Even if
>>> the scale is simply discarded already, using a location will probably
>>> NOT give the expected result
>>
>> It depends on what your expectations are. For the discrete
>> distributions, all the loc parameter means is this, as documented:
>>
>> ?pmf(x; loc) -> pmf(x-loc)
>>
>> That's it. I don't know why you would expect anything else.
>
> Because using a location parameter, you change the support domain.
> Back to the example of a Poisson distribution with loc=1, the support
> domain is now x>=1, which amounts to truncating the zeroes.

I don't understand why you go through all of these contortions. It
does not amount to truncation at all. It just shifts the distribution.

> The mean
> of a zero-truncated Poisson with parameter pr should be pr/(1-exp(-
> pr)), but we end up with pr+1. Not the expected result.

Because you are expecting that the operation is equivalent to
something that it is not.

  pmf(x; loc) -> pmf(x-loc)

Nothing more. It is definitely *not* the same thing as setting all
x<loc to 0 and renormalizing.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From josef.pktd at gmail.com  Thu Aug  6 18:16:13 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 6 Aug 2009 18:16:13 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
Message-ID: <1cd32cbb0908061516g3cbab79ax2798b322a3c2fdf@mail.gmail.com>

On Thu, Aug 6, 2009 at 6:02 PM, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 6, 2009, at 5:49 PM, Robert Kern wrote:
>
>> On Thu, Aug 6, 2009 at 16:43, Pierre GM<pgmdevlist at gmail.com> wrote:
>>> Even if
>>> the scale is simply discarded already, using a location will probably
>>> NOT give the expected result
>>
>> It depends on what your expectations are. For the discrete
>> distributions, all the loc parameter means is this, as documented:
>>
>> ?pmf(x; loc) -> pmf(x-loc)
>>
>> That's it. I don't know why you would expect anything else.
>
> Because using a location parameter, you change the support domain.
> Back to the example of a Poisson distribution with loc=1, the support
> domain is now x>=1, which amounts to truncating the zeroes. The mean
> of a zero-truncated Poisson with parameter pr should be pr/(1-exp(-
> pr)), but we end up with pr+1. Not the expected result.
> I think it's a source of confusion to keep a location parameter for
> discrete distributions. it'd be worth to implement method to allow
> truncation, but just a loc parameter doesn't do it.
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

loc just shifts the distribution on the real/integer line.

except for the fit method (which doesn't exist for discrete
distribution), I don't see any real disadvantage to having loc in
there as an option, but I guess in many cases it won't be very useful
either.  I think there are also discrete distribution with unbound
support +/- inf for which a loc shift would make sense.

The big advantage of the current setup, as Robert said, is
consistency, both in the implementation and in code that goes over all
(or a large set of) distribution(s).

But for a long time, I have been all in favor of "fixing" the fit
method, and possibly introduce a semi-frozen distribution class, but
for this I don't see why we should special case location. fixing loc
is the main use case, but for example estimation with the scale
parameter fixed is also a common use case.

Josef


From robert.kern at gmail.com  Thu Aug  6 18:21:49 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 6 Aug 2009 17:21:49 -0500
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com> 
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com> 
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com> 
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com> 
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
Message-ID: <3d375d730908061521s166051f2h6e1e7579b8018bfe@mail.gmail.com>

On Thu, Aug 6, 2009 at 17:02, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 6, 2009, at 5:49 PM, Robert Kern wrote:
>
>> On Thu, Aug 6, 2009 at 16:43, Pierre GM<pgmdevlist at gmail.com> wrote:
>>> Even if
>>> the scale is simply discarded already, using a location will probably
>>> NOT give the expected result
>>
>> It depends on what your expectations are. For the discrete
>> distributions, all the loc parameter means is this, as documented:
>>
>> ?pmf(x; loc) -> pmf(x-loc)
>>
>> That's it. I don't know why you would expect anything else.
>
> Because using a location parameter, you change the support domain.

It should be noted that the location parameter changes the support
domain *as a consequence* of the above transformation. Changing the
support domain (and holding everything else fixed) is not the defining
characteristic of the location parameter.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Thu Aug  6 18:23:12 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 6 Aug 2009 17:23:12 -0500
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <1cd32cbb0908061516g3cbab79ax2798b322a3c2fdf@mail.gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com> 
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com> 
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com> 
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com> 
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
	<1cd32cbb0908061516g3cbab79ax2798b322a3c2fdf@mail.gmail.com>
Message-ID: <3d375d730908061523m712e9470pf0c577e1efa2ed56@mail.gmail.com>

On Thu, Aug 6, 2009 at 17:16, <josef.pktd at gmail.com> wrote:

> But for a long time, I have been all in favor of "fixing" the fit
> method,

I don't think anyone's *against* fixing the fit method. No one's found
the time or motivation to actually do it, though.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From pgmdevlist at gmail.com  Thu Aug  6 18:37:07 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Thu, 6 Aug 2009 18:37:07 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <3d375d730908061521s166051f2h6e1e7579b8018bfe@mail.gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
	<3d375d730908061521s166051f2h6e1e7579b8018bfe@mail.gmail.com>
Message-ID: <DAD6D052-A155-4039-8DB5-015CE9488E85@gmail.com>


On Aug 6, 2009, at 6:21 PM, Robert Kern wrote:
>
> It should be noted that the location parameter changes the support
> domain *as a consequence* of the above transformation. Changing the
> support domain (and holding everything else fixed) is not the defining
> characteristic of the location parameter.

Got the point. I'll make a mental note to mention that in the docs.

I'm switching to "meh" mode: I still think that allowing for the shift  
can lead to some troubles on the user, and I'd be in favor to modify  
_fix_loc_scale or something like that to force loc=0 on discrete  
distributions with support on positive integers, but I'll certainly  
not lose any sleep other that...
In any case, thx a lot to y'all for your comments.



From josef.pktd at gmail.com  Thu Aug  6 20:17:47 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 6 Aug 2009 20:17:47 -0400
Subject: [SciPy-dev] stats.distributions.poisson loc parameter : is it
	wise ?
In-Reply-To: <DAD6D052-A155-4039-8DB5-015CE9488E85@gmail.com>
References: <D13A9F2B-798B-4B90-96A9-BB96416196B6@gmail.com>
	<fa510ff80908061421m5471a912see1a874e82fa265@mail.gmail.com>
	<3d375d730908061434j21d72c8dk36fb7d72cb8a9092@mail.gmail.com>
	<FE6FE7A1-FF31-46B2-AB9B-481DED165494@gmail.com>
	<3d375d730908061449s6fae1a57p3f416dc41910f833@mail.gmail.com>
	<9D0FD21F-CE10-499A-971B-BDA3C0F1CEFE@gmail.com>
	<3d375d730908061521s166051f2h6e1e7579b8018bfe@mail.gmail.com>
	<DAD6D052-A155-4039-8DB5-015CE9488E85@gmail.com>
Message-ID: <1cd32cbb0908061717i37deee4x542e21cd146c2691@mail.gmail.com>

On Thu, Aug 6, 2009 at 6:37 PM, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 6, 2009, at 6:21 PM, Robert Kern wrote:
>>
>> It should be noted that the location parameter changes the support
>> domain *as a consequence* of the above transformation. Changing the
>> support domain (and holding everything else fixed) is not the defining
>> characteristic of the location parameter.
>
> Got the point. I'll make a mental note to mention that in the docs.
>
> I'm switching to "meh" mode: I still think that allowing for the shift
> can lead to some troubles on the user, and I'd be in favor to modify
> _fix_loc_scale or something like that to force loc=0 on discrete
> distributions with support on positive integers, but I'll certainly
> not lose any sleep other that...
> In any case, thx a lot to y'all for your comments.
>

I agree that loc for distribution with a finite upper or lower support
bound is confusing, at least at the beginning. It took me a while to
figure out why I get some strange results with some distributions when
I ran a fit over all of them until I realized that the support is
shifted when loc is estimated. But I think this is mostly a
documentation problem.
(I still have an unresolved problem with vonmises which doesn't define
it's support points, but I don't know anything at all about circular
distributions.)

Below is a prototype for a semi-frozen class, essentially an adapted
version of the frozen class, that fixes only the location loc. (copy
and paste errors still possible)

However, this doesn't do anything different than the current
implementation if you ignore the loc keyword. It also has the same
uninformative signature which could be improved.
The only real advantage I see, is, when the fit method is adjusted to
take some of the parameters as fixed.

Josef

import numpy as np
from scipy import stats


class rv_frozenloc(object):
    def __init__(self, dist, loc=0):
        self.loc = loc
        self.dist = dist
    def pdf(self,x,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.pdf(x,*args,**kwds)
    def cdf(self,x,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.cdf(x,*args,**kwds)
    def ppf(self,q,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.ppf(q,*args,**kwds)
    def isf(self,q,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.isf(q,*args,**kwds)
    def rvs(self, size=None,*args,**kwds):
        kwds.update({'loc':self.loc, 'size':size})
        return self.dist.rvs(*self.args,**kwds)
    def sf(self,x,*args,**kwds):
        return self.dist.sf(x,*args,**kwds)
    def stats(self, moments='mv',*args,**kwds):
        kwds.update({'loc':self.loc, 'moments':moments})
        return self.dist.stats(*args,**kwds)
    def moment(self,n,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.moment(n,*args,**kwds)
    def entropy(self,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.entropy(*args,**kwds)
    def pmf(self,k,*args,**kwds):
        kwds.update({'loc':self.loc})
        return self.dist.pmf(k,*args,**kwds)


def freezeloc(dist, loc=0):
        return rv_frozenloc(dist, loc=loc)


poiss = freezeloc(stats.poisson)

print poiss.pmf(np.arange(10),5)
print poiss.cdf(np.arange(10),5)
print poiss.cdf(np.arange(10),5, loc=5) #this ignores loc but doesn't
raise warning (yet)
print stats.poisson.cdf(np.arange(10),5, loc=5)
poiss5 = freezeloc(stats.poisson, loc=5)
print poiss5.cdf(np.arange(10),5)
norm0 = freezeloc(stats.norm, loc=1)
print norm0.stats()
norm0.stats(loc=0)  # loc is ignored but doesn't raise warning (yet)
print norm0.stats(scale=np.sqrt(2))


From david at ar.media.kyoto-u.ac.jp  Fri Aug  7 09:13:41 2009
From: david at ar.media.kyoto-u.ac.jp (David Cournapeau)
Date: Fri, 07 Aug 2009 22:13:41 +0900
Subject: [SciPy-dev] Last numpy required for scipy
Message-ID: <4A7C2885.1090308@ar.media.kyoto-u.ac.jp>

Hi,

    While fixing the neighborhood iterators, I once again broke the
trunk API - so for last scipy trunk (r5893), you need last numpy
(r7297). Hopefully, this will be the last time this is needed,

sorry for the inconvenience,

cheers,

David


From d_l_goldsmith at yahoo.com  Sat Aug  8 02:42:58 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 7 Aug 2009 23:42:58 -0700 (PDT)
Subject: [SciPy-dev] Where is numpy.generic defined?
Message-ID: <843554.18671.qm@web52105.mail.re2.yahoo.com>




      


From d_l_goldsmith at yahoo.com  Mon Aug 10 15:45:51 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Mon, 10 Aug 2009 12:45:51 -0700 (PDT)
Subject: [SciPy-dev] summer marathon skypecon this week?
Message-ID: <105079.43514.qm@web52108.mail.re2.yahoo.com>

Wed. 16:00 UTC has been suggested - if there's even interest; how does this suit people?

DG


      


From ralf.gommers at googlemail.com  Mon Aug 10 22:38:17 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Mon, 10 Aug 2009 22:38:17 -0400
Subject: [SciPy-dev] summer marathon skypecon this week?
In-Reply-To: <105079.43514.qm@web52108.mail.re2.yahoo.com>
References: <105079.43514.qm@web52108.mail.re2.yahoo.com>
Message-ID: <dde7764a0908101938i5a80e9c7rafe73b82e2bae7fd@mail.gmail.com>

That would work for me.

Ralf


On Mon, Aug 10, 2009 at 3:45 PM, David Goldsmith <d_l_goldsmith at yahoo.com>wrote:

> Wed. 16:00 UTC has been suggested - if there's even interest; how does this
> suit people?
>
> DG
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090810/050c354e/attachment.html>

From d_l_goldsmith at yahoo.com  Tue Aug 11 00:23:15 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Mon, 10 Aug 2009 21:23:15 -0700 (PDT)
Subject: [SciPy-dev] summer marathon skypecon this week?
Message-ID: <89338.9696.qm@web52110.mail.re2.yahoo.com>

OK, thanks, Ralf.

DG

--- On Mon, 8/10/09, Ralf Gommers <ralf.gommers at googlemail.com> wrote:

> From: Ralf Gommers <ralf.gommers at googlemail.com>
> Subject: Re: [SciPy-dev] summer marathon skypecon this week?
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Monday, August 10, 2009, 7:38 PM
> That would work for me.
> 
> Ralf
> 
> 
> On Mon, Aug 10, 2009 at 3:45 PM,
> David Goldsmith <d_l_goldsmith at yahoo.com>
> wrote:
> 
> Wed. 16:00 UTC has been suggested -
> if there's even interest; how does this suit people?
> 
> 
> 
> 
> DG
> 
> 
> 
> 
> 
> 
> 
> _______________________________________________
> 
> Scipy-dev mailing list
> 
> Scipy-dev at scipy.org
> 
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 
> 
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Tue Aug 11 00:25:16 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Mon, 10 Aug 2009 21:25:16 -0700 (PDT)
Subject: [SciPy-dev] summer marathon skypecon this week?
Message-ID: <97534.82743.qm@web52106.mail.re2.yahoo.com>

Whoops, didn't mean that to go to the whole list - sorry.

DG

--- On Mon, 8/10/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:

> From: David Goldsmith <d_l_goldsmith at yahoo.com>
> Subject: Re: [SciPy-dev] summer marathon skypecon this week?
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Monday, August 10, 2009, 9:23 PM
> OK, thanks, Ralf.
> 
> DG
> 
> --- On Mon, 8/10/09, Ralf Gommers <ralf.gommers at googlemail.com>
> wrote:
> 
> > From: Ralf Gommers <ralf.gommers at googlemail.com>
> > Subject: Re: [SciPy-dev] summer marathon skypecon this
> week?
> > To: "SciPy Developers List" <scipy-dev at scipy.org>
> > Date: Monday, August 10, 2009, 7:38 PM
> > That would work for me.
> > 
> > Ralf
> > 
> > 
> > On Mon, Aug 10, 2009 at 3:45 PM,
> > David Goldsmith <d_l_goldsmith at yahoo.com>
> > wrote:
> > 
> > Wed. 16:00 UTC has been suggested -
> > if there's even interest; how does this suit people?
> > 
> > 
> > 
> > 
> > DG
> > 
> > 
> > 
> > 
> > 
> > 
> > 
> > _______________________________________________
> > 
> > Scipy-dev mailing list
> > 
> > Scipy-dev at scipy.org
> > 
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > 
> > 
> > 
> > 
> > -----Inline Attachment Follows-----
> > 
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > 
> 
> 
> ? ? ? 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Tue Aug 11 15:05:06 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 12:05:06 -0700 (PDT)
Subject: [SciPy-dev] Summer Marathon Skypecon tomorrow, Wed., Aug. 12,
	16:00 UTC
Message-ID: <315280.34538.qm@web52102.mail.re2.yahoo.com>

OK, have two positive RSVP's , so we're on for 16:00 UTC tomorrow.

DG


      


From HAWRYLA at novachem.com  Tue Aug 11 15:10:39 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Tue, 11 Aug 2009 13:10:39 -0600
Subject: [SciPy-dev] can I help with the docs?
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF264@CHMAILMBX01.novachem.com>

I'd like to help with the docs project. I have registered a user name of
ahawryluk on the Wiki.
Although the current focus is cleaning up the docstrings on the numpy
module, I'm most interested in writing some introductory material for
the NumPy User Guide.

Andrew Hawryluk
NOVA Chemicals Research & Technology Centre
Calgary, Canada





From d_l_goldsmith at yahoo.com  Tue Aug 11 15:15:39 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 12:15:39 -0700 (PDT)
Subject: [SciPy-dev] can I help with the docs?
Message-ID: <734509.78574.qm@web52101.mail.re2.yahoo.com>

Hi, Andrew, and thanks for your offer to help.  Can you be more specific vis-a-vis what you have in mind?

David Goldsmith, Technical Editor
Olympia, WA

--- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:

> From: Andrew Hawryluk <HAWRYLA at novachem.com>
> Subject: [SciPy-dev] can I help with the docs?
> To: scipy-dev at scipy.org
> Date: Tuesday, August 11, 2009, 12:10 PM
> I'd like to help with the docs
> project. I have registered a user name of
> ahawryluk on the Wiki.
> Although the current focus is cleaning up the docstrings on
> the numpy
> module, I'm most interested in writing some introductory
> material for
> the NumPy User Guide.
> 
> Andrew Hawryluk
> NOVA Chemicals Research & Technology Centre
> Calgary, Canada
> 
> 
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From HAWRYLA at novachem.com  Tue Aug 11 16:08:30 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Tue, 11 Aug 2009 14:08:30 -0600
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <734509.78574.qm@web52101.mail.re2.yahoo.com>
References: <734509.78574.qm@web52101.mail.re2.yahoo.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>

Good question. I'll let you know what I was thinking, and then you can
tell me if I've misunderstood the documentation objectives.

Each year I train a new set of engineering internship students in our
department, and one of the things they learn is Python.  Upon arriving
here they have taken a single course on C++ and another on
Matlab/Mathematica/Excel etc., but no Python.  I usually start them off
reading "Dive Into Python", but I'd also like a single place to send
them to get a brief tour of proper NumPy use.  The main NumPy & SciPy
docs are excellent (and improving rapidly), but they start a bit too
abruptly for my needs. For example, the NumPy User Guide currently start
with installation instructions, and then the next page of body text is a
table of all the available array types.

I would like something that briefly explains what NumPy and SciPy are,
and why/when arrays are better than lists/dicts, perhaps followed by a
brief tour of some common NumPy tricks, before diving into the more
detailed sections.

One possible table of contents would be

Introduction
  What is NumPy?
  Building/Installing
  Short Tour
  How to find documentation
NumPy Basics
  ...

What direction were you thinking the User Guide would take?

Andrew

 
> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of David Goldsmith
> Sent: 11 Aug 2009 1:16 PM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] can I help with the docs?
> 
> Hi, Andrew, and thanks for your offer to help.  Can you be more
> specific vis-a-vis what you have in mind?
> 
> David Goldsmith, Technical Editor
> Olympia, WA
> 
> --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:
> 
> > From: Andrew Hawryluk <HAWRYLA at novachem.com>
> > Subject: [SciPy-dev] can I help with the docs?
> > To: scipy-dev at scipy.org
> > Date: Tuesday, August 11, 2009, 12:10 PM I'd like to help with the
> > docs project. I have registered a user name of ahawryluk on the
Wiki.
> > Although the current focus is cleaning up the docstrings on the
numpy
> > module, I'm most interested in writing some introductory material
for
> > the NumPy User Guide.
> >
> > Andrew Hawryluk
> > NOVA Chemicals Research & Technology Centre Calgary, Canada
> >
> >
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> 
> 
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From vanforeest at gmail.com  Tue Aug 11 17:06:52 2009
From: vanforeest at gmail.com (nicky van foreest)
Date: Tue, 11 Aug 2009 23:06:52 +0200
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>
References: <734509.78574.qm@web52101.mail.re2.yahoo.com>
	<48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>
Message-ID: <fa510ff80908111406h3f9a7a1j6d57e6f5692440e7@mail.gmail.com>

Hi Andrew,

I do not want to interfere with the scipy doc project... However,
after having read your ideas, might this cover your needs, at least in
part?

http://johnstachurski.net/lectures/index.html

bye

Nicky

2009/8/11 Andrew Hawryluk <HAWRYLA at novachem.com>:
> Good question. I'll let you know what I was thinking, and then you can
> tell me if I've misunderstood the documentation objectives.
>
> Each year I train a new set of engineering internship students in our
> department, and one of the things they learn is Python. ?Upon arriving
> here they have taken a single course on C++ and another on
> Matlab/Mathematica/Excel etc., but no Python. ?I usually start them off
> reading "Dive Into Python", but I'd also like a single place to send
> them to get a brief tour of proper NumPy use. ?The main NumPy & SciPy
> docs are excellent (and improving rapidly), but they start a bit too
> abruptly for my needs. For example, the NumPy User Guide currently start
> with installation instructions, and then the next page of body text is a
> table of all the available array types.
>
> I would like something that briefly explains what NumPy and SciPy are,
> and why/when arrays are better than lists/dicts, perhaps followed by a
> brief tour of some common NumPy tricks, before diving into the more
> detailed sections.
>
> One possible table of contents would be
>
> Introduction
> ?What is NumPy?
> ?Building/Installing
> ?Short Tour
> ?How to find documentation
> NumPy Basics
> ?...
>
> What direction were you thinking the User Guide would take?
>
> Andrew
>
>
>> -----Original Message-----
>> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
>> On Behalf Of David Goldsmith
>> Sent: 11 Aug 2009 1:16 PM
>> To: SciPy Developers List
>> Subject: Re: [SciPy-dev] can I help with the docs?
>>
>> Hi, Andrew, and thanks for your offer to help. ?Can you be more
>> specific vis-a-vis what you have in mind?
>>
>> David Goldsmith, Technical Editor
>> Olympia, WA
>>
>> --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:
>>
>> > From: Andrew Hawryluk <HAWRYLA at novachem.com>
>> > Subject: [SciPy-dev] can I help with the docs?
>> > To: scipy-dev at scipy.org
>> > Date: Tuesday, August 11, 2009, 12:10 PM I'd like to help with the
>> > docs project. I have registered a user name of ahawryluk on the
> Wiki.
>> > Although the current focus is cleaning up the docstrings on the
> numpy
>> > module, I'm most interested in writing some introductory material
> for
>> > the NumPy User Guide.
>> >
>> > Andrew Hawryluk
>> > NOVA Chemicals Research & Technology Centre Calgary, Canada
>> >
>> >
>> >
>> > _______________________________________________
>> > Scipy-dev mailing list
>> > Scipy-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>>
>>
>>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Tue Aug 11 18:17:09 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 15:17:09 -0700 (PDT)
Subject: [SciPy-dev] can I help with the docs?
Message-ID: <697806.15047.qm@web52105.mail.re2.yahoo.com>

--- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:

> I would like something that briefly explains what NumPy and
> SciPy are,
> and why/when arrays are better than lists/dicts, perhaps
> followed by a
> brief tour of some common NumPy tricks, before diving into
> the more
> detailed sections.
> 
> One possible table of contents would be
> 
> Introduction
> ? What is NumPy?
> ? Building/Installing
> ? Short Tour
> ? How to find documentation
> NumPy Basics
> ? ...
> 
> What direction were you thinking the User Guide would
> take?

Good question, especially as I don't think there's as clear a vision on that (certainly not as a matter of consensus) as yours is for what you want/need.  Personally, my "advice" to you would be: draft the document/sections you would like to see added - then you'll have what you want regardless of how/when it gets added to the "official" User Guide - and then post it (or a link to it) here to be vetted, if you want.  That said, 

http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst/

is the link to the Wiki (pydocweb) interface for editing the official User Guide, and you're more than welcome to add your content that way; of course, for better or worse, that's a Wiki, so whatever you add there will be subject to review and edit (but that's going to be true for any contribution to the official doc, of course).  Note that there you'll find a link to a section titled "NumPy Basics" w/ further links to sub-sections "types," "array creation," "indexing," and so forth - if that content is *not* what you've already read (there's a lot of doc in the Wiki - anything that's not past the proof stage, basically - that's pending inclusion in anything official) be sure to look it over, as it may already contain some (much?) of what you're after.  Thanks again,

DG

PS: If your original email was (primarily) intended to have someone grant you edit permissions, I'm afraid I haven't been granted that power yet, so you'll have to wait until someone who does, does.

> 
> Andrew
> 
>  
> > -----Original Message-----
> > From: scipy-dev-bounces at scipy.org
> [mailto:scipy-dev-bounces at scipy.org]
> > On Behalf Of David Goldsmith
> > Sent: 11 Aug 2009 1:16 PM
> > To: SciPy Developers List
> > Subject: Re: [SciPy-dev] can I help with the docs?
> > 
> > Hi, Andrew, and thanks for your offer to help.?
> Can you be more
> > specific vis-a-vis what you have in mind?
> > 
> > David Goldsmith, Technical Editor
> > Olympia, WA
> > 
> > --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com>
> wrote:
> > 
> > > From: Andrew Hawryluk <HAWRYLA at novachem.com>
> > > Subject: [SciPy-dev] can I help with the docs?
> > > To: scipy-dev at scipy.org
> > > Date: Tuesday, August 11, 2009, 12:10 PM I'd like
> to help with the
> > > docs project. I have registered a user name of
> ahawryluk on the
> Wiki.
> > > Although the current focus is cleaning up the
> docstrings on the
> numpy
> > > module, I'm most interested in writing some
> introductory material
> for
> > > the NumPy User Guide.
> > >
> > > Andrew Hawryluk
> > > NOVA Chemicals Research & Technology Centre
> Calgary, Canada
> > >
> > >
> > >
> > > _______________________________________________
> > > Scipy-dev mailing list
> > > Scipy-dev at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > >
> > 
> > 
> > 
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Tue Aug 11 19:02:15 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 16:02:15 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
Message-ID: <377127.33034.qm@web52105.mail.re2.yahoo.com>

First, a "reality check" question:

0) Is Windows (DOS) Terminal capable of rendering unicode?

Unless the answer is "No," my real question:

1) Does chararray.capitalize() capitalize non-Roman letters that have different lower-case and upper-case forms (e.g., the Greek letters)?  If "yes," are there any exceptions (e.g., Russian letters)?

Thanks!

DG


      


From HAWRYLA at novachem.com  Tue Aug 11 19:13:49 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Tue, 11 Aug 2009 17:13:49 -0600
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <697806.15047.qm@web52105.mail.re2.yahoo.com>
References: <697806.15047.qm@web52105.mail.re2.yahoo.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF267@CHMAILMBX01.novachem.com>

That sounds good. I will draft up some ideas, and submit them for comment / review / merciless editing.
Yes, the 'NumPy Basics' section covers most of the important ground, so I will re-read it before I suggest anything.

Thanks,
Andrew



> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of David Goldsmith
> Sent: 11 Aug 2009 4:17 PM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] can I help with the docs?
> 
> --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:
> 
> > I would like something that briefly explains what NumPy and SciPy
> are,
> > and why/when arrays are better than lists/dicts, perhaps followed by
> a
> > brief tour of some common NumPy tricks, before diving into the more
> > detailed sections.
> >
> > One possible table of contents would be
> >
> > Introduction
> > ? What is NumPy?
> > ? Building/Installing
> > ? Short Tour
> > ? How to find documentation
> > NumPy Basics
> > ? ...
> >
> > What direction were you thinking the User Guide would take?
> 
> Good question, especially as I don't think there's as clear a vision on
> that (certainly not as a matter of consensus) as yours is for what you
> want/need.  Personally, my "advice" to you would be: draft the
> document/sections you would like to see added - then you'll have what
> you want regardless of how/when it gets added to the "official" User
> Guide - and then post it (or a link to it) here to be vetted, if you
> want.  That said,
> 
> http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst/
> 
> is the link to the Wiki (pydocweb) interface for editing the official
> User Guide, and you're more than welcome to add your content that way;
> of course, for better or worse, that's a Wiki, so whatever you add
> there will be subject to review and edit (but that's going to be true
> for any contribution to the official doc, of course).  Note that there
> you'll find a link to a section titled "NumPy Basics" w/ further links
> to sub-sections "types," "array creation," "indexing," and so forth -
> if that content is *not* what you've already read (there's a lot of doc
> in the Wiki - anything that's not past the proof stage, basically -
> that's pending inclusion in anything official) be sure to look it over,
> as it may already contain some (much?) of what you're after.  Thanks
> again,
> 
> DG
> 
> PS: If your original email was (primarily) intended to have someone
> grant you edit permissions, I'm afraid I haven't been granted that
> power yet, so you'll have to wait until someone who does, does.
> 
> >
> > Andrew
> >
> >
> > > -----Original Message-----
> > > From: scipy-dev-bounces at scipy.org
> > [mailto:scipy-dev-bounces at scipy.org]
> > > On Behalf Of David Goldsmith
> > > Sent: 11 Aug 2009 1:16 PM
> > > To: SciPy Developers List
> > > Subject: Re: [SciPy-dev] can I help with the docs?
> > >
> > > Hi, Andrew, and thanks for your offer to help.
> > Can you be more
> > > specific vis-a-vis what you have in mind?
> > >
> > > David Goldsmith, Technical Editor
> > > Olympia, WA
> > >
> > > --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com>
> > wrote:
> > >
> > > > From: Andrew Hawryluk <HAWRYLA at novachem.com>
> > > > Subject: [SciPy-dev] can I help with the docs?
> > > > To: scipy-dev at scipy.org
> > > > Date: Tuesday, August 11, 2009, 12:10 PM I'd like
> > to help with the
> > > > docs project. I have registered a user name of
> > ahawryluk on the
> > Wiki.
> > > > Although the current focus is cleaning up the
> > docstrings on the
> > numpy
> > > > module, I'm most interested in writing some
> > introductory material
> > for
> > > > the NumPy User Guide.
> > > >
> > > > Andrew Hawryluk
> > > > NOVA Chemicals Research & Technology Centre
> > Calgary, Canada
> > > >
> > > >
> > > >
> > > > _______________________________________________
> > > > Scipy-dev mailing list
> > > > Scipy-dev at scipy.org
> > > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > > >
> > >
> > >
> > >
> > > _______________________________________________
> > > Scipy-dev mailing list
> > > Scipy-dev at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> 
> 
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From HAWRYLA at novachem.com  Tue Aug 11 19:15:48 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Tue, 11 Aug 2009 17:15:48 -0600
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <fa510ff80908111406h3f9a7a1j6d57e6f5692440e7@mail.gmail.com>
References: <734509.78574.qm@web52101.mail.re2.yahoo.com><48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>
	<fa510ff80908111406h3f9a7a1j6d57e6f5692440e7@mail.gmail.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF268@CHMAILMBX01.novachem.com>

Thanks for the link! I will certainly include that in my list of helpful training materials.

Andrew

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of nicky van foreest
> Sent: 11 Aug 2009 3:07 PM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] can I help with the docs?
> 
> Hi Andrew,
> 
> I do not want to interfere with the scipy doc project... However, after
> having read your ideas, might this cover your needs, at least in part?
> 
> http://johnstachurski.net/lectures/index.html
> 
> bye
> 
> Nicky
> 
> 2009/8/11 Andrew Hawryluk <HAWRYLA at novachem.com>:
> > Good question. I'll let you know what I was thinking, and then you
> can
> > tell me if I've misunderstood the documentation objectives.
> >
> > Each year I train a new set of engineering internship students in our
> > department, and one of the things they learn is Python. ?Upon
> arriving
> > here they have taken a single course on C++ and another on
> > Matlab/Mathematica/Excel etc., but no Python. ?I usually start them
> > off reading "Dive Into Python", but I'd also like a single place to
> > send them to get a brief tour of proper NumPy use. ?The main NumPy &
> > SciPy docs are excellent (and improving rapidly), but they start a
> bit
> > too abruptly for my needs. For example, the NumPy User Guide
> currently
> > start with installation instructions, and then the next page of body
> > text is a table of all the available array types.
> >
> > I would like something that briefly explains what NumPy and SciPy
> are,
> > and why/when arrays are better than lists/dicts, perhaps followed by
> a
> > brief tour of some common NumPy tricks, before diving into the more
> > detailed sections.
> >
> > One possible table of contents would be
> >
> > Introduction
> > ?What is NumPy?
> > ?Building/Installing
> > ?Short Tour
> > ?How to find documentation
> > NumPy Basics
> > ?...
> >
> > What direction were you thinking the User Guide would take?
> >
> > Andrew
> >
> >
> >> -----Original Message-----
> >> From: scipy-dev-bounces at scipy.org
> >> [mailto:scipy-dev-bounces at scipy.org]
> >> On Behalf Of David Goldsmith
> >> Sent: 11 Aug 2009 1:16 PM
> >> To: SciPy Developers List
> >> Subject: Re: [SciPy-dev] can I help with the docs?
> >>
> >> Hi, Andrew, and thanks for your offer to help. ?Can you be more
> >> specific vis-a-vis what you have in mind?
> >>
> >> David Goldsmith, Technical Editor
> >> Olympia, WA
> >>
> >> --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:
> >>
> >> > From: Andrew Hawryluk <HAWRYLA at novachem.com>
> >> > Subject: [SciPy-dev] can I help with the docs?
> >> > To: scipy-dev at scipy.org
> >> > Date: Tuesday, August 11, 2009, 12:10 PM I'd like to help with the
> >> > docs project. I have registered a user name of ahawryluk on the
> > Wiki.
> >> > Although the current focus is cleaning up the docstrings on the
> > numpy
> >> > module, I'm most interested in writing some introductory material
> > for
> >> > the NumPy User Guide.
> >> >
> >> > Andrew Hawryluk
> >> > NOVA Chemicals Research & Technology Centre Calgary, Canada
> >> >
> >> >
> >> >
> >> > _______________________________________________
> >> > Scipy-dev mailing list
> >> > Scipy-dev at scipy.org
> >> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >> >
> >>
> >>
> >>
> >> _______________________________________________
> >> Scipy-dev mailing list
> >> Scipy-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From d_l_goldsmith at yahoo.com  Tue Aug 11 19:30:45 2009
From: d_l_goldsmith at yahoo.com (d_l_goldsmith at yahoo.com)
Date: Tue, 11 Aug 2009 16:30:45 -0700 (PDT)
Subject: [SciPy-dev] 'nuther Q re: chararray
Message-ID: <164127.69670.qm@web52103.mail.re2.yahoo.com>

>From "Guide to Numpy":

"Perhaps the easiest way to create a chararray is to use self.view(chararray) where self is an ndarray of string or unicode data-type."? 

OK, but what is "(chararray)"?? In particular, my best guesses yielded:

>>> c = np.empty((2,2), dtype=unicode)
>>> c
array([[u'\u0f01\ude68', u'\uf880\udc00'],
? ? ???[u'\uf0c0\udc00', u'\u5300']],
? ? ? dtype='<U1')
>>> c.view(chararray) # can't mean it literally, right?
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
NameError: name 'chararray' is not defined
>>> c.view(C) # does the expression assign a value to the argument?
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
NameError: name 'C' is not defined
>>> C = np.chararray((2,2)) # this is most logical
>>> c.view(C)
Traceback (most recent call last):
? File "<stdin>", line 1, in <module>
ValueError: Dtype must be a numpy data-type

So, what does Travis mean?  Thanks,

DG


      


From d_l_goldsmith at yahoo.com  Tue Aug 11 19:32:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 16:32:20 -0700 (PDT)
Subject: [SciPy-dev] can I help with the docs?
Message-ID: <140993.83164.qm@web52112.mail.re2.yahoo.com>

You're very welcome!  Thank you for helping!

DG

--- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:

> From: Andrew Hawryluk <HAWRYLA at novachem.com>
> Subject: Re: [SciPy-dev] can I help with the docs?
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 11, 2009, 4:13 PM
> That sounds good. I will draft up
> some ideas, and submit them for comment / review / merciless
> editing.
> Yes, the 'NumPy Basics' section covers most of the
> important ground, so I will re-read it before I suggest
> anything.
> 
> Thanks,
> Andrew
> 
> 
> 
> > -----Original Message-----
> > From: scipy-dev-bounces at scipy.org
> [mailto:scipy-dev-bounces at scipy.org]
> > On Behalf Of David Goldsmith
> > Sent: 11 Aug 2009 4:17 PM
> > To: SciPy Developers List
> > Subject: Re: [SciPy-dev] can I help with the docs?
> > 
> > --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com>
> wrote:
> > 
> > > I would like something that briefly explains what
> NumPy and SciPy
> > are,
> > > and why/when arrays are better than lists/dicts,
> perhaps followed by
> > a
> > > brief tour of some common NumPy tricks, before
> diving into the more
> > > detailed sections.
> > >
> > > One possible table of contents would be
> > >
> > > Introduction
> > > ? What is NumPy?
> > > ? Building/Installing
> > > ? Short Tour
> > > ? How to find documentation
> > > NumPy Basics
> > > ? ...
> > >
> > > What direction were you thinking the User Guide
> would take?
> > 
> > Good question, especially as I don't think there's as
> clear a vision on
> > that (certainly not as a matter of consensus) as yours
> is for what you
> > want/need.? Personally, my "advice" to you would
> be: draft the
> > document/sections you would like to see added - then
> you'll have what
> > you want regardless of how/when it gets added to the
> "official" User
> > Guide - and then post it (or a link to it) here to be
> vetted, if you
> > want.? That said,
> > 
> > http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst/
> > 
> > is the link to the Wiki (pydocweb) interface for
> editing the official
> > User Guide, and you're more than welcome to add your
> content that way;
> > of course, for better or worse, that's a Wiki, so
> whatever you add
> > there will be subject to review and edit (but that's
> going to be true
> > for any contribution to the official doc, of
> course).? Note that there
> > you'll find a link to a section titled "NumPy Basics"
> w/ further links
> > to sub-sections "types," "array creation," "indexing,"
> and so forth -
> > if that content is *not* what you've already read
> (there's a lot of doc
> > in the Wiki - anything that's not past the proof
> stage, basically -
> > that's pending inclusion in anything official) be sure
> to look it over,
> > as it may already contain some (much?) of what you're
> after.? Thanks
> > again,
> > 
> > DG
> > 
> > PS: If your original email was (primarily) intended to
> have someone
> > grant you edit permissions, I'm afraid I haven't been
> granted that
> > power yet, so you'll have to wait until someone who
> does, does.
> > 
> > >
> > > Andrew
> > >
> > >
> > > > -----Original Message-----
> > > > From: scipy-dev-bounces at scipy.org
> > > [mailto:scipy-dev-bounces at scipy.org]
> > > > On Behalf Of David Goldsmith
> > > > Sent: 11 Aug 2009 1:16 PM
> > > > To: SciPy Developers List
> > > > Subject: Re: [SciPy-dev] can I help with the
> docs?
> > > >
> > > > Hi, Andrew, and thanks for your offer to
> help.
> > > Can you be more
> > > > specific vis-a-vis what you have in mind?
> > > >
> > > > David Goldsmith, Technical Editor
> > > > Olympia, WA
> > > >
> > > > --- On Tue, 8/11/09, Andrew Hawryluk <HAWRYLA at novachem.com>
> > > wrote:
> > > >
> > > > > From: Andrew Hawryluk <HAWRYLA at novachem.com>
> > > > > Subject: [SciPy-dev] can I help with
> the docs?
> > > > > To: scipy-dev at scipy.org
> > > > > Date: Tuesday, August 11, 2009, 12:10
> PM I'd like
> > > to help with the
> > > > > docs project. I have registered a user
> name of
> > > ahawryluk on the
> > > Wiki.
> > > > > Although the current focus is cleaning
> up the
> > > docstrings on the
> > > numpy
> > > > > module, I'm most interested in writing
> some
> > > introductory material
> > > for
> > > > > the NumPy User Guide.
> > > > >
> > > > > Andrew Hawryluk
> > > > > NOVA Chemicals Research &
> Technology Centre
> > > Calgary, Canada
> > > > >
> > > > >
> > > > >
> > > > >
> _______________________________________________
> > > > > Scipy-dev mailing list
> > > > > Scipy-dev at scipy.org
> > > > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > > > >
> > > >
> > > >
> > > >
> > > >
> _______________________________________________
> > > > Scipy-dev mailing list
> > > > Scipy-dev at scipy.org
> > > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > > _______________________________________________
> > > Scipy-dev mailing list
> > > Scipy-dev at scipy.org
> > > http://mail.scipy.org/mailman/listinfo/scipy-dev
> > >
> > 
> > 
> > 
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Tue Aug 11 19:36:10 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 19:36:10 -0400
Subject: [SciPy-dev] 'nuther Q re: chararray
In-Reply-To: <164127.69670.qm@web52103.mail.re2.yahoo.com>
References: <164127.69670.qm@web52103.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908111636qd8a54d8l6a873b078971f3ec@mail.gmail.com>

On Tue, Aug 11, 2009 at 7:30 PM, <d_l_goldsmith at yahoo.com> wrote:
> >From "Guide to Numpy":
>
> "Perhaps the easiest way to create a chararray is to use self.view(chararray) where self is an ndarray of string or unicode data-type."
>
> OK, but what is "(chararray)"?? In particular, my best guesses yielded:
>
>>>> c = np.empty((2,2), dtype=unicode)
>>>> c
> array([[u'\u0f01\ude68', u'\uf880\udc00'],
> ? ? ???[u'\uf0c0\udc00', u'\u5300']],
> ? ? ? dtype='<U1')
>>>> c.view(chararray) # can't mean it literally, right?
> Traceback (most recent call last):
> ? File "<stdin>", line 1, in <module>
> NameError: name 'chararray' is not defined
>>>> c.view(C) # does the expression assign a value to the argument?
> Traceback (most recent call last):
> ? File "<stdin>", line 1, in <module>
> NameError: name 'C' is not defined
>>>> C = np.chararray((2,2)) # this is most logical
>>>> c.view(C)
> Traceback (most recent call last):
> ? File "<stdin>", line 1, in <module>
> ValueError: Dtype must be a numpy data-type
>
> So, what does Travis mean? ?Thanks,
>
> DG
>
>

with this type of error I usually check the namespace

>>> import numpy as np
>>> np.chararray
<class 'numpy.core.defchararray.chararray'>
>>> c = np.empty((2,2), dtype=unicode)
>>> c.view(np.chararray)
chararray([['', u'\x04'],
       [u'\x01', u'\uf3db\ude20']],
      dtype='<U1')

But I have no idea what chararrays are

Josef

>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Tue Aug 11 19:49:32 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 16:49:32 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
Message-ID: <902989.11820.qm@web52110.mail.re2.yahoo.com>

OK, may have answered Q1 myself: unless I'm misunderstanding what I'm seeing, what I'm finding is that capitalize() does nothing at all if the chararray is of dtype unicode - correct?  Thanks,

DG

--- On Tue, 8/11/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:

> From: David Goldsmith <d_l_goldsmith at yahoo.com>
> Subject: Some Q's vis-a-vis Numpy unicode support
> To: scipy-dev at scipy.org
> Date: Tuesday, August 11, 2009, 4:02 PM
> First, a "reality check" question:
> 
> 0) Is Windows (DOS) Terminal capable of rendering unicode?
> 
> Unless the answer is "No," my real question:
> 
> 1) Does chararray.capitalize() capitalize non-Roman letters
> that have different lower-case and upper-case forms (e.g.,
> the Greek letters)?? If "yes," are there any exceptions
> (e.g., Russian letters)?
> 
> Thanks!
> 
> DG
> 
> 
> ? ? ? 
> 


      


From d_l_goldsmith at yahoo.com  Tue Aug 11 19:50:46 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 16:50:46 -0700 (PDT)
Subject: [SciPy-dev] 'nuther Q re: chararray
Message-ID: <880273.12857.qm@web52110.mail.re2.yahoo.com>

Doh! Jackpot, thanks!

DG

--- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
> Subject: Re: [SciPy-dev] 'nuther Q re: chararray
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 11, 2009, 4:36 PM
> On Tue, Aug 11, 2009 at 7:30 PM,
> <d_l_goldsmith at yahoo.com>
> wrote:
> > >From "Guide to Numpy":
> >
> > "Perhaps the easiest way to create a chararray is to
> use self.view(chararray) where self is an ndarray of string
> or unicode data-type."
> >
> > OK, but what is "(chararray)"?? In particular, my
> best guesses yielded:
> >
> >>>> c = np.empty((2,2), dtype=unicode)
> >>>> c
> > array([[u'\u0f01\ude68', u'\uf880\udc00'],
> > ? ? ???[u'\uf0c0\udc00', u'\u5300']],
> > ? ? ? dtype='<U1')
> >>>> c.view(chararray) # can't mean it
> literally, right?
> > Traceback (most recent call last):
> > ? File "<stdin>", line 1, in <module>
> > NameError: name 'chararray' is not defined
> >>>> c.view(C) # does the expression assign a
> value to the argument?
> > Traceback (most recent call last):
> > ? File "<stdin>", line 1, in <module>
> > NameError: name 'C' is not defined
> >>>> C = np.chararray((2,2)) # this is most
> logical
> >>>> c.view(C)
> > Traceback (most recent call last):
> > ? File "<stdin>", line 1, in <module>
> > ValueError: Dtype must be a numpy data-type
> >
> > So, what does Travis mean? ?Thanks,
> >
> > DG
> >
> >
> 
> with this type of error I usually check the namespace
> 
> >>> import numpy as np
> >>> np.chararray
> <class 'numpy.core.defchararray.chararray'>
> >>> c = np.empty((2,2), dtype=unicode)
> >>> c.view(np.chararray)
> chararray([['', u'\x04'],
> ? ? ???[u'\x01',
> u'\uf3db\ude20']],
> ? ? ? dtype='<U1')
> 
> But I have no idea what chararrays are
> 
> Josef
> 
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From pgmdevlist at gmail.com  Tue Aug 11 20:24:17 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Tue, 11 Aug 2009 20:24:17 -0400
Subject: [SciPy-dev] 'nuther Q re: chararray
In-Reply-To: <164127.69670.qm@web52103.mail.re2.yahoo.com>
References: <164127.69670.qm@web52103.mail.re2.yahoo.com>
Message-ID: <8FD5A7ED-1F76-4701-A528-F61D1138739A@gmail.com>


On Aug 11, 2009, at 7:30 PM, d_l_goldsmith at yahoo.com wrote:

>> From "Guide to Numpy":
>
> "Perhaps the easiest way to create a chararray is to use  
> self.view(chararray) where self is an ndarray of string or unicode  
> data-type."
>
> OK, but what is "(chararray)"?  In particular, my best guesses  
> yielded:

numpy.char.chararray

Check defcharray.py in numpy.core



From josef.pktd at gmail.com  Tue Aug 11 20:41:33 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 20:41:33 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <902989.11820.qm@web52110.mail.re2.yahoo.com>
References: <902989.11820.qm@web52110.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908111741u59d3dd38o2212a77d42a8a8d5@mail.gmail.com>

On Tue, Aug 11, 2009 at 7:49 PM, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> OK, may have answered Q1 myself: unless I'm misunderstanding what I'm seeing, what I'm finding is that capitalize() does nothing at all if the chararray is of dtype unicode - correct? ?Thanks,


>>> b
chararray(u'\xe9',
      dtype='<U1')
>>> b.capitalize()
chararray(u'\xc9',
      dtype='<U1')

see http://stackoverflow.com/questions/1006450/capitalizing-non-ascii-words-in-python



>
> DG
>
> --- On Tue, 8/11/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:
>
>> From: David Goldsmith <d_l_goldsmith at yahoo.com>
>> Subject: Some Q's vis-a-vis Numpy unicode support
>> To: scipy-dev at scipy.org
>> Date: Tuesday, August 11, 2009, 4:02 PM
>> First, a "reality check" question:
>>
>> 0) Is Windows (DOS) Terminal capable of rendering unicode?

not by default ( in US english at least)
but the code page number can be changed, which I never tried

>help graftabl
Enable Windows to display an extended character set in graphics mode.

GRAFTABL [xxx]
GRAFTABL /STATUS

   xxx      Specifies a code page number.
   /STATUS  Displays the current code page selected for use with GRAFTABL.



from python session in windows command shell (it prints correctly in
case mail doesn't render it)
>>> print u'\xe9'
?
>>> print u'\xe9'.capitalize()
?
>>> u'\xe9'.capitalize()
u'\xc9'
>>>


but I cannot print any numpy.chararrays without getting
>>> c= np.array(u'\xe9','<U1')
>>> print c
....
UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
position 0: ordinal not in range(128)

(this is in Idle, with cp1252 I think)

the usual encode, decode problems with unicode, which take several
hours of trial and error and reading docs to figure out.

Josef

>>
>> Unless the answer is "No," my real question:
>>
>> 1) Does chararray.capitalize() capitalize non-Roman letters
>> that have different lower-case and upper-case forms (e.g.,
>> the Greek letters)?? If "yes," are there any exceptions
>> (e.g., Russian letters)?
>>
>> Thanks!
>>
>> DG
>>
>>
>>
>>
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From josef.pktd at gmail.com  Tue Aug 11 21:03:18 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 21:03:18 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908111741u59d3dd38o2212a77d42a8a8d5@mail.gmail.com>
References: <902989.11820.qm@web52110.mail.re2.yahoo.com>
	<1cd32cbb0908111741u59d3dd38o2212a77d42a8a8d5@mail.gmail.com>
Message-ID: <1cd32cbb0908111803m33f140adwe4f633bee5f6cf7f@mail.gmail.com>

On Tue, Aug 11, 2009 at 8:41 PM, <josef.pktd at gmail.com> wrote:
> On Tue, Aug 11, 2009 at 7:49 PM, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>> OK, may have answered Q1 myself: unless I'm misunderstanding what I'm seeing, what I'm finding is that capitalize() does nothing at all if the chararray is of dtype unicode - correct? ?Thanks,
>
>
>>>> b
> chararray(u'\xe9',
> ? ? ?dtype='<U1')
>>>> b.capitalize()
> chararray(u'\xc9',
> ? ? ?dtype='<U1')
>
> see http://stackoverflow.com/questions/1006450/capitalizing-non-ascii-words-in-python
>
>
>
>>
>> DG
>>
>> --- On Tue, 8/11/09, David Goldsmith <d_l_goldsmith at yahoo.com> wrote:
>>
>>> From: David Goldsmith <d_l_goldsmith at yahoo.com>
>>> Subject: Some Q's vis-a-vis Numpy unicode support
>>> To: scipy-dev at scipy.org
>>> Date: Tuesday, August 11, 2009, 4:02 PM
>>> First, a "reality check" question:
>>>
>>> 0) Is Windows (DOS) Terminal capable of rendering unicode?
>
> not by default ( in US english at least)
> but the code page number can be changed, which I never tried
>
>>help graftabl
> Enable Windows to display an extended character set in graphics mode.
>
> GRAFTABL [xxx]
> GRAFTABL /STATUS
>
> ? xxx ? ? ?Specifies a code page number.
> ? /STATUS ?Displays the current code page selected for use with GRAFTABL.
>
>
>
> from python session in windows command shell (it prints correctly in
> case mail doesn't render it)
>>>> print u'\xe9'
> ?
>>>> print u'\xe9'.capitalize()
> ?
>>>> u'\xe9'.capitalize()
> u'\xc9'
>>>>
>
>
> but I cannot print any numpy.chararrays without getting
>>>> c= np.array(u'\xe9','<U1')
>>>> print c
> ....
> UnicodeEncodeError: 'ascii' codec can't encode character u'\xe9' in
> position 0: ordinal not in range(128)
>
> (this is in Idle, with cp1252 I think)
>
> the usual encode, decode problems with unicode, which take several
> hours of trial and error and reading docs to figure out.


actually this works (in Idle)

>>> b = np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>>> print b.encode('cp1252')[0]
?
>>> print b.capitalize().encode('cp1252')[0]
?
>>> print b[0].encode('cp1252')
?


this looks like a bug ? or is it a known limitation that chararrays
cannot be 0-d

>>> b0= np.array(u'\xe9','<U1').view(np.chararray)
>>> print b0.encode('cp1252')
Traceback (most recent call last):
  File "<pyshell#47>", line 1, in <module>
    print b0.encode('cp1252')
  File "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
line 217, in encode
    return self._generalmethod('encode', broadcast(self, encoding, errors))
  File "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
line 162, in _generalmethod
    newarr[:] = res
ValueError: cannot slice a 0-d array


>
> Josef
>
>>>
>>> Unless the answer is "No," my real question:
>>>
>>> 1) Does chararray.capitalize() capitalize non-Roman letters
>>> that have different lower-case and upper-case forms (e.g.,
>>> the Greek letters)?? If "yes," are there any exceptions
>>> (e.g., Russian letters)?
>>>
>>> Thanks!
>>>
>>> DG
>>>
>>>
>>>
>>>
>>
>>
>>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>


From d_l_goldsmith at yahoo.com  Tue Aug 11 22:28:32 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 19:28:32 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
Message-ID: <877871.14415.qm@web52104.mail.re2.yahoo.com>

Thanks, Josef.  This may just be an artifact of working in a DOS Terminal (but your example, though not printing the accented e, did at least print something different for b vs. b.capitalize()), or it may be because I don't know the right encoding to use, but I tried your code w/ what I found on Wikipedia to be the unicode for the Greek letter delta, namely, u'\x03b04', with both 'cp1252' and 'iso8859-7' encoding (the latter being inferred from the same Wikipedia article) and here's what I get:

>>> b = np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>>> print b.encode('cp1252')[0]
?
>>> print b.capitalize().encode('cp1252')[0]
?
>>> print b.encode('iso8859-7')[0]
?
>>> print b.capitalize().encode('iso8859-7')[0]
?

i.e., no difference.  If I'm doing something wrong, please let me know; otherwise, for the purpose of documenting chararray.capitalize() - which is my ultimate goal - is there any rhyme or reason behind which unicode characters capitalize() works on and which it doesn't? 

Thanks,

DG
--- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> actually this works (in Idle)
> 
> >>> b =
> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
> >>> print b.encode('cp1252')[0]
> ?
> >>> print b.capitalize().encode('cp1252')[0]
> ?
> >>> print b[0].encode('cp1252')
> ?
> 
> 
> this looks like a bug ? or is it a known limitation that
> chararrays
> cannot be 0-d
> 
> >>> b0=
> np.array(u'\xe9','<U1').view(np.chararray)
> >>> print b0.encode('cp1252')
> Traceback (most recent call last):
> ? File "<pyshell#47>", line 1, in
> <module>
> ? ? print b0.encode('cp1252')
> ? File
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> line 217, in encode
> ? ? return self._generalmethod('encode',
> broadcast(self, encoding, errors))
> ? File
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> line 162, in _generalmethod
> ? ? newarr[:] = res
> ValueError: cannot slice a 0-d array
> 
> 
> >
> > Josef
> >
> >>>
> >>> Unless the answer is "No," my real question:
> >>>
> >>> 1) Does chararray.capitalize() capitalize
> non-Roman letters
> >>> that have different lower-case and upper-case
> forms (e.g.,
> >>> the Greek letters)?? If "yes," are there any
> exceptions
> >>> (e.g., Russian letters)?
> >>>
> >>> Thanks!
> >>>
> >>> DG
> >>>
> >>>
> >>>
> >>>
> >>
> >>
> >>
> >> _______________________________________________
> >> Scipy-dev mailing list
> >> Scipy-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Tue Aug 11 23:18:14 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 23:18:14 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <877871.14415.qm@web52104.mail.re2.yahoo.com>
References: <877871.14415.qm@web52104.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908112018x35546e80u1e42d18548dd659a@mail.gmail.com>

On Tue, Aug 11, 2009 at 10:28 PM, David
Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> Thanks, Josef. ?This may just be an artifact of working in a DOS Terminal (but your example, though not printing the accented e, did at least print something different for b vs. b.capitalize()), or it may be because I don't know the right encoding to use, but I tried your code w/ what I found on Wikipedia to be the unicode for the Greek letter delta, namely, u'\x03b04', with both 'cp1252' and 'iso8859-7' encoding (the latter being inferred from the same Wikipedia article) and here's what I get:
>
>>>> b = np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>>>> print b.encode('cp1252')[0]
> ?
>>>> print b.capitalize().encode('cp1252')[0]
> ?
>>>> print b.encode('iso8859-7')[0]
> ?
>>>> print b.capitalize().encode('iso8859-7')[0]
> ?
>
> i.e., no difference. ?If I'm doing something wrong, please let me know; otherwise, for the purpose of documenting chararray.capitalize() - which is my ultimate goal - is there any rhyme or reason behind which unicode characters capitalize() works on and which it doesn't?
>
> Thanks,
>
> DG
> --- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>
>> actually this works (in Idle)
>>
>> >>> b =
>> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>> >>> print b.encode('cp1252')[0]
>> ?
>> >>> print b.capitalize().encode('cp1252')[0]
>> ?
>> >>> print b[0].encode('cp1252')
>> ?
>>
>>
>> this looks like a bug ? or is it a known limitation that
>> chararrays
>> cannot be 0-d
>>
>> >>> b0=
>> np.array(u'\xe9','<U1').view(np.chararray)
>> >>> print b0.encode('cp1252')
>> Traceback (most recent call last):
>> ? File "<pyshell#47>", line 1, in
>> <module>
>> ? ? print b0.encode('cp1252')
>> ? File
>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>> line 217, in encode
>> ? ? return self._generalmethod('encode',
>> broadcast(self, encoding, errors))
>> ? File
>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>> line 162, in _generalmethod
>> ? ? newarr[:] = res
>> ValueError: cannot slice a 0-d array
>>
>>
>> >
>> > Josef
>> >
>> >>>
>> >>> Unless the answer is "No," my real question:
>> >>>
>> >>> 1) Does chararray.capitalize() capitalize
>> non-Roman letters
>> >>> that have different lower-case and upper-case
>> forms (e.g.,
>> >>> the Greek letters)?? If "yes," are there any
>> exceptions
>> >>> (e.g., Russian letters)?

I think yes, exceptions are languages for which no capital letters
exist, Cantonese(Chinese) ?
http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
  ??? google search for 03B04,

>> >>>
>> >>> Thanks!
>> >>>
>> >>> DG
>> >>>
>> >>>

I have problems finding the correct codes for the characters and
usually need a word processor.

To me it looks like your character is not a greek delta

>>> print u'\x03b04'
b04
>>> print u'\u03b04'
?4
>>> print u'\u03b4'
?

I don't know what it is since it doesn't render to anything meaningful

I managed to get the greek delta through the html code for it &#948; from page:
http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4


running this script:


# -*- coding: utf-8 -*-

sd = u'?'
print sd

b = np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
print b[0]
print repr(b[0])
print b.capitalize()[0]
print repr(b.capitalize()[0])

***********
prints this in my Idle shell
>>>
?
?
u'\u03b4'
?
u'\u0394'

delta is correctly capitalized


Josef


From josef.pktd at gmail.com  Tue Aug 11 23:40:24 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 23:40:24 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908112018x35546e80u1e42d18548dd659a@mail.gmail.com>
References: <877871.14415.qm@web52104.mail.re2.yahoo.com>
	<1cd32cbb0908112018x35546e80u1e42d18548dd659a@mail.gmail.com>
Message-ID: <1cd32cbb0908112040j78c28f04nb07bdf786252305@mail.gmail.com>

On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com> wrote:
> On Tue, Aug 11, 2009 at 10:28 PM, David
> Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>> Thanks, Josef. ?This may just be an artifact of working in a DOS Terminal (but your example, though not printing the accented e, did at least print something different for b vs. b.capitalize()), or it may be because I don't know the right encoding to use, but I tried your code w/ what I found on Wikipedia to be the unicode for the Greek letter delta, namely, u'\x03b04', with both 'cp1252' and 'iso8859-7' encoding (the latter being inferred from the same Wikipedia article) and here's what I get:
>>
>>>>> b = np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>>>>> print b.encode('cp1252')[0]
>> ?
>>>>> print b.capitalize().encode('cp1252')[0]
>> ?
>>>>> print b.encode('iso8859-7')[0]
>> ?
>>>>> print b.capitalize().encode('iso8859-7')[0]
>> ?
>>
>> i.e., no difference. ?If I'm doing something wrong, please let me know; otherwise, for the purpose of documenting chararray.capitalize() - which is my ultimate goal - is there any rhyme or reason behind which unicode characters capitalize() works on and which it doesn't?
>>
>> Thanks,
>>
>> DG
>> --- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>>
>>> actually this works (in Idle)
>>>
>>> >>> b =
>>> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>>> >>> print b.encode('cp1252')[0]
>>> ?
>>> >>> print b.capitalize().encode('cp1252')[0]
>>> ?
>>> >>> print b[0].encode('cp1252')
>>> ?
>>>
>>>
>>> this looks like a bug ? or is it a known limitation that
>>> chararrays
>>> cannot be 0-d
>>>
>>> >>> b0=
>>> np.array(u'\xe9','<U1').view(np.chararray)
>>> >>> print b0.encode('cp1252')
>>> Traceback (most recent call last):
>>> ? File "<pyshell#47>", line 1, in
>>> <module>
>>> ? ? print b0.encode('cp1252')
>>> ? File
>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>> line 217, in encode
>>> ? ? return self._generalmethod('encode',
>>> broadcast(self, encoding, errors))
>>> ? File
>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>> line 162, in _generalmethod
>>> ? ? newarr[:] = res
>>> ValueError: cannot slice a 0-d array
>>>
>>>
>>> >
>>> > Josef
>>> >
>>> >>>
>>> >>> Unless the answer is "No," my real question:
>>> >>>
>>> >>> 1) Does chararray.capitalize() capitalize
>>> non-Roman letters
>>> >>> that have different lower-case and upper-case
>>> forms (e.g.,
>>> >>> the Greek letters)?? If "yes," are there any
>>> exceptions
>>> >>> (e.g., Russian letters)?
>
> I think yes, exceptions are languages for which no capital letters
> exist, Cantonese(Chinese) ?
> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
> ???? google search for 03B04,
>
>>> >>>
>>> >>> Thanks!
>>> >>>
>>> >>> DG
>>> >>>
>>> >>>
>
> I have problems finding the correct codes for the characters and
> usually need a word processor.
>
> To me it looks like your character is not a greek delta
>
>>>> print u'\x03b04'
> ?b04
>>>> print u'\u03b04'
> ?4
>>>> print u'\u03b4'
> ?
>
> I don't know what it is since it doesn't render to anything meaningful
>
> I managed to get the greek delta through the html code for it &#948; from page:
> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
>
>
> running this script:
>
>
> # -*- coding: utf-8 -*-
>
> sd = u'?'
> print sd
>
> b = np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
> print b[0]
> print repr(b[0])
> print b.capitalize()[0]
> print repr(b.capitalize()[0])
>
> ***********
> prints this in my Idle shell
>>>>
> ?
> ?
> u'\u03b4'
> ?
> u'\u0394'
>
> delta is correctly capitalized
>
>
> Josef
>


trying without copy and past non-Ascii characters
the page at
http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4

also has the utf8 code \xCE\xB4,  everything looks ok starting from this.

Josef

>>> '\xCE\xB4'.decode('utf8')
u'\u03b4'
>>> print '\xCE\xB4'.decode('utf8')
?
>>> print '\xCE\xB4'.decode('utf8').capitalize()
?
>>> b = np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
>>> b
chararray([u'\u03b4', u'\u03b4'],
      dtype='<U1')
>>> print b[0]
?
>>> print b.capitalize()[0]
?


From josef.pktd at gmail.com  Tue Aug 11 23:59:59 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 11 Aug 2009 23:59:59 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908112040j78c28f04nb07bdf786252305@mail.gmail.com>
References: <877871.14415.qm@web52104.mail.re2.yahoo.com>
	<1cd32cbb0908112018x35546e80u1e42d18548dd659a@mail.gmail.com>
	<1cd32cbb0908112040j78c28f04nb07bdf786252305@mail.gmail.com>
Message-ID: <1cd32cbb0908112059t1d1c2edfnca30e3ccf63c7f3c@mail.gmail.com>

On Tue, Aug 11, 2009 at 11:40 PM, <josef.pktd at gmail.com> wrote:
> On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com> wrote:
>> On Tue, Aug 11, 2009 at 10:28 PM, David
>> Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>>> Thanks, Josef. ?This may just be an artifact of working in a DOS Terminal (but your example, though not printing the accented e, did at least print something different for b vs. b.capitalize()), or it may be because I don't know the right encoding to use, but I tried your code w/ what I found on Wikipedia to be the unicode for the Greek letter delta, namely, u'\x03b04', with both 'cp1252' and 'iso8859-7' encoding (the latter being inferred from the same Wikipedia article) and here's what I get:
>>>
>>>>>> b = np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>>>>>> print b.encode('cp1252')[0]
>>> ?
>>>>>> print b.capitalize().encode('cp1252')[0]
>>> ?
>>>>>> print b.encode('iso8859-7')[0]
>>> ?
>>>>>> print b.capitalize().encode('iso8859-7')[0]
>>> ?
>>>
>>> i.e., no difference. ?If I'm doing something wrong, please let me know; otherwise, for the purpose of documenting chararray.capitalize() - which is my ultimate goal - is there any rhyme or reason behind which unicode characters capitalize() works on and which it doesn't?
>>>
>>> Thanks,
>>>
>>> DG
>>> --- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>>>
>>>> actually this works (in Idle)
>>>>
>>>> >>> b =
>>>> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>>>> >>> print b.encode('cp1252')[0]
>>>> ?
>>>> >>> print b.capitalize().encode('cp1252')[0]
>>>> ?
>>>> >>> print b[0].encode('cp1252')
>>>> ?
>>>>
>>>>
>>>> this looks like a bug ? or is it a known limitation that
>>>> chararrays
>>>> cannot be 0-d
>>>>
>>>> >>> b0=
>>>> np.array(u'\xe9','<U1').view(np.chararray)
>>>> >>> print b0.encode('cp1252')
>>>> Traceback (most recent call last):
>>>> ? File "<pyshell#47>", line 1, in
>>>> <module>
>>>> ? ? print b0.encode('cp1252')
>>>> ? File
>>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>>> line 217, in encode
>>>> ? ? return self._generalmethod('encode',
>>>> broadcast(self, encoding, errors))
>>>> ? File
>>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>>> line 162, in _generalmethod
>>>> ? ? newarr[:] = res
>>>> ValueError: cannot slice a 0-d array
>>>>
>>>>
>>>> >
>>>> > Josef
>>>> >
>>>> >>>
>>>> >>> Unless the answer is "No," my real question:
>>>> >>>
>>>> >>> 1) Does chararray.capitalize() capitalize
>>>> non-Roman letters
>>>> >>> that have different lower-case and upper-case
>>>> forms (e.g.,
>>>> >>> the Greek letters)?? If "yes," are there any
>>>> exceptions
>>>> >>> (e.g., Russian letters)?
>>
>> I think yes, exceptions are languages for which no capital letters
>> exist, Cantonese(Chinese) ?
>> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
>> ???? google search for 03B04,
>>
>>>> >>>
>>>> >>> Thanks!
>>>> >>>
>>>> >>> DG
>>>> >>>
>>>> >>>
>>
>> I have problems finding the correct codes for the characters and
>> usually need a word processor.
>>
>> To me it looks like your character is not a greek delta
>>
>>>>> print u'\x03b04'
>> ?b04
>>>>> print u'\u03b04'
>> ?4
>>>>> print u'\u03b4'
>> ?
>>
>> I don't know what it is since it doesn't render to anything meaningful
>>
>> I managed to get the greek delta through the html code for it &#948; from page:
>> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
>>
>>
>> running this script:
>>
>>
>> # -*- coding: utf-8 -*-
>>
>> sd = u'?'
>> print sd
>>
>> b = np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
>> print b[0]
>> print repr(b[0])
>> print b.capitalize()[0]
>> print repr(b.capitalize()[0])
>>
>> ***********
>> prints this in my Idle shell
>>>>>
>> ?
>> ?
>> u'\u03b4'
>> ?
>> u'\u0394'
>>
>> delta is correctly capitalized
>>
>>
>> Josef
>>
>
>
> trying without copy and past non-Ascii characters
> the page at
> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
>
> also has the utf8 code \xCE\xB4, ?everything looks ok starting from this.
>
> Josef
>
>>>> '\xCE\xB4'.decode('utf8')
> u'\u03b4'
>>>> print '\xCE\xB4'.decode('utf8')
> ?
>>>> print '\xCE\xB4'.decode('utf8').capitalize()
> ?
>>>> b = np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
>>>> b
> chararray([u'\u03b4', u'\u03b4'],
> ? ? ?dtype='<U1')
>>>> print b[0]
> ?
>>>> print b.capitalize()[0]
> ?
>

and for the fun of it,
a Russian (cyrillic) character that capitalizes

>>> print '\xD0\xB9'.decode('utf8')
?
>>> print '\xD0\xB9'.decode('utf8').capitalize()
?
>>> '\xD0\xB9'.decode('utf8')
u'\u0439'
>>> '\xD0\xB9'.decode('utf8').capitalize()
u'\u0419'


and a german letter that doesn't have a capitalized version

>>> print '\xC3\x9F'.decode('utf8').capitalize()
?
>>> print '\xC3\x9F'.decode('utf8')
?
>>> '\xC3\x9F'.decode('utf8')
u'\xdf'
>>> '\xC3\x9F'.decode('utf8').capitalize()
u'\xdf'

and here's a nice picture of unicode 03B04
http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04

and here are all unicode characters (although my browser doesn't
display most of them)
http://www.isthisthingon.org/unicode/allchars1.php


I hope this helps,

Josef


From josef.pktd at gmail.com  Wed Aug 12 00:18:58 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 12 Aug 2009 00:18:58 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908112059t1d1c2edfnca30e3ccf63c7f3c@mail.gmail.com>
References: <877871.14415.qm@web52104.mail.re2.yahoo.com>
	<1cd32cbb0908112018x35546e80u1e42d18548dd659a@mail.gmail.com>
	<1cd32cbb0908112040j78c28f04nb07bdf786252305@mail.gmail.com>
	<1cd32cbb0908112059t1d1c2edfnca30e3ccf63c7f3c@mail.gmail.com>
Message-ID: <1cd32cbb0908112118i2ea11905i40b8f644cfa8bd4b@mail.gmail.com>

On Tue, Aug 11, 2009 at 11:59 PM, <josef.pktd at gmail.com> wrote:
> On Tue, Aug 11, 2009 at 11:40 PM, <josef.pktd at gmail.com> wrote:
>> On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com> wrote:
>>> On Tue, Aug 11, 2009 at 10:28 PM, David
>>> Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>>>> Thanks, Josef. ?This may just be an artifact of working in a DOS Terminal (but your example, though not printing the accented e, did at least print something different for b vs. b.capitalize()), or it may be because I don't know the right encoding to use, but I tried your code w/ what I found on Wikipedia to be the unicode for the Greek letter delta, namely, u'\x03b04', with both 'cp1252' and 'iso8859-7' encoding (the latter being inferred from the same Wikipedia article) and here's what I get:
>>>>
>>>>>>> b = np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>>>>>>> print b.encode('cp1252')[0]
>>>> ?
>>>>>>> print b.capitalize().encode('cp1252')[0]
>>>> ?
>>>>>>> print b.encode('iso8859-7')[0]
>>>> ?
>>>>>>> print b.capitalize().encode('iso8859-7')[0]
>>>> ?
>>>>
>>>> i.e., no difference. ?If I'm doing something wrong, please let me know; otherwise, for the purpose of documenting chararray.capitalize() - which is my ultimate goal - is there any rhyme or reason behind which unicode characters capitalize() works on and which it doesn't?
>>>>
>>>> Thanks,
>>>>
>>>> DG
>>>> --- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>>>>
>>>>> actually this works (in Idle)
>>>>>
>>>>> >>> b =
>>>>> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>>>>> >>> print b.encode('cp1252')[0]
>>>>> ?
>>>>> >>> print b.capitalize().encode('cp1252')[0]
>>>>> ?
>>>>> >>> print b[0].encode('cp1252')
>>>>> ?
>>>>>
>>>>>
>>>>> this looks like a bug ? or is it a known limitation that
>>>>> chararrays
>>>>> cannot be 0-d
>>>>>
>>>>> >>> b0=
>>>>> np.array(u'\xe9','<U1').view(np.chararray)
>>>>> >>> print b0.encode('cp1252')
>>>>> Traceback (most recent call last):
>>>>> ? File "<pyshell#47>", line 1, in
>>>>> <module>
>>>>> ? ? print b0.encode('cp1252')
>>>>> ? File
>>>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>>>> line 217, in encode
>>>>> ? ? return self._generalmethod('encode',
>>>>> broadcast(self, encoding, errors))
>>>>> ? File
>>>>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>>>>> line 162, in _generalmethod
>>>>> ? ? newarr[:] = res
>>>>> ValueError: cannot slice a 0-d array
>>>>>
>>>>>
>>>>> >
>>>>> > Josef
>>>>> >
>>>>> >>>
>>>>> >>> Unless the answer is "No," my real question:
>>>>> >>>
>>>>> >>> 1) Does chararray.capitalize() capitalize
>>>>> non-Roman letters
>>>>> >>> that have different lower-case and upper-case
>>>>> forms (e.g.,
>>>>> >>> the Greek letters)?? If "yes," are there any
>>>>> exceptions
>>>>> >>> (e.g., Russian letters)?
>>>
>>> I think yes, exceptions are languages for which no capital letters
>>> exist, Cantonese(Chinese) ?
>>> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
>>> ???? google search for 03B04,
>>>
>>>>> >>>
>>>>> >>> Thanks!
>>>>> >>>
>>>>> >>> DG
>>>>> >>>
>>>>> >>>
>>>
>>> I have problems finding the correct codes for the characters and
>>> usually need a word processor.
>>>
>>> To me it looks like your character is not a greek delta
>>>
>>>>>> print u'\x03b04'
>>> ?b04
>>>>>> print u'\u03b04'
>>> ?4
>>>>>> print u'\u03b4'
>>> ?
>>>
>>> I don't know what it is since it doesn't render to anything meaningful
>>>
>>> I managed to get the greek delta through the html code for it &#948; from page:
>>> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
>>>
>>>
>>> running this script:
>>>
>>>
>>> # -*- coding: utf-8 -*-
>>>
>>> sd = u'?'
>>> print sd
>>>
>>> b = np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
>>> print b[0]
>>> print repr(b[0])
>>> print b.capitalize()[0]
>>> print repr(b.capitalize()[0])
>>>
>>> ***********
>>> prints this in my Idle shell
>>>>>>
>>> ?
>>> ?
>>> u'\u03b4'
>>> ?
>>> u'\u0394'
>>>
>>> delta is correctly capitalized
>>>
>>>
>>> Josef
>>>
>>
>>
>> trying without copy and past non-Ascii characters
>> the page at
>> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
>>
>> also has the utf8 code \xCE\xB4, ?everything looks ok starting from this.
>>
>> Josef
>>
>>>>> '\xCE\xB4'.decode('utf8')
>> u'\u03b4'
>>>>> print '\xCE\xB4'.decode('utf8')
>> ?
>>>>> print '\xCE\xB4'.decode('utf8').capitalize()
>> ?
>>>>> b = np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
>>>>> b
>> chararray([u'\u03b4', u'\u03b4'],
>> ? ? ?dtype='<U1')
>>>>> print b[0]
>> ?
>>>>> print b.capitalize()[0]
>> ?
>>
>
> and for the fun of it,
> a Russian (cyrillic) character that capitalizes
>
>>>> print '\xD0\xB9'.decode('utf8')
> ?
>>>> print '\xD0\xB9'.decode('utf8').capitalize()
> ?
>>>> '\xD0\xB9'.decode('utf8')
> u'\u0439'
>>>> '\xD0\xB9'.decode('utf8').capitalize()
> u'\u0419'
>
>
> and a german letter that doesn't have a capitalized version
>
>>>> print '\xC3\x9F'.decode('utf8').capitalize()
> ?
>>>> print '\xC3\x9F'.decode('utf8')
> ?
>>>> '\xC3\x9F'.decode('utf8')
> u'\xdf'
>>>> '\xC3\x9F'.decode('utf8').capitalize()
> u'\xdf'
>
> and here's a nice picture of unicode 03B04
> http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04
>
> and here are all unicode characters (although my browser doesn't
> display most of them)
> http://www.isthisthingon.org/unicode/allchars1.php
>
>
> I hope this helps,
>
> Josef
>

and then there is also

>>> b = np.array([u'\u03b4\u03b4', u'\u03b4\u03b4'],'<U2').view(np.chararray)
>>> print b.capitalize()
[u'\u0394\u03b4' u'\u0394\u03b4']
>>> print b.capitalize()[0]
??
>>> print b.upper()[0]
??
>>> print b.upper().lower()[0]
??
>>> print b.title()[0]
??

that's enough fun for the night

Josef


From d_l_goldsmith at yahoo.com  Wed Aug 12 01:45:17 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 11 Aug 2009 22:45:17 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908112059t1d1c2edfnca30e3ccf63c7f3c@mail.gmail.com>
Message-ID: <941048.92622.qm@web52101.mail.re2.yahoo.com>

Actually, since you seem so into it ;-) can you write me a little script (just 'cause it seems like you could do it faster) to print all the unicode characters u such that u == u.capitalize()?

DG

--- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 11, 2009, 8:59 PM
> On Tue, Aug 11, 2009 at 11:40 PM,
> <josef.pktd at gmail.com>
> wrote:
> > On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com>
> wrote:
> >> On Tue, Aug 11, 2009 at 10:28 PM, David
> >> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> >>> Thanks, Josef. ?This may just be an artifact
> of working in a DOS Terminal (but your example, though not
> printing the accented e, did at least print something
> different for b vs. b.capitalize()), or it may be because I
> don't know the right encoding to use, but I tried your code
> w/ what I found on Wikipedia to be the unicode for the Greek
> letter delta, namely, u'\x03b04', with both 'cp1252' and
> 'iso8859-7' encoding (the latter being inferred from the
> same Wikipedia article) and here's what I get:
> >>>
> >>>>>> b =
> np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
> >>>>>> print b.encode('cp1252')[0]
> >>> ?
> >>>>>> print
> b.capitalize().encode('cp1252')[0]
> >>> ?
> >>>>>> print b.encode('iso8859-7')[0]
> >>> ?
> >>>>>> print
> b.capitalize().encode('iso8859-7')[0]
> >>> ?
> >>>
> >>> i.e., no difference. ?If I'm doing something
> wrong, please let me know; otherwise, for the purpose of
> documenting chararray.capitalize() - which is my ultimate
> goal - is there any rhyme or reason behind which unicode
> characters capitalize() works on and which it doesn't?
> >>>
> >>> Thanks,
> >>>
> >>> DG
> >>> --- On Tue, 8/11/09, josef.pktd at gmail.com
> <josef.pktd at gmail.com>
> wrote:
> >>>
> >>>> actually this works (in Idle)
> >>>>
> >>>> >>> b =
> >>>>
> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
> >>>> >>> print b.encode('cp1252')[0]
> >>>> ?
> >>>> >>> print
> b.capitalize().encode('cp1252')[0]
> >>>> ?
> >>>> >>> print b[0].encode('cp1252')
> >>>> ?
> >>>>
> >>>>
> >>>> this looks like a bug ? or is it a known
> limitation that
> >>>> chararrays
> >>>> cannot be 0-d
> >>>>
> >>>> >>> b0=
> >>>>
> np.array(u'\xe9','<U1').view(np.chararray)
> >>>> >>> print b0.encode('cp1252')
> >>>> Traceback (most recent call last):
> >>>> ? File "<pyshell#47>", line 1, in
> >>>> <module>
> >>>> ? ? print b0.encode('cp1252')
> >>>> ? File
> >>>>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >>>> line 217, in encode
> >>>> ? ? return
> self._generalmethod('encode',
> >>>> broadcast(self, encoding, errors))
> >>>> ? File
> >>>>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >>>> line 162, in _generalmethod
> >>>> ? ? newarr[:] = res
> >>>> ValueError: cannot slice a 0-d array
> >>>>
> >>>>
> >>>> >
> >>>> > Josef
> >>>> >
> >>>> >>>
> >>>> >>> Unless the answer is "No," my
> real question:
> >>>> >>>
> >>>> >>> 1) Does
> chararray.capitalize() capitalize
> >>>> non-Roman letters
> >>>> >>> that have different
> lower-case and upper-case
> >>>> forms (e.g.,
> >>>> >>> the Greek letters)?? If
> "yes," are there any
> >>>> exceptions
> >>>> >>> (e.g., Russian letters)?
> >>
> >> I think yes, exceptions are languages for which no
> capital letters
> >> exist, Cantonese(Chinese) ?
> >> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
> >> ???? google search for 03B04,
> >>
> >>>> >>>
> >>>> >>> Thanks!
> >>>> >>>
> >>>> >>> DG
> >>>> >>>
> >>>> >>>
> >>
> >> I have problems finding the correct codes for the
> characters and
> >> usually need a word processor.
> >>
> >> To me it looks like your character is not a greek
> delta
> >>
> >>>>> print u'\x03b04'
> >> ?b04
> >>>>> print u'\u03b04'
> >> ?4
> >>>>> print u'\u03b4'
> >> ?
> >>
> >> I don't know what it is since it doesn't render to
> anything meaningful
> >>
> >> I managed to get the greek delta through the html
> code for it ? from page:
> >> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
> >>
> >>
> >> running this script:
> >>
> >>
> >> # -*- coding: utf-8 -*-
> >>
> >> sd = u'?'
> >> print sd
> >>
> >> b =
> np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
> >> print b[0]
> >> print repr(b[0])
> >> print b.capitalize()[0]
> >> print repr(b.capitalize()[0])
> >>
> >> ***********
> >> prints this in my Idle shell
> >>>>>
> >> ?
> >> ?
> >> u'\u03b4'
> >> ?
> >> u'\u0394'
> >>
> >> delta is correctly capitalized
> >>
> >>
> >> Josef
> >>
> >
> >
> > trying without copy and past non-Ascii characters
> > the page at
> > http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
> >
> > also has the utf8 code \xCE\xB4, ?everything looks ok
> starting from this.
> >
> > Josef
> >
> >>>> '\xCE\xB4'.decode('utf8')
> > u'\u03b4'
> >>>> print '\xCE\xB4'.decode('utf8')
> > ?
> >>>> print
> '\xCE\xB4'.decode('utf8').capitalize()
> > ?
> >>>> b =
> np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
> >>>> b
> > chararray([u'\u03b4', u'\u03b4'],
> > ? ? ?dtype='<U1')
> >>>> print b[0]
> > ?
> >>>> print b.capitalize()[0]
> > ?
> >
> 
> and for the fun of it,
> a Russian (cyrillic) character that capitalizes
> 
> >>> print '\xD0\xB9'.decode('utf8')
> ?
> >>> print '\xD0\xB9'.decode('utf8').capitalize()
> ?
> >>> '\xD0\xB9'.decode('utf8')
> u'\u0439'
> >>> '\xD0\xB9'.decode('utf8').capitalize()
> u'\u0419'
> 
> 
> and a german letter that doesn't have a capitalized
> version
> 
> >>> print '\xC3\x9F'.decode('utf8').capitalize()
> ?
> >>> print '\xC3\x9F'.decode('utf8')
> ?
> >>> '\xC3\x9F'.decode('utf8')
> u'\xdf'
> >>> '\xC3\x9F'.decode('utf8').capitalize()
> u'\xdf'
> 
> and here's a nice picture of unicode 03B04
> http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04
> 
> and here are all unicode characters (although my browser
> doesn't
> display most of them)
> http://www.isthisthingon.org/unicode/allchars1.php
> 
> 
> I hope this helps,
> 
> Josef
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Wed Aug 12 09:54:09 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 12 Aug 2009 09:54:09 -0400
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <941048.92622.qm@web52101.mail.re2.yahoo.com>
References: <1cd32cbb0908112059t1d1c2edfnca30e3ccf63c7f3c@mail.gmail.com>
	<941048.92622.qm@web52101.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908120654j29577d1dmb9fa207efbbc2f1a@mail.gmail.com>

On Wed, Aug 12, 2009 at 1:45 AM, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> Actually, since you seem so into it ;-)
This was just a refresher, I struggled much more the first time I
tried to use non-english filenames and files.

> can you write me a little script (just 'cause it seems like you could do it faster) to print all the unicode characters u such >that u == u.capitalize()?

u == u.capitalize()    that' s for most of them,
The webpage lists 89,674 unicode characters and I didn't want to try
all of them.

Below are the unicode characters in the first 1000 for which u != u.capitalize()

josef

-----------------------------

print unichr(30)

maxcode = 1000 # I don't want to try 38000
start = 0  # 38000  is boring
umany = np.array([unichr(i) for i in xrange(start,start+maxcode)],
                 '<U1').view(np.chararray)

capmask = (umany != umany.capitalize())
umanycap = umany[capmask]


print umany
print capmask
print '%2.2f percent differ in capitalize' %
(np.sum(capmask)/float(len(umany))*100)

for i in xrange(len(umanycap)):
    try:
        print umanycap[i],
    except:
        print "\n%r doesn't print" % umanycap[i]
--------------

>
> DG
>
> --- On Tue, 8/11/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>
>> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
>> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
>> To: "SciPy Developers List" <scipy-dev at scipy.org>
>> Date: Tuesday, August 11, 2009, 8:59 PM
>> On Tue, Aug 11, 2009 at 11:40 PM,
>> <josef.pktd at gmail.com>
>> wrote:
>> > On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com>
>> wrote:
>> >> On Tue, Aug 11, 2009 at 10:28 PM, David
>> >> Goldsmith<d_l_goldsmith at yahoo.com>
>> wrote:
>> >>> Thanks, Josef. ?This may just be an artifact
>> of working in a DOS Terminal (but your example, though not
>> printing the accented e, did at least print something
>> different for b vs. b.capitalize()), or it may be because I
>> don't know the right encoding to use, but I tried your code
>> w/ what I found on Wikipedia to be the unicode for the Greek
>> letter delta, namely, u'\x03b04', with both 'cp1252' and
>> 'iso8859-7' encoding (the latter being inferred from the
>> same Wikipedia article) and here's what I get:
>> >>>
>> >>>>>> b =
>> np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
>> >>>>>> print b.encode('cp1252')[0]
>> >>> ?
>> >>>>>> print
>> b.capitalize().encode('cp1252')[0]
>> >>> ?
>> >>>>>> print b.encode('iso8859-7')[0]
>> >>> ?
>> >>>>>> print
>> b.capitalize().encode('iso8859-7')[0]
>> >>> ?
>> >>>
>> >>> i.e., no difference. ?If I'm doing something
>> wrong, please let me know; otherwise, for the purpose of
>> documenting chararray.capitalize() - which is my ultimate
>> goal - is there any rhyme or reason behind which unicode
>> characters capitalize() works on and which it doesn't?
>> >>>
>> >>> Thanks,
>> >>>
>> >>> DG
>> >>> --- On Tue, 8/11/09, josef.pktd at gmail.com
>> <josef.pktd at gmail.com>
>> wrote:
>> >>>
>> >>>> actually this works (in Idle)
>> >>>>
>> >>>> >>> b =
>> >>>>
>> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
>> >>>> >>> print b.encode('cp1252')[0]
>> >>>> ?
>> >>>> >>> print
>> b.capitalize().encode('cp1252')[0]
>> >>>> ?
>> >>>> >>> print b[0].encode('cp1252')
>> >>>> ?
>> >>>>
>> >>>>
>> >>>> this looks like a bug ? or is it a known
>> limitation that
>> >>>> chararrays
>> >>>> cannot be 0-d
>> >>>>
>> >>>> >>> b0=
>> >>>>
>> np.array(u'\xe9','<U1').view(np.chararray)
>> >>>> >>> print b0.encode('cp1252')
>> >>>> Traceback (most recent call last):
>> >>>> ? File "<pyshell#47>", line 1, in
>> >>>> <module>
>> >>>> ? ? print b0.encode('cp1252')
>> >>>> ? File
>> >>>>
>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>> >>>> line 217, in encode
>> >>>> ? ? return
>> self._generalmethod('encode',
>> >>>> broadcast(self, encoding, errors))
>> >>>> ? File
>> >>>>
>> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
>> >>>> line 162, in _generalmethod
>> >>>> ? ? newarr[:] = res
>> >>>> ValueError: cannot slice a 0-d array
>> >>>>
>> >>>>
>> >>>> >
>> >>>> > Josef
>> >>>> >
>> >>>> >>>
>> >>>> >>> Unless the answer is "No," my
>> real question:
>> >>>> >>>
>> >>>> >>> 1) Does
>> chararray.capitalize() capitalize
>> >>>> non-Roman letters
>> >>>> >>> that have different
>> lower-case and upper-case
>> >>>> forms (e.g.,
>> >>>> >>> the Greek letters)?? If
>> "yes," are there any
>> >>>> exceptions
>> >>>> >>> (e.g., Russian letters)?
>> >>
>> >> I think yes, exceptions are languages for which no
>> capital letters
>> >> exist, Cantonese(Chinese) ?
>> >> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
>> >> ???? google search for 03B04,
>> >>
>> >>>> >>>
>> >>>> >>> Thanks!
>> >>>> >>>
>> >>>> >>> DG
>> >>>> >>>
>> >>>> >>>
>> >>
>> >> I have problems finding the correct codes for the
>> characters and
>> >> usually need a word processor.
>> >>
>> >> To me it looks like your character is not a greek
>> delta
>> >>
>> >>>>> print u'\x03b04'
>> >> ?b04
>> >>>>> print u'\u03b04'
>> >> ?4
>> >>>>> print u'\u03b4'
>> >> ?
>> >>
>> >> I don't know what it is since it doesn't render to
>> anything meaningful
>> >>
>> >> I managed to get the greek delta through the html
>> code for it ? from page:
>> >> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
>> >>
>> >>
>> >> running this script:
>> >>
>> >>
>> >> # -*- coding: utf-8 -*-
>> >>
>> >> sd = u'?'
>> >> print sd
>> >>
>> >> b =
>> np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
>> >> print b[0]
>> >> print repr(b[0])
>> >> print b.capitalize()[0]
>> >> print repr(b.capitalize()[0])
>> >>
>> >> ***********
>> >> prints this in my Idle shell
>> >>>>>
>> >> ?
>> >> ?
>> >> u'\u03b4'
>> >> ?
>> >> u'\u0394'
>> >>
>> >> delta is correctly capitalized
>> >>
>> >>
>> >> Josef
>> >>
>> >
>> >
>> > trying without copy and past non-Ascii characters
>> > the page at
>> > http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
>> >
>> > also has the utf8 code \xCE\xB4, ?everything looks ok
>> starting from this.
>> >
>> > Josef
>> >
>> >>>> '\xCE\xB4'.decode('utf8')
>> > u'\u03b4'
>> >>>> print '\xCE\xB4'.decode('utf8')
>> > ?
>> >>>> print
>> '\xCE\xB4'.decode('utf8').capitalize()
>> > ?
>> >>>> b =
>> np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
>> >>>> b
>> > chararray([u'\u03b4', u'\u03b4'],
>> > ? ? ?dtype='<U1')
>> >>>> print b[0]
>> > ?
>> >>>> print b.capitalize()[0]
>> > ?
>> >
>>
>> and for the fun of it,
>> a Russian (cyrillic) character that capitalizes
>>
>> >>> print '\xD0\xB9'.decode('utf8')
>> ?
>> >>> print '\xD0\xB9'.decode('utf8').capitalize()
>> ?
>> >>> '\xD0\xB9'.decode('utf8')
>> u'\u0439'
>> >>> '\xD0\xB9'.decode('utf8').capitalize()
>> u'\u0419'
>>
>>
>> and a german letter that doesn't have a capitalized
>> version
>>
>> >>> print '\xC3\x9F'.decode('utf8').capitalize()
>> ?
>> >>> print '\xC3\x9F'.decode('utf8')
>> ?
>> >>> '\xC3\x9F'.decode('utf8')
>> u'\xdf'
>> >>> '\xC3\x9F'.decode('utf8').capitalize()
>> u'\xdf'
>>
>> and here's a nice picture of unicode 03B04
>> http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04
>>
>> and here are all unicode characters (although my browser
>> doesn't
>> display most of them)
>> http://www.isthisthingon.org/unicode/allchars1.php
>>
>>
>> I hope this helps,
>>
>> Josef
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Wed Aug 12 10:47:54 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 12 Aug 2009 07:47:54 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908120654j29577d1dmb9fa207efbbc2f1a@mail.gmail.com>
Message-ID: <546271.90032.qm@web52109.mail.re2.yahoo.com>

Awesome, thanks!

DG

--- On Wed, 8/12/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Wednesday, August 12, 2009, 6:54 AM
> On Wed, Aug 12, 2009 at 1:45 AM,
> David Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > Actually, since you seem so into it ;-)
> This was just a refresher, I struggled much more the first
> time I
> tried to use non-english filenames and files.
> 
> > can you write me a little script (just 'cause it seems
> like you could do it faster) to print all the unicode
> characters u such >that u == u.capitalize()?
> 
> u == u.capitalize()? ? that' s for most of them,
> The webpage lists 89,674 unicode characters and I didn't
> want to try
> all of them.
> 
> Below are the unicode characters in the first 1000 for
> which u != u.capitalize()
> 
> josef
> 
> -----------------------------
> 
> print unichr(30)
> 
> maxcode = 1000 # I don't want to try 38000
> start = 0? # 38000? is boring
> umany = np.array([unichr(i) for i in
> xrange(start,start+maxcode)],
> ? ? ? ? ? ? ?
> ???'<U1').view(np.chararray)
> 
> capmask = (umany != umany.capitalize())
> umanycap = umany[capmask]
> 
> 
> print umany
> print capmask
> print '%2.2f percent differ in capitalize' %
> (np.sum(capmask)/float(len(umany))*100)
> 
> for i in xrange(len(umanycap)):
> ? ? try:
> ? ? ? ? print umanycap[i],
> ? ? except:
> ? ? ? ? print "\n%r doesn't print" %
> umanycap[i]
> --------------
> 
> >
> > DG
> >
> > --- On Tue, 8/11/09, josef.pktd at gmail.com
> <josef.pktd at gmail.com>
> wrote:
> >
> >> From: josef.pktd at gmail.com
> <josef.pktd at gmail.com>
> >> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy
> unicode support
> >> To: "SciPy Developers List" <scipy-dev at scipy.org>
> >> Date: Tuesday, August 11, 2009, 8:59 PM
> >> On Tue, Aug 11, 2009 at 11:40 PM,
> >> <josef.pktd at gmail.com>
> >> wrote:
> >> > On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com>
> >> wrote:
> >> >> On Tue, Aug 11, 2009 at 10:28 PM, David
> >> >> Goldsmith<d_l_goldsmith at yahoo.com>
> >> wrote:
> >> >>> Thanks, Josef. ?This may just be an
> artifact
> >> of working in a DOS Terminal (but your example,
> though not
> >> printing the accented e, did at least print
> something
> >> different for b vs. b.capitalize()), or it may be
> because I
> >> don't know the right encoding to use, but I tried
> your code
> >> w/ what I found on Wikipedia to be the unicode for
> the Greek
> >> letter delta, namely, u'\x03b04', with both
> 'cp1252' and
> >> 'iso8859-7' encoding (the latter being inferred
> from the
> >> same Wikipedia article) and here's what I get:
> >> >>>
> >> >>>>>> b =
> >>
> np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
> >> >>>>>> print
> b.encode('cp1252')[0]
> >> >>> ?
> >> >>>>>> print
> >> b.capitalize().encode('cp1252')[0]
> >> >>> ?
> >> >>>>>> print
> b.encode('iso8859-7')[0]
> >> >>> ?
> >> >>>>>> print
> >> b.capitalize().encode('iso8859-7')[0]
> >> >>> ?
> >> >>>
> >> >>> i.e., no difference. ?If I'm doing
> something
> >> wrong, please let me know; otherwise, for the
> purpose of
> >> documenting chararray.capitalize() - which is my
> ultimate
> >> goal - is there any rhyme or reason behind which
> unicode
> >> characters capitalize() works on and which it
> doesn't?
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>> DG
> >> >>> --- On Tue, 8/11/09, josef.pktd at gmail.com
> >> <josef.pktd at gmail.com>
> >> wrote:
> >> >>>
> >> >>>> actually this works (in Idle)
> >> >>>>
> >> >>>> >>> b =
> >> >>>>
> >>
> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
> >> >>>> >>> print
> b.encode('cp1252')[0]
> >> >>>> ?
> >> >>>> >>> print
> >> b.capitalize().encode('cp1252')[0]
> >> >>>> ?
> >> >>>> >>> print
> b[0].encode('cp1252')
> >> >>>> ?
> >> >>>>
> >> >>>>
> >> >>>> this looks like a bug ? or is it
> a known
> >> limitation that
> >> >>>> chararrays
> >> >>>> cannot be 0-d
> >> >>>>
> >> >>>> >>> b0=
> >> >>>>
> >> np.array(u'\xe9','<U1').view(np.chararray)
> >> >>>> >>> print
> b0.encode('cp1252')
> >> >>>> Traceback (most recent call
> last):
> >> >>>> ? File "<pyshell#47>",
> line 1, in
> >> >>>> <module>
> >> >>>> ? ? print b0.encode('cp1252')
> >> >>>> ? File
> >> >>>>
> >>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >> >>>> line 217, in encode
> >> >>>> ? ? return
> >> self._generalmethod('encode',
> >> >>>> broadcast(self, encoding,
> errors))
> >> >>>> ? File
> >> >>>>
> >>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >> >>>> line 162, in _generalmethod
> >> >>>> ? ? newarr[:] = res
> >> >>>> ValueError: cannot slice a 0-d
> array
> >> >>>>
> >> >>>>
> >> >>>> >
> >> >>>> > Josef
> >> >>>> >
> >> >>>> >>>
> >> >>>> >>> Unless the answer is
> "No," my
> >> real question:
> >> >>>> >>>
> >> >>>> >>> 1) Does
> >> chararray.capitalize() capitalize
> >> >>>> non-Roman letters
> >> >>>> >>> that have different
> >> lower-case and upper-case
> >> >>>> forms (e.g.,
> >> >>>> >>> the Greek
> letters)?? If
> >> "yes," are there any
> >> >>>> exceptions
> >> >>>> >>> (e.g., Russian
> letters)?
> >> >>
> >> >> I think yes, exceptions are languages for
> which no
> >> capital letters
> >> >> exist, Cantonese(Chinese) ?
> >> >> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
> >> >> ???? google search for 03B04,
> >> >>
> >> >>>> >>>
> >> >>>> >>> Thanks!
> >> >>>> >>>
> >> >>>> >>> DG
> >> >>>> >>>
> >> >>>> >>>
> >> >>
> >> >> I have problems finding the correct codes
> for the
> >> characters and
> >> >> usually need a word processor.
> >> >>
> >> >> To me it looks like your character is not
> a greek
> >> delta
> >> >>
> >> >>>>> print u'\x03b04'
> >> >> ?b04
> >> >>>>> print u'\u03b04'
> >> >> ?4
> >> >>>>> print u'\u03b4'
> >> >> ?
> >> >>
> >> >> I don't know what it is since it doesn't
> render to
> >> anything meaningful
> >> >>
> >> >> I managed to get the greek delta through
> the html
> >> code for it ? from page:
> >> >> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
> >> >>
> >> >>
> >> >> running this script:
> >> >>
> >> >>
> >> >> # -*- coding: utf-8 -*-
> >> >>
> >> >> sd = u'?'
> >> >> print sd
> >> >>
> >> >> b =
> >>
> np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
> >> >> print b[0]
> >> >> print repr(b[0])
> >> >> print b.capitalize()[0]
> >> >> print repr(b.capitalize()[0])
> >> >>
> >> >> ***********
> >> >> prints this in my Idle shell
> >> >>>>>
> >> >> ?
> >> >> ?
> >> >> u'\u03b4'
> >> >> ?
> >> >> u'\u0394'
> >> >>
> >> >> delta is correctly capitalized
> >> >>
> >> >>
> >> >> Josef
> >> >>
> >> >
> >> >
> >> > trying without copy and past non-Ascii
> characters
> >> > the page at
> >> > http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
> >> >
> >> > also has the utf8 code \xCE\xB4, ?everything
> looks ok
> >> starting from this.
> >> >
> >> > Josef
> >> >
> >> >>>> '\xCE\xB4'.decode('utf8')
> >> > u'\u03b4'
> >> >>>> print '\xCE\xB4'.decode('utf8')
> >> > ?
> >> >>>> print
> >> '\xCE\xB4'.decode('utf8').capitalize()
> >> > ?
> >> >>>> b =
> >>
> np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
> >> >>>> b
> >> > chararray([u'\u03b4', u'\u03b4'],
> >> > ? ? ?dtype='<U1')
> >> >>>> print b[0]
> >> > ?
> >> >>>> print b.capitalize()[0]
> >> > ?
> >> >
> >>
> >> and for the fun of it,
> >> a Russian (cyrillic) character that capitalizes
> >>
> >> >>> print '\xD0\xB9'.decode('utf8')
> >> ?
> >> >>> print
> '\xD0\xB9'.decode('utf8').capitalize()
> >> ?
> >> >>> '\xD0\xB9'.decode('utf8')
> >> u'\u0439'
> >> >>>
> '\xD0\xB9'.decode('utf8').capitalize()
> >> u'\u0419'
> >>
> >>
> >> and a german letter that doesn't have a
> capitalized
> >> version
> >>
> >> >>> print
> '\xC3\x9F'.decode('utf8').capitalize()
> >> ?
> >> >>> print '\xC3\x9F'.decode('utf8')
> >> ?
> >> >>> '\xC3\x9F'.decode('utf8')
> >> u'\xdf'
> >> >>>
> '\xC3\x9F'.decode('utf8').capitalize()
> >> u'\xdf'
> >>
> >> and here's a nice picture of unicode 03B04
> >> http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04
> >>
> >> and here are all unicode characters (although my
> browser
> >> doesn't
> >> display most of them)
> >> http://www.isthisthingon.org/unicode/allchars1.php
> >>
> >>
> >> I hope this helps,
> >>
> >> Josef
> >> _______________________________________________
> >> Scipy-dev mailing list
> >> Scipy-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>
> >
> >
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Wed Aug 12 11:23:19 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 12 Aug 2009 08:23:19 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <1cd32cbb0908120654j29577d1dmb9fa207efbbc2f1a@mail.gmail.com>
Message-ID: <962671.76050.qm@web52102.mail.re2.yahoo.com>

--- On Wed, 8/12/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> David Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > Actually, since you seem so into it ;-)
> This was just a refresher, I struggled much more the first
> time I
> tried to use non-english filenames and files.
> 
> > can you write me a little script (just 'cause it seems
> like you could do it faster) to print all the unicode
> characters u such >that u == u.capitalize()?
> 
> u == u.capitalize()? ? that' s for most of them,
> The webpage lists 89,674 unicode characters and I didn't
> want to try
> all of them.

Thanks again: I guess I did a "narrow Python build" (must be the msi default) cause I had to restrict maxcode to 16**4, but doing so gives umanycap.size = 946.  "u == u.capitalize()    that' s for most of them," indeed!

DG

> 
> Below are the unicode characters in the first 1000 for
> which u != u.capitalize()
> 
> josef
> 
> -----------------------------
> 
> print unichr(30)
> 
> maxcode = 1000 # I don't want to try 38000
> start = 0? # 38000? is boring
> umany = np.array([unichr(i) for i in
> xrange(start,start+maxcode)],
> ? ? ? ? ? ? ?
> ???'<U1').view(np.chararray)
> 
> capmask = (umany != umany.capitalize())
> umanycap = umany[capmask]
> 
> 
> print umany
> print capmask
> print '%2.2f percent differ in capitalize' %
> (np.sum(capmask)/float(len(umany))*100)
> 
> for i in xrange(len(umanycap)):
> ? ? try:
> ? ? ? ? print umanycap[i],
> ? ? except:
> ? ? ? ? print "\n%r doesn't print" %
> umanycap[i]
> --------------
> 
> >
> > DG
> >
> > --- On Tue, 8/11/09, josef.pktd at gmail.com
> <josef.pktd at gmail.com>
> wrote:
> >
> >> From: josef.pktd at gmail.com
> <josef.pktd at gmail.com>
> >> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy
> unicode support
> >> To: "SciPy Developers List" <scipy-dev at scipy.org>
> >> Date: Tuesday, August 11, 2009, 8:59 PM
> >> On Tue, Aug 11, 2009 at 11:40 PM,
> >> <josef.pktd at gmail.com>
> >> wrote:
> >> > On Tue, Aug 11, 2009 at 11:18 PM, <josef.pktd at gmail.com>
> >> wrote:
> >> >> On Tue, Aug 11, 2009 at 10:28 PM, David
> >> >> Goldsmith<d_l_goldsmith at yahoo.com>
> >> wrote:
> >> >>> Thanks, Josef. ?This may just be an
> artifact
> >> of working in a DOS Terminal (but your example,
> though not
> >> printing the accented e, did at least print
> something
> >> different for b vs. b.capitalize()), or it may be
> because I
> >> don't know the right encoding to use, but I tried
> your code
> >> w/ what I found on Wikipedia to be the unicode for
> the Greek
> >> letter delta, namely, u'\x03b04', with both
> 'cp1252' and
> >> 'iso8859-7' encoding (the latter being inferred
> from the
> >> same Wikipedia article) and here's what I get:
> >> >>>
> >> >>>>>> b =
> >>
> np.array([u'\x03b04',u'\x03b04'],'<U1').view(np.chararray)
> >> >>>>>> print
> b.encode('cp1252')[0]
> >> >>> ?
> >> >>>>>> print
> >> b.capitalize().encode('cp1252')[0]
> >> >>> ?
> >> >>>>>> print
> b.encode('iso8859-7')[0]
> >> >>> ?
> >> >>>>>> print
> >> b.capitalize().encode('iso8859-7')[0]
> >> >>> ?
> >> >>>
> >> >>> i.e., no difference. ?If I'm doing
> something
> >> wrong, please let me know; otherwise, for the
> purpose of
> >> documenting chararray.capitalize() - which is my
> ultimate
> >> goal - is there any rhyme or reason behind which
> unicode
> >> characters capitalize() works on and which it
> doesn't?
> >> >>>
> >> >>> Thanks,
> >> >>>
> >> >>> DG
> >> >>> --- On Tue, 8/11/09, josef.pktd at gmail.com
> >> <josef.pktd at gmail.com>
> >> wrote:
> >> >>>
> >> >>>> actually this works (in Idle)
> >> >>>>
> >> >>>> >>> b =
> >> >>>>
> >>
> np.array([u'\xe9',u'\xe9'],'<U1').view(np.chararray)
> >> >>>> >>> print
> b.encode('cp1252')[0]
> >> >>>> ?
> >> >>>> >>> print
> >> b.capitalize().encode('cp1252')[0]
> >> >>>> ?
> >> >>>> >>> print
> b[0].encode('cp1252')
> >> >>>> ?
> >> >>>>
> >> >>>>
> >> >>>> this looks like a bug ? or is it
> a known
> >> limitation that
> >> >>>> chararrays
> >> >>>> cannot be 0-d
> >> >>>>
> >> >>>> >>> b0=
> >> >>>>
> >> np.array(u'\xe9','<U1').view(np.chararray)
> >> >>>> >>> print
> b0.encode('cp1252')
> >> >>>> Traceback (most recent call
> last):
> >> >>>> ? File "<pyshell#47>",
> line 1, in
> >> >>>> <module>
> >> >>>> ? ? print b0.encode('cp1252')
> >> >>>> ? File
> >> >>>>
> >>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >> >>>> line 217, in encode
> >> >>>> ? ? return
> >> self._generalmethod('encode',
> >> >>>> broadcast(self, encoding,
> errors))
> >> >>>> ? File
> >> >>>>
> >>
> "C:\Programs\Python25\Lib\site-packages\numpy\core\defchararray.py",
> >> >>>> line 162, in _generalmethod
> >> >>>> ? ? newarr[:] = res
> >> >>>> ValueError: cannot slice a 0-d
> array
> >> >>>>
> >> >>>>
> >> >>>> >
> >> >>>> > Josef
> >> >>>> >
> >> >>>> >>>
> >> >>>> >>> Unless the answer is
> "No," my
> >> real question:
> >> >>>> >>>
> >> >>>> >>> 1) Does
> >> chararray.capitalize() capitalize
> >> >>>> non-Roman letters
> >> >>>> >>> that have different
> >> lower-case and upper-case
> >> >>>> forms (e.g.,
> >> >>>> >>> the Greek
> letters)?? If
> >> "yes," are there any
> >> >>>> exceptions
> >> >>>> >>> (e.g., Russian
> letters)?
> >> >>
> >> >> I think yes, exceptions are languages for
> which no
> >> capital letters
> >> >> exist, Cantonese(Chinese) ?
> >> >> http://www.isthisthingon.org/unicode/index.phtml?page=03&subpage=B&glyph=03B04
> >> >> ???? google search for 03B04,
> >> >>
> >> >>>> >>>
> >> >>>> >>> Thanks!
> >> >>>> >>>
> >> >>>> >>> DG
> >> >>>> >>>
> >> >>>> >>>
> >> >>
> >> >> I have problems finding the correct codes
> for the
> >> characters and
> >> >> usually need a word processor.
> >> >>
> >> >> To me it looks like your character is not
> a greek
> >> delta
> >> >>
> >> >>>>> print u'\x03b04'
> >> >> ?b04
> >> >>>>> print u'\u03b04'
> >> >> ?4
> >> >>>>> print u'\u03b4'
> >> >> ?
> >> >>
> >> >> I don't know what it is since it doesn't
> render to
> >> anything meaningful
> >> >>
> >> >> I managed to get the greek delta through
> the html
> >> code for it ? from page:
> >> >> http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&hilite=003B4
> >> >>
> >> >>
> >> >> running this script:
> >> >>
> >> >>
> >> >> # -*- coding: utf-8 -*-
> >> >>
> >> >> sd = u'?'
> >> >> print sd
> >> >>
> >> >> b =
> >>
> np.array([u'\u03b4',u'\u0394'],'<U1').view(np.chararray)
> >> >> print b[0]
> >> >> print repr(b[0])
> >> >> print b.capitalize()[0]
> >> >> print repr(b.capitalize()[0])
> >> >>
> >> >> ***********
> >> >> prints this in my Idle shell
> >> >>>>>
> >> >> ?
> >> >> ?
> >> >> u'\u03b4'
> >> >> ?
> >> >> u'\u0394'
> >> >>
> >> >> delta is correctly capitalized
> >> >>
> >> >>
> >> >> Josef
> >> >>
> >> >
> >> >
> >> > trying without copy and past non-Ascii
> characters
> >> > the page at
> >> > http://www.isthisthingon.org/unicode/index.phtml?page=00&subpage=3&glyph=003B4
> >> >
> >> > also has the utf8 code \xCE\xB4, ?everything
> looks ok
> >> starting from this.
> >> >
> >> > Josef
> >> >
> >> >>>> '\xCE\xB4'.decode('utf8')
> >> > u'\u03b4'
> >> >>>> print '\xCE\xB4'.decode('utf8')
> >> > ?
> >> >>>> print
> >> '\xCE\xB4'.decode('utf8').capitalize()
> >> > ?
> >> >>>> b =
> >>
> np.array(['\xCE\xB4'.decode('utf8'),'\xCE\xB4'.decode('utf8')],'<U1').view(np.chararray)
> >> >>>> b
> >> > chararray([u'\u03b4', u'\u03b4'],
> >> > ? ? ?dtype='<U1')
> >> >>>> print b[0]
> >> > ?
> >> >>>> print b.capitalize()[0]
> >> > ?
> >> >
> >>
> >> and for the fun of it,
> >> a Russian (cyrillic) character that capitalizes
> >>
> >> >>> print '\xD0\xB9'.decode('utf8')
> >> ?
> >> >>> print
> '\xD0\xB9'.decode('utf8').capitalize()
> >> ?
> >> >>> '\xD0\xB9'.decode('utf8')
> >> u'\u0439'
> >> >>>
> '\xD0\xB9'.decode('utf8').capitalize()
> >> u'\u0419'
> >>
> >>
> >> and a german letter that doesn't have a
> capitalized
> >> version
> >>
> >> >>> print
> '\xC3\x9F'.decode('utf8').capitalize()
> >> ?
> >> >>> print '\xC3\x9F'.decode('utf8')
> >> ?
> >> >>> '\xC3\x9F'.decode('utf8')
> >> u'\xdf'
> >> >>>
> '\xC3\x9F'.decode('utf8').capitalize()
> >> u'\xdf'
> >>
> >> and here's a nice picture of unicode 03B04
> >> http://www.cns11643.gov.tw/seeker/english/showfont.jsp?ucode=03B04
> >>
> >> and here are all unicode characters (although my
> browser
> >> doesn't
> >> display most of them)
> >> http://www.isthisthingon.org/unicode/allchars1.php
> >>
> >>
> >> I hope this helps,
> >>
> >> Josef
> >> _______________________________________________
> >> Scipy-dev mailing list
> >> Scipy-dev at scipy.org
> >> http://mail.scipy.org/mailman/listinfo/scipy-dev
> >>
> >
> >
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Wed Aug 12 12:03:09 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 12 Aug 2009 09:03:09 -0700 (PDT)
Subject: [SciPy-dev] Skypecon happening now
Message-ID: <31619.33443.qm@web52108.mail.re2.yahoo.com>

Text me if you want in.

DG


      


From robert.kern at gmail.com  Wed Aug 12 12:10:12 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 12 Aug 2009 11:10:12 -0500
Subject: [SciPy-dev] Where is numpy.generic defined?
In-Reply-To: <843554.18671.qm@web52105.mail.re2.yahoo.com>
References: <843554.18671.qm@web52105.mail.re2.yahoo.com>
Message-ID: <3d375d730908120910r681d788ehdbbed2c6a89f48d0@mail.gmail.com>

In the generated C code. It's just the abstract base class for the scalar types.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Wed Aug 12 12:45:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 12 Aug 2009 09:45:20 -0700 (PDT)
Subject: [SciPy-dev] Where is numpy.generic defined?
In-Reply-To: <3d375d730908120910r681d788ehdbbed2c6a89f48d0@mail.gmail.com>
Message-ID: <244549.61722.qm@web52108.mail.re2.yahoo.com>

Thanks, Robert.

"It's just the abstract base class for the scalar types."

I understood that; so, are it's attributes "empty" virtuals "waiting" to be overridden?  If yes, are there any exceptions (i.e., "substantive" attributes that should actually have non-trivial docstrings)?  Indeed, is it your opinion that all attributes of the generic namespace should actually be classified as "Unimportant" (w.r.t. to the doc Wiki)?

Thanks again,

DG

--- On Wed, 8/12/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] Where is numpy.generic defined?
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Wednesday, August 12, 2009, 9:10 AM
> In the generated C code. It's just
> the abstract base class for the scalar types.
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From gael.varoquaux at normalesup.org  Thu Aug 13 05:58:22 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 13 Aug 2009 11:58:22 +0200
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>
References: <734509.78574.qm@web52101.mail.re2.yahoo.com>
	<48C01AE7354EC240A26F19CEB995E943033AF265@CHMAILMBX01.novachem.com>
Message-ID: <20090813095822.GG30160@phare.normalesup.org>

On Tue, Aug 11, 2009 at 02:08:30PM -0600, Andrew Hawryluk wrote:
> Each year I train a new set of engineering internship students in our
> department, and one of the things they learn is Python.  Upon arriving
> here they have taken a single course on C++ and another on
> Matlab/Mathematica/Excel etc., but no Python.  I usually start them off
> reading "Dive Into Python", but I'd also like a single place to send
> them to get a brief tour of proper NumPy use.  The main NumPy & SciPy
> docs are excellent (and improving rapidly), but they start a bit too
> abruptly for my needs. For example, the NumPy User Guide currently start
> with installation instructions, and then the next page of body text is a
> table of all the available array types.

> I would like something that briefly explains what NumPy and SciPy are,
> and why/when arrays are better than lists/dicts, perhaps followed by a
> brief tour of some common NumPy tricks, before diving into the more
> detailed sections.

> One possible table of contents would be

> Introduction
>   What is NumPy?
>   Building/Installing
>   Short Tour
>   How to find documentation
> NumPy Basics
>   ...

> What direction were you thinking the User Guide would take?

Hey Andrew,

Good documentation, including documentation targetting newcomers, as you
point out, is paramount. I have also found that it is a tremendous amount
of work, because it requires reworking and revisiting the complete set of
available documentation all the time, to make sure that you have a
consistent set, that fits together in a narrative way, but also that
drive the reader to where he wants quickly.

I would very much like the numpy user guide, that you can edit on the
docwiki, to be that documentation. The reason being that it is a shared
project (it is easy for people to edit it), and thus it is easier to work
together and gather momentuml, that it is the 'official' documentation,
and thus will get more exposure, and that it is stored internally in a
format (sphinx) that makes it possible to produce an online doc as well
as a printed doc.

So I would say: just go ahead, and make your proposed changes in the
docwiki. They seem very sensible. If you have any problems, or want
review, ask on the mailing list. I say ask, I could say 'yell'. People
are all struggling with day-to-day life, deadlines, and many projects, so
you will probably not get as much review as you would like. However, in
my experience, such contributions grow the available material and end up
making it better all the time.

One thing that we have to be careful about, is not to repeat ourselves
too much in the documentation. If we do, the documentation can become a
huge maze that noones read. I believe the answer to this problem is to
cross link a lot, and to try to improve existing articles, when
available.

Thanks a lot for your interest,

Ga?l



From gael.varoquaux at normalesup.org  Thu Aug 13 13:52:42 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 13 Aug 2009 19:52:42 +0200
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF264@CHMAILMBX01.novachem.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF264@CHMAILMBX01.novachem.com>
Message-ID: <20090813175242.GR4037@phare.normalesup.org>

On Tue, Aug 11, 2009 at 01:10:39PM -0600, Andrew Hawryluk wrote:
> I'd like to help with the docs project. I have registered a user name of
> ahawryluk on the Wiki.
> Although the current focus is cleaning up the docstrings on the numpy
> module, I'm most interested in writing some introductory material for
> the NumPy User Guide.

Hey Andrew,

I have added you to the editor list, you can now edit docs on the
docwiki. Sorry for the slow reply, I have been on holidays lately.

Ga?l


From d_l_goldsmith at yahoo.com  Thu Aug 13 15:56:04 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Thu, 13 Aug 2009 12:56:04 -0700 (PDT)
Subject: [SciPy-dev] Summer Marathon "Wrap-up" BoF
Message-ID: <986988.71834.qm@web52108.mail.re2.yahoo.com>

Hi!  So, one thing that came out of yesterday's Skypecon was the idea for a SciPyCon BoF for "wrapping-up" the Summer Marathon; if we were to have that next Wed. at 19:00 UTC (noon local time), i.e., at the "regularly scheduled" Skypecon time, would there be anyone who would want to participate but couldn't because of the selection of that time?

DG

PS: I'd try to arrange it so that it is a Skypecon as well as a live face-to-face.


      


From HAWRYLA at novachem.com  Thu Aug 13 16:34:40 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 13 Aug 2009 14:34:40 -0600
Subject: [SciPy-dev] can I help with the docs?
In-Reply-To: <20090813175242.GR4037@phare.normalesup.org>
References: <48C01AE7354EC240A26F19CEB995E943033AF264@CHMAILMBX01.novachem.com>
	<20090813175242.GR4037@phare.normalesup.org>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF272@CHMAILMBX01.novachem.com>

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of Gael Varoquaux
> Sent: 13 Aug 2009 11:53 AM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] can I help with the docs?
> 
> Hey Andrew,
> 
> I have added you to the editor list, you can now edit docs on the
> docwiki. Sorry for the slow reply, I have been on holidays lately.
> 
> Ga?l

No problem, and welcome back. Thanks for the encouragement on the docs!

Andrew


From dwf at cs.toronto.edu  Thu Aug 13 18:32:15 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Thu, 13 Aug 2009 18:32:15 -0400
Subject: [SciPy-dev] Sphinx-in-LaTeX do's and don'ts
Message-ID: <9F997718-45B8-4528-87D7-2114C1F07B27@cs.toronto.edu>

I looked at the scipy.maxentropy docstring on the doc site and  
realized that that equation should really be typeset. The trouble is  
that doing so properly leads to some pretty unreadable LaTeX.

As a compromise, I added the "plaintext" version of the equation in a  
reST comment above the LaTeX:

	http://docs.scipy.org/scipy/docs/scipy.maxentropy/

do people think this is a reasonable compromise?

David


From josef.pktd at gmail.com  Thu Aug 13 23:15:42 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 13 Aug 2009 23:15:42 -0400
Subject: [SciPy-dev] Sphinx-in-LaTeX do's and don'ts
In-Reply-To: <9F997718-45B8-4528-87D7-2114C1F07B27@cs.toronto.edu>
References: <9F997718-45B8-4528-87D7-2114C1F07B27@cs.toronto.edu>
Message-ID: <1cd32cbb0908132015j1e570ca5re8bd0968c2efdb6d@mail.gmail.com>

On Thu, Aug 13, 2009 at 6:32 PM, David Warde-Farley<dwf at cs.toronto.edu> wrote:
> I looked at the scipy.maxentropy docstring on the doc site and
> realized that that equation should really be typeset. The trouble is
> that doing so properly leads to some pretty unreadable LaTeX.
>
> As a compromise, I added the "plaintext" version of the equation in a
> reST comment above the LaTeX:
>
> ? ? ? ?http://docs.scipy.org/scipy/docs/scipy.maxentropy/
>
> do people think this is a reasonable compromise?
>
> David
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

Do you really need all the fancy latex in a docstring? It seems all
you need is a dot/inner product, and the over arrow for vectors is not
really necessary.
Your type setting looks good, but for a docstring it seems overkill.
In econometrics/statistics for example  X^T X is perfectly understood
as a matrix/dot product if you know that X are matrices or2-d arrays.

Actually, with the typesetting of the equation (transpose and arrow) I
start to worry that I misunderstand the dimension in the definition.
If theta is a 1d array and f is a vector of functions with the same
number of elements then \theta \dot \f should be clear enough to
indicate the innerproduct.
In analogy, this looks very much like a multinomial logit with the
vector of functions f replacing the role of explanatory variables X.

p(y=j|X) = exp(X_j beta)/sum_i{exp(X_i beta)}   where X_j and beta are
both 1-d vectors with same length.

Similarly in the kernel literature in machine learning, I have seen
mostly a simple notation for inner products to represent a linear
combination of feature or kernel functions.

My recommendation would be to keep the latex in the docstrings simple
and readable. Latex in rst files for more extended explanations are a
different story.


as a BTW:

When I tried to figure out what the maxentropy subpackage is doing, I
found it quite impossible from a quick reading of the docs, tests and
examples. The examples are good but don't explain why it works, as far
as I remember.

Recently, I stumbled upon two introductory references, where I
understood for the first time a little bit of the theory behind
maxentropy, especially the connection to maximum likelihood
estimation.

http://www.stat.washington.edu/courses/stat592/winter04/public_html/handouts.html

and
http://www.cs.cmu.edu/afs/cs/user/aberger/www/maxent.html
especially
http://www.cs.cmu.edu/afs/cs/user/aberger/www/ps/compling.ps


Sorry for the longer than intended mail, but I'm always happy when a
(statistics/machine learning related) package, that I thought might be
orphaned, gets some attention.

Josef


From pav at iki.fi  Fri Aug 14 03:46:26 2009
From: pav at iki.fi (Pauli Virtanen)
Date: Fri, 14 Aug 2009 07:46:26 +0000 (UTC)
Subject: [SciPy-dev] Summer Marathon "Wrap-up" BoF
References: <986988.71834.qm@web52108.mail.re2.yahoo.com>
Message-ID: <h634oi$b21$1@ger.gmane.org>

Hi David,

Thu, 13 Aug 2009 12:56:04 -0700, David Goldsmith kirjoitti:
> Hi!  So, one thing that came out of yesterday's Skypecon was the idea
> for a SciPyCon BoF for "wrapping-up" the Summer Marathon; if we were to
> have that next Wed. at 19:00 UTC (noon local time), i.e., at the
> "regularly scheduled" Skypecon time, would there be anyone who would
> want to participate but couldn't because of the selection of that time?

I was just about to propose this same BOF. I think since many people who 
have been involved in this are present there, this is indeed a good 
opportunity for a BOF. I went forward and added it to http://scipy.org/
SciPy2009/BoF, marking you tentatively as the coordinator. Please change 
it as you see fit.

I also added a few topics that might be of interest:

 * Wrapping up this summer's marathon.
 * What next: Scipy? User guides?
 * Drafting a TOC for the Numpy User Guide?

There are probably more. If we run out, we can start building a bike shed 
and paint it with different colors (each bring your own! :)

	Pauli



From d_l_goldsmith at yahoo.com  Fri Aug 14 15:07:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 12:07:20 -0700 (PDT)
Subject: [SciPy-dev] Summer Marathon "Wrap-up" BoF
In-Reply-To: <h634oi$b21$1@ger.gmane.org>
Message-ID: <213173.19652.qm@web52108.mail.re2.yahoo.com>

Perfect, Pauli, thanks!  I confess to not yet having read the BoF Wiki: are there "instructions" there for how to arrange for a location?

DG

--- On Fri, 8/14/09, Pauli Virtanen <pav at iki.fi> wrote:

> From: Pauli Virtanen <pav at iki.fi>
> Subject: Re: [SciPy-dev] Summer Marathon "Wrap-up" BoF
> To: scipy-dev at scipy.org
> Date: Friday, August 14, 2009, 12:46 AM
> Hi David,
> 
> Thu, 13 Aug 2009 12:56:04 -0700, David Goldsmith
> kirjoitti:
> > Hi!? So, one thing that came out of yesterday's
> Skypecon was the idea
> > for a SciPyCon BoF for "wrapping-up" the Summer
> Marathon; if we were to
> > have that next Wed. at 19:00 UTC (noon local time),
> i.e., at the
> > "regularly scheduled" Skypecon time, would there be
> anyone who would
> > want to participate but couldn't because of the
> selection of that time?
> 
> I was just about to propose this same BOF. I think since
> many people who 
> have been involved in this are present there, this is
> indeed a good 
> opportunity for a BOF. I went forward and added it to http://scipy.org/
> SciPy2009/BoF, marking you tentatively as the coordinator.
> Please change 
> it as you see fit.
> 
> I also added a few topics that might be of interest:
> 
>  * Wrapping up this summer's marathon.
>  * What next: Scipy? User guides?
>  * Drafting a TOC for the Numpy User Guide?
> 
> There are probably more. If we run out, we can start
> building a bike shed 
> and paint it with different colors (each bring your own!
> :)
> 
> ??? Pauli
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Fri Aug 14 15:36:21 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 12:36:21 -0700 (PDT)
Subject: [SciPy-dev] Can't login at BoF Wiki
Message-ID: <52058.18205.qm@web52106.mail.re2.yahoo.com>

Hi!  I can't login at the BoF Wiki: evidently, I can't remember my pwd, and when I try to use the form to get it sent to me, it says my email is not on file (I tried all three that I have); I thought maybe I hadn't established an account there yet, but it says my name is already registered (is there another SciPy David Goldsmith - wouldn't surprise me, actually)?  Please help!

DG


      


From gael.varoquaux at normalesup.org  Fri Aug 14 16:44:09 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 14 Aug 2009 22:44:09 +0200
Subject: [SciPy-dev] Can't login at BoF Wiki
In-Reply-To: <52058.18205.qm@web52106.mail.re2.yahoo.com>
References: <52058.18205.qm@web52106.mail.re2.yahoo.com>
Message-ID: <20090814204409.GB7131@phare.normalesup.org>

On Fri, Aug 14, 2009 at 12:36:21PM -0700, David Goldsmith wrote:
> Hi!  I can't login at the BoF Wiki: evidently, I can't remember my pwd, and when I try to use the form to get it sent to me, it says my email is not on file (I tried all three that I have); I thought maybe I hadn't established an account there yet, but it says my name is already registered (is there another SciPy David Goldsmith - wouldn't surprise me, actually)?  Please help!

Which wiki are we talking about (url). I don't have control on all wikis,
and if it is the moin, unless some of the few people who know have time,
it is a lost cause.

Ga?l


From d_l_goldsmith at yahoo.com  Fri Aug 14 16:51:53 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 13:51:53 -0700 (PDT)
Subject: [SciPy-dev] Can't login at BoF Wiki
In-Reply-To: <20090814204409.GB7131@phare.normalesup.org>
Message-ID: <767657.75044.qm@web52110.mail.re2.yahoo.com>

http://scipy.org/SciPy2009/BoF

DG

--- On Fri, 8/14/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Can't login at BoF Wiki
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 14, 2009, 1:44 PM
> On Fri, Aug 14, 2009 at 12:36:21PM
> -0700, David Goldsmith wrote:
> > Hi!? I can't login at the BoF Wiki: evidently, I
> can't remember my pwd, and when I try to use the form to get
> it sent to me, it says my email is not on file (I tried all
> three that I have); I thought maybe I hadn't established an
> account there yet, but it says my name is already registered
> (is there another SciPy David Goldsmith - wouldn't surprise
> me, actually)?? Please help!
> 
> Which wiki are we talking about (url). I don't have control
> on all wikis,
> and if it is the moin, unless some of the few people who
> know have time,
> it is a lost cause.
> 
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From dwf at cs.toronto.edu  Fri Aug 14 16:52:37 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Fri, 14 Aug 2009 20:52:37 -0000
Subject: [SciPy-dev] Can't login at BoF Wiki
In-Reply-To: <20090814204409.GB7131@phare.normalesup.org>
References: <52058.18205.qm@web52106.mail.re2.yahoo.com>
	<20090814204409.GB7131@phare.normalesup.org>
Message-ID: <CFB4C43A-C101-477E-B9F4-A53BB5060A01@cs.toronto.edu>

He means the wiki page on Scipy.org. Pauli cleared up the spam word  
issue so I guess he has appropriate access?

DWF

On 14-Aug-09, at 4:44 PM, Gael Varoquaux  
<gael.varoquaux at normalesup.org> wrote:

> On Fri, Aug 14, 2009 at 12:36:21PM -0700, David Goldsmith wrote:
>> Hi!  I can't login at the BoF Wiki: evidently, I can't remember my  
>> pwd, and when I try to use the form to get it sent to me, it says  
>> my email is not on file (I tried all three that I have); I thought  
>> maybe I hadn't established an account there yet, but it says my  
>> name is already registered (is there another SciPy David Goldsmith  
>> - wouldn't surprise me, actually)?  Please help!
>
> Which wiki are we talking about (url). I don't have control on all  
> wikis,
> and if it is the moin, unless some of the few people who know have  
> time,
> it is a lost cause.
>
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From d_l_goldsmith at yahoo.com  Fri Aug 14 17:02:04 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 14:02:04 -0700 (PDT)
Subject: [SciPy-dev] Ground xport to/from airport
Message-ID: <97035.89451.qm@web52105.mail.re2.yahoo.com>

Hi!? Kind of late for me to be asking, I know, but any advice on ("public") ground transportation to/from LAX (I'm staying at the Vagabond), e.g., cab fare, group shuttle, or city transit, would be appreciated.? Thanks!

DG

PS: I'm arriving Monday @ 2:32pm and departing Sunday @ 8:10pm in case anyone arriving/departing similar times can give me a ride...



      


From gael.varoquaux at normalesup.org  Fri Aug 14 17:05:41 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 14 Aug 2009 23:05:41 +0200
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <97035.89451.qm@web52105.mail.re2.yahoo.com>
References: <97035.89451.qm@web52105.mail.re2.yahoo.com>
Message-ID: <20090814210541.GA15669@phare.normalesup.org>

On Fri, Aug 14, 2009 at 02:02:04PM -0700, David Goldsmith wrote:
> Hi!? Kind of late for me to be asking, I know, but any advice on
> ("public") ground transportation to/from LAX (I'm staying at the
> Vagabond), e.g., cab fare, group shuttle, or city transit, would be
> appreciated.? Thanks!

I tend to take the shuttle to Union station, and than the metro to Del
Mar station.

G.


From d_l_goldsmith at yahoo.com  Fri Aug 14 17:16:56 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 14:16:56 -0700 (PDT)
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <20090814210541.GA15669@phare.normalesup.org>
Message-ID: <576503.15078.qm@web52109.mail.re2.yahoo.com>

Thanks, Gael - for ref., how much ($) does that run?

DG

--- On Fri, 8/14/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Ground xport to/from airport
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 14, 2009, 2:05 PM
> On Fri, Aug 14, 2009 at 02:02:04PM
> -0700, David Goldsmith wrote:
> > Hi!? Kind of late for me to be asking, I know, but
> any advice on
> > ("public") ground transportation to/from LAX (I'm
> staying at the
> > Vagabond), e.g., cab fare, group shuttle, or city
> transit, would be
> > appreciated.? Thanks!
> 
> I tend to take the shuttle to Union station, and than the
> metro to Del
> Mar station.
> 
> G.
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From gael.varoquaux at normalesup.org  Fri Aug 14 17:18:33 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 14 Aug 2009 23:18:33 +0200
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <576503.15078.qm@web52109.mail.re2.yahoo.com>
References: <20090814210541.GA15669@phare.normalesup.org>
	<576503.15078.qm@web52109.mail.re2.yahoo.com>
Message-ID: <20090814211833.GB15669@phare.normalesup.org>

On Fri, Aug 14, 2009 at 02:16:56PM -0700, David Goldsmith wrote:
> Thanks, Gael - for ref., how much ($) does that run?

I would say $20 from airport to Union, and less then $10 to Del Mar.

G.


From pgmdevlist at gmail.com  Fri Aug 14 17:19:57 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Fri, 14 Aug 2009 17:19:57 -0400
Subject: [SciPy-dev] GLMs ?
Message-ID: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>

All,
What's the status of GLMs in scipy ? Does anybody have some code they  
could point me to ?
Thanks a lot in advance
P.


From robert.kern at gmail.com  Fri Aug 14 17:22:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Aug 2009 16:22:19 -0500
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <20090814211833.GB15669@phare.normalesup.org>
References: <20090814210541.GA15669@phare.normalesup.org>
	<576503.15078.qm@web52109.mail.re2.yahoo.com> 
	<20090814211833.GB15669@phare.normalesup.org>
Message-ID: <3d375d730908141422x79cdb8a0xf9e90402d7095dde@mail.gmail.com>

On Fri, Aug 14, 2009 at 16:18, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Fri, Aug 14, 2009 at 02:16:56PM -0700, David Goldsmith wrote:
>> Thanks, Gael - for ref., how much ($) does that run?
>
> I would say $20 from airport to Union, and less then $10 to Del Mar.

This will be useful to people:

  http://transitguide.caltech.edu/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Fri Aug 14 17:24:00 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 14 Aug 2009 16:24:00 -0500
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
Message-ID: <3d375d730908141424v68f2c326x89efd5c407bafbac@mail.gmail.com>

On Fri, Aug 14, 2009 at 16:19, Pierre GM<pgmdevlist at gmail.com> wrote:
> All,
> What's the status of GLMs in scipy ? Does anybody have some code they
> could point me to ?

The old scipy.stats.models code had GLMs IIRC. I thought there was a
SoC project working on it outside of scipy at this time.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From jsseabold at gmail.com  Fri Aug 14 17:27:40 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Fri, 14 Aug 2009 17:27:40 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
Message-ID: <c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>

On Fri, Aug 14, 2009 at 5:19 PM, Pierre GM<pgmdevlist at gmail.com> wrote:
> All,
> What's the status of GLMs in scipy ? Does anybody have some code they
> could point me to ?
> Thanks a lot in advance
> P.

GLMs got a lot of attention as part of my GSoC.  You can find the
(still in flux) code here

<http://bazaar.launchpad.net/~scipystats/%2Bjunk/skipper-working/files/head%3A/nipy/fixes/scipy/stats/models/>

You can find pretty much all of the usage in tests/test_glm.py (to be
moved to the examples in the next week) and the docs should be
somewhat informative if a bit incomplete at the moment.

Some info on GLM here <http://scipystats.blogspot.com/>

I am trying to polish up a few last details then next week is going to
be spent all on finishing documentation (and finishing the draft of
some tutorials -- brief theory, usage, formulas, notes on where we
differ with commercial packages/R and why) and deciding on how to
package the models code etc. as the GSoC officially ends next week,
but I think Josef and I will continue to work on extending the code
(towards econometrics) after this.

Let me know if there are any questions.  We will be soliciting
feedback once it's finished up and packaged to be distributed, though
we haven't approached the best way to do this now.

Cheers,

Skipper


From gael.varoquaux at normalesup.org  Fri Aug 14 17:28:31 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 14 Aug 2009 23:28:31 +0200
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <3d375d730908141422x79cdb8a0xf9e90402d7095dde@mail.gmail.com>
References: <20090814210541.GA15669@phare.normalesup.org>
	<576503.15078.qm@web52109.mail.re2.yahoo.com>
	<20090814211833.GB15669@phare.normalesup.org>
	<3d375d730908141422x79cdb8a0xf9e90402d7095dde@mail.gmail.com>
Message-ID: <20090814212831.GA18720@phare.normalesup.org>

On Fri, Aug 14, 2009 at 04:22:19PM -0500, Robert Kern wrote:
> On Fri, Aug 14, 2009 at 16:18, Gael
> Varoquaux<gael.varoquaux at normalesup.org> wrote:
> > On Fri, Aug 14, 2009 at 02:16:56PM -0700, David Goldsmith wrote:
> >> Thanks, Gael - for ref., how much ($) does that run?

> > I would say $20 from airport to Union, and less then $10 to Del Mar.

> This will be useful to people:

>   http://transitguide.caltech.edu/

So, I was way out of guess: the LAWA shuttle is $3.

Ga?l


From pgmdevlist at gmail.com  Fri Aug 14 17:46:45 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Fri, 14 Aug 2009 17:46:45 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
Message-ID: <8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>


On Aug 14, 2009, at 5:27 PM, Skipper Seabold wrote:

> On Fri, Aug 14, 2009 at 5:19 PM, Pierre GM<pgmdevlist at gmail.com>  
> wrote:
>> All,
>> What's the status of GLMs in scipy ? Does anybody have some code they
>> could point me to ?
>> Thanks a lot in advance
>> P.
>
> GLMs got a lot of attention as part of my GSoC.  You can find the
> (still in flux) code here
>
> <http://bazaar.launchpad.net/~scipystats/%2Bjunk/skipper-working/files/head%3A/nipy/fixes/scipy/stats/models/ 
> >

Fab'.
FYI, I need to fit Tweedie distributions to precipitation series. I  
have already coded the distributions in the scipy standard, and now I  
need to estimate the parameters...
Thanks again
P.


From d_l_goldsmith at yahoo.com  Fri Aug 14 18:28:23 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 15:28:23 -0700 (PDT)
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <3d375d730908141422x79cdb8a0xf9e90402d7095dde@mail.gmail.com>
Message-ID: <646135.90596.qm@web52111.mail.re2.yahoo.com>

Thanks, Robert!!!

DG

--- On Fri, 8/14/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] Ground xport to/from airport
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 14, 2009, 2:22 PM
> On Fri, Aug 14, 2009 at 16:18, Gael
> Varoquaux<gael.varoquaux at normalesup.org>
> wrote:
> > On Fri, Aug 14, 2009 at 02:16:56PM -0700, David
> Goldsmith wrote:
> >> Thanks, Gael - for ref., how much ($) does that
> run?
> >
> > I would say $20 from airport to Union, and less then
> $10 to Del Mar.
> 
> This will be useful to people:
> 
> ? http://transitguide.caltech.edu/
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Fri Aug 14 19:29:04 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 14 Aug 2009 19:29:04 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
Message-ID: <1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>

On Fri, Aug 14, 2009 at 5:46 PM, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 14, 2009, at 5:27 PM, Skipper Seabold wrote:
>
>> On Fri, Aug 14, 2009 at 5:19 PM, Pierre GM<pgmdevlist at gmail.com>
>> wrote:
>>> All,
>>> What's the status of GLMs in scipy ? Does anybody have some code they
>>> could point me to ?
>>> Thanks a lot in advance
>>> P.
>>
>> GLMs got a lot of attention as part of my GSoC. ?You can find the
>> (still in flux) code here
>>
>> <http://bazaar.launchpad.net/~scipystats/%2Bjunk/skipper-working/files/head%3A/nipy/fixes/scipy/stats/models/
>> >
>
> Fab'.
> FYI, I need to fit Tweedie distributions to precipitation series. I
> have already coded the distributions in the scipy standard, and now I
> need to estimate the parameters...
> Thanks again
> P.
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

What's GLM ? Generalized, not General Linear Model, just to make sure
we are talking about the same thing. I don't see how GLM can help you
directly to estimate the parameters of a distribution, unless it is
one of the exponential family distributions covered with a standard
Generalized Linear Model implementation.

I don't know the Tweedie distribution, but from the reference Skipper
gave you, you can see GLM covers gaussian, binomial, gamma and a few
others. The main point of GLM is to regress the mean/expectation of
the distribution on some explanatory variables.

Are you trying to estimate parameters of the distribution themselves,
or parameters of the distribution as function of some explanatory
variables? In the first case, GLM won't be of much help.

Josef


From strawman at astraw.com  Fri Aug 14 22:01:17 2009
From: strawman at astraw.com (Andrew Straw)
Date: Fri, 14 Aug 2009 19:01:17 -0700
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <20090814212831.GA18720@phare.normalesup.org>
References: <20090814210541.GA15669@phare.normalesup.org>	<576503.15078.qm@web52109.mail.re2.yahoo.com>	<20090814211833.GB15669@phare.normalesup.org>	<3d375d730908141422x79cdb8a0xf9e90402d7095dde@mail.gmail.com>
	<20090814212831.GA18720@phare.normalesup.org>
Message-ID: <4A8616ED.2000105@astraw.com>

(This is a re-send from an email that didn't appear to make it to the list.)

I have some more concise instructions for the impatient. Recently
verified (3 days ago). LAX to Caltech:

Flyaway bus to Union Station ($7). Get on in the "green zone" in the
arrival level. Pay when you get off.
http://www.lawa.org/welcome_LAX.aspx?id=292

Gold line to Pasadena (Fillmore, Del Mar, or Lake Station are all
roughly similar distance to Caltech -- probably a 15-20 minute walk).
$1.25 http://www.metro.net/riding_metro/gold_line.htm

Optionally, one could take bus 177 from the Del Mar station to Caltech.
(I don't know that price.)


From d_l_goldsmith at yahoo.com  Fri Aug 14 22:58:48 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 14 Aug 2009 19:58:48 -0700 (PDT)
Subject: [SciPy-dev] Ground xport to/from airport
In-Reply-To: <4A8616ED.2000105@astraw.com>
Message-ID: <68204.24578.qm@web52106.mail.re2.yahoo.com>

Thanks!!!

DG

--- On Fri, 8/14/09, Andrew Straw <strawman at astraw.com> wrote:

> From: Andrew Straw <strawman at astraw.com>
> Subject: Re: [SciPy-dev] Ground xport to/from airport
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 14, 2009, 7:01 PM
> (This is a re-send from an email that
> didn't appear to make it to the list.)
> 
> I have some more concise instructions for the impatient.
> Recently
> verified (3 days ago). LAX to Caltech:
> 
> Flyaway bus to Union Station ($7). Get on in the "green
> zone" in the
> arrival level. Pay when you get off.
> http://www.lawa.org/welcome_LAX.aspx?id=292
> 
> Gold line to Pasadena (Fillmore, Del Mar, or Lake Station
> are all
> roughly similar distance to Caltech -- probably a 15-20
> minute walk).
> $1.25 http://www.metro.net/riding_metro/gold_line.htm
> 
> Optionally, one could take bus 177 from the Del Mar station
> to Caltech.
> (I don't know that price.)
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From dwf at cs.toronto.edu  Sat Aug 15 03:00:35 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 15 Aug 2009 03:00:35 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
Message-ID: <545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>


On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:

>> Fab'.
>> FYI, I need to fit Tweedie distributions to precipitation series. I
>> have already coded the distributions in the scipy standard, and now I
>> need to estimate the parameters...
>> Thanks again

As I understand it, the Tweedie distributions are a further  
generalization of the exponential family. Are you saying that your  
parametric assumption is that they are Tweedie but not any of the  
standard ones like Gaussian, Poisson, Gamma?

> Are you trying to estimate parameters of the distribution themselves,
> or parameters of the distribution as function of some explanatory
> variables? In the first case, GLM won't be of much help.

Is it that you have samples of a (nonstandard) Tweedie random variable  
that you want to regress on explanatory variables?
You can probably do it by gradient descent but I don't foresee it  
being pretty and probably not even convex. Either way, a GLM package  
probably won't  help.

David


From pgmdevlist at gmail.com  Sat Aug 15 03:32:46 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Sat, 15 Aug 2009 03:32:46 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
	<545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
Message-ID: <575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com>


On Aug 15, 2009, at 3:00 AM, David Warde-Farley wrote:

>
> On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:
>
>>> Fab'.
>>> FYI, I need to fit Tweedie distributions to precipitation series. I
>>> have already coded the distributions in the scipy standard, and  
>>> now I
>>> need to estimate the parameters...
>>> Thanks again
>
> As I understand it, the Tweedie distributions are a further
> generalization of the exponential family.

Indeed.

> Are you saying that your
> parametric assumption is that they are Tweedie but not any of the
> standard ones like Gaussian, Poisson, Gamma?

Yes, something intermediate between Poisson and Gamma, with a variance  
proportional to the mean to a power 1<=p<=2.

>> Are you trying to estimate parameters of the distribution themselves,
>> or parameters of the distribution as function of some explanatory
>> variables? In the first case, GLM won't be of much help.
>
> Is it that you have samples of a (nonstandard) Tweedie random variable
> that you want to regress on explanatory variables?
> You can probably do it by gradient descent but I don't foresee it
> being pretty and probably not even convex. Either way, a GLM package
> probably won't  help.

I'm not sure yet whether GLMs are the way to go to my particular  
problem. I'm trying to reproduce an approach to model precipitation  
patterns (keeping track of both the number and intensities of rainfall  
events) described in several papers. I know that at term, I'll have to  
introduce extra variables and then GLMs will be the way to go. I just  
wanted to check what algorithms were already available.
Thanks a lot for your comments.


From josef.pktd at gmail.com  Sat Aug 15 07:35:17 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 15 Aug 2009 07:35:17 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
	<545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
	<575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com>
Message-ID: <1cd32cbb0908150435l5f105c57x28d98258ebccc186@mail.gmail.com>

On Sat, Aug 15, 2009 at 3:32 AM, Pierre GM<pgmdevlist at gmail.com> wrote:
>
> On Aug 15, 2009, at 3:00 AM, David Warde-Farley wrote:
>
>>
>> On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:
>>
>>>> Fab'.
>>>> FYI, I need to fit Tweedie distributions to precipitation series. I
>>>> have already coded the distributions in the scipy standard, and
>>>> now I
>>>> need to estimate the parameters...
>>>> Thanks again
>>
>> As I understand it, the Tweedie distributions are a further
>> generalization of the exponential family.
>
> Indeed.
>
>> Are you saying that your
>> parametric assumption is that they are Tweedie but not any of the
>> standard ones like Gaussian, Poisson, Gamma?
>
> Yes, something intermediate between Poisson and Gamma, with a variance
> proportional to the mean to a power 1<=p<=2.
>
>>> Are you trying to estimate parameters of the distribution themselves,
>>> or parameters of the distribution as function of some explanatory
>>> variables? In the first case, GLM won't be of much help.
>>
>> Is it that you have samples of a (nonstandard) Tweedie random variable
>> that you want to regress on explanatory variables?
>> You can probably do it by gradient descent but I don't foresee it
>> being pretty and probably not even convex. Either way, a GLM package
>> probably won't ?help.
>
> I'm not sure yet whether GLMs are the way to go to my particular
> problem. I'm trying to reproduce an approach to model precipitation
> patterns (keeping track of both the number and intensities of rainfall
> events) described in several papers. I know that at term, I'll have to
> introduce extra variables and then GLMs will be the way to go. I just
> wanted to check what algorithms were already available.
> Thanks a lot for your comments.

Using models.GLM could be as easy as adding a new distribution to the
family. The main algorithm is (supposed to be) independent of the
distribution, and all distribution specific code is supposed to be in
family.

If Tweedie is like Poisson and Gamma, mainly with a different variance
function, then I think it *should* work with very little work.

If you try this, then this would be a good check for how general our
implementation is, and whether there are still some hidden,
distribution specific assumptions left.

And it will be good if we soon have more eyes on the models code,
because I don't think we have settled on a good API yet.

Josef




> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From josef.pktd at gmail.com  Sat Aug 15 09:12:04 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 15 Aug 2009 09:12:04 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <1cd32cbb0908150435l5f105c57x28d98258ebccc186@mail.gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
	<545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
	<575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com>
	<1cd32cbb0908150435l5f105c57x28d98258ebccc186@mail.gmail.com>
Message-ID: <1cd32cbb0908150612p3ee28f42nfc71a27bda6642d0@mail.gmail.com>

On Sat, Aug 15, 2009 at 7:35 AM, <josef.pktd at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 3:32 AM, Pierre GM<pgmdevlist at gmail.com> wrote:
>>
>> On Aug 15, 2009, at 3:00 AM, David Warde-Farley wrote:
>>
>>>
>>> On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:
>>>
>>>>> Fab'.
>>>>> FYI, I need to fit Tweedie distributions to precipitation series. I
>>>>> have already coded the distributions in the scipy standard, and
>>>>> now I
>>>>> need to estimate the parameters...
>>>>> Thanks again
>>>
>>> As I understand it, the Tweedie distributions are a further
>>> generalization of the exponential family.
>>
>> Indeed.
>>
>>> Are you saying that your
>>> parametric assumption is that they are Tweedie but not any of the
>>> standard ones like Gaussian, Poisson, Gamma?
>>
>> Yes, something intermediate between Poisson and Gamma, with a variance
>> proportional to the mean to a power 1<=p<=2.
>>
>>>> Are you trying to estimate parameters of the distribution themselves,
>>>> or parameters of the distribution as function of some explanatory
>>>> variables? In the first case, GLM won't be of much help.
>>>
>>> Is it that you have samples of a (nonstandard) Tweedie random variable
>>> that you want to regress on explanatory variables?
>>> You can probably do it by gradient descent but I don't foresee it
>>> being pretty and probably not even convex. Either way, a GLM package
>>> probably won't ?help.
>>
>> I'm not sure yet whether GLMs are the way to go to my particular
>> problem. I'm trying to reproduce an approach to model precipitation
>> patterns (keeping track of both the number and intensities of rainfall
>> events) described in several papers. I know that at term, I'll have to
>> introduce extra variables and then GLMs will be the way to go. I just
>> wanted to check what algorithms were already available.
>> Thanks a lot for your comments.
>
> Using models.GLM could be as easy as adding a new distribution to the
> family. The main algorithm is (supposed to be) independent of the
> distribution, and all distribution specific code is supposed to be in
> family.
>
> If Tweedie is like Poisson and Gamma, mainly with a different variance
> function, then I think it *should* work with very little work.
>
> If you try this, then this would be a good check for how general our
> implementation is, and whether there are still some hidden,
> distribution specific assumptions left.
>
> And it will be good if we soon have more eyes on the models code,
> because I don't think we have settled on a good API yet.
>
> Josef
>

I should have done a bit of homework first.

The tweedie family of distributions looks very interesting, and it
should fit in both glm and maximum likelihood framework. R/S has
tweedie in GLM and ML in fbasics.

So I would be very interested in seeing it both in models.glm and in
scipy.stats.distributions. However, in the short term, I see two
potential issues

Wikipedia: "Apart from the four special cases identified above, their
probability density function have no closed form. However, software is
available that enables the accurate computation of the Tweedie
densities (and probability distribution functions)"

Currently the models code is in pure python, which makes distribution
as a standalone package much easier, until the dust has settled, and
models is reintegrated into scipy. Do you have the numerical
calculations in python or compile, fortran,C? I didn't find tweedie in
hydroclimpy.

Wikipedia: "For 1 < p < 2, the distribution is continuous on the
positive reals, plus an added mass (exact zero) at Y = 0"

The generic framework for the distributions in stats.distributions
doesn't handle, currently, distributions that have continuous and
discrete support (masspoints). In some cases, this can be extended by
delegation, but we could think about to handle the mixed case.

Josef

>
>
>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>


From bsouthey at gmail.com  Sat Aug 15 09:19:04 2009
From: bsouthey at gmail.com (Bruce Southey)
Date: Sat, 15 Aug 2009 08:19:04 -0500
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <1cd32cbb0908150612p3ee28f42nfc71a27bda6642d0@mail.gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com>
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com>
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com>
	<545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
	<575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com>
	<1cd32cbb0908150435l5f105c57x28d98258ebccc186@mail.gmail.com>
	<1cd32cbb0908150612p3ee28f42nfc71a27bda6642d0@mail.gmail.com>
Message-ID: <bbcd77d00908150619m2250797fwc3507498821ad8bb@mail.gmail.com>

On Sat, Aug 15, 2009 at 8:12 AM, <josef.pktd at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 7:35 AM, <josef.pktd at gmail.com> wrote:
>> On Sat, Aug 15, 2009 at 3:32 AM, Pierre GM<pgmdevlist at gmail.com> wrote:
>>>
>>> On Aug 15, 2009, at 3:00 AM, David Warde-Farley wrote:
>>>
>>>>
>>>> On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:
>>>>
>>>>>> Fab'.
>>>>>> FYI, I need to fit Tweedie distributions to precipitation series. I
>>>>>> have already coded the distributions in the scipy standard, and
>>>>>> now I
>>>>>> need to estimate the parameters...
>>>>>> Thanks again
>>>>
>>>> As I understand it, the Tweedie distributions are a further
>>>> generalization of the exponential family.
>>>
>>> Indeed.
>>>
>>>> Are you saying that your
>>>> parametric assumption is that they are Tweedie but not any of the
>>>> standard ones like Gaussian, Poisson, Gamma?
>>>
>>> Yes, something intermediate between Poisson and Gamma, with a variance
>>> proportional to the mean to a power 1<=p<=2.
>>>
>>>>> Are you trying to estimate parameters of the distribution themselves,
>>>>> or parameters of the distribution as function of some explanatory
>>>>> variables? In the first case, GLM won't be of much help.
>>>>
>>>> Is it that you have samples of a (nonstandard) Tweedie random variable
>>>> that you want to regress on explanatory variables?
>>>> You can probably do it by gradient descent but I don't foresee it
>>>> being pretty and probably not even convex. Either way, a GLM package
>>>> probably won't ?help.
>>>
>>> I'm not sure yet whether GLMs are the way to go to my particular
>>> problem. I'm trying to reproduce an approach to model precipitation
>>> patterns (keeping track of both the number and intensities of rainfall
>>> events) described in several papers. I know that at term, I'll have to
>>> introduce extra variables and then GLMs will be the way to go. I just
>>> wanted to check what algorithms were already available.
>>> Thanks a lot for your comments.
>>
>> Using models.GLM could be as easy as adding a new distribution to the
>> family. The main algorithm is (supposed to be) independent of the
>> distribution, and all distribution specific code is supposed to be in
>> family.
>>
>> If Tweedie is like Poisson and Gamma, mainly with a different variance
>> function, then I think it *should* work with very little work.
>>
>> If you try this, then this would be a good check for how general our
>> implementation is, and whether there are still some hidden,
>> distribution specific assumptions left.
>>
>> And it will be good if we soon have more eyes on the models code,
>> because I don't think we have settled on a good API yet.
>>
>> Josef
>>
>
> I should have done a bit of homework first.
>
> The tweedie family of distributions looks very interesting, and it
> should fit in both glm and maximum likelihood framework. R/S has
> tweedie in GLM and ML in fbasics.
>
> So I would be very interested in seeing it both in models.glm and in
> scipy.stats.distributions. However, in the short term, I see two
> potential issues
>
> Wikipedia: "Apart from the four special cases identified above, their
> probability density function have no closed form. However, software is
> available that enables the accurate computation of the Tweedie
> densities (and probability distribution functions)"
>
> Currently the models code is in pure python, which makes distribution
> as a standalone package much easier, until the dust has settled, and
> models is reintegrated into scipy. Do you have the numerical
> calculations in python or compile, fortran,C? I didn't find tweedie in
> hydroclimpy.
>
> Wikipedia: "For 1 < p < 2, the distribution is continuous on the
> positive reals, plus an added mass (exact zero) at Y = 0"
>
> The generic framework for the distributions in stats.distributions
> doesn't handle, currently, distributions that have continuous and
> discrete support (masspoints). In some cases, this can be extended by
> delegation, but we could think about to handle the mixed case.
>
> Josef
>

Hi,
Technically generalized linear models only apply to a exponential
family of distributions. However, it does work for some other like
negative binomial so you have quazi-likelihoods. Typically you require
the link and inverse link functions to do it and nonlinear modeling or
iteratively reweighted least squares to solve. See Wikipedia:
http://en.wikipedia.org/wiki/Generalized_linear_model

While I have not used the Tweedie formulation the wikipedia page
http://en.wikipedia.org/wiki/Tweedie_distributions notes that these do
not have closed form. Technically this is a problem but there is an R
package:
http://cran.r-project.org/web/packages/tweedie/index.html

Given the relationships on the wikipedia, I would tend to favor using
the special cases first and see what support you have for these.

Bruce


From emmanuelle.gouillart at normalesup.org  Sat Aug 15 09:59:15 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Sat, 15 Aug 2009 15:59:15 +0200
Subject: [SciPy-dev] change print statements to print functions in docstrings
Message-ID: <20090815135915.GA32560@phare.normalesup.org>

	Hi,

	I've started changing the print statements in numpy's doctrings
to print functions on the doc wiki, to meet the requirements of Python
3.0. Any objections or comments?

	Cheers,

	Emmanuelle


From jsseabold at gmail.com  Sat Aug 15 10:27:25 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 15 Aug 2009 10:27:25 -0400
Subject: [SciPy-dev] GLMs ?
In-Reply-To: <bbcd77d00908150619m2250797fwc3507498821ad8bb@mail.gmail.com>
References: <B659019B-1AE9-492E-8A79-A506E4CF6FA9@gmail.com> 
	<c048da1c0908141427r128ff008p16df03cca90b78ed@mail.gmail.com> 
	<8B8A63F2-D5EE-476D-915D-F840FCB6A76D@gmail.com>
	<1cd32cbb0908141629w3b667cfbxce843ea95fd5757b@mail.gmail.com> 
	<545F7424-8F43-4279-952A-380C180B09CD@cs.toronto.edu>
	<575EF48D-8290-4194-9F32-8D76CF9DAD45@gmail.com> 
	<1cd32cbb0908150435l5f105c57x28d98258ebccc186@mail.gmail.com> 
	<1cd32cbb0908150612p3ee28f42nfc71a27bda6642d0@mail.gmail.com> 
	<bbcd77d00908150619m2250797fwc3507498821ad8bb@mail.gmail.com>
Message-ID: <c048da1c0908150727p6de62c5fnba917f11f066631c@mail.gmail.com>

On Sat, Aug 15, 2009 at 9:19 AM, Bruce Southey<bsouthey at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 8:12 AM, <josef.pktd at gmail.com> wrote:
>> On Sat, Aug 15, 2009 at 7:35 AM, <josef.pktd at gmail.com> wrote:
>>> On Sat, Aug 15, 2009 at 3:32 AM, Pierre GM<pgmdevlist at gmail.com> wrote:
>>>>
>>>> On Aug 15, 2009, at 3:00 AM, David Warde-Farley wrote:
>>>>
>>>>>
>>>>> On 14-Aug-09, at 7:29 PM, josef.pktd at gmail.com wrote:
>>>>>
>>>>>>> Fab'.
>>>>>>> FYI, I need to fit Tweedie distributions to precipitation series. I
>>>>>>> have already coded the distributions in the scipy standard, and
>>>>>>> now I
>>>>>>> need to estimate the parameters...
>>>>>>> Thanks again
>>>>>
>>>>> As I understand it, the Tweedie distributions are a further
>>>>> generalization of the exponential family.
>>>>
>>>> Indeed.
>>>>
>>>>> Are you saying that your
>>>>> parametric assumption is that they are Tweedie but not any of the
>>>>> standard ones like Gaussian, Poisson, Gamma?
>>>>
>>>> Yes, something intermediate between Poisson and Gamma, with a variance
>>>> proportional to the mean to a power 1<=p<=2.
>>>>
>>>>>> Are you trying to estimate parameters of the distribution themselves,
>>>>>> or parameters of the distribution as function of some explanatory
>>>>>> variables? In the first case, GLM won't be of much help.
>>>>>
>>>>> Is it that you have samples of a (nonstandard) Tweedie random variable
>>>>> that you want to regress on explanatory variables?
>>>>> You can probably do it by gradient descent but I don't foresee it
>>>>> being pretty and probably not even convex. Either way, a GLM package
>>>>> probably won't ?help.
>>>>
>>>> I'm not sure yet whether GLMs are the way to go to my particular
>>>> problem. I'm trying to reproduce an approach to model precipitation
>>>> patterns (keeping track of both the number and intensities of rainfall
>>>> events) described in several papers. I know that at term, I'll have to
>>>> introduce extra variables and then GLMs will be the way to go. I just
>>>> wanted to check what algorithms were already available.
>>>> Thanks a lot for your comments.
>>>
>>> Using models.GLM could be as easy as adding a new distribution to the
>>> family. The main algorithm is (supposed to be) independent of the
>>> distribution, and all distribution specific code is supposed to be in
>>> family.
>>>
>>> If Tweedie is like Poisson and Gamma, mainly with a different variance
>>> function, then I think it *should* work with very little work.
>>>
>>> If you try this, then this would be a good check for how general our
>>> implementation is, and whether there are still some hidden,
>>> distribution specific assumptions left.
>>>
>>> And it will be good if we soon have more eyes on the models code,
>>> because I don't think we have settled on a good API yet.
>>>
>>> Josef
>>>
>>
>> I should have done a bit of homework first.
>>
>> The tweedie family of distributions looks very interesting, and it
>> should fit in both glm and maximum likelihood framework. R/S has
>> tweedie in GLM and ML in fbasics.
>>
>> So I would be very interested in seeing it both in models.glm and in
>> scipy.stats.distributions. However, in the short term, I see two
>> potential issues
>>
>> Wikipedia: "Apart from the four special cases identified above, their
>> probability density function have no closed form. However, software is
>> available that enables the accurate computation of the Tweedie
>> densities (and probability distribution functions)"
>>
>> Currently the models code is in pure python, which makes distribution
>> as a standalone package much easier, until the dust has settled, and
>> models is reintegrated into scipy. Do you have the numerical
>> calculations in python or compile, fortran,C? I didn't find tweedie in
>> hydroclimpy.
>>
>> Wikipedia: "For 1 < p < 2, the distribution is continuous on the
>> positive reals, plus an added mass (exact zero) at Y = 0"
>>
>> The generic framework for the distributions in stats.distributions
>> doesn't handle, currently, distributions that have continuous and
>> discrete support (masspoints). In some cases, this can be extended by
>> delegation, but we could think about to handle the mixed case.
>>
>> Josef
>>
>
> Hi,
> Technically generalized linear models only apply to a exponential
> family of distributions. However, it does work for some other like
> negative binomial so you have quazi-likelihoods. Typically you require
> the link and inverse link functions to do it and nonlinear modeling or
> iteratively reweighted least squares to solve. See Wikipedia:
> http://en.wikipedia.org/wiki/Generalized_linear_model
>

This is indeed the case.

It is my understanding that the negative binomial distribution is
thought of as a full member of the exponential family even though it
is a two-parameter exponential (since the ancillary parameter is
assumed to be nonstochastic for our code at the moment) and our
approach is then not quasi-likelihood.  I wanted to extend the GLMs to
allow for estimation of the variance function, etc. in a
quasi-likelihood framework, but I didn't have the time.

Our approach uses iteratively reweighted least squares, which is
really only ~5 lines of code if you have access to weighted least
squares.

Having a quick look at the Tweedie distribution (I came across them in
the literature as Poisson-Gamma mixture models, which includes the
negative binomial), it shouldn't be any problem to include support for
them in the current framework under assumptions on the power in the
variance, if this will suit your needs.  Once I've finished the
documentation it should be pretty clear on how to add distributions.
It took almost no time to add support for the inverse gaussian and
negative binomial distributions.  As mentioned above you only need to
define the variance function, the link function, its inverse and its
derivative (could be done numerically).

Skipper


From ralf.gommers at googlemail.com  Sat Aug 15 11:14:13 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Sat, 15 Aug 2009 11:14:13 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815135915.GA32560@phare.normalesup.org>
References: <20090815135915.GA32560@phare.normalesup.org>
Message-ID: <dde7764a0908150814x2167d886yc237b6914c26d25e@mail.gmail.com>

On Sat, Aug 15, 2009 at 9:59 AM, Emmanuelle Gouillart <
emmanuelle.gouillart at normalesup.org> wrote:

>        Hi,
>
>        I've started changing the print statements in numpy's doctrings
> to print functions on the doc wiki, to meet the requirements of Python
> 3.0. Any objections or comments?


No objections here, but wouldn't it be a lot easier to do this on the whole
code base at once (with 2to3 for example) rather than in the doc wiki? Saves
you a lot of effort and makes sure we don't have both forms at the same time
for a while.

Cheers,
Ralf


>
>        Cheers,
>
>        Emmanuelle
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/e26d36fb/attachment.html>

From gael.varoquaux at normalesup.org  Sat Aug 15 11:20:18 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 15 Aug 2009 17:20:18 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <dde7764a0908150814x2167d886yc237b6914c26d25e@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<dde7764a0908150814x2167d886yc237b6914c26d25e@mail.gmail.com>
Message-ID: <20090815152018.GK16976@phare.normalesup.org>

On Sat, Aug 15, 2009 at 11:14:13AM -0400, Ralf Gommers wrote:
>    No objections here, but wouldn't it be a lot easier to do this on the
>    whole code base at once (with 2to3 for example) rather than in the doc
>    wiki? Saves you a lot of effort and makes sure we don't have both forms at
>    the same time for a while.

Will 2to3 act on the docstrings?

But anyhow, IMHO, this is more a question of enforcing good practice on
beginners by example than actual code. We will not be running 2to3
anytime soon, but we can start teaching (by example) people to use
parenthesis on print statements now.

We came up with this idea as I was preparing the tutorial for scipy. We
realized that teaching beginners to use 'print(1)' was probably a good
idea. 

Disclamer: I am sitting 2 meters (6 feet) away from Emmanuelle, working
on docs and tutorial too.

Ga?l


From charlesr.harris at gmail.com  Sat Aug 15 11:37:59 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 15 Aug 2009 09:37:59 -0600
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815135915.GA32560@phare.normalesup.org>
References: <20090815135915.GA32560@phare.normalesup.org>
Message-ID: <e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>

On Sat, Aug 15, 2009 at 7:59 AM, Emmanuelle Gouillart <
emmanuelle.gouillart at normalesup.org> wrote:

>        Hi,
>
>        I've started changing the print statements in numpy's doctrings
> to print functions on the doc wiki, to meet the requirements of Python
> 3.0. Any objections or comments?
>

This can lead to some oddities:

In [6]: print(1,2,3)
(1, 2, 3)


I think it might be better to find out what  2to3 does in these
circumstances first.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/7f94c239/attachment.html>

From emmanuelle.gouillart at normalesup.org  Sat Aug 15 11:54:56 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Sat, 15 Aug 2009 17:54:56 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
Message-ID: <20090815155456.GB32560@phare.normalesup.org>

On Sat, Aug 15, 2009 at 09:37:59AM -0600, Charles R Harris wrote:
>    In [6]: print(1,2,3)
>    (1, 2, 3)
>    ?

Right, so I'm not modifying the print statement with commas, to be sure
that we get what's expected with Python 2.x. Having the statement as well
as the function in the doc is not very satisfying, but these changes may
encourage people to use print(). As Ga?l said, this is mostly for
pedagogical purpose.

BTW, with my Ipython (0.11.bzr.r1205), 

In [1]: print 1, 2, 3
------> print(1, 2, 3)
(1, 2, 3)


Emmanuelle






From ralf.gommers at googlemail.com  Sat Aug 15 12:24:48 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Sat, 15 Aug 2009 12:24:48 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
Message-ID: <dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com>

On Sat, Aug 15, 2009 at 11:37 AM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Sat, Aug 15, 2009 at 7:59 AM, Emmanuelle Gouillart <
> emmanuelle.gouillart at normalesup.org> wrote:
>
>>        Hi,
>>
>>        I've started changing the print statements in numpy's doctrings
>> to print functions on the doc wiki, to meet the requirements of Python
>> 3.0. Any objections or comments?
>>
>
> This can lead to some oddities:
>
> In [6]: print(1,2,3)
> (1, 2, 3)
>
>
> I think it might be better to find out what  2to3 does in these
> circumstances first.
>

>From the 2to3 docs:
"2to3 can also refactor doctests. To enable this mode, use the -d
f**<http://docs.python.org/using/cmdline.html#cmdoption-d>lag.
Note that *only* doctests will be refactored. This also doesn?t require the
module to be valid Python. For example, doctest like examples in a reST
document could also be refactored with this option."

Anyway, the change can also be made with a good editor and search-replace. I
am all for changing this, my point was simply that the doc wiki is not the
best tool to make changes like these (not that I don't love the doc wiki,
it's great).

Cheers,
Ralf


> Chuck
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/3d5abf6c/attachment.html>

From charlesr.harris at gmail.com  Sat Aug 15 12:44:01 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 15 Aug 2009 10:44:01 -0600
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
	<dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com>
Message-ID: <e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>

On Sat, Aug 15, 2009 at 10:24 AM, Ralf Gommers
<ralf.gommers at googlemail.com>wrote:

>
>
> On Sat, Aug 15, 2009 at 11:37 AM, Charles R Harris <
> charlesr.harris at gmail.com> wrote:
>
>>
>>
>> On Sat, Aug 15, 2009 at 7:59 AM, Emmanuelle Gouillart <
>> emmanuelle.gouillart at normalesup.org> wrote:
>>
>>>        Hi,
>>>
>>>        I've started changing the print statements in numpy's doctrings
>>> to print functions on the doc wiki, to meet the requirements of Python
>>> 3.0. Any objections or comments?
>>>
>>
>> This can lead to some oddities:
>>
>> In [6]: print(1,2,3)
>> (1, 2, 3)
>>
>>
>> I think it might be better to find out what  2to3 does in these
>> circumstances first.
>>
>
> From the 2to3 docs:
> "2to3 can also refactor doctests. To enable this mode, use the -d f**<http://docs.python.org/using/cmdline.html#cmdoption-d>lag.
> Note that *only* doctests will be refactored. This also doesn?t require
> the module to be valid Python. For example, doctest like examples in a reST
> document could also be refactored with this option."
>
> Anyway, the change can also be made with a good editor and search-replace.
> I am all for changing this, my point was simply that the doc wiki is not the
> best tool to make changes like these (not that I don't love the doc wiki,
> it's great).
>

I'm not convinced it is a good idea, even pedagogically. First, all the
python books I have are for the python-2.x versions, and second it saddles
the students with having to make a distinction between printing single items
vs printing multiple items. It's better, I think, to wait until we have a
python 3.x version of numpy.

That said, it would be interesting if someone ran some tests using 2to3 just
to see how it works.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/69e3135b/attachment.html>

From gael.varoquaux at normalesup.org  Sat Aug 15 12:48:35 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 15 Aug 2009 18:48:35 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
	<dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com>
	<e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>
Message-ID: <20090815164835.GL16976@phare.normalesup.org>

On Sat, Aug 15, 2009 at 10:44:01AM -0600, Charles R Harris wrote:
>    I'm not convinced it is a good idea, even pedagogically. First, all the
>    python books I have are for the python-2.x versions, and second it saddles
>    the students with having to make a distinction between printing single
>    items vs printing multiple items. It's better, I think, to wait until we
>    have a python 3.x version of numpy.

But, the real issue here is that students confronted with IPython will
have to deal with this. At least on some systems, amongst others Ubuntu
Jaunty (python 2.6, IPython 0.9.1):

In [1]: print 1, 2, 3
------> print(1, 2, 3)
(1, 2, 3)

Ga?l


From d_l_goldsmith at yahoo.com  Sat Aug 15 12:50:56 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 09:50:56 -0700 (PDT)
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>
Message-ID: <759973.20819.qm@web52111.mail.re2.yahoo.com>

As a curious "innocent bystander" that only goes along with earth shattering software change "kicking and screaming" ;-) why did the Python 3 PTB decide to change such a fundamental construct?

DG

--- On Sat, 8/15/09, Charles R Harris <charlesr.harris at gmail.com> wrote:

> From: Charles R Harris <charlesr.harris at gmail.com>
> Subject: Re: [SciPy-dev] change print statements to print functions in docstrings
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Saturday, August 15, 2009, 9:44 AM
> 
> 
> On Sat, Aug 15, 2009 at 10:24 AM,
> Ralf Gommers <ralf.gommers at googlemail.com>
> wrote:
> 
> 
> 
> On
> Sat, Aug 15, 2009 at 11:37 AM, Charles R Harris <charlesr.harris at gmail.com>
> wrote:
> 
> 
> 
> 
> On Sat, Aug 15, 2009 at 7:59
> AM, Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>
> wrote:
> 
> 
> 
>  ? ? ? ?Hi,
> 
> 
> 
>  ? ? ? ?I've started changing the print statements
> in numpy's doctrings
> 
> to print functions on the doc wiki, to meet the
> requirements of Python
> 
> 3.0. Any objections or comments?
> 
> 
> This can lead to some oddities:
> 
> In [6]: print(1,2,3)
> (1, 2, 3)
> ?
> 
> I think it might be better to find out what? 2to3 does in
> these circumstances first.
> 
> 
> 
> From the 2to3 docs:
> "2to3 can also refactor doctests.  To enable this
> mode, use the -d flag.
>  Note that only doctests will be refactored.  This
> also doesn?t require
> the module to be valid Python.  For example, doctest like
> examples in a reST
> document could also be refactored with this option."
> 
> Anyway, the change can also be made with a good editor and
> search-replace. I am all for changing this, my point was
> simply that the doc wiki is not the best tool to make
> changes like these (not that I don't love the doc wiki,
> it's great).
> 
> 
> 
> I'm not convinced it is a good idea, even
> pedagogically. First, all the python books I have are for
> the python-2.x versions, and second it saddles the students
> with having to make a distinction between printing single
> items vs printing multiple items. It's better, I think,
> to wait until we have a python 3.x version of numpy.
> 
> 
> That said, it would be interesting if someone ran some
> tests using 2to3 just to see how it works.
> 
> Chuck 
> 
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From rmay31 at gmail.com  Sat Aug 15 13:02:05 2009
From: rmay31 at gmail.com (Ryan May)
Date: Sat, 15 Aug 2009 12:02:05 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <759973.20819.qm@web52111.mail.re2.yahoo.com>
References: <e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com> 
	<759973.20819.qm@web52111.mail.re2.yahoo.com>
Message-ID: <c7b95a020908151002t2df52cd5x3d4472febba66390@mail.gmail.com>

On Sat, Aug 15, 2009 at 11:50 AM, David Goldsmith
<d_l_goldsmith at yahoo.com>wrote:

> As a curious "innocent bystander" that only goes along with earth
> shattering software change "kicking and screaming" ;-) why did the Python 3
> PTB decide to change such a fundamental construct?
>

One reason is that by making it a function, it's easy to replace it with a
different, custom function.  It also (IMO) seems a bit more pythonic.
Remeber, the pain of breakage was kind of the point of 3.0. :)

Ryan

--
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/c03af224/attachment.html>

From ralf.gommers at googlemail.com  Sat Aug 15 13:14:48 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Sat, 15 Aug 2009 13:14:48 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com>
	<dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com>
	<e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com>
Message-ID: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>

On Sat, Aug 15, 2009 at 12:44 PM, Charles R Harris <
charlesr.harris at gmail.com> wrote:

>
>
> On Sat, Aug 15, 2009 at 10:24 AM, Ralf Gommers <
> ralf.gommers at googlemail.com> wrote:
>
>>
>>
>> On Sat, Aug 15, 2009 at 11:37 AM, Charles R Harris <
>> charlesr.harris at gmail.com> wrote:
>>
>>>
>>>
>>> On Sat, Aug 15, 2009 at 7:59 AM, Emmanuelle Gouillart <
>>> emmanuelle.gouillart at normalesup.org> wrote:
>>>
>>>>        Hi,
>>>>
>>>>        I've started changing the print statements in numpy's doctrings
>>>> to print functions on the doc wiki, to meet the requirements of Python
>>>> 3.0. Any objections or comments?
>>>>
>>>
>>> This can lead to some oddities:
>>>
>>> In [6]: print(1,2,3)
>>> (1, 2, 3)
>>>
>>>
>>> I think it might be better to find out what  2to3 does in these
>>> circumstances first.
>>>
>>
>> From the 2to3 docs:
>> "2to3 can also refactor doctests. To enable this mode, use the -d f**<http://docs.python.org/using/cmdline.html#cmdoption-d>lag.
>> Note that *only* doctests will be refactored. This also doesn?t require
>> the module to be valid Python. For example, doctest like examples in a reST
>> document could also be refactored with this option."
>>
>> Anyway, the change can also be made with a good editor and search-replace.
>> I am all for changing this, my point was simply that the doc wiki is not the
>> best tool to make changes like these (not that I don't love the doc wiki,
>> it's great).
>>
>
> I'm not convinced it is a good idea, even pedagogically. First, all the
> python books I have are for the python-2.x versions, and second it saddles
> the students with having to make a distinction between printing single items
> vs printing multiple items. It's better, I think, to wait until we have a
> python 3.x version of numpy.
>

Okay fair enough. Change or no change, I have no strong opinion, as long as
it is done consistently for all docstrings.

>
> That said, it would be interesting if someone ran some tests using 2to3
> just to see how it works.
>

It seems to work out of the box on many files, but chokes on a few. Maybe it
tries to execute some problematic doctests, I'm not sure. Probably easy to
fix. Here are some examples:

ralf at ralf-desktop:~/python/numpy/numpy/core$ 2to3 -d records.py
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
--- records.py (original)
+++ records.py (refactored)
@@ -473,7 +473,7 @@
     >>> x2=np.array(['a','dd','xyz','12'])
     >>> x3=np.array([1.1,2,3,4])
     >>> r = np.core.records.fromarrays([x1,x2,x3],names='a,b,c')
-    >>> print r[1]
+    >>> print(r[1])
     (2, 'dd', 2.0)
     >>> x1[1]=34
     >>> r.a
@@ -552,15 +552,15 @@

     >>> r=np.core.records.fromrecords([(456,'dbe',1.2),(2,'de',1.3)],
     ... names='col1,col2,col3')
-    >>> print r[0]
+    >>> print(r[0])
     (456, 'dbe', 1.2)
     >>> r.col1
     array([456,   2])
     >>> r.col2
     chararray(['dbe', 'de'],
           dtype='|S3')
-    >>> import cPickle
-    >>> print cPickle.loads(cPickle.dumps(r))
+    >>> import pickle
+    >>> print(cPickle.loads(cPickle.dumps(r)))
     [(456, 'dbe', 1.2) (2, 'de', 1.3)]
     """

@@ -647,7 +647,7 @@
     >>> fd.seek(0)
     >>> r=np.core.records.fromfile(fd, formats='f8,i4,a5', shape=10,
     ... byteorder='<')
-    >>> print r[5]
+    >>> print(r[5])
     (0.5, 10, 'abcde')
     >>> r.shape
     (10,)
RefactoringTool: Files that need to be modified:
RefactoringTool: records.py



ralf at ralf-desktop:~/python/numpy/numpy/core$ 2to3 -d numerictypes.py
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
RefactoringTool: No files need to be modified.




ralf at ralf-desktop:~/python/numpy/numpy/core$ 2to3 -d fromnumeric.py
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
--- fromnumeric.py (original)
+++ fromnumeric.py (refactored)
@@ -941,15 +941,15 @@
     to ``reshape(-1)``:

     >>> x = np.array([[1, 2, 3], [4, 5, 6]])
-    >>> print x.reshape(-1)
+    >>> print(x.reshape(-1))
     [1  2  3  4  5  6]

-    >>> print np.ravel(x)
+    >>> print(np.ravel(x))
     [1  2  3  4  5  6]

     When flattening using Fortran-order, however, we see

-    >>> print np.ravel(x, order='F')
+    >>> print(np.ravel(x, order='F'))
     [1 4 2 5 3 6]

     """
RefactoringTool: Files that need to be modified:
RefactoringTool: fromnumeric.py




ralf at ralf-desktop:~/python/numpy/numpy/core$ 2to3 -d numeric.py
RefactoringTool: Skipping implicit fixer: buffer
RefactoringTool: Skipping implicit fixer: idioms
RefactoringTool: Skipping implicit fixer: set_literal
RefactoringTool: Skipping implicit fixer: ws_comma
Traceback (most recent call last):
  File "/usr/bin/2to3", line 6, in <module>
    sys.exit(main("lib2to3.fixes"))
  File "/usr/lib/python2.6/lib2to3/main.py", line 129, in main
    rt.refactor(args, options.write, options.doctests_only)
  File "/usr/lib/python2.6/lib2to3/refactor.py", line 196, in refactor
    self.refactor_file(dir_or_file, write, doctests_only)
  File "/usr/lib/python2.6/lib2to3/refactor.py", line 229, in refactor_file
    output = self.refactor_docstring(input, filename)
  File "/usr/lib/python2.6/lib2to3/refactor.py", line 406, in
refactor_docstring
    indent, filename))
  File "/usr/lib/python2.6/lib2to3/refactor.py", line 426, in
refactor_doctest
    if self.log.isEnabledFor(logging.DEBUG):
AttributeError: 'StdoutRefactoringTool' object has no attribute 'log'



Cheers,
Ralf



>
> Chuck
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/bcb46aae/attachment.html>

From d_l_goldsmith at yahoo.com  Sat Aug 15 14:18:46 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 11:18:46 -0700 (PDT)
Subject: [SciPy-dev] P3 - my education [was: change print statements to
	print functions in docstrings]
In-Reply-To: <c7b95a020908151002t2df52cd5x3d4472febba66390@mail.gmail.com>
Message-ID: <243486.65769.qm@web52104.mail.re2.yahoo.com>

--- On Sat, 8/15/09, Ryan May <rmay31 at gmail.com> wrote:

> As a curious "innocent bystander" that only goes
> along with earth shattering software change "kicking
> and screaming" ;-) why did the Python 3 PTB decide to
> change such a fundamental construct?
> 
> 
> One reason is that by making it a function, it's easy
> to replace it with a different, custom function.? It also

But isn't that the purpose an object's __str__ method being bound to print by default?  Unless, I guess, you want to usurp print globally; but then can't you still just assign your replacement to print in the top level namespace and be done w/it?  Personally, if I were among the PTB, I'd need more...

> (IMO) seems a bit more pythonic.? 

Um, why?  Perhaps it stems from one's personal preference regarding the way one prefers to program?  Having "grown-up" a procedural programmer, I'm now an OO "true believer," though I feel that one of Python's greatest strengths is that it fully supports "going both ways"; because of that, I feel neither function nor method has the edge in being more Pythonic.  (I wonder, however, what the dominant mode is in practice - certainly OO in GUI/Event Driven programming, no?)

> Remeber, the pain of
> breakage was kind of the point of 3.0. :)

Please explain.

DG

PS: In recognition that this branch of this thread is OT, everyone please feel free to email me your thoughts off-list.

> 
> 
> 
> Ryan? 
> ?
> --
> Ryan May
> Graduate Research Assistant
> School of Meteorology
> University of Oklahoma
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From njs at pobox.com  Sat Aug 15 15:18:35 2009
From: njs at pobox.com (Nathaniel Smith)
Date: Sat, 15 Aug 2009 12:18:35 -0700
Subject: [SciPy-dev] P3 - my education [was: change print statements to
	print functions in docstrings]
In-Reply-To: <243486.65769.qm@web52104.mail.re2.yahoo.com>
References: <c7b95a020908151002t2df52cd5x3d4472febba66390@mail.gmail.com>
	<243486.65769.qm@web52104.mail.re2.yahoo.com>
Message-ID: <961fa2b40908151218g4e4946cdtb7632f591b67cf71@mail.gmail.com>

(Replying in public in the vain -- in both senses -- hope that this
might finish the discussion.)

> Um, why? ?Perhaps it stems from one's personal preference regarding the way
> one prefers to program? ?Having "grown-up" a procedural programmer, I'm now
> an OO "true believer," though I feel that one of Python's greatest strengths is
> that it fully supports "going both ways"; because of that, I feel neither function
> nor method has the edge in being more Pythonic.

In 2.x, print is neither a function nor a method. It's a statement
with its own syntax, comparable to "for" or "try" or "if". (And it's
actually relatively bizarre syntax -- there's a special construct
involving ">>" for directing print output to a file that's totally
unlike anything else in Python, etc.) This is confusing, especially
for newcomers, because the first 'function-like' operation they learn
follows its own rules that don't generalize to anything else.

Also, this makes it possible to have variables, methods, functions in
a local scope that are named "print", which might be handy sometimes:
  class Printer:
    def print(self, document): # <-- in 2.x this is a syntax error
      ...

In any case, it is the way it is, at this point -- 3.0 was released
last year :-).

-- Nathaniel


From rmay31 at gmail.com  Sat Aug 15 15:43:01 2009
From: rmay31 at gmail.com (Ryan May)
Date: Sat, 15 Aug 2009 14:43:01 -0500
Subject: [SciPy-dev] P3 - my education [was: change print statements to
	print functions in docstrings]
In-Reply-To: <243486.65769.qm@web52104.mail.re2.yahoo.com>
References: <c7b95a020908151002t2df52cd5x3d4472febba66390@mail.gmail.com> 
	<243486.65769.qm@web52104.mail.re2.yahoo.com>
Message-ID: <c7b95a020908151243s20eb2b25v55b29d9c77a355d7@mail.gmail.com>

On Sat, Aug 15, 2009 at 1:18 PM, David Goldsmith <d_l_goldsmith at yahoo.com>wrote:

> --- On Sat, 8/15/09, Ryan May <rmay31 at gmail.com> wrote:
>
> > As a curious "innocent bystander" that only goes
> > along with earth shattering software change "kicking
> > and screaming" ;-) why did the Python 3 PTB decide to
> > change such a fundamental construct?
> >
> >
> > One reason is that by making it a function, it's easy
> > to replace it with a different, custom function.  It also
>
> But isn't that the purpose an object's __str__ method being bound to print
> by default?  Unless, I guess, you want to usurp print globally; but then
> can't you still just assign your replacement to print in the top level
> namespace and be done w/it?  Personally, if I were among the PTB, I'd need
> more...
>

If you have print as a function, you can change it to a logging function
just by redefining print.  With print as a statement, the only way you can
get this is by using a custom printing function from the start.  Also,
__str__ doesn't put anything on the screen--it does what the name says, it
converts to a string.  That's orthogonal to the issue of print as a
function.


>
> > (IMO) seems a bit more pythonic.
>
> Um, why?  Perhaps it stems from one's personal preference regarding the way
> one prefers to program?  Having "grown-up" a procedural programmer, I'm now
> an OO "true believer," though I feel that one of Python's greatest strengths
> is that it fully supports "going both ways"; because of that, I feel neither
> function nor method has the edge in being more Pythonic.  (I wonder,
> however, what the dominant mode is in practice - certainly OO in GUI/Event
> Driven programming, no?)
>

Procedural vs. OO has nothing to do with this either.  It's merely a change
from language *syntax* to a built-in function.  To me, a function seems a
more natural place for such *functionality*, not a statement/syntax.

    print a,  #Suppresses newline

    print >>file, a #Prints to fileobject a

These other uses are a bit cryptic (especially new line suppression).  These
are reminicent of Perl and aren't as expressive and explicit as modern
python.  And have you ever tried to print a bunch of things using something
other than spaces to separate them?  You *can't* without making the string
yourself.  These are warts and they only continue to exist because they date
back to early python and were very painful to change.


> > Remeber, the pain of
> > breakage was kind of the point of 3.0. :)
>
> Please explain.


3.0 is the time to do *any* breaking changes.  If they were ever going to
change something like this, this was the sole opportunity.  If you were
starting from scratch and not worried about the pain of changes, would you
have problems with print as a function? Would you actually favor the print
statement?  The design goal of Python 3.0 was to make the best python
possible with no mind to backwards compatibility.  As I've heard several
times, there *will not* be a 4.0 ilke this.

Ryan


> DG
>
> PS: In recognition that this branch of this thread is OT, everyone please
> feel free to email me your thoughts off-list.
>
> >
> >
> >
> > Ryan
> >
> > --
> > Ryan May
> > Graduate Research Assistant
> > School of Meteorology
> > University of Oklahoma
> >
> >
> > -----Inline Attachment Follows-----
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>



-- 
Ryan May
Graduate Research Assistant
School of Meteorology
University of Oklahoma
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/78a5cc69/attachment.html>

From jh at physics.ucf.edu  Sat Aug 15 18:05:31 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sat, 15 Aug 2009 18:05:31 -0400
Subject: [SciPy-dev] Summer Marathon "Wrap-up" BoF
In-Reply-To: <213173.19652.qm@web52108.mail.re2.yahoo.com> (message from David
	Goldsmith on Fri, 14 Aug 2009 12:07:20 -0700 (PDT))
References: <213173.19652.qm@web52108.mail.re2.yahoo.com>
Message-ID: <wl8prawn36c.fsf@glup.physics.ucf.edu>

I'd like to be at the Doc and Astro BoFs, and at the Organization BoF
I just posted.  My flight gets into LAX at 1:39 pm Wednesday, so
realistically I won't be sure to be at Caltech until 5 or so
Wednesday.

--jh--

Date: Fri, 14 Aug 2009 12:07:20 -0700 (PDT)
From: David Goldsmith <d_l_goldsmith at yahoo.com>
Subject: Re: [SciPy-dev] Summer Marathon "Wrap-up" BoF
To: SciPy Developers List <scipy-dev at scipy.org>

Perfect, Pauli, thanks!  I confess to not yet having read the BoF Wiki: are there "instructions" there for how to arrange for a location?

DG

--- On Fri, 8/14/09, Pauli Virtanen <pav at iki.fi> wrote:

> From: Pauli Virtanen <pav at iki.fi>
> Subject: Re: [SciPy-dev] Summer Marathon "Wrap-up" BoF
> To: scipy-dev at scipy.org
> Date: Friday, August 14, 2009, 12:46 AM
> Hi David,
> 
> Thu, 13 Aug 2009 12:56:04 -0700, David Goldsmith
> kirjoitti:
> > Hi!? So, one thing that came out of yesterday's
> Skypecon was the idea
> > for a SciPyCon BoF for "wrapping-up" the Summer
> Marathon; if we were to
> > have that next Wed. at 19:00 UTC (noon local time),
> i.e., at the
> > "regularly scheduled" Skypecon time, would there be
> anyone who would
> > want to participate but couldn't because of the
> selection of that time?
> 
> I was just about to propose this same BOF. I think since
> many people who 
> have been involved in this are present there, this is
> indeed a good 
> opportunity for a BOF. I went forward and added it to http://scipy.org/
> SciPy2009/BoF, marking you tentatively as the coordinator.
> Please change 
> it as you see fit.
> 
> I also added a few topics that might be of interest:
> 
>  * Wrapping up this summer's marathon.
>  * What next: Scipy? User guides?
>  * Drafting a TOC for the Numpy User Guide?
> 
> There are probably more. If we run out, we can start
> building a bike shed 
> and paint it with different colors (each bring your own!
> :)
> 
> ??? Pauli
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From jh at physics.ucf.edu  Sat Aug 15 18:08:15 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sat, 15 Aug 2009 18:08:15 -0400
Subject: [SciPy-dev] doc T-shirts
Message-ID: <wl8ocqgn31s.fsf@glup.physics.ucf.edu>

If you have written 1000 words on the doc wiki and have not yet gotten
a Doc Project T-shirt, please email me off-list with your mailing
address.  If you will be at SciPy09, let me know and I'll bring it.

--jh--


From d_l_goldsmith at yahoo.com  Sat Aug 15 18:29:48 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 15:29:48 -0700 (PDT)
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
Message-ID: <663365.4887.qm@web52105.mail.re2.yahoo.com>

--- On Sat, 8/15/09, Ralf Gommers <ralf.gommers at googlemail.com> wrote:

> On Sat, Aug 15, 2009 at 12:44 PM,
> Charles R Harris <charlesr.harris at gmail.com>
> wrote:
> I'm not convinced it is a good idea, even
> pedagogically. First, all the python books I have are for
> the python-2.x versions, and second it saddles the students
> with having to make a distinction between printing single
> items vs printing multiple items. It's better, I think,
> to wait until we have a python 3.x version of numpy.
> 
> Okay fair enough. Change or no change, I have no strong
> opinion, as long as it is done consistently for all
> docstrings. 
> 
At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.

DG

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From gael.varoquaux at normalesup.org  Sat Aug 15 18:33:46 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 16 Aug 2009 00:33:46 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <663365.4887.qm@web52105.mail.re2.yahoo.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
Message-ID: <20090815223346.GF31205@phare.normalesup.org>

On Sat, Aug 15, 2009 at 03:29:48PM -0700, David Goldsmith wrote:
> --- On Sat, 8/15/09, Ralf Gommers <ralf.gommers at googlemail.com> wrote:

> > On Sat, Aug 15, 2009 at 12:44 PM,
> > Charles R Harris <charlesr.harris at gmail.com>
> > wrote:
> > I'm not convinced it is a good idea, even
> > pedagogically. First, all the python books I have are for
> > the python-2.x versions, and second it saddles the students
> > with having to make a distinction between printing single
> > items vs printing multiple items. It's better, I think,
> > to wait until we have a python 3.x version of numpy.

> > Okay fair enough. Change or no change, I have no strong
> > opinion, as long as it is done consistently for all
> > docstrings. 

> At this point in this thread, I've reached the "strong opinion" that
> I'm +1 w/ Charles: speaking as someone who's gung-ho to make the
> docstrings as pedagogically useful as possible, I feel code-docstring
> consistency trumps this - IMO, the docstrings should not be be changed
> prior to the code itself being changed.

But how does this fit with the fact that people using IPython are seeing
this change (I believe that's the third time I ask the question)?

That what worries me most, docstrings or no docstrings. I came on the
problem while testing my tutorial on IPython, as a naive user.

Ga?l


From emmanuelle.gouillart at normalesup.org  Sat Aug 15 18:37:12 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Sun, 16 Aug 2009 00:37:12 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <663365.4887.qm@web52105.mail.re2.yahoo.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
Message-ID: <20090815223712.GB7514@phare.normalesup.org>

> At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.

OK, you can revert the changes very easily as I paid attention to save 
versions including only the modifications of print statements.

Emmanuelle


From stefan at sun.ac.za  Sat Aug 15 18:47:21 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 15 Aug 2009 15:47:21 -0700
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815223712.GB7514@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com> 
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
Message-ID: <9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>

2009/8/15 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
>> At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.
>
> OK, you can revert the changes very easily as I paid attention to save
> versions including only the modifications of print statements.

Don't revert those changes.  Using "print" as a function is more
consistent in style (one of the reasons it's been changed in 3.0).
Doing it now saves us a lot of trouble later on.

Regards
St?fan


From robert.kern at gmail.com  Sat Aug 15 18:57:45 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Aug 2009 17:57:45 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com> 
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org> 
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
Message-ID: <3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>

2009/8/15 St?fan van der Walt <stefan at sun.ac.za>:
> 2009/8/15 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
>>> At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.
>>
>> OK, you can revert the changes very easily as I paid attention to save
>> versions including only the modifications of print statements.
>
> Don't revert those changes. ?Using "print" as a function is more
> consistent in style (one of the reasons it's been changed in 3.0).
> Doing it now saves us a lot of trouble later on.

print is simply not a function in 2.x. Abusing the Python 2.x syntax
to pretend like it is a function is terribly inconsistent when showing
Python 2.x code. It is bad style to use print() in Python 2.x. The
semantics are *not* the same when executing Python 3.x print() code in
Python 2.x. numpy does not support Python 3.x, yet.

+1 on reverting the changes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From charlesr.harris at gmail.com  Sat Aug 15 18:58:11 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 15 Aug 2009 16:58:11 -0600
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
Message-ID: <e06186140908151558h6fa81356x5096846595d72257@mail.gmail.com>

2009/8/15 St?fan van der Walt <stefan at sun.ac.za>

> 2009/8/15 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
> >> At this point in this thread, I've reached the "strong opinion" that I'm
> +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as
> pedagogically useful as possible, I feel code-docstring consistency trumps
> this - IMO, the docstrings should not be be changed prior to the code itself
> being changed.
> >
> > OK, you can revert the changes very easily as I paid attention to save
> > versions including only the modifications of print statements.
>
> Don't revert those changes.  Using "print" as a function is more
> consistent in style (one of the reasons it's been changed in 3.0).
> Doing it now saves us a lot of trouble later on.
>

It doesn't save us trouble, 2to3 will make the needed changes, nor is it
consistent in style IMHO. As to ipython, I would call the behaviour some are
seeing a bug for python versions < 3.0.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/d8d70f07/attachment.html>

From robert.kern at gmail.com  Sat Aug 15 19:02:24 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Aug 2009 18:02:24 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815164835.GL16976@phare.normalesup.org>
References: <20090815135915.GA32560@phare.normalesup.org>
	<e06186140908150837x6a19cc46u5111a64e2675f84f@mail.gmail.com> 
	<dde7764a0908150924y69deb298qb13560db657012c2@mail.gmail.com> 
	<e06186140908150944yc5f9d81webcfefa514c3ac3a@mail.gmail.com> 
	<20090815164835.GL16976@phare.normalesup.org>
Message-ID: <3d375d730908151602t28f1ae8ao26909b902fc2f3ae@mail.gmail.com>

On Sat, Aug 15, 2009 at 11:48, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Sat, Aug 15, 2009 at 10:44:01AM -0600, Charles R Harris wrote:
>> ? ?I'm not convinced it is a good idea, even pedagogically. First, all the
>> ? ?python books I have are for the python-2.x versions, and second it saddles
>> ? ?the students with having to make a distinction between printing single
>> ? ?items vs printing multiple items. It's better, I think, to wait until we
>> ? ?have a python 3.x version of numpy.
>
> But, the real issue here is that students confronted with IPython will
> have to deal with this. At least on some systems, amongst others Ubuntu
> Jaunty (python 2.6, IPython 0.9.1):
>
> In [1]: print 1, 2, 3
> ------> print(1, 2, 3)
> (1, 2, 3)

This is a bug in IPython's autocall feature. It should not be
translating print statements.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ralf.gommers at googlemail.com  Sat Aug 15 19:02:57 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Sat, 15 Aug 2009 19:02:57 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815223346.GF31205@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223346.GF31205@phare.normalesup.org>
Message-ID: <dde7764a0908151602i2962068w5c52ea4002b74d2a@mail.gmail.com>

On Sat, Aug 15, 2009 at 6:33 PM, Gael Varoquaux <
gael.varoquaux at normalesup.org> wrote:

> On Sat, Aug 15, 2009 at 03:29:48PM -0700, David Goldsmith wrote:
> > --- On Sat, 8/15/09, Ralf Gommers <ralf.gommers at googlemail.com> wrote:
>
> > > On Sat, Aug 15, 2009 at 12:44 PM,
> > > Charles R Harris <charlesr.harris at gmail.com>
> > > wrote:
> > > I'm not convinced it is a good idea, even
> > > pedagogically. First, all the python books I have are for
> > > the python-2.x versions, and second it saddles the students
> > > with having to make a distinction between printing single
> > > items vs printing multiple items. It's better, I think,
> > > to wait until we have a python 3.x version of numpy.
>
> > > Okay fair enough. Change or no change, I have no strong
> > > opinion, as long as it is done consistently for all
> > > docstrings.
>
> > At this point in this thread, I've reached the "strong opinion" that
> > I'm +1 w/ Charles: speaking as someone who's gung-ho to make the
> > docstrings as pedagogically useful as possible, I feel code-docstring
> > consistency trumps this - IMO, the docstrings should not be be changed
> > prior to the code itself being changed.
>
> But how does this fit with the fact that people using IPython are seeing
> this change (I believe that's the third time I ask the question)?
>
> That what worries me most, docstrings or no docstrings. I came on the
> problem while testing my tutorial on IPython, as a naive user.


Changing the examples does not solve the inconsistency for all platforms and
users. As Charles said, most books and tutorials use the older syntax.
You'll eventually have to explain to new users the Python 2.x / 3.x
situation and its incompatibilities. Face to face in a tutorial is maybe
better for that than having your students confused later at home.

Cheers,
Ralf



>
>
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090815/bd6af62e/attachment.html>

From stefan at sun.ac.za  Sat Aug 15 19:03:40 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 15 Aug 2009 16:03:40 -0700
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com> 
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org> 
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com> 
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
Message-ID: <9457e7c80908151603jbadb014m856ff0f097e569fa@mail.gmail.com>

2009/8/15 Robert Kern <robert.kern at gmail.com>:
> print is simply not a function in 2.x. Abusing the Python 2.x syntax
> to pretend like it is a function is terribly inconsistent when showing
> Python 2.x code. It is bad style to use print() in Python 2.x. The
> semantics are *not* the same when executing Python 3.x print() code in
> Python 2.x. numpy does not support Python 3.x, yet.
>
> +1 on reverting the changes.

I don't feel strong enough about this to argue, so go ahead.

Cheers
St?fan


From d_l_goldsmith at yahoo.com  Sat Aug 15 19:05:28 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 23:05:28 +0000 (GMT)
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815223346.GF31205@phare.normalesup.org>
Message-ID: <700580.31449.qm@web52108.mail.re2.yahoo.com>

--- On Sat, 8/15/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> But how does this fit with the fact that people using
> IPython are seeing
> this change (I believe that's the third time I ask the

I don't know: how universally used is IPython?  I'm only installing it now on account of the tutorials; otherwise, so far, I've seen no need to and probably wouldn't, but am in the minority?  I honestly don't know (and suspect that no one has the global usage statistics to say for sure).  In other words, are we going to let the "problems" arising in one specialty package govern the behavior for the baseline library?

DG

> question)?
> 
> That what worries me most, docstrings or no docstrings. I
> came on the
> problem while testing my tutorial on IPython, as a naive
> user.
> 
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Sat Aug 15 19:07:54 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 16:07:54 -0700 (PDT)
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815223712.GB7514@phare.normalesup.org>
Message-ID: <332640.6013.qm@web52111.mail.re2.yahoo.com>

Thanks, Emmanuelle, but I don't mean to be a dictator - I genuinely see mine as a "vote" and am "merely" expressing my view on the matter...Perhaps this should be added to the Doc BoF agenda?

DG

--- On Sat, 8/15/09, Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org> wrote:

> From: Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>
> Subject: Re: [SciPy-dev] change print statements to print functions in docstrings
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Saturday, August 15, 2009, 3:37 PM
> > At this point in this thread,
> I've reached the "strong opinion" that I'm +1 w/ Charles:
> speaking as someone who's gung-ho to make the docstrings as
> pedagogically useful as possible, I feel code-docstring
> consistency trumps this - IMO, the docstrings should not be
> be changed prior to the code itself being changed.
> 
> OK, you can revert the changes very easily as I paid
> attention to save 
> versions including only the modifications of print
> statements.
> 
> Emmanuelle
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From gael.varoquaux at normalesup.org  Sat Aug 15 19:08:29 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sun, 16 Aug 2009 01:08:29 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
Message-ID: <20090815230829.GA30494@phare.normalesup.org>

On Sat, Aug 15, 2009 at 05:57:45PM -0500, Robert Kern wrote:
> 2009/8/15 St?fan van der Walt <stefan at sun.ac.za>:
> > 2009/8/15 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
> >>> At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.

> >> OK, you can revert the changes very easily as I paid attention to save
> >> versions including only the modifications of print statements.

> > Don't revert those changes. ?Using "print" as a function is more
> > consistent in style (one of the reasons it's been changed in 3.0).
> > Doing it now saves us a lot of trouble later on.

> print is simply not a function in 2.x. Abusing the Python 2.x syntax
> to pretend like it is a function is terribly inconsistent when showing
> Python 2.x code.

Well, actually, it seems that this is not completely true in 2.6. Print
seems to be both a statement and a builtin function (for this to work, I
can only think that the Python devs added some custom logic to the
parser). This is what confuses IPython. The strange thing is that they
behave differently. You can test this outside of IPython.

So you are advocating no transition period between print as a statement,
and print as a function, because if we are going to bite a bullet, we
might as well understand what we bite, right?

Ga?l


From robert.kern at gmail.com  Sat Aug 15 19:14:56 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Aug 2009 18:14:56 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815230829.GA30494@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com> 
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org> 
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com> 
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com> 
	<20090815230829.GA30494@phare.normalesup.org>
Message-ID: <3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>

On Sat, Aug 15, 2009 at 18:08, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Sat, Aug 15, 2009 at 05:57:45PM -0500, Robert Kern wrote:
>> 2009/8/15 St?fan van der Walt <stefan at sun.ac.za>:
>> > 2009/8/15 Emmanuelle Gouillart <emmanuelle.gouillart at normalesup.org>:
>> >>> At this point in this thread, I've reached the "strong opinion" that I'm +1 w/ Charles: speaking as someone who's gung-ho to make the docstrings as pedagogically useful as possible, I feel code-docstring consistency trumps this - IMO, the docstrings should not be be changed prior to the code itself being changed.
>
>> >> OK, you can revert the changes very easily as I paid attention to save
>> >> versions including only the modifications of print statements.
>
>> > Don't revert those changes. ?Using "print" as a function is more
>> > consistent in style (one of the reasons it's been changed in 3.0).
>> > Doing it now saves us a lot of trouble later on.
>
>> print is simply not a function in 2.x. Abusing the Python 2.x syntax
>> to pretend like it is a function is terribly inconsistent when showing
>> Python 2.x code.
>
> Well, actually, it seems that this is not completely true in 2.6. Print
> seems to be both a statement and a builtin function (for this to work, I
> can only think that the Python devs added some custom logic to the
> parser). This is what confuses IPython. The strange thing is that they
> behave differently. You can test this outside of IPython.

No, it is *not* a function in Python 2.x. There is no custom logic.
The fact that "print(1,2,3)" executes is a side effect of the grammar
not requiring a spacing between "print" and the expression "(1,2,3)".
It is equivalent to "print (1,2,3)" in Python 2.x and "print((1,2,3))"
in Python 3.x. Note that it is *not* equivalent to "print(1,2,3)" in
Python 3.x.

The special support for print as a function in Python 2.6 is *only*
active when one does a __future__ import.

http://docs.python.org/whatsnew/2.6.html#pep-3105-print-as-a-function

> So you are advocating no transition period between print as a statement,
> and print as a function, because if we are going to bite a bullet, we
> might as well understand what we bite, right?

Yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Sat Aug 15 20:01:33 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Aug 2009 19:01:33 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090815230829.GA30494@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com> 
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org> 
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com> 
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com> 
	<20090815230829.GA30494@phare.normalesup.org>
Message-ID: <3d375d730908151701j47d9d6d4l10b07766d3110a37@mail.gmail.com>

On Sat, Aug 15, 2009 at 18:08, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:

> So you are advocating no transition period between print as a statement,
> and print as a function, because if we are going to bite a bullet, we
> might as well understand what we bite, right?

If we were to drop support for Python 2.5, or make a slight fork for
2.6/3.x (not likely, but conceivable), I would be in favor of using
the 3.x print() syntax with the appropriate __future__ import noted in
the preface.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From robert.kern at gmail.com  Sat Aug 15 20:09:37 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 15 Aug 2009 19:09:37 -0500
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <700580.31449.qm@web52108.mail.re2.yahoo.com>
References: <20090815223346.GF31205@phare.normalesup.org>
	<700580.31449.qm@web52108.mail.re2.yahoo.com>
Message-ID: <3d375d730908151709k792e15b7n9c2ee579c4ee29c2@mail.gmail.com>

On Sat, Aug 15, 2009 at 18:05, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> --- On Sat, 8/15/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
>
>> But how does this fit with the fact that people using
>> IPython are seeing
>> this change (I believe that's the third time I ask the
>
> I don't know: how universally used is IPython? ?I'm only installing it now on account of the tutorials; otherwise, so far, I've seen no need to and probably wouldn't, but am in the minority? ?I honestly don't know (and suspect that no one has the global usage statistics to say for sure). ?In other words, are we going to let the "problems" arising in one specialty package govern the behavior for the baseline library?

It's not universal (nothing is), but it is immensely popular among our
install-base. You see no need for it *now*, but just wait until you've
taken it out for a spin for a while.

Its features are important to consider sometimes. For example, some of
the decisions about the formatting of the documentation (particularly
readable plaintext math) were driven by the large use of IPython's "?"
feature for reading docstrings (the builtin help() is similar in
functionality but more cumbersome; I doubt I would have been as
insistent about readable plaintext if the alternatives were just
reading the HTML and help()).

However, I don't think incidental bugs in IPython should drive design choices.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From josef.pktd at gmail.com  Sat Aug 15 20:34:25 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 15 Aug 2009 20:34:25 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <3d375d730908151709k792e15b7n9c2ee579c4ee29c2@mail.gmail.com>
References: <20090815223346.GF31205@phare.normalesup.org>
	<700580.31449.qm@web52108.mail.re2.yahoo.com>
	<3d375d730908151709k792e15b7n9c2ee579c4ee29c2@mail.gmail.com>
Message-ID: <1cd32cbb0908151734v1fb5b128v3be5067713f66c8d@mail.gmail.com>

On Sat, Aug 15, 2009 at 8:09 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Sat, Aug 15, 2009 at 18:05, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>> --- On Sat, 8/15/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:
>>
>>> But how does this fit with the fact that people using
>>> IPython are seeing
>>> this change (I believe that's the third time I ask the
>>
>> I don't know: how universally used is IPython? ?I'm only installing it now on account of the tutorials; otherwise, so far, I've seen no need to and probably wouldn't, but am in the minority? ?I honestly don't know (and suspect that no one has the global usage statistics to say for sure). ?In other words, are we going to let the "problems" arising in one specialty package govern the behavior for the baseline library?
>
> It's not universal (nothing is), but it is immensely popular among our
> install-base. You see no need for it *now*, but just wait until you've
> taken it out for a spin for a while.
>
> Its features are important to consider sometimes. For example, some of
> the decisions about the formatting of the documentation (particularly
> readable plaintext math) were driven by the large use of IPython's "?"
> feature for reading docstrings (the builtin help() is similar in
> functionality but more cumbersome; I doubt I would have been as
> insistent about readable plaintext if the alternatives were just
> reading the HTML and help()).
>
> However, I don't think incidental bugs in IPython should drive design choices.
>
> --
> Robert Kern

Just out of curiosity, do you have a rough guesstimate on the fraction
on Windows users in the install base, and if a sizable fraction of
them is using ipython?

At least on the mailing list, Windows users seem to be a small
minority, but that could also be a strong selection bias.
I'm one of the silent minority that never got friendly with ipython
even after several tries.

Josef


>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
> ?-- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Sat Aug 15 21:27:21 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 18:27:21 -0700 (PDT)
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <1cd32cbb0908151734v1fb5b128v3be5067713f66c8d@mail.gmail.com>
Message-ID: <841938.69776.qm@web52111.mail.re2.yahoo.com>

I'm sorry, I honestly didn't foresee my comments touching such a nerve; really now, turning this thread into a debate on the merits of IPython - well, that really should be a separate thread (at the very least), IMHO. :-)

DG

--- On Sat, 8/15/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
> Subject: Re: [SciPy-dev] change print statements to print functions in docstrings
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Saturday, August 15, 2009, 5:34 PM
> On Sat, Aug 15, 2009 at 8:09 PM,
> Robert Kern<robert.kern at gmail.com>
> wrote:
> > On Sat, Aug 15, 2009 at 18:05, David Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> >> --- On Sat, 8/15/09, Gael Varoquaux <gael.varoquaux at normalesup.org>
> wrote:
> >>
> >>> But how does this fit with the fact that
> people using
> >>> IPython are seeing
> >>> this change (I believe that's the third time I
> ask the
> >>
> >> I don't know: how universally used is IPython?
> ?I'm only installing it now on account of the tutorials;
> otherwise, so far, I've seen no need to and probably
> wouldn't, but am in the minority? ?I honestly don't know
> (and suspect that no one has the global usage statistics to
> say for sure). ?In other words, are we going to let the
> "problems" arising in one specialty package govern the
> behavior for the baseline library?
> >
> > It's not universal (nothing is), but it is immensely
> popular among our
> > install-base. You see no need for it *now*, but just
> wait until you've
> > taken it out for a spin for a while.
> >
> > Its features are important to consider sometimes. For
> example, some of
> > the decisions about the formatting of the
> documentation (particularly
> > readable plaintext math) were driven by the large use
> of IPython's "?"
> > feature for reading docstrings (the builtin help() is
> similar in
> > functionality but more cumbersome; I doubt I would
> have been as
> > insistent about readable plaintext if the alternatives
> were just
> > reading the HTML and help()).
> >
> > However, I don't think incidental bugs in IPython
> should drive design choices.
> >
> > --
> > Robert Kern
> 
> Just out of curiosity, do you have a rough guesstimate on
> the fraction
> on Windows users in the install base, and if a sizable
> fraction of
> them is using ipython?
> 
> At least on the mailing list, Windows users seem to be a
> small
> minority, but that could also be a strong selection bias.
> I'm one of the silent minority that never got friendly with
> ipython
> even after several tries.
> 
> Josef
> 
> 
> >
> > "I have come to believe that the whole world is an
> enigma, a harmless
> > enigma that is made terrible by our own mad attempt to
> interpret it as
> > though it had an underlying truth."
> > ?-- Umberto Eco
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Sat Aug 15 21:39:49 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sat, 15 Aug 2009 21:39:49 -0400
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <841938.69776.qm@web52111.mail.re2.yahoo.com>
References: <1cd32cbb0908151734v1fb5b128v3be5067713f66c8d@mail.gmail.com>
	<841938.69776.qm@web52111.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908151839m2ca00502v4483363fc3632431@mail.gmail.com>

On Sat, Aug 15, 2009 at 9:27 PM, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> I'm sorry, I honestly didn't foresee my comments touching such a nerve; really now, turning this thread into a debate on the merits of IPython - well, that really should be a separate thread (at the very least), IMHO. :-)

I'm not discussing the merits of IPython, editors and shells are
mostly a taste question, and it's pretty useless to argue about taste.
There are users that like Eclipse and Idle and Spyder, and there are
users who like the commandline, Vims and Emacs and other things that I
only ever read about in newsgroups but never seen in real life. And
there are lots if others ....

I just wanted a rough idea of how "lonely" I am as one of the few
known Windows users. But we are pretty well treated with installers
and htmlhelp. (thanks to David and Pauli)

Josef

>
> DG
>
> --- On Sat, 8/15/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
>
>> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
>> Subject: Re: [SciPy-dev] change print statements to print functions in docstrings
>> To: "SciPy Developers List" <scipy-dev at scipy.org>
>> Date: Saturday, August 15, 2009, 5:34 PM
>> On Sat, Aug 15, 2009 at 8:09 PM,
>> Robert Kern<robert.kern at gmail.com>
>> wrote:
>> > On Sat, Aug 15, 2009 at 18:05, David Goldsmith<d_l_goldsmith at yahoo.com>
>> wrote:
>> >> --- On Sat, 8/15/09, Gael Varoquaux <gael.varoquaux at normalesup.org>
>> wrote:
>> >>
>> >>> But how does this fit with the fact that
>> people using
>> >>> IPython are seeing
>> >>> this change (I believe that's the third time I
>> ask the
>> >>
>> >> I don't know: how universally used is IPython?
>> ?I'm only installing it now on account of the tutorials;
>> otherwise, so far, I've seen no need to and probably
>> wouldn't, but am in the minority? ?I honestly don't know
>> (and suspect that no one has the global usage statistics to
>> say for sure). ?In other words, are we going to let the
>> "problems" arising in one specialty package govern the
>> behavior for the baseline library?
>> >
>> > It's not universal (nothing is), but it is immensely
>> popular among our
>> > install-base. You see no need for it *now*, but just
>> wait until you've
>> > taken it out for a spin for a while.
>> >
>> > Its features are important to consider sometimes. For
>> example, some of
>> > the decisions about the formatting of the
>> documentation (particularly
>> > readable plaintext math) were driven by the large use
>> of IPython's "?"
>> > feature for reading docstrings (the builtin help() is
>> similar in
>> > functionality but more cumbersome; I doubt I would
>> have been as
>> > insistent about readable plaintext if the alternatives
>> were just
>> > reading the HTML and help()).
>> >
>> > However, I don't think incidental bugs in IPython
>> should drive design choices.
>> >
>> > --
>> > Robert Kern
>>
>> Just out of curiosity, do you have a rough guesstimate on
>> the fraction
>> on Windows users in the install base, and if a sizable
>> fraction of
>> them is using ipython?
>>
>> At least on the mailing list, Windows users seem to be a
>> small
>> minority, but that could also be a strong selection bias.
>> I'm one of the silent minority that never got friendly with
>> ipython
>> even after several tries.
>>
>> Josef
>>
>>
>> >
>> > "I have come to believe that the whole world is an
>> enigma, a harmless
>> > enigma that is made terrible by our own mad attempt to
>> interpret it as
>> > though it had an underlying truth."
>> > ?-- Umberto Eco
>> > _______________________________________________
>> > Scipy-dev mailing list
>> > Scipy-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From d_l_goldsmith at yahoo.com  Sun Aug 16 02:32:11 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 15 Aug 2009 23:32:11 -0700 (PDT)
Subject: [SciPy-dev] Documentation BoF resched
Message-ID: <364921.7061.qm@web52104.mail.re2.yahoo.com>

Wed. Lunch - out

Thurs. Lunch (12:30-2:30), Friday Lunch (same), or nominate.

DG


      


From emmanuelle.gouillart at normalesup.org  Sun Aug 16 05:17:56 2009
From: emmanuelle.gouillart at normalesup.org (Emmanuelle Gouillart)
Date: Sun, 16 Aug 2009 11:17:56 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
	<20090815230829.GA30494@phare.normalesup.org>
	<3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
Message-ID: <20090816091756.GA13497@phare.normalesup.org>

On Sat, Aug 15, 2009 at 06:14:56PM -0500, Robert Kern wrote:
[...]
> >> print is simply not a function in 2.x. Abusing the Python 2.x syntax
> >> to pretend like it is a function is terribly inconsistent when showing
> >> Python 2.x code.
[...]

All right, I was quite convinced by Robert's argument about print not
being a function in 2.x, so I reverted my changes.

Emmanuelle


From jh at physics.ucf.edu  Sun Aug 16 10:40:51 2009
From: jh at physics.ucf.edu (Joe Harrington)
Date: Sun, 16 Aug 2009 10:40:51 -0400
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8fxcc6aua.fsf@glup.physics.ucf.edu> (message from Joe
	Harrington on Fri, 31 Jul 2009 13:06:37 -0400)
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
Message-ID: <wl8bpmfn7nw.fsf@glup.physics.ucf.edu>

I've finally had time to look at all the replies to this thread.
There were dozens, so rather than quoting and responding to everyone
individually, I'll summarize.  The short version is that due to an
early misunderstanding, we spent a lot of bandwidth generating
agreement that masqueraded as dissent!  In the end, I think we have
general agreement (and no specific dissent) to the idea of an
organization dedicated to development of scientific tools in Python
and gathering and disbursing funds to that end.  We even agree on our
major priorities.  So, I propose that we move forward with planning.
There's a BoF proposal at the end.


Here's the longer version:

1. Objection: The mission statement stuffs too much into one package.
The scipy package doesn't need a GUI!  (Long post by Gael 2009-08-01
22:52:16, shorter one by Robert 2009-08-04 19:37:01, many others.)

My apologies to these fine gentlemen and others who discussed on this
threadlet, but this was a bit of a bandwidth waster since I started my
proposed mission statement with "(The toolstack)", not "SciPy" or
"scipy".  Of course nobody would go to such lengths just for one
package, nor propose stuffing so much into it that exists elsewhere
and is in wide use already (GUIs, interactive shells, etc.).  We're
talking broadly about scientific use of Python.

Robert proposed a name change to avoid such ambiguity.  SciPython?
SciPyStack? Py4Sci?  Scientific Python is taken.  I really prefer
SciPy, as it has branding already, but perhaps SciPyStack is ok
informally.  I think we're stuck with SciPy for formal docs, web site,
etc, just like JPL (which has not studied the propulsion of jets or
rocket engines for decades).  What this means is:

a. POSTERS BE CLEAR: specify package or toolstack when you talk about
scipy.  Use "SciPy" for the toolstack and "scipy" for the package, but
don't rely on that alone.  (note: I did this!)

b. RESPONDENTS BE CAREFUL: double-check what the poster wrote before
replying if it's about "scipy" or "SciPy".


2. It's important for the package structure to be light.

Yes!  I am not proposing to change the package structure at all.
People need to be able to pick and choose, and it needs to be light
for many reasons, such as OLPC.

However, as a practical matter, I know of *nobody* who is a heavy user
and who does not install a significant number of packages.  We install
about 15 python-related packages now for our group.  It has become a
nightmare that takes my very experienced system manager, an Ubuntu
developer with a PhD in computer science, several days.  Basically, if
you want everything current (e.g., to get recent docs in numpy, or HDF
libraries that actually work), it is hard to do a consistent build
without doing a lot of patching.  Clearly, most potential users cannot
tolerate that, or even do it.

So, I would like to see packaging *coordination* such that a
monolithic install is as trivial for the user as it is to install one
package.  From my discussion with hundreds of users who are sitting on
the sidelines in my discipline alone, this and docs are essentially
what they are waiting for.  Done right, I think most of the relevant
package authors would welcome the opportunity to coordinate (but I
don't speak for them).  Exactly what and how is a matter to discuss
but let's get the overall project structure settled first.


3. This is going to be a lot of work, particularly IDEs and GUIs!  I
don't want to burn out or hurt my career.

People should not burn out or hurt their careers on service projects!
It's the first rule of academia.  There are tons of workers who will
happily contribute small bits if they were served in nice-sized chunks
and integrated by someone when finished.  There are lots more willing
to work for pay, or even partial pay.  This proposal is a way of
moving to that model, which might also be called "many hands make
light work".  I think the doc project proves the viability of the
paid-coordinator model.  For IDEs and GUIs, there are good starts
already.  With enough momentum, we can directly fund development to
provide something better.


4. Why not use Sage/EPD/etc.?

Those solve the monolithic packaging problem, usually inelegantly but
that's the only way to do it today.  There is plenty broken in our own
house before we even get to the monolithic packaging problem, like
missing documentation, code cleanups, API
stabilization/rationalization, and getting packages to build together
for all platforms.

Once that's done, Sage's and our goals might well merge.  Still, Sage
has its own focus, and it is not scientific modeling and data
analysis.  EPD focuses on Windows.  My ideal would be that our
much-improved packaging makes rolling a monolithic distro for a
particular purpose much easier, in some cases as easy as publishing a
meta-package that pulls in what you want as dependencies.  Then STScI
can release an astronomy distro, someone else can release a
neuroscience distro, and Sage can release their thing for math, all
benefitting from a toolstack that builds cleanly together.


5. Packaging is hard.  What we need is (long description of packaging
needs)...

What we need is fully-automated builds that populate PPAs on all
platforms for every version and a nightly snapshot of every package,
and tests run nightly that show they still work together.  There is a
tool that does this.  It was funded by the US National Science
Foundation and is required for applicants to many of their grant
programs.  It is called metronome, formerly NMI Build and Test Suite:

http://nmi.cs.wisc.edu/

At last count, they build on 46 platforms.  Getting there will be
hard.  That is what money is for.

Story: When I was a freshman in 1984, there was a free student
computing system at MIT called Multics that was run by a student
group.  Your account had a certain amount of "money", which it charged
for CPU usage and printing.  When you ran out, you asked for more
"money" and got it for free.  There was a sign on the door to the
group's office that said, "If you need more money, use the
request-extension command."  But, it was done with funky colors and
words going every which way, and to me it initially read, "If you need
more, use money, the request-extension command."  I've looked at money
in a different light ever since...


Gael Varoquaux 2009-08-01 22:52:16 GMT writes:

> Specifically, I would love to see an official umbrella project for
> BSD-licensed tools for building scientific projects with Python. As the
> "scipy" name is well branded (through the website, and the conference),
> we could call this the 'scipy project'. I would personally like to limit
> wheel reinvention and have preferred solutions for the various bricks (I
> am thinking of the unfortunate Chaco versus Matplotlib situation, where I
> have to depend on both libraries that complement each other). 

This is exactly what I am proposing.  Pretty much everything else in
the message was based on a misunderstandings of my intent about
package vs. toolstack.  I would not limit it to BSD-licensed tools,
but would want that to continue to be a requirement for the core
stuff, and likely for grants we would write.  In other words, if a
benefactor came along wanting to give some cash to a field-specific
project that was under GPL, fine, I'd be glad to funnel their money to
the developers.

> first, as Robert points out,
> telling somebody what to do will not achieve anything. I am already way
> too busy scratching my own itches. 

It works well if you are paying them.  What is amazing (witness the
doc project) is that if just one person is paid to organize an area,
lots of people flock to the project and pitch in doing small tasks.
Not everyone.  Not even most people.  But enough.  That is what the
funding is for.  It's the request_extension command!  Specifically,
extension of effort on the part of someone who would otherwise find
other uses for their time.

> Second, who will find the time to take care of this?

I've been doing it since Spring 2008 for the doc project.  Hopefully a
few others will join me so we can write some grants, start a funding
organization, and launch something more permanent and far-reaching.

I've proposed a BoF on this topic.  I immodestly think it could be the
most important of the meeting.  I propose Thursday at 8:30 (I think
that 2.5 hours for dinner is too much and that we should start the
BoFs much earlier, like at 7:30, so we can do an early and a late set
of BoFs.  I'm not sure what the reception is.  Is it dinner?  Or just
a delay in the start of dinner?).  Alternatively, we can do it Friday
over lunch, though that depends on getting some box lunches.  Proposed
format:

Organization, Funding, and Future Direction of SciPy
(coordinator: Joe Harrington, sergeant-at-arms: David Goldsmith)

I'd like to spend a strict 10 minutes on each of these, cutting off
discussion and moving on after each item. After all 6 items, we can
continue discussion on any item:

    * What are our long-term goals?
    * What are our current strengths and weaknesses?
    * How is our current Steering Committee/grass-roots model working?
    * What would we do with funding? Would it require a change in how
        the community operates? 
    * How can we get funding?
    * In the large, how should we proceed?

--jh--


From cohen at lpta.in2p3.fr  Sun Aug 16 13:01:13 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Sun, 16 Aug 2009 19:01:13 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8bpmfn7nw.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<wl8bpmfn7nw.fsf@glup.physics.ucf.edu>
Message-ID: <4A883B59.9020109@lpta.in2p3.fr>

Hi Joe, just one quick comment : I really think that you cannot use 
scipy name without certainly creating misunderstandings down the line. 
It is crazy in my mind to rely on 2 upper/lowercases to differentiate 2 
different "objects". I do not like the difference package/toolstack 
either. For one thing you may have more confusion coming from the non 
English native speakers than you really wish!
Why no Py4Science? It does convey the ultimate goal of this effort, and 
I only saw it in the context of ipython and matplotlib : first hit from 
google is http://ipython.scipy.org/moin/Py4Science which was a practical 
workshop in python usage for scientific work (I think content still 
lives in matplotlib SVN).

anyway, my two cents.....

Johann

Joe Harrington wrote:
> I've finally had time to look at all the replies to this thread.
> There were dozens, so rather than quoting and responding to everyone
> individually, I'll summarize.  The short version is that due to an
> early misunderstanding, we spent a lot of bandwidth generating
> agreement that masqueraded as dissent!  In the end, I think we have
> general agreement (and no specific dissent) to the idea of an
> organization dedicated to development of scientific tools in Python
> and gathering and disbursing funds to that end.  We even agree on our
> major priorities.  So, I propose that we move forward with planning.
> There's a BoF proposal at the end.
>
>
> Here's the longer version:
>
> 1. Objection: The mission statement stuffs too much into one package.
> The scipy package doesn't need a GUI!  (Long post by Gael 2009-08-01
> 22:52:16, shorter one by Robert 2009-08-04 19:37:01, many others.)
>
> My apologies to these fine gentlemen and others who discussed on this
> threadlet, but this was a bit of a bandwidth waster since I started my
> proposed mission statement with "(The toolstack)", not "SciPy" or
> "scipy".  Of course nobody would go to such lengths just for one
> package, nor propose stuffing so much into it that exists elsewhere
> and is in wide use already (GUIs, interactive shells, etc.).  We're
> talking broadly about scientific use of Python.
>
> Robert proposed a name change to avoid such ambiguity.  SciPython?
> SciPyStack? Py4Sci?  Scientific Python is taken.  I really prefer
> SciPy, as it has branding already, but perhaps SciPyStack is ok
> informally.  I think we're stuck with SciPy for formal docs, web site,
> etc, just like JPL (which has not studied the propulsion of jets or
> rocket engines for decades).  What this means is:
>
> a. POSTERS BE CLEAR: specify package or toolstack when you talk about
> scipy.  Use "SciPy" for the toolstack and "scipy" for the package, but
> don't rely on that alone.  (note: I did this!)
>
> b. RESPONDENTS BE CAREFUL: double-check what the poster wrote before
> replying if it's about "scipy" or "SciPy".
>
>
> 2. It's important for the package structure to be light.
>
> Yes!  I am not proposing to change the package structure at all.
> People need to be able to pick and choose, and it needs to be light
> for many reasons, such as OLPC.
>
> However, as a practical matter, I know of *nobody* who is a heavy user
> and who does not install a significant number of packages.  We install
> about 15 python-related packages now for our group.  It has become a
> nightmare that takes my very experienced system manager, an Ubuntu
> developer with a PhD in computer science, several days.  Basically, if
> you want everything current (e.g., to get recent docs in numpy, or HDF
> libraries that actually work), it is hard to do a consistent build
> without doing a lot of patching.  Clearly, most potential users cannot
> tolerate that, or even do it.
>
> So, I would like to see packaging *coordination* such that a
> monolithic install is as trivial for the user as it is to install one
> package.  From my discussion with hundreds of users who are sitting on
> the sidelines in my discipline alone, this and docs are essentially
> what they are waiting for.  Done right, I think most of the relevant
> package authors would welcome the opportunity to coordinate (but I
> don't speak for them).  Exactly what and how is a matter to discuss
> but let's get the overall project structure settled first.
>
>
> 3. This is going to be a lot of work, particularly IDEs and GUIs!  I
> don't want to burn out or hurt my career.
>
> People should not burn out or hurt their careers on service projects!
> It's the first rule of academia.  There are tons of workers who will
> happily contribute small bits if they were served in nice-sized chunks
> and integrated by someone when finished.  There are lots more willing
> to work for pay, or even partial pay.  This proposal is a way of
> moving to that model, which might also be called "many hands make
> light work".  I think the doc project proves the viability of the
> paid-coordinator model.  For IDEs and GUIs, there are good starts
> already.  With enough momentum, we can directly fund development to
> provide something better.
>
>
> 4. Why not use Sage/EPD/etc.?
>
> Those solve the monolithic packaging problem, usually inelegantly but
> that's the only way to do it today.  There is plenty broken in our own
> house before we even get to the monolithic packaging problem, like
> missing documentation, code cleanups, API
> stabilization/rationalization, and getting packages to build together
> for all platforms.
>
> Once that's done, Sage's and our goals might well merge.  Still, Sage
> has its own focus, and it is not scientific modeling and data
> analysis.  EPD focuses on Windows.  My ideal would be that our
> much-improved packaging makes rolling a monolithic distro for a
> particular purpose much easier, in some cases as easy as publishing a
> meta-package that pulls in what you want as dependencies.  Then STScI
> can release an astronomy distro, someone else can release a
> neuroscience distro, and Sage can release their thing for math, all
> benefitting from a toolstack that builds cleanly together.
>
>
> 5. Packaging is hard.  What we need is (long description of packaging
> needs)...
>
> What we need is fully-automated builds that populate PPAs on all
> platforms for every version and a nightly snapshot of every package,
> and tests run nightly that show they still work together.  There is a
> tool that does this.  It was funded by the US National Science
> Foundation and is required for applicants to many of their grant
> programs.  It is called metronome, formerly NMI Build and Test Suite:
>
> http://nmi.cs.wisc.edu/
>
> At last count, they build on 46 platforms.  Getting there will be
> hard.  That is what money is for.
>
> Story: When I was a freshman in 1984, there was a free student
> computing system at MIT called Multics that was run by a student
> group.  Your account had a certain amount of "money", which it charged
> for CPU usage and printing.  When you ran out, you asked for more
> "money" and got it for free.  There was a sign on the door to the
> group's office that said, "If you need more money, use the
> request-extension command."  But, it was done with funky colors and
> words going every which way, and to me it initially read, "If you need
> more, use money, the request-extension command."  I've looked at money
> in a different light ever since...
>
>
> Gael Varoquaux 2009-08-01 22:52:16 GMT writes:
>
>   
>> Specifically, I would love to see an official umbrella project for
>> BSD-licensed tools for building scientific projects with Python. As the
>> "scipy" name is well branded (through the website, and the conference),
>> we could call this the 'scipy project'. I would personally like to limit
>> wheel reinvention and have preferred solutions for the various bricks (I
>> am thinking of the unfortunate Chaco versus Matplotlib situation, where I
>> have to depend on both libraries that complement each other). 
>>     
>
> This is exactly what I am proposing.  Pretty much everything else in
> the message was based on a misunderstandings of my intent about
> package vs. toolstack.  I would not limit it to BSD-licensed tools,
> but would want that to continue to be a requirement for the core
> stuff, and likely for grants we would write.  In other words, if a
> benefactor came along wanting to give some cash to a field-specific
> project that was under GPL, fine, I'd be glad to funnel their money to
> the developers.
>
>   
>> first, as Robert points out,
>> telling somebody what to do will not achieve anything. I am already way
>> too busy scratching my own itches. 
>>     
>
> It works well if you are paying them.  What is amazing (witness the
> doc project) is that if just one person is paid to organize an area,
> lots of people flock to the project and pitch in doing small tasks.
> Not everyone.  Not even most people.  But enough.  That is what the
> funding is for.  It's the request_extension command!  Specifically,
> extension of effort on the part of someone who would otherwise find
> other uses for their time.
>
>   
>> Second, who will find the time to take care of this?
>>     
>
> I've been doing it since Spring 2008 for the doc project.  Hopefully a
> few others will join me so we can write some grants, start a funding
> organization, and launch something more permanent and far-reaching.
>
> I've proposed a BoF on this topic.  I immodestly think it could be the
> most important of the meeting.  I propose Thursday at 8:30 (I think
> that 2.5 hours for dinner is too much and that we should start the
> BoFs much earlier, like at 7:30, so we can do an early and a late set
> of BoFs.  I'm not sure what the reception is.  Is it dinner?  Or just
> a delay in the start of dinner?).  Alternatively, we can do it Friday
> over lunch, though that depends on getting some box lunches.  Proposed
> format:
>
> Organization, Funding, and Future Direction of SciPy
> (coordinator: Joe Harrington, sergeant-at-arms: David Goldsmith)
>
> I'd like to spend a strict 10 minutes on each of these, cutting off
> discussion and moving on after each item. After all 6 items, we can
> continue discussion on any item:
>
>     * What are our long-term goals?
>     * What are our current strengths and weaknesses?
>     * How is our current Steering Committee/grass-roots model working?
>     * What would we do with funding? Would it require a change in how
>         the community operates? 
>     * How can we get funding?
>     * In the large, how should we proceed?
>
> --jh--
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>   


From robert.kern at gmail.com  Sun Aug 16 13:09:07 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sun, 16 Aug 2009 12:09:07 -0500
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <wl8bpmfn7nw.fsf@glup.physics.ucf.edu>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>
	<wl8bpmfn7nw.fsf@glup.physics.ucf.edu>
Message-ID: <3d375d730908161009y6d16848eq388af2a2888d4326@mail.gmail.com>

On Sun, Aug 16, 2009 at 09:40, Joe Harrington<jh at physics.ucf.edu> wrote:

>?EPD focuses on Windows.

Uh, no.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From cohen at lpta.in2p3.fr  Sun Aug 16 13:09:07 2009
From: cohen at lpta.in2p3.fr (Johann Cohen-Tanugi)
Date: Sun, 16 Aug 2009 19:09:07 +0200
Subject: [SciPy-dev] SciPy Foundation
In-Reply-To: <3d375d730908161009y6d16848eq388af2a2888d4326@mail.gmail.com>
References: <wl8fxcc6aua.fsf@glup.physics.ucf.edu>	<wl8bpmfn7nw.fsf@glup.physics.ucf.edu>
	<3d375d730908161009y6d16848eq388af2a2888d4326@mail.gmail.com>
Message-ID: <4A883D33.9080108@lpta.in2p3.fr>

Maybe a confusion with Python(x,y)? I believe it has a very short 
release cycle (at least in the recent past), which seems to be what Joe 
would like to see happening for the toolstack, so that many different 
flavors of science can easily have an updated set of packages that meet 
their needs. I guess EPD has a much longer release cycle.....

Johann


Robert Kern wrote:
> On Sun, Aug 16, 2009 at 09:40, Joe Harrington<jh at physics.ucf.edu> wrote:
>
>   
>>  EPD focuses on Windows.
>>     
>
> Uh, no.
>
>   


From d_l_goldsmith at yahoo.com  Mon Aug 17 01:35:07 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 16 Aug 2009 22:35:07 -0700 (PDT)
Subject: [SciPy-dev] Don't understand these results
Message-ID: <609231.20172.qm@web52102.mail.re2.yahoo.com>

>>> c = np.array(['aAaAaA', '  aA  ', 'abBABba']).view(np.chararray); c
chararray(['aAaAaA', '  aA', 'abBABba'],
      dtype='|S7')
>>> c.count('aA', end=3)
array([3, 1, 0])
>>> c.count('aA', end=2)
array([3, 1, 0])
>>> c.count('aA', end=1)
array([3, 1, 0])
>>> c.count('aA', end=0)
array([3, 1, 0])

???

DG


      


From pgmdevlist at gmail.com  Mon Aug 17 02:19:36 2009
From: pgmdevlist at gmail.com (Pierre GM)
Date: Mon, 17 Aug 2009 02:19:36 -0400
Subject: [SciPy-dev] Don't understand these results
In-Reply-To: <609231.20172.qm@web52102.mail.re2.yahoo.com>
References: <609231.20172.qm@web52102.mail.re2.yahoo.com>
Message-ID: <A75569B4-8849-446E-93FE-EE22160837CE@gmail.com>


On Aug 17, 2009, at 1:35 AM, David Goldsmith wrote:

>>>> c = np.array(['aAaAaA', '  aA  ', 'abBABba']).view(np.chararray); c
> chararray(['aAaAaA', '  aA', 'abBABba'],
>      dtype='|S7')
>>>> c.count('aA', end=3)
> array([3, 1, 0])
>
> ???

Have you checked the code ?

The input of the method ('aA',start,end)  are checked for consistency.  
However, a problem is that as soon any value is 0 or None or False,  
the following values are not checked and just discarded (that's line  
171 in defchararray).
When you call the `count` method, `start` defaults to None, meaning  
that the `end` parameter is never taken into account.

That's clearly a bug, actually 2 bugs:
* `start` should default to 0 in that case (you need it in the `count`  
method of strings)
* when one of the parameter is not None, it should be taken into  
account.
 >>> if not chk or (chk.dtype is object_ and chk.item() is None):
should become
 >>> if (chk is None) of (chk.dtype is object_ and chk.item() is None):

I don't have time to spend on fixing that right now, sorry. Try to  
fill a ticket on the numpy trac.
Many thanks for reporting.







From d_l_goldsmith at yahoo.com  Mon Aug 17 02:34:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sun, 16 Aug 2009 23:34:20 -0700 (PDT)
Subject: [SciPy-dev] Don't understand these results
In-Reply-To: <A75569B4-8849-446E-93FE-EE22160837CE@gmail.com>
Message-ID: <276765.36101.qm@web52105.mail.re2.yahoo.com>

--- On Sun, 8/16/09, Pierre GM <pgmdevlist at gmail.com> wrote:

> On Aug 17, 2009, at 1:35 AM, David Goldsmith wrote:
> 
> >>>> c = np.array(['aAaAaA', '? aA?
> ', 'abBABba']).view(np.chararray); c
> > chararray(['aAaAaA', '? aA', 'abBABba'],
> >? ? ? dtype='|S7')
> >>>> c.count('aA', end=3)
> > array([3, 1, 0])
> >
> > ???
> 
> Have you checked the code ?

I "take the 5th." ;-)

> The input of the method ('aA',start,end)? are checked
> for consistency.? 
> However, a problem is that as soon any value is 0 or None
> or False,? 
> the following values are not checked and just discarded
> (that's line? 
> 171 in defchararray).
> When you call the `count` method, `start` defaults to None,
> meaning? 
> that the `end` parameter is never taken into account.

That would explain it.

> That's clearly a bug, actually 2 bugs:
> * `start` should default to 0 in that case (you need it in
> the `count`? 
> method of strings)
> * when one of the parameter is not None, it should be taken
> into? 
> account.
>  >>> if not chk or (chk.dtype is object_ and
> chk.item() is None):
> should become
>  >>> if (chk is None) of (chk.dtype is object_ and
> chk.item() is None):
> 
> I don't have time to spend on fixing that right now, sorry.

NP, understood 100%

> Try to? 
> fill a ticket on the numpy trac.

Will do, presently.

> Many thanks for reporting.

You're welcome.  (Never sure anymore whether odd behavior is a bug or a feature...)

DG

> 
> 
> 
> 
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From gael.varoquaux at normalesup.org  Mon Aug 17 02:46:47 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 17 Aug 2009 08:46:47 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
	<20090815230829.GA30494@phare.normalesup.org>
	<3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
Message-ID: <20090817064647.GA19761@phare.normalesup.org>

On Sat, Aug 15, 2009 at 06:14:56PM -0500, Robert Kern wrote:
> No, it is *not* a function in Python 2.x. There is no custom logic.
> The fact that "print(1,2,3)" executes is a side effect of the grammar
> not requiring a spacing between "print" and the expression "(1,2,3)".
> It is equivalent to "print (1,2,3)" in Python 2.x and "print((1,2,3))"
> in Python 3.x. Note that it is *not* equivalent to "print(1,2,3)" in
> Python 3.x.

OK, I am confused by the fact that __builtins__ as a print function that
matches the Python 3.x print function. You can see this doing 
'$ pydoc __builtin__.print' and I believe this is what is confusing
IPython too. What you are saying is that that function does not kick in
if we don't do the __future__ import, right?

Cheers,

Ga?l


From cournape at gmail.com  Mon Aug 17 03:38:03 2009
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 17 Aug 2009 00:38:03 -0700
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <1cd32cbb0908151734v1fb5b128v3be5067713f66c8d@mail.gmail.com>
References: <20090815223346.GF31205@phare.normalesup.org>
	<700580.31449.qm@web52108.mail.re2.yahoo.com>
	<3d375d730908151709k792e15b7n9c2ee579c4ee29c2@mail.gmail.com>
	<1cd32cbb0908151734v1fb5b128v3be5067713f66c8d@mail.gmail.com>
Message-ID: <5b8d13220908170038m7b584ebfuafaf64f385498749@mail.gmail.com>

> Just out of curiosity, do you have a rough guesstimate on the fraction
> on Windows users in the install base, and if a sizable fraction of
> them is using ipython?

The only number we have is the the download number from sourceforge.
For numpy 1.3.0, each python version on windows got ~ 15 000
downloads, the tar.gz 15 000 as well, and mac os x ~ 5000. I would
suspect many people on Linux to get installation from packages/sources
as well, but certainly, windows is not a small portion of our users I
would say.

cheers,

David


From aisaac at american.edu  Mon Aug 17 08:36:07 2009
From: aisaac at american.edu (Alan G Isaac)
Date: Mon, 17 Aug 2009 08:36:07 -0400
Subject: [SciPy-dev] change print statements to print
	functions	in	docstrings
In-Reply-To: <20090817064647.GA19761@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>	<663365.4887.qm@web52105.mail.re2.yahoo.com>	<20090815223712.GB7514@phare.normalesup.org>	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>	<20090815230829.GA30494@phare.normalesup.org>	<3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
	<20090817064647.GA19761@phare.normalesup.org>
Message-ID: <4A894EB7.3020000@american.edu>

On 8/17/2009 2:46 AM Gael Varoquaux apparently wrote:
> OK, I am confused by the fact that __builtins__ as a print function that
> matches the Python 3.x print function. You can see this doing 
> '$ pydoc __builtin__.print' and I believe this is what is confusing
> IPython too. What you are saying is that that function does not kick in
> if we don't do the __future__ import, right?


http://docs.python.org/whatsnew/2.6.html#pep-3105-print-as-a-function

fwiw,
Alan Isaac



From gael.varoquaux at normalesup.org  Mon Aug 17 10:26:26 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Mon, 17 Aug 2009 16:26:26 +0200
Subject: [SciPy-dev] change print statements to print functions
	in	docstrings
In-Reply-To: <4A894EB7.3020000@american.edu>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
	<20090815230829.GA30494@phare.normalesup.org>
	<3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
	<20090817064647.GA19761@phare.normalesup.org>
	<4A894EB7.3020000@american.edu>
Message-ID: <20090817142626.GA30571@phare.normalesup.org>

On Mon, Aug 17, 2009 at 08:36:07AM -0400, Alan G Isaac wrote:
> On 8/17/2009 2:46 AM Gael Varoquaux apparently wrote:
> > OK, I am confused by the fact that __builtins__ as a print function that
> > matches the Python 3.x print function. You can see this doing 
> > '$ pydoc __builtin__.print' and I believe this is what is confusing
> > IPython too. What you are saying is that that function does not kick in
> > if we don't do the __future__ import, right?


> http://docs.python.org/whatsnew/2.6.html#pep-3105-print-as-a-function

Thank you Alan. I should have refered to that document earlier, it is
indeed quite clear.

Ga?l


From matthew.brett at gmail.com  Mon Aug 17 14:26:21 2009
From: matthew.brett at gmail.com (Matthew Brett)
Date: Mon, 17 Aug 2009 11:26:21 -0700
Subject: [SciPy-dev] change print statements to print functions in
	docstrings
In-Reply-To: <20090817142626.GA30571@phare.normalesup.org>
References: <dde7764a0908151014m15b6aed2x7fed3fe3aa17d4bd@mail.gmail.com>
	<663365.4887.qm@web52105.mail.re2.yahoo.com>
	<20090815223712.GB7514@phare.normalesup.org>
	<9457e7c80908151547nccf79c7h6f2c6495196df375@mail.gmail.com>
	<3d375d730908151557l1a194a04t6fd6258b575ba327@mail.gmail.com>
	<20090815230829.GA30494@phare.normalesup.org>
	<3d375d730908151614o1c8e9840w294130625259bf85@mail.gmail.com>
	<20090817064647.GA19761@phare.normalesup.org>
	<4A894EB7.3020000@american.edu>
	<20090817142626.GA30571@phare.normalesup.org>
Message-ID: <1e2af89e0908171126r2dfd6dcfp1832bd39f4ae407c@mail.gmail.com>

On Mon, Aug 17, 2009 at 7:26 AM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Mon, Aug 17, 2009 at 08:36:07AM -0400, Alan G Isaac wrote:
>> On 8/17/2009 2:46 AM Gael Varoquaux apparently wrote:
>> > OK, I am confused by the fact that __builtins__ as a print function that
>> > matches the Python 3.x print function. You can see this doing
>> > '$ pydoc __builtin__.print' and I believe this is what is confusing
>> > IPython too. What you are saying is that that function does not kick in
>> > if we don't do the __future__ import, right?

It does seem that IPython autocall is detecting print as being a
callable, in python 2.6, but it's not because of the from __future__
import I don't think

In [1]: type(print) # print is a statement -> error
------------------------------------------------------------
   File "<ipython console>", line 1
     type(print) # print is a statement -> error
              ^
SyntaxError: invalid syntax


In [2]: print 1,2
------> print(1,2)
(1, 2)

In [3]: autocall 0
Automatic calling is: OFF

In [4]: print 1, 2
1 2

In [5]: from __future__ import print_function

In [6]: type(print)
Out[6]: <type 'builtin_function_or_method'>

In [7]: print 1, 2
------------------------------------------------------------
   File "<ipython console>", line 1
     print 1, 2 # of course it's an error
           ^
SyntaxError: invalid syntax


In [8]: print(1, 2)
1 2

See you,

Matthew


From matthieu.brucher at gmail.com  Tue Aug 18 11:09:24 2009
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Tue, 18 Aug 2009 17:09:24 +0200
Subject: [SciPy-dev] Optimization scikits (reloaded)
Message-ID: <e76aa17f0908180809u5a2779dbt86eaca0608b45d80@mail.gmail.com>

Hi,

I've taken some time to extract the generic optimization framework
from the old openopt scikit. It's now available as
scikits.optimization.

I didn't start cleaning the tutorial on the trac website, perhaps it
could be done somewhere else (the new portal? my own blog?).
The next step is to fix the manifold learning sub-scikit so that it
uses the new location. Then, openopt will be removed from the svn.

I don't know if I'mm be able to put much in developing new modules,
but who knows? Maybe one day I'll have to use it for my work...
If you have any questions, fill free to ask me.

Matthieu Brucher
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From Scott.Daniels at Acm.Org  Tue Aug 18 11:27:35 2009
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue, 18 Aug 2009 08:27:35 -0700
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <962671.76050.qm@web52102.mail.re2.yahoo.com>
References: <1cd32cbb0908120654j29577d1dmb9fa207efbbc2f1a@mail.gmail.com>
	<962671.76050.qm@web52102.mail.re2.yahoo.com>
Message-ID: <h6egor$950$1@ger.gmane.org>

More tools to explore unicode:
     unicodedata.name
     sys.maxunicode

 >>> import unicodedata
 >>> for n in xrange(32, 300):
     ch = unichr(n)
     if ch.lower() == ch.upper():
         print n, unicodedata.name(ch, 'Unknown')

 >>> text = u'\N{COLON}\N{COPYRIGHT SIGN}\N{INVERTED QUESTION MARK}'
 >>> print len(text)
3
 >>> [ord(ch) for ch in text]
[58, 169, 191]

--Scott David Daniels
Scott.Daniels at Acm.Org



From josef.pktd at gmail.com  Tue Aug 18 11:19:55 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 18 Aug 2009 11:19:55 -0400
Subject: [SciPy-dev] Optimization scikits (reloaded)
In-Reply-To: <e76aa17f0908180809u5a2779dbt86eaca0608b45d80@mail.gmail.com>
References: <e76aa17f0908180809u5a2779dbt86eaca0608b45d80@mail.gmail.com>
Message-ID: <1cd32cbb0908180819k1f265b9fv10b36dfad7134ed7@mail.gmail.com>

On Tue, Aug 18, 2009 at 11:09 AM, Matthieu
Brucher<matthieu.brucher at gmail.com> wrote:
> Hi,
>
> I've taken some time to extract the generic optimization framework
> from the old openopt scikit. It's now available as
> scikits.optimization.
>
> I didn't start cleaning the tutorial on the trac website, perhaps it
> could be done somewhere else (the new portal? my own blog?).
> The next step is to fix the manifold learning sub-scikit so that it
> uses the new location. Then, openopt will be removed from the svn.
>
> I don't know if I'mm be able to put much in developing new modules,
> but who knows? Maybe one day I'll have to use it for my work...
> If you have any questions, fill free to ask me.
>
> Matthieu Brucher
> --
> Information System Engineer, Ph.D.
> Website: http://matthieu-brucher.developpez.com/
> Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

Could you also clarify the boost dependency of the manifold learning
sub-scikits?
I had a hard time building it and was/am not sure if boost is a
requirement. I didn't see anything in the installation instructions.

However, I haven't looked or tried in a long time, so my information
might be outdated.

Thanks,

Josef


From matthieu.brucher at gmail.com  Tue Aug 18 11:30:00 2009
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Tue, 18 Aug 2009 17:30:00 +0200
Subject: [SciPy-dev] Optimization scikits (reloaded)
In-Reply-To: <1cd32cbb0908180819k1f265b9fv10b36dfad7134ed7@mail.gmail.com>
References: <e76aa17f0908180809u5a2779dbt86eaca0608b45d80@mail.gmail.com>
	<1cd32cbb0908180819k1f265b9fv10b36dfad7134ed7@mail.gmail.com>
Message-ID: <e76aa17f0908180830n15204b5du7af659204f0f8cd@mail.gmail.com>

> Could you also clarify the boost dependency of the manifold learning
> sub-scikits?

Will do ;)

> I had a hard time building it and was/am not sure if boost is a
> requirement. I didn't see anything in the installation instructions.

Indeed :| I don't even recall why I was using it :| I'll check this.

> However, I haven't looked or tried in a long time, so my information
> might be outdated.

Not much I fear, as I didn't update it in a very long time (for me, it
is finished in that matter).

Matthieu
-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From matthieu.brucher at gmail.com  Tue Aug 18 11:34:22 2009
From: matthieu.brucher at gmail.com (Matthieu Brucher)
Date: Tue, 18 Aug 2009 17:34:22 +0200
Subject: [SciPy-dev] Optimization scikits (reloaded)
In-Reply-To: <e76aa17f0908180830n15204b5du7af659204f0f8cd@mail.gmail.com>
References: <e76aa17f0908180809u5a2779dbt86eaca0608b45d80@mail.gmail.com>
	<1cd32cbb0908180819k1f265b9fv10b36dfad7134ed7@mail.gmail.com>
	<e76aa17f0908180830n15204b5du7af659204f0f8cd@mail.gmail.com>
Message-ID: <e76aa17f0908180834y639553aapf7ebb0d37f6e7ba2@mail.gmail.com>

2009/8/18 Matthieu Brucher <matthieu.brucher at gmail.com>:
>> Could you also clarify the boost dependency of the manifold learning
>> sub-scikits?
>
> Will do ;)
>
>> I had a hard time building it and was/am not sure if boost is a
>> requirement. I didn't see anything in the installation instructions.
>
> Indeed :| I don't even recall why I was using it :| I'll check this.

OK, it's in the neighbor class, I use a shared pointer. In the future,
I'll switch to TR1's shared pointer.

>> However, I haven't looked or tried in a long time, so my information
>> might be outdated.
>
> Not much I fear, as I didn't update it in a very long time (for me, it
> is finished in that matter).
>
> Matthieu
> --
> Information System Engineer, Ph.D.
> Website: http://matthieu-brucher.developpez.com/
> Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
> LinkedIn: http://www.linkedin.com/in/matthieubrucher
>



-- 
Information System Engineer, Ph.D.
Website: http://matthieu-brucher.developpez.com/
Blogs: http://matt.eifelle.com and http://blog.developpez.com/?blog=92
LinkedIn: http://www.linkedin.com/in/matthieubrucher


From d_l_goldsmith at yahoo.com  Tue Aug 18 11:43:49 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 08:43:49 -0700 (PDT)
Subject: [SciPy-dev] def __array__
Message-ID: <965455.25848.qm@web52109.mail.re2.yahoo.com>

Is this in the compiled C code also?

Someone (I think it was Pierre) recently asked me if I checked the code to see how something worked.  I "took the fifth," but I feel I needn't have: the real reason I haven't made a habit of it recently is (and maybe this is just due to the late stage we're at in the doc game), more often than not, the explanation I seek is buried in the compiled C code, and thus unfindable by me, at least with the technology I have at my disposal, and thus I'm at the "mercy" of the list for various explanations.

To wit: trying to flesh out the details of how chararray.argsort works - I go examine the code and all it does is call self.__array__.  chararray doesn't appear to have an __array__ method, and I can't find one in ndarray either, so after checking all other leads, I finally remember that the main def of ndarray is in the compiled C code.  What's an innocent to do?

DG

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From robert.kern at gmail.com  Tue Aug 18 11:48:41 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 18 Aug 2009 08:48:41 -0700
Subject: [SciPy-dev] def __array__
In-Reply-To: <965455.25848.qm@web52109.mail.re2.yahoo.com>
References: <965455.25848.qm@web52109.mail.re2.yahoo.com>
Message-ID: <3d375d730908180848k14e08383gc0aaff94c8cfc32b@mail.gmail.com>

On Tue, Aug 18, 2009 at 08:43, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> Is this in the compiled C code also?
>
> Someone (I think it was Pierre) recently asked me if I checked the code to see how something worked. ?I "took the fifth," but I feel I needn't have: the real reason I haven't made a habit of it recently is (and maybe this is just due to the late stage we're at in the doc game), more often than not, the explanation I seek is buried in the compiled C code, and thus unfindable by me, at least with the technology I have at my disposal, and thus I'm at the "mercy" of the list for various explanations.
>
> To wit: trying to flesh out the details of how chararray.argsort works - I go examine the code and all it does is call self.__array__. ?chararray doesn't appear to have an __array__ method, and I can't find one in ndarray either, so after checking all other leads, I finally remember that the main def of ndarray is in the compiled C code. ?What's an innocent to do?

[~]$ cd ~/svn/numpy/numpy/core/src
[src]$ grep -r __array__ .
....
# AHA!
./multiarray/methods.c:    {"__array__", (PyCFunction)array_getarray,
....
[src]$ grep -n array_getarray multiarray/methods.c
793:array_getarray(PyArrayObject *self, PyObject *args)
2032:    {"__array__", (PyCFunction)array_getarray,


And there you can see that ndarray.__array__ implementation is on line
793 of multiarray/methods.c .

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Tue Aug 18 12:13:53 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 09:13:53 -0700 (PDT)
Subject: [SciPy-dev] def __array__
In-Reply-To: <3d375d730908180848k14e08383gc0aaff94c8cfc32b@mail.gmail.com>
Message-ID: <378054.61472.qm@web52110.mail.re2.yahoo.com>

Hi, Robert.  I see you and I are paying about the same amount of attention to the ndarray discussion. ;-)

So, you saw me ask Stefan where to get the source, which I now have (again). :-)

I searched for __array__ and argsort, found plenty of 'em, but couldn't at a glance see how it treats character data, but now at least I have something substantive to study; thanks!

DG

--- On Tue, 8/18/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] def __array__
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 18, 2009, 8:48 AM
> On Tue, Aug 18, 2009 at 08:43, David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > Is this in the compiled C code also?
> >
> > Someone (I think it was Pierre) recently asked me if I
> checked the code to see how something worked. ?I "took the
> fifth," but I feel I needn't have: the real reason I haven't
> made a habit of it recently is (and maybe this is just due
> to the late stage we're at in the doc game), more often than
> not, the explanation I seek is buried in the compiled C
> code, and thus unfindable by me, at least with the
> technology I have at my disposal, and thus I'm at the
> "mercy" of the list for various explanations.
> >
> > To wit: trying to flesh out the details of how
> chararray.argsort works - I go examine the code and all it
> does is call self.__array__. ?chararray doesn't appear to
> have an __array__ method, and I can't find one in ndarray
> either, so after checking all other leads, I finally
> remember that the main def of ndarray is in the compiled C
> code. ?What's an innocent to do?
> 
> [~]$ cd ~/svn/numpy/numpy/core/src
> [src]$ grep -r __array__ .
> ....
> # AHA!
> ./multiarray/methods.c:? ? {"__array__",
> (PyCFunction)array_getarray,
> ....
> [src]$ grep -n array_getarray multiarray/methods.c
> 793:array_getarray(PyArrayObject *self, PyObject *args)
> 2032:? ? {"__array__",
> (PyCFunction)array_getarray,
> 
> 
> And there you can see that ndarray.__array__ implementation
> is on line
> 793 of multiarray/methods.c .
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From d_l_goldsmith at yahoo.com  Tue Aug 18 12:18:22 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 09:18:22 -0700 (PDT)
Subject: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
In-Reply-To: <h6egor$950$1@ger.gmane.org>
Message-ID: <789032.51097.qm@web52105.mail.re2.yahoo.com>

Thanks!

DG

--- On Tue, 8/18/09, Scott David Daniels <Scott.Daniels at Acm.Org> wrote:

> From: Scott David Daniels <Scott.Daniels at Acm.Org>
> Subject: Re: [SciPy-dev] Some Q's vis-a-vis Numpy unicode support
> To: scipy-dev at scipy.org
> Date: Tuesday, August 18, 2009, 8:27 AM
> More tools to explore unicode:
> ? ???unicodedata.name
> ? ???sys.maxunicode
> 
>  >>> import unicodedata
>  >>> for n in xrange(32, 300):
> ? ???ch = unichr(n)
> ? ???if ch.lower() == ch.upper():
> ? ? ? ???print n,
> unicodedata.name(ch, 'Unknown')
> 
>  >>> text = u'\N{COLON}\N{COPYRIGHT
> SIGN}\N{INVERTED QUESTION MARK}'
>  >>> print len(text)
> 3
>  >>> [ord(ch) for ch in text]
> [58, 169, 191]
> 
> --Scott David Daniels
> Scott.Daniels at Acm.Org
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Tue Aug 18 13:09:54 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 10:09:54 -0700 (PDT)
Subject: [SciPy-dev] def __array__
In-Reply-To: <3d375d730908180848k14e08383gc0aaff94c8cfc32b@mail.gmail.com>
Message-ID: <164975.71843.qm@web52102.mail.re2.yahoo.com>

OK, I downloaded and unpacked numpy-1.3.0rc2.tar.gz and directory numpy\core\src\multiarray doesn't exist.  There's a src\multiarraymodule.c; that doesn't contain a single '__array__'; it does contain an argsort, however, and scrolling up, I see that's in PyArray_LexSort, w/ a preceding comment that confirms that the sorting is lexicographic, so I have the answer to my question.  But I'm left wondering why my directory structure is different from yours, Robert, and what problems that may cause me down the line.

DG

--- On Tue, 8/18/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] def __array__
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 18, 2009, 8:48 AM
> On Tue, Aug 18, 2009 at 08:43, David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > Is this in the compiled C code also?
> >
> > Someone (I think it was Pierre) recently asked me if I
> checked the code to see how something worked. ?I "took the
> fifth," but I feel I needn't have: the real reason I haven't
> made a habit of it recently is (and maybe this is just due
> to the late stage we're at in the doc game), more often than
> not, the explanation I seek is buried in the compiled C
> code, and thus unfindable by me, at least with the
> technology I have at my disposal, and thus I'm at the
> "mercy" of the list for various explanations.
> >
> > To wit: trying to flesh out the details of how
> chararray.argsort works - I go examine the code and all it
> does is call self.__array__. ?chararray doesn't appear to
> have an __array__ method, and I can't find one in ndarray
> either, so after checking all other leads, I finally
> remember that the main def of ndarray is in the compiled C
> code. ?What's an innocent to do?
> 
> [~]$ cd ~/svn/numpy/numpy/core/src
> [src]$ grep -r __array__ .
> ....
> # AHA!
> ./multiarray/methods.c:? ? {"__array__",
> (PyCFunction)array_getarray,
> ....
> [src]$ grep -n array_getarray multiarray/methods.c
> 793:array_getarray(PyArrayObject *self, PyObject *args)
> 2032:? ? {"__array__",
> (PyCFunction)array_getarray,
> 
> 
> And there you can see that ndarray.__array__ implementation
> is on line
> 793 of multiarray/methods.c .
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From robert.kern at gmail.com  Tue Aug 18 13:12:44 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 18 Aug 2009 10:12:44 -0700
Subject: [SciPy-dev] def __array__
In-Reply-To: <164975.71843.qm@web52102.mail.re2.yahoo.com>
References: <3d375d730908180848k14e08383gc0aaff94c8cfc32b@mail.gmail.com> 
	<164975.71843.qm@web52102.mail.re2.yahoo.com>
Message-ID: <3d375d730908181012k4b4e5aa9v9d860b2988528823@mail.gmail.com>

On Tue, Aug 18, 2009 at 10:09, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> OK, I downloaded and unpacked numpy-1.3.0rc2.tar.gz and directory numpy\core\src\multiarray doesn't exist. ?There's a src\multiarraymodule.c; that doesn't contain a single '__array__'; it does contain an argsort, however, and scrolling up, I see that's in PyArray_LexSort, w/ a preceding comment that confirms that the sorting is lexicographic, so I have the answer to my question. ?But I'm left wondering why my directory structure is different from yours, Robert, and what problems that may cause me down the line.

There was a restructuring of the source files into more manageable
pieces after the 1.3 release.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Tue Aug 18 13:16:31 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 10:16:31 -0700 (PDT)
Subject: [SciPy-dev] def __array__
In-Reply-To: <3d375d730908181012k4b4e5aa9v9d860b2988528823@mail.gmail.com>
Message-ID: <347663.69606.qm@web52109.mail.re2.yahoo.com>

I.e., you still have the pre-1.3 direc struc for your local copy?

DG

--- On Tue, 8/18/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] def __array__
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 18, 2009, 10:12 AM
> On Tue, Aug 18, 2009 at 10:09, David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > OK, I downloaded and unpacked numpy-1.3.0rc2.tar.gz
> and directory numpy\core\src\multiarray doesn't exist.
> ?There's a src\multiarraymodule.c; that doesn't contain a
> single '__array__'; it does contain an argsort, however, and
> scrolling up, I see that's in PyArray_LexSort, w/ a
> preceding comment that confirms that the sorting is
> lexicographic, so I have the answer to my question. ?But
> I'm left wondering why my directory structure is different
> from yours, Robert, and what problems that may cause me down
> the line.
> 
> There was a restructuring of the source files into more
> manageable
> pieces after the 1.3 release.
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From robert.kern at gmail.com  Tue Aug 18 13:18:06 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Tue, 18 Aug 2009 10:18:06 -0700
Subject: [SciPy-dev] def __array__
In-Reply-To: <347663.69606.qm@web52109.mail.re2.yahoo.com>
References: <3d375d730908181012k4b4e5aa9v9d860b2988528823@mail.gmail.com> 
	<347663.69606.qm@web52109.mail.re2.yahoo.com>
Message-ID: <3d375d730908181018t7e06ad45p8d64daf69f0d7007@mail.gmail.com>

On Tue, Aug 18, 2009 at 10:16, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> I.e., you still have the pre-1.3 direc struc for your local copy?

No, I have an SVN checkout post-1.3. The restructuring happened after
the 1.3 release.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Tue Aug 18 13:54:35 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 10:54:35 -0700 (PDT)
Subject: [SciPy-dev] def __array__
In-Reply-To: <3d375d730908181018t7e06ad45p8d64daf69f0d7007@mail.gmail.com>
Message-ID: <647790.87928.qm@web52109.mail.re2.yahoo.com>

Ah, glad I clarified...

DG

--- On Tue, 8/18/09, Robert Kern <robert.kern at gmail.com> wrote:

> From: Robert Kern <robert.kern at gmail.com>
> Subject: Re: [SciPy-dev] def __array__
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Tuesday, August 18, 2009, 10:18 AM
> On Tue, Aug 18, 2009 at 10:16, David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > I.e., you still have the pre-1.3 direc struc for your
> local copy?
> 
> No, I have an SVN checkout post-1.3. The restructuring
> happened after
> the 1.3 release.
> 
> -- 
> Robert Kern
> 
> "I have come to believe that the whole world is an enigma,
> a harmless
> enigma that is made terrible by our own mad attempt to
> interpret it as
> though it had an underlying truth."
> ? -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From d_l_goldsmith at yahoo.com  Tue Aug 18 20:01:37 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 18 Aug 2009 17:01:37 -0700 (PDT)
Subject: [SciPy-dev] Force trailing ws in chararray
Message-ID: <420526.71579.qm@web52106.mail.re2.yahoo.com>

The np.array().view(np.chararray) way to create a chararray appears to automatically strip trailing whitespace:

>>> c=np.array(['aAaAaA','  aA  ','abBABba'],dtype='S7')
>>> c[1]; len(c[1])
'  aA  '
6
>>> c=c.view(np.chararray)
>>> c[1]; len(c[1])
'  aA'
4

Is there a way to suppress this behavior?  If not, bug or feature?

Thanks,

DG


      


From d_l_goldsmith at yahoo.com  Wed Aug 19 11:34:42 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 19 Aug 2009 08:34:42 -0700 (PDT)
Subject: [SciPy-dev] Plea for help
Message-ID: <900719.6663.qm@web52110.mail.re2.yahoo.com>

Does anyone have a detailed knowledge/understanding of chararray?  Much of what is left to document has literally zero documentation from which to begin; I'm having mixed luck figuring things out using trial-and-error; I'm finding the code to be more "buggy" than average; and Travis - original author of most of it - doesn't recollect it well, says it was added in haste, and is presently busy.  And frankly, why is this even in NumPy: IMO, it should be deprecated, or moved to SciPy.

DG

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From chanley at stsci.edu  Wed Aug 19 11:55:18 2009
From: chanley at stsci.edu (Christopher Hanley)
Date: Wed, 19 Aug 2009 11:55:18 -0400
Subject: [SciPy-dev] Plea for help
In-Reply-To: <900719.6663.qm@web52110.mail.re2.yahoo.com>
References: <900719.6663.qm@web52110.mail.re2.yahoo.com>
Message-ID: <1B38FD88-BDD4-49E0-91FA-6CB2DC240B08@stsci.edu>

This module was created to provide backward compatibility with  
numarray.  It's functionality is modeled after the numarray.strings  
module.  You can find some online documentation for that module here:

http://stsdas.stsci.edu/numarray/numarray-1.5.html/node42.html

I believe that this module should be kept as part of numpy.  There are  
going to be projects that ported their code from numarray to numpy  
that depend on this module.

Chris

-- 
Christopher Hanley
Senior Systems Software Engineer
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4338

On Aug 19, 2009, at 11:34 AM, David Goldsmith wrote:

> Does anyone have a detailed knowledge/understanding of chararray?   
> Much of what is left to document has literally zero documentation  
> from which to begin; I'm having mixed luck figuring things out using  
> trial-and-error; I'm finding the code to be more "buggy" than  
> average; and Travis - original author of most of it - doesn't  
> recollect it well, says it was added in haste, and is presently  
> busy.  And frankly, why is this even in NumPy: IMO, it should be  
> deprecated, or moved to SciPy.
>
> DG
>
> __________________________________________________
> Do You Yahoo!?
> Tired of spam?  Yahoo! Mail has the best spam protection around
> http://mail.yahoo.com
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev



From josef.pktd at gmail.com  Wed Aug 19 18:41:46 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 19 Aug 2009 18:41:46 -0400
Subject: [SciPy-dev] stats.models report/preannouncement
Message-ID: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com>

GSOC Report
===========

Background
----------

The GSOC project for stats.models is close to it's end.
We are currently finishing up some final changes and
improvements to the docstrings. We intend to "release" the
package next week. As a reminder, the GSOC project
"stats.models" had as a target to correct and update
Jonathan Taylor's statistical models (currently found in
NiPy) for eventual re-inclusion in SciPy.

Current Status
--------------

models is a pure python package.

models only includes:

* regression: mainly OLS and generalized least squares, GLS
 including weighted least squares and least squares with AR
 errors.
* glm: generalized linear models
* rlm: robust linear models
* datasets: for examples and tests

The other code which we didn't have enough time to verify and fix
was moved to a sandbox folder. The formula framework is not used any
more (in the verified code). Only the verified part would go into
scipy.

Compared to the original code, the class structure and some of the
method arguments have changed. Additional estimation
results, e.g. test statistics have been included.

Most importantly, almost every result has been verified with at least
one other statistical package, R, Stata and SAS. The guiding principal
for the rewrite was that all numbers have to be verified, even if we
don't manage to cover everything. There are a few remaining issues,
that we hope to clear up by next week. Not all parts of the code have
been tested for unexpected inputs. We are currently adding checks for,
and conversions of array types and dimension. Additionally, many of
the tests call rpy to compare the results directly with R. We use an
extended wrapper for R models in the test suite. This provides greater
flexibility writing new test cases, but will eventually be replaced by
hard coded expected results.

The code is written for plain NumPy arrays.

We have also included several datasets from the public domain and by
permission for the tests and examples.  The datasets follow
fairly closely David C's datasets proposal in
scikits.learn, with some small modifications. The datasets
are set up so that it is easy to add more datasets.

Looking Forward
---------------

The current question is, what will be the near future for
"models"? We would like to distribute it as a standalone
package to gain experience with the API, and to allow us to
make changes without being committed to backwards
compatibility. It will also give us the opportunity to find
and kill some remaining bugs, and fill some holes in our
test suite.

We can either package it as a scikit or as a independent
package distributed through pypi. Also depending on the feedback,
it could go into scipy 0.8 to close the gap in the stats area,
but with a warning that there might still be some changes to
the API, or we could wait for 0.9, if 0.8 is coming out
soon.

Earlier this summer, there was a discussion on the nipy
mailing list on the structure of the API and about possible
additional methods. The release as a standalone package or
scikit should give us the opportunity to discuss any changes
in the design or API and make adjustments to the code
without having to go through the scipy installation and
release cycle constraints.

We are also discussing future inclusion of additional
models.  Separate, and only partially dependent on the
models code, we would like to create a package that can be
used as a staging ground for new models. We will focus on
models that are closer to our area, mainly econometrics and
time series analysis, and less "pure" statistics.  There has
also been some interest by others to add additional models
or new cases to existing models. This could also be the
location for the part of "models", that we moved in the
sandbox, until they can be fixed and verified. Whether
additional models are included in scipy or remain separate
can be discussed as they mature.

So, how should we package "models" for next week? And how
soon should we plan to include models in scipy.

Skipper, Josef, and Alan


From wflynny at gmail.com  Wed Aug 19 19:09:41 2009
From: wflynny at gmail.com (Bill Flynn)
Date: Wed, 19 Aug 2009 16:09:41 -0700
Subject: [SciPy-dev] Mpfit optimizer
Message-ID: <ed6abe4e0908191609s2a7e0684p8f1e326b657076d3@mail.gmail.com>

Hi,

I received an old python version of the mpfit optimizer earlier this summer.
My advisor had previously been in contact with SciPy about including this
optimizer in the optimize package. He was sent to following message with
tips to help improve the package:

"""

> ---------- Forwarded message ----------

From: Pauli Virtanen <pav at iki.fi>

Date: Sat, May 9, 2009 at 11:55 AM

Subject: Re: [SciPy-dev] adding mpfit to scipy optimize (Please respond)

To: scipy-dev at scipy.org


>
> Fri, 08 May 2009 16:09:35 -0400, william ratcliff wrote:


> > Hi!  For a long time, there has been a standard package used by IDL

> users for fitting functions to data called MPFIT:

> http://cow.physics.wisc.edu/~craigm/idl/fitting.html

> <http://cow.physics.wisc.edu/%7Ecraigm/idl/fitting.html>

[clip]

> http://drop.io/mpfitdrop


> Nice!


> However, I see some points where the code could be improved for better

reusability and maintainability.


> If we lived in an ideal world with infinite time to polish everything,

I'd like to see all of the points below addressed before landing this to

Scipy. But since this would be lots of error-prone work, it's probably

best to try to reach some compromise.


> Given these constraints, I'd still like to see at least the coding style

and error handling fixed (which probably are not too difficult to

change), in addition to having better test coverage. The rest could come

later, even if we accrue yet more technical debt with this...


>
> First, broad maintenance concerns:


> - We already have `leastsq` from MINPACK. Having two MINPACK-derived

 least squares fitting routines is not good.


>  So, I'd perhaps like to see the `leastsq` core part extracted out of

 MPFIT, and the MPFIT interface implemented on top of it as a thin

 wrapper, or the other way around.


>  Maybe, if the modifications made on MINPACK are small, they can be

 backported to the Fortran code and MPFIT can be reimplemented on top

 of `leastsq`.


>  Any thoughts on this?


> - What is the performance of the Python implementation as compared to the

 Fortran code? For small data sets, the Python code is probably much

 slower, but for large datasets is the performance is comparable?


> - Much of the code is Fortran written in Python: long routines,

 goto-equivalents, 6-letter variable names.


>  Good commenting offset this, though.


>
> Then more specific points of refactoring:


> - The code implements QR factorization with column pivoting from scratch.


>  Perhaps some of this could be replaced with suitable LAPACK routines,

 or with stuff from scipy.linalg. (Cf. DGEQPF, DGEQP3)


>  I'm not sure whether there's something suitable for qrsolve in LAPACK;

 the triangular system solver could be replaced with DTRTRS.


>  Anyway, it might be useful to refactor qrfac and qrsolve out of MPFIT;

 there may be other applications when it's useful to be able to solve

 ||(A + I lambda) x - b||_2 = min! efficiently for multiple different

 `lambda` in sequence.


> - fdjac2 should probably be refactored out of MPFIT; other optimization

 algorithms that need Jacobians computed by forward differences can then

 use it. Do we already have something similar in Scipy already?


> - `enorm` should be replaced with BLAS xNRM2; it does appropriate scaling

 automatically and is probably much faster.


>  enorm = scipy.lib.blas.get_blas_funcs(['nrm2'], [some_array])


> - The long mpfit.__init__ routine should be split into smaller parts,

 the function body is too long. I'm not sure exactly what parts, though.


>  Perhaps at least the covariance matrix computation and input argument

 parsing should be separated.


> - "self.errmsg = 'ERROR ...; return'".

 Probably should raise exceptions instead, at least for errors in input

 parameters.


>  In general, I think the error handling should make better use of the

 Python exception and warning system; using `print` is not a correct

 way to signal an error in library code.


> - I'm not sure about implementing everything in a class. This tends to

 couple tightly parts of the code that wouldn't really need such a strong

 coupling.


> - Does it work with complex-valued inputs?


>
> Numpy issues:


> - Use numpy.finfo instead of machar


> - Lots of nonzero/put/take combos, probably dates from Numeric.


>  I think these should be replaced with boolean array indexing, to

 enhance readability and performance.


> - numpy.min/max -> amin/amax


>
> Minor stylistic issues in the code:


> - `catch_msg` is not used for anything


> - The original fortran documentation could be moved into the method

 docstrings.


> - "if foo:", not "if (foo):"


> - if foo:

    bar


>  not


>  if foo: bar


> - "return foo", not "return(foo)"


> - "# comment", not "## comment"


> - It's not necessary to "import types"; you can just use

 isinstance(x, float)


>
> [clip]

> I have written a nose-test compatible test.  Could someone look at it

> and tell me if it meets the scipy style before I continue adding more

> tests from Craig's test-suite for the C version of his program?


> The test style is OK; if it works with nosetests, it should be OK.

Suggest using "import numpy as np", though, to stay in line with the rest

of the code.


> --

Pauli Virtanen


> _______________________________________________

Scipy-dev mailing list

Scipy-dev at scipy.org

http://mail.scipy.org/mailman/listinfo/scipy-dev


"""

We cleaned up most of these issues and would like SciPy to reconsider the
package. Here is a short list of things we feel we accomplished:

- Minor stylic issues cleaned up
- Numpy issues (still a lot of nonzero calls but no more put/take combos)
- There is an approx_jacobian that uses a forward diff method in
scipy.optimize.fmin_slsq. We were not sure whether we should still factor
out our jacobian approximation. We also just checked for jacobian
approximators in SciPy - we haven't checked to see which is better, cleaner,
faster, etc.
- enorm fixed
- long init routine split in half
- Exceptions included (although a lot of them to show specific errors - this
can be changed)

Things that still need to be fixed:

- Complex numbers. We didn't have this in mind in during implementation and
still haven't thoroughly tested.
- Still written as a class. What is the best alternative? Just a simple
function?
- QR from scratch.
- Performance testing vs. Fortran code.

Please let us know if there is anymore work you would like done on this
optimizer. We use it for our own applications and it would be nice to have
it included in SciPy for others to use. Thanks for your time.

Bill Flynn
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090819/356d0df2/attachment.html>

From d_l_goldsmith at yahoo.com  Wed Aug 19 19:16:53 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 19 Aug 2009 16:16:53 -0700 (PDT)
Subject: [SciPy-dev] Documentation BoF Time/Place confirmation
In-Reply-To: <20090819222125.GE10463@phare.normalesup.org>
Message-ID: <161838.28376.qm@web52104.mail.re2.yahoo.com>

Thanks, Gael, for being able to accommodate my time preference; so, from the Schedule page, we see that the *Documentation BoF* is confirmed for: 

Friday, August 21, 1:30 - 2:30 PDT (20:30-21:30 UTC, UIAM), Powell Booth, Room 100.

Gael: do you know if we'll have internet in there?  Wired?  Wireless?  To those who couldn't make it this year: I'll do my best to make it a Skypecon.  

Thanks again,

DG

--- On Wed, 8/19/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-User] Scipy 2009 BOFs time/location
> To: "SciPy Users List" <scipy-user at scipy.org>
> Date: Wednesday, August 19, 2009, 3:21 PM
> On Wed, Aug 19, 2009 at 11:25:07PM
> +0200, Gael Varoquaux wrote:
> > On Wed, Aug 19, 2009 at 01:05:04PM -0400, Christopher
> Hanley wrote:
> > > Have any decisions been made on the time/location
> for any of the BOFs? 
> > > at Scipy this year?? Are there going to be
> any tonight or will they? 
> > > all be tomorrow and Friday?
> 
> > Yes, I have been doing a terrible job about that.
> 
> > So, we have a lot of BoFs. It is quite a struggle to
> get everything fit
> > together without too much overlap, especially with the
> different
> > constraints. Here is a suggestion:
> 
> OK, I was able to get an extra room. So I have rescheduled
> BoFs to avoid
> having BoFs during the reception on Thursday evening. The
> schedule page
> has now the information.
> 
> http://conference.scipy.org/schedule
> 
> Ga?l
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-user
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From gael.varoquaux at normalesup.org  Wed Aug 19 19:27:22 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Thu, 20 Aug 2009 01:27:22 +0200
Subject: [SciPy-dev] Documentation BoF Time/Place confirmation
In-Reply-To: <161838.28376.qm@web52104.mail.re2.yahoo.com>
References: <20090819222125.GE10463@phare.normalesup.org>
	<161838.28376.qm@web52104.mail.re2.yahoo.com>
Message-ID: <20090819232722.GB21059@phare.normalesup.org>

On Wed, Aug 19, 2009 at 04:16:53PM -0700, David Goldsmith wrote:
> Thanks, Gael, for being able to accommodate my time preference; so, from the Schedule page, we see that the *Documentation BoF* is confirmed for: 

> Friday, August 21, 1:30 - 2:30 PDT (20:30-21:30 UTC, UIAM), Powell Booth, Room 100.

> Gael: do you know if we'll have internet in there?  Wired?  Wireless?  To those who couldn't make it this year: I'll do my best to make it a Skypecon.  

Wireless should work well.

Ga?l


From d_l_goldsmith at yahoo.com  Wed Aug 19 19:29:35 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 19 Aug 2009 16:29:35 -0700 (PDT)
Subject: [SciPy-dev] Restaurant Wiki?
Message-ID: <622262.32786.qm@web52104.mail.re2.yahoo.com>

Hi!  I appear to have deleted the email: wasn't there a wiki started for CalTech area dining suggestions?  What's the URL?  Thanks!

DG


      


From d_l_goldsmith at yahoo.com  Wed Aug 19 19:30:03 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 19 Aug 2009 16:30:03 -0700 (PDT)
Subject: [SciPy-dev] Documentation BoF Time/Place confirmation
In-Reply-To: <20090819232722.GB21059@phare.normalesup.org>
Message-ID: <412524.91998.qm@web52102.mail.re2.yahoo.com>

Great, thanks!

DG

--- On Wed, 8/19/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Documentation BoF Time/Place confirmation
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Wednesday, August 19, 2009, 4:27 PM
> On Wed, Aug 19, 2009 at 04:16:53PM
> -0700, David Goldsmith wrote:
> > Thanks, Gael, for being able to accommodate my time
> preference; so, from the Schedule page, we see that the
> *Documentation BoF* is confirmed for: 
> 
> > Friday, August 21, 1:30 - 2:30 PDT (20:30-21:30 UTC,
> UIAM), Powell Booth, Room 100.
> 
> > Gael: do you know if we'll have internet in
> there?? Wired?? Wireless?? To those who
> couldn't make it this year: I'll do my best to make it a
> Skypecon.? 
> 
> Wireless should work well.
> 
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From robert.kern at gmail.com  Wed Aug 19 19:46:45 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 19 Aug 2009 16:46:45 -0700
Subject: [SciPy-dev] Restaurant Wiki?
In-Reply-To: <622262.32786.qm@web52104.mail.re2.yahoo.com>
References: <622262.32786.qm@web52104.mail.re2.yahoo.com>
Message-ID: <3d375d730908191646r3988a75yc8e9139b62728613@mail.gmail.com>

On Wed, Aug 19, 2009 at 16:29, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> Hi! ?I appear to have deleted the email: wasn't there a wiki started for CalTech area dining suggestions? ?What's the URL? ?Thanks!

http://conference.scipy.org/food

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From wflynny at gmail.com  Wed Aug 19 20:12:23 2009
From: wflynny at gmail.com (Bill Flynn)
Date: Wed, 19 Aug 2009 17:12:23 -0700
Subject: [SciPy-dev] Mpfit optimizer
In-Reply-To: <ed6abe4e0908191609s2a7e0684p8f1e326b657076d3@mail.gmail.com>
References: <ed6abe4e0908191609s2a7e0684p8f1e326b657076d3@mail.gmail.com>
Message-ID: <ed6abe4e0908191712v40dba608jadda360982ebe6a8@mail.gmail.com>

And I realize that I forgot to attach our files. We have them in our svn
repo at
svn checkout *http*://
spinwaves.googlecode.com/svn/trunk/spinwaves/utilities/mpfit

We have the mpfit.py file and some tests for it as well.

Thanks!

Bill Flynn


On Wed, Aug 19, 2009 at 4:09 PM, Bill Flynn <wflynny at gmail.com> wrote:

> Hi,
>
> I received an old python version of the mpfit optimizer earlier this
> summer. My advisor had previously been in contact with SciPy about including
> this optimizer in the optimize package. He was sent to following message
> with tips to help improve the package:
>
> """
>
>> ---------- Forwarded message ----------
>
> From: Pauli Virtanen <pav at iki.fi>
>
> Date: Sat, May 9, 2009 at 11:55 AM
>
> Subject: Re: [SciPy-dev] adding mpfit to scipy optimize (Please respond)
>
> To: scipy-dev at scipy.org
>
>
>>
>> Fri, 08 May 2009 16:09:35 -0400, william ratcliff wrote:
>
>
>> > Hi!  For a long time, there has been a standard package used by IDL
>
> > users for fitting functions to data called MPFIT:
>
> > http://cow.physics.wisc.edu/~craigm/idl/fitting.html
>
> > <http://cow.physics.wisc.edu/%7Ecraigm/idl/fitting.html>
>
> [clip]
>
> > http://drop.io/mpfitdrop
>
>
>> Nice!
>
>
>> However, I see some points where the code could be improved for better
>
> reusability and maintainability.
>
>
>> If we lived in an ideal world with infinite time to polish everything,
>
> I'd like to see all of the points below addressed before landing this to
>
> Scipy. But since this would be lots of error-prone work, it's probably
>
> best to try to reach some compromise.
>
>
>> Given these constraints, I'd still like to see at least the coding style
>
> and error handling fixed (which probably are not too difficult to
>
> change), in addition to having better test coverage. The rest could come
>
> later, even if we accrue yet more technical debt with this...
>
>
>>
>> First, broad maintenance concerns:
>
>
>> - We already have `leastsq` from MINPACK. Having two MINPACK-derived
>
>  least squares fitting routines is not good.
>
>
>>  So, I'd perhaps like to see the `leastsq` core part extracted out of
>
>  MPFIT, and the MPFIT interface implemented on top of it as a thin
>
>  wrapper, or the other way around.
>
>
>>  Maybe, if the modifications made on MINPACK are small, they can be
>
>  backported to the Fortran code and MPFIT can be reimplemented on top
>
>  of `leastsq`.
>
>
>>  Any thoughts on this?
>
>
>> - What is the performance of the Python implementation as compared to the
>
>  Fortran code? For small data sets, the Python code is probably much
>
>  slower, but for large datasets is the performance is comparable?
>
>
>> - Much of the code is Fortran written in Python: long routines,
>
>  goto-equivalents, 6-letter variable names.
>
>
>>  Good commenting offset this, though.
>
>
>>
>> Then more specific points of refactoring:
>
>
>> - The code implements QR factorization with column pivoting from scratch.
>
>
>>  Perhaps some of this could be replaced with suitable LAPACK routines,
>
>  or with stuff from scipy.linalg. (Cf. DGEQPF, DGEQP3)
>
>
>>  I'm not sure whether there's something suitable for qrsolve in LAPACK;
>
>  the triangular system solver could be replaced with DTRTRS.
>
>
>>  Anyway, it might be useful to refactor qrfac and qrsolve out of MPFIT;
>
>  there may be other applications when it's useful to be able to solve
>
>  ||(A + I lambda) x - b||_2 = min! efficiently for multiple different
>
>  `lambda` in sequence.
>
>
>> - fdjac2 should probably be refactored out of MPFIT; other optimization
>
>  algorithms that need Jacobians computed by forward differences can then
>
>  use it. Do we already have something similar in Scipy already?
>
>
>> - `enorm` should be replaced with BLAS xNRM2; it does appropriate scaling
>
>  automatically and is probably much faster.
>
>
>>  enorm = scipy.lib.blas.get_blas_funcs(['nrm2'], [some_array])
>
>
>> - The long mpfit.__init__ routine should be split into smaller parts,
>
>  the function body is too long. I'm not sure exactly what parts, though.
>
>
>>  Perhaps at least the covariance matrix computation and input argument
>
>  parsing should be separated.
>
>
>> - "self.errmsg = 'ERROR ...; return'".
>
>  Probably should raise exceptions instead, at least for errors in input
>
>  parameters.
>
>
>>  In general, I think the error handling should make better use of the
>
>  Python exception and warning system; using `print` is not a correct
>
>  way to signal an error in library code.
>
>
>> - I'm not sure about implementing everything in a class. This tends to
>
>  couple tightly parts of the code that wouldn't really need such a strong
>
>  coupling.
>
>
>> - Does it work with complex-valued inputs?
>
>
>>
>> Numpy issues:
>
>
>> - Use numpy.finfo instead of machar
>
>
>> - Lots of nonzero/put/take combos, probably dates from Numeric.
>
>
>>  I think these should be replaced with boolean array indexing, to
>
>  enhance readability and performance.
>
>
>> - numpy.min/max -> amin/amax
>
>
>>
>> Minor stylistic issues in the code:
>
>
>> - `catch_msg` is not used for anything
>
>
>> - The original fortran documentation could be moved into the method
>
>  docstrings.
>
>
>> - "if foo:", not "if (foo):"
>
>
>> - if foo:
>
>     bar
>
>
>>  not
>
>
>>  if foo: bar
>
>
>> - "return foo", not "return(foo)"
>
>
>> - "# comment", not "## comment"
>
>
>> - It's not necessary to "import types"; you can just use
>
>  isinstance(x, float)
>
>
>>
>> [clip]
>
> > I have written a nose-test compatible test.  Could someone look at it
>
> > and tell me if it meets the scipy style before I continue adding more
>
> > tests from Craig's test-suite for the C version of his program?
>
>
>> The test style is OK; if it works with nosetests, it should be OK.
>
> Suggest using "import numpy as np", though, to stay in line with the rest
>
> of the code.
>
>
>> --
>
> Pauli Virtanen
>
>
>> _______________________________________________
>
> Scipy-dev mailing list
>
> Scipy-dev at scipy.org
>
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
> """
>
> We cleaned up most of these issues and would like SciPy to reconsider the
> package. Here is a short list of things we feel we accomplished:
>
> - Minor stylic issues cleaned up
> - Numpy issues (still a lot of nonzero calls but no more put/take combos)
> - There is an approx_jacobian that uses a forward diff method in
> scipy.optimize.fmin_slsq. We were not sure whether we should still factor
> out our jacobian approximation. We also just checked for jacobian
> approximators in SciPy - we haven't checked to see which is better, cleaner,
> faster, etc.
> - enorm fixed
> - long init routine split in half
> - Exceptions included (although a lot of them to show specific errors -
> this can be changed)
>
> Things that still need to be fixed:
>
> - Complex numbers. We didn't have this in mind in during implementation and
> still haven't thoroughly tested.
> - Still written as a class. What is the best alternative? Just a simple
> function?
> - QR from scratch.
> - Performance testing vs. Fortran code.
>
> Please let us know if there is anymore work you would like done on this
> optimizer. We use it for our own applications and it would be nice to have
> it included in SciPy for others to use. Thanks for your time.
>
> Bill Flynn
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090819/af942acf/attachment.html>

From robert.kern at gmail.com  Wed Aug 19 20:16:08 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 19 Aug 2009 17:16:08 -0700
Subject: [SciPy-dev] Mpfit optimizer
In-Reply-To: <ed6abe4e0908191712v40dba608jadda360982ebe6a8@mail.gmail.com>
References: <ed6abe4e0908191609s2a7e0684p8f1e326b657076d3@mail.gmail.com> 
	<ed6abe4e0908191712v40dba608jadda360982ebe6a8@mail.gmail.com>
Message-ID: <3d375d730908191716v6e14db05x57484e1c8ddef91e@mail.gmail.com>

On Wed, Aug 19, 2009 at 17:12, Bill Flynn<wflynny at gmail.com> wrote:
> And I realize that I forgot to attach our files. We have them in our svn
> repo at
> svn
> checkout?http://spinwaves.googlecode.com/svn/trunk/spinwaves/utilities/mpfit
> We have the mpfit.py file and some tests for it as well.

Actually:

svn checkount http://spinwaves.googlecode.com/svn/trunk/spinwaves/utilities/

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From stefan at sun.ac.za  Wed Aug 19 20:28:44 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 19 Aug 2009 17:28:44 -0700
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com>
Message-ID: <9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com>

Hi guys,

2009/8/19 josef.pktd at gmail.com <josef.pktd at gmail.com>:
> Most importantly, almost every result has been verified with at least
> one other statistical package, R, Stata and SAS. The guiding principal
> for the rewrite was that all numbers have to be verified, even if we
> don't manage to cover everything. There are a few remaining issues,
> that we hope to clear up by next week. Not all parts of the code have
> been tested for unexpected inputs. We are currently adding checks for,
> and conversions of array types and dimension. Additionally, many of
> the tests call rpy to compare the results directly with R. We use an
> extended wrapper for R models in the test suite. This provides greater
> flexibility writing new test cases, but will eventually be replaced by
> hard coded expected results.
>
> The code is written for plain NumPy arrays.

Thanks for all your hard work!

> We can either package it as a scikit or as a independent
> package distributed through pypi.

SciKits are also distributed through pypi.  It's basically just a
naming/namespace convention.  All scikits.* packages from pypi are
also listed automatically on

http://scikits.appspot.com

Cheers
St?fan


From josef.pktd at gmail.com  Wed Aug 19 21:05:33 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 19 Aug 2009 21:05:33 -0400
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com>
	<9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com>
Message-ID: <1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com>

2009/8/19 St?fan van der Walt <stefan at sun.ac.za>:
> Hi guys,
>
> 2009/8/19 josef.pktd at gmail.com <josef.pktd at gmail.com>:
>> Most importantly, almost every result has been verified with at least
>> one other statistical package, R, Stata and SAS. The guiding principal
>> for the rewrite was that all numbers have to be verified, even if we
>> don't manage to cover everything. There are a few remaining issues,
>> that we hope to clear up by next week. Not all parts of the code have
>> been tested for unexpected inputs. We are currently adding checks for,
>> and conversions of array types and dimension. Additionally, many of
>> the tests call rpy to compare the results directly with R. We use an
>> extended wrapper for R models in the test suite. This provides greater
>> flexibility writing new test cases, but will eventually be replaced by
>> hard coded expected results.
>>
>> The code is written for plain NumPy arrays.
>
> Thanks for all your hard work!
>
>> We can either package it as a scikit or as a independent
>> package distributed through pypi.
>
> SciKits are also distributed through pypi. ?It's basically just a
> naming/namespace convention. ?All scikits.* packages from pypi are
> also listed automatically on
>
> http://scikits.appspot.com

Thanks,

Is it worth setting up a scikits if the code goes into scipy in a few months?
I never looked at how high the setup costs for a scikits are. A plain
python package looks easier. However, I have no experience in
distributing a package.

Josef


>
> Cheers
> St?fan
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From robert.kern at gmail.com  Wed Aug 19 21:07:55 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 19 Aug 2009 18:07:55 -0700
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com> 
	<9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com> 
	<1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com>
Message-ID: <3d375d730908191807v60b3feb7nd362c21d5cbc1127@mail.gmail.com>

On Wed, Aug 19, 2009 at 18:05, <josef.pktd at gmail.com> wrote:
> 2009/8/19 St?fan van der Walt <stefan at sun.ac.za>:
>> Hi guys,
>>
>> 2009/8/19 josef.pktd at gmail.com <josef.pktd at gmail.com>:
>>> Most importantly, almost every result has been verified with at least
>>> one other statistical package, R, Stata and SAS. The guiding principal
>>> for the rewrite was that all numbers have to be verified, even if we
>>> don't manage to cover everything. There are a few remaining issues,
>>> that we hope to clear up by next week. Not all parts of the code have
>>> been tested for unexpected inputs. We are currently adding checks for,
>>> and conversions of array types and dimension. Additionally, many of
>>> the tests call rpy to compare the results directly with R. We use an
>>> extended wrapper for R models in the test suite. This provides greater
>>> flexibility writing new test cases, but will eventually be replaced by
>>> hard coded expected results.
>>>
>>> The code is written for plain NumPy arrays.
>>
>> Thanks for all your hard work!
>>
>>> We can either package it as a scikit or as a independent
>>> package distributed through pypi.
>>
>> SciKits are also distributed through pypi. ?It's basically just a
>> naming/namespace convention. ?All scikits.* packages from pypi are
>> also listed automatically on
>>
>> http://scikits.appspot.com
>
> Thanks,
>
> Is it worth setting up a scikits if the code goes into scipy in a few months?
> I never looked at how high the setup costs for a scikits are. A plain
> python package looks easier. However, I have no experience in
> distributing a package.

It's always a chunk of work. It's no worse with scikits.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From stefan at sun.ac.za  Wed Aug 19 21:10:30 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Wed, 19 Aug 2009 18:10:30 -0700
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com> 
	<9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com> 
	<1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com>
Message-ID: <9457e7c80908191810v27350417y68535297138fe312@mail.gmail.com>

Hi Josef

2009/8/19 josef.pktd at gmail.com <josef.pktd at gmail.com>:
> Is it worth setting up a scikits if the code goes into scipy in a few months?
> I never looked at how high the setup costs for a scikits are. A plain
> python package looks easier. However, I have no experience in
> distributing a package.

It's very easy... have a look at

http://scikits.appspot.com/example

Check out the source, update the setup.py, add your source and voil?.

Cheers
St?fan


From jsseabold at gmail.com  Wed Aug 19 21:10:05 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Wed, 19 Aug 2009 21:10:05 -0400
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <3d375d730908191807v60b3feb7nd362c21d5cbc1127@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com> 
	<9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com> 
	<1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com> 
	<3d375d730908191807v60b3feb7nd362c21d5cbc1127@mail.gmail.com>
Message-ID: <c048da1c0908191810t3556d1e3kda9e12b4c4b5c9a8@mail.gmail.com>

On Wed, Aug 19, 2009 at 9:07 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Wed, Aug 19, 2009 at 18:05, <josef.pktd at gmail.com> wrote:
>> 2009/8/19 St?fan van der Walt <stefan at sun.ac.za>:
>>> Hi guys,
>>>
>>> 2009/8/19 josef.pktd at gmail.com <josef.pktd at gmail.com>:
>>>> Most importantly, almost every result has been verified with at least
>>>> one other statistical package, R, Stata and SAS. The guiding principal
>>>> for the rewrite was that all numbers have to be verified, even if we
>>>> don't manage to cover everything. There are a few remaining issues,
>>>> that we hope to clear up by next week. Not all parts of the code have
>>>> been tested for unexpected inputs. We are currently adding checks for,
>>>> and conversions of array types and dimension. Additionally, many of
>>>> the tests call rpy to compare the results directly with R. We use an
>>>> extended wrapper for R models in the test suite. This provides greater
>>>> flexibility writing new test cases, but will eventually be replaced by
>>>> hard coded expected results.
>>>>
>>>> The code is written for plain NumPy arrays.
>>>
>>> Thanks for all your hard work!
>>>
>>>> We can either package it as a scikit or as a independent
>>>> package distributed through pypi.
>>>
>>> SciKits are also distributed through pypi. ?It's basically just a
>>> naming/namespace convention. ?All scikits.* packages from pypi are
>>> also listed automatically on
>>>
>>> http://scikits.appspot.com
>>
>> Thanks,
>>
>> Is it worth setting up a scikits if the code goes into scipy in a few months?
>> I never looked at how high the setup costs for a scikits are. A plain
>> python package looks easier. However, I have no experience in
>> distributing a package.
>
> It's always a chunk of work. It's no worse with scikits.
>

Is it any better? ;)

-Skipper


From robert.kern at gmail.com  Wed Aug 19 21:18:05 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 19 Aug 2009 18:18:05 -0700
Subject: [SciPy-dev] stats.models report/preannouncement
In-Reply-To: <c048da1c0908191810t3556d1e3kda9e12b4c4b5c9a8@mail.gmail.com>
References: <1cd32cbb0908191541i46e0f39ck5df358c2bdd98eda@mail.gmail.com> 
	<9457e7c80908191728j1044dc9erfc8b56edf2d4c3af@mail.gmail.com> 
	<1cd32cbb0908191805sf92ea67le360caab4d5dc6f4@mail.gmail.com> 
	<3d375d730908191807v60b3feb7nd362c21d5cbc1127@mail.gmail.com> 
	<c048da1c0908191810t3556d1e3kda9e12b4c4b5c9a8@mail.gmail.com>
Message-ID: <3d375d730908191818x300af908p73f2f4fa4427926b@mail.gmail.com>

On Wed, Aug 19, 2009 at 18:10, Skipper Seabold<jsseabold at gmail.com> wrote:
> On Wed, Aug 19, 2009 at 9:07 PM, Robert Kern<robert.kern at gmail.com> wrote:
>> On Wed, Aug 19, 2009 at 18:05, <josef.pktd at gmail.com> wrote:

>>> Is it worth setting up a scikits if the code goes into scipy in a few months?
>>> I never looked at how high the setup costs for a scikits are. A plain
>>> python package looks easier. However, I have no experience in
>>> distributing a package.
>>
>> It's always a chunk of work. It's no worse with scikits.
>>
>
> Is it any better? ;)

For this kind of thing, yes.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From ralf.gommers at googlemail.com  Wed Aug 19 22:42:44 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Wed, 19 Aug 2009 22:42:44 -0400
Subject: [SciPy-dev] Plea for help
In-Reply-To: <1B38FD88-BDD4-49E0-91FA-6CB2DC240B08@stsci.edu>
References: <900719.6663.qm@web52110.mail.re2.yahoo.com>
	<1B38FD88-BDD4-49E0-91FA-6CB2DC240B08@stsci.edu>
Message-ID: <dde7764a0908191942h1bf2172cq1d3f1245fdac1ea3@mail.gmail.com>

On Wed, Aug 19, 2009 at 11:55 AM, Christopher Hanley <chanley at stsci.edu>wrote:

> This module was created to provide backward compatibility with
> numarray.  It's functionality is modeled after the numarray.strings
> module.  You can find some online documentation for that module here:
>
> http://stsdas.stsci.edu/numarray/numarray-1.5.html/node42.html
>
> I believe that this module should be kept as part of numpy.  There are
> going to be projects that ported their code from numarray to numpy
> that depend on this module.


That's a valid reason for keeping it. On the other hand, it gets very little
use, is pretty much undocumented and probably more buggy than most other
parts of NumPy. There is also not a really good use-case for it (otherwise
new users would ask about it). And as Stefan pointed out when asking this
same question last year, object arrays would do nicely if one needs this
functionality.

Would it make sense to put clearly in the docs that this module exists for
backwards compatibility reasons, and is not recommended for new development?

Ralf



>
> Chris
>
> --
> Christopher Hanley
> Senior Systems Software Engineer
> Space Telescope Science Institute
> 3700 San Martin Drive
> Baltimore MD, 21218
> (410) 338-4338
>
> On Aug 19, 2009, at 11:34 AM, David Goldsmith wrote:
>
> > Does anyone have a detailed knowledge/understanding of chararray?
> > Much of what is left to document has literally zero documentation
> > from which to begin; I'm having mixed luck figuring things out using
> > trial-and-error; I'm finding the code to be more "buggy" than
> > average; and Travis - original author of most of it - doesn't
> > recollect it well, says it was added in haste, and is presently
> > busy.  And frankly, why is this even in NumPy: IMO, it should be
> > deprecated, or moved to SciPy.
> >
> > DG
> >
> > __________________________________________________
> > Do You Yahoo!?
> > Tired of spam?  Yahoo! Mail has the best spam protection around
> > http://mail.yahoo.com
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090819/1e0babf3/attachment.html>

From d_l_goldsmith at yahoo.com  Wed Aug 19 23:03:16 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Wed, 19 Aug 2009 20:03:16 -0700 (PDT)
Subject: [SciPy-dev] Plea for help
In-Reply-To: <dde7764a0908191942h1bf2172cq1d3f1245fdac1ea3@mail.gmail.com>
Message-ID: <749298.14041.qm@web52106.mail.re2.yahoo.com>

I'm going to take it a step further: "breakage" is always the deterrent to change, and yet "change we must" (i.e., "adapt or die").  It's certainly not without precedent - even within Numpy, I believe - for things (though perhaps not whole namespaces) to be deemed "to-be-deprecated," have a warning to this effect established in one x.[even #].0 release, and then be removed by the x.[even # + 2 or + 4].0 release.  How has deprecation in Numpy worked in the past - by dictum, vote, or consensus?

DG

--- On Wed, 8/19/09, Ralf Gommers <ralf.gommers at googlemail.com> wrote:

> From: Ralf Gommers <ralf.gommers at googlemail.com>
> Subject: Re: [SciPy-dev] Plea for help
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Wednesday, August 19, 2009, 7:42 PM
> 
> 
> On Wed, Aug 19, 2009 at 11:55 AM,
> Christopher Hanley <chanley at stsci.edu>
> wrote:
> 
> This module was created to provide backward compatibility
> with
> 
> numarray. ?It's functionality is modeled after the
> numarray.strings
> 
> module. ?You can find some online documentation for that
> module here:
> 
> 
> 
> http://stsdas.stsci.edu/numarray/numarray-1.5.html/node42.html
> 
> 
> 
> I believe that this module should be kept as part of numpy.
> ?There are
> 
> going to be projects that ported their code from numarray
> to numpy
> 
> that depend on this module.
> That's a valid reason for keeping it. On the other
> hand, it gets very little use, is pretty much undocumented
> and probably more buggy than most other parts of NumPy.
> There is also not a really good use-case for it (otherwise
> new users would ask about it). And as Stefan pointed out
> when asking this same question last year, object arrays
> would do nicely if one needs this functionality.
> 
> 
> Would it make sense to put clearly in the docs that this
> module exists for backwards compatibility reasons, and is
> not recommended for new development?
> 
> Ralf
> 
> 
> 
> 
> 
> 
> 
> Chris
> 
> 
> 
> --
> 
> Christopher Hanley
> 
> Senior Systems Software Engineer
> 
> Space Telescope Science Institute
> 
> 3700 San Martin Drive
> 
> Baltimore MD, 21218
> 
> (410) 338-4338
> 
> 
> 
> On Aug 19, 2009, at 11:34 AM, David Goldsmith wrote:
> 
> 
> 
> > Does anyone have a detailed knowledge/understanding of
> chararray?
> 
> > Much of what is left to document has literally zero
> documentation
> 
> > from which to begin; I'm having mixed luck
> figuring things out using
> 
> > trial-and-error; I'm finding the code to be more
> "buggy" than
> 
> > average; and Travis - original author of most of it -
> doesn't
> 
> > recollect it well, says it was added in haste, and is
> presently
> 
> > busy. ?And frankly, why is this even in NumPy: IMO,
> it should be
> 
> > deprecated, or moved to SciPy.
> 
> >
> 
> > DG
> 
> >
> 
> > __________________________________________________
> 
> > Do You Yahoo!?
> 
> > Tired of spam? ?Yahoo! Mail has the best spam
> protection around
> 
> > http://mail.yahoo.com
> 
> > _______________________________________________
> 
> > Scipy-dev mailing list
> 
> > Scipy-dev at scipy.org
> 
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> 
> 
> 
> _______________________________________________
> 
> Scipy-dev mailing list
> 
> Scipy-dev at scipy.org
> 
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 
> 
> 
> 
> -----Inline Attachment Follows-----
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From robert.kern at gmail.com  Thu Aug 20 01:35:10 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 19 Aug 2009 22:35:10 -0700
Subject: [SciPy-dev] Plea for help
In-Reply-To: <749298.14041.qm@web52106.mail.re2.yahoo.com>
References: <dde7764a0908191942h1bf2172cq1d3f1245fdac1ea3@mail.gmail.com> 
	<749298.14041.qm@web52106.mail.re2.yahoo.com>
Message-ID: <3d375d730908192235u786e48q23780ee01c29ab52@mail.gmail.com>

On Wed, Aug 19, 2009 at 20:03, David Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> I'm going to take it a step further: "breakage" is always the deterrent to change, and yet "change we must" (i.e., "adapt or die"). ?It's certainly not without precedent - even within Numpy, I believe - for things (though perhaps not whole namespaces) to be deemed "to-be-deprecated," have a warning to this effect established in one x.[even #].0 release, and then be removed by the x.[even # + 2 or + 4].0 release. ?How has deprecation in Numpy worked in the past - by dictum, vote, or consensus?

Consensus or dictum without major objection. Voting is pointless
except to inform one of those.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From jsalvati at u.washington.edu  Thu Aug 20 03:20:08 2009
From: jsalvati at u.washington.edu (John Salvatier)
Date: Thu, 20 Aug 2009 00:20:08 -0700
Subject: [SciPy-dev] Questions about scikits: 1) What is the best way to
	translate nosetests into numpy testing framework? 2) How can
	I get an svn account?
Message-ID: <113e17f20908200020q67278b89y18ef033ff184df7c@mail.gmail.com>

Hello All,

My name is John Salvatier and I am building the scikit
scikit.bvp_solver<http://pypi.python.org/pypi/scikits.bvp_solver/>which
is for solving two-point boundary value problem.

Anyway, I have two questions about how scikits work:

1) What is the best way to translate tests which are arranged so for nose to
the configuration required for scikits
http://www.scipy.org/scipy/scikits/wiki/ScikitsForDevelopers#Testing? Is
there any easy way?

2) How can I get an svn account for svn.scipy.org/svn/?

Best Regards,
John Salvatier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090820/9ad399e8/attachment.html>

From niall.moran at gmail.com  Thu Aug 20 05:52:46 2009
From: niall.moran at gmail.com (Niall Moran)
Date: Thu, 20 Aug 2009 10:52:46 +0100
Subject: [SciPy-dev] Please help!
Message-ID: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>

Hi,

I am trying to find the lowest eigenvalues and eigenvectors of a
sparse complex matrix using
scipy. I could not find any suitable looking routines in the stable
version 0.6.0 so I have compiled the
and installed the development version 0.8.0.dev5798 and have been
trying to use the scipy.sparse.linalg.eigen routine which calls the
relevant fortran arpack routines.
This is working great for real matrices but is not working for complex
matrices. Even when all the matrix
elements are real and the matrix is just of complex type. The error I get is
"RuntimeError: Error info=-9999 in arpack".
on line 224 of .../python2.6/dist-packages/scipy/sparse/linalg/eigen/arpack/arpack.py.

The matrix I am using is of type complex128 so the arpack function
called is znaupd
(http://www.caam.rice.edu/software/ARPACK/UG/node138.html). Looking at
the documentation for a return value of info=-9999

"= -9999: Could not build an Arnoldi factorization.
c                   User input error highly likely.  Please
c                   check actual array dimensions and layout.
c                   IPARAM(5) returns the size of the current Arnoldi
c                   factorization."

suggests a user input error is likely. IPARAM(5) has a value of 4. The
page for znaupd (linked above) gives details on the expected size and
type of each of the arguments. I have went through each of these in
the arpack.py file and checked to make sure the python code is
creating the correct size arrays and everything looks fine so I am at
a complete loss as to what is going wrong as the real version is
working fine.

The only thing I have been thinking is that maybe there is something
different about the way python and fortran store complex arrays and
maybe there is some problem in the crossover. Can anyone suggest how I
may debug this further?

Please see the attached source (test_complex.py) which creates a
simple 16x16 matrix with all real elements and tries to find the
eigenvalues when this is stored as a complex sparse matrix and when it
is stored as a real sparse matrix.

Niall.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test_complex.py
Type: application/octet-stream
Size: 2700 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090820/01565668/attachment.obj>

From chanley at stsci.edu  Thu Aug 20 09:50:22 2009
From: chanley at stsci.edu (Christopher Hanley)
Date: Thu, 20 Aug 2009 09:50:22 -0400
Subject: [SciPy-dev] Plea for help
In-Reply-To: <3d375d730908192235u786e48q23780ee01c29ab52@mail.gmail.com>
References: <dde7764a0908191942h1bf2172cq1d3f1245fdac1ea3@mail.gmail.com>
	<749298.14041.qm@web52106.mail.re2.yahoo.com>
	<3d375d730908192235u786e48q23780ee01c29ab52@mail.gmail.com>
Message-ID: <8E59D101-D0AE-485B-A5BD-AF04C516F5CE@stsci.edu>

Hi,

I'd like to respectfully request that we move any discussion of what  
to do with the numpy.char module to the numpy list.

I'm a little concerned about some of the assumptions that are being  
made about the number of users of the module.  I would also like to  
better understand the reasons for wanting to dump it.  Let me be  
clear.  I'm not opposed to change.  However breaking other people's  
code just for the sake of change seems like a poor reason and a mean  
thing to do to our customers.

Thank you for your time and help,
Chris


-- 
Christopher Hanley
Senior Systems Software Engineer
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4338

On Aug 20, 2009, at 1:35 AM, Robert Kern wrote:

> On Wed, Aug 19, 2009 at 20:03, David  
> Goldsmith<d_l_goldsmith at yahoo.com> wrote:
>> I'm going to take it a step further: "breakage" is always the  
>> deterrent to change, and yet "change we must" (i.e., "adapt or  
>> die").  It's certainly not without precedent - even within Numpy, I  
>> believe - for things (though perhaps not whole namespaces) to be  
>> deemed "to-be-deprecated," have a warning to this effect  
>> established in one x.[even #].0 release, and then be removed by the  
>> x.[even # + 2 or + 4].0 release.  How has deprecation in Numpy  
>> worked in the past - by dictum, vote, or consensus?
>
> Consensus or dictum without major objection. Voting is pointless
> except to inform one of those.
>
> -- 
> Robert Kern
>
> "I have come to believe that the whole world is an enigma, a harmless
> enigma that is made terrible by our own mad attempt to interpret it as
> though it had an underlying truth."
>  -- Umberto Eco
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev



From robert.kern at gmail.com  Thu Aug 20 10:46:22 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 20 Aug 2009 07:46:22 -0700
Subject: [SciPy-dev] Questions about scikits: 1) What is the best way to
	translate nosetests into numpy testing framework? 2) How can
	I get an svn account?
In-Reply-To: <113e17f20908200020q67278b89y18ef033ff184df7c@mail.gmail.com>
References: <113e17f20908200020q67278b89y18ef033ff184df7c@mail.gmail.com>
Message-ID: <3d375d730908200746of4a60acuaf35199304ac00e7@mail.gmail.com>

On Thu, Aug 20, 2009 at 00:20, John Salvatier<jsalvati at u.washington.edu> wrote:
> Hello All,
>
> My name is John Salvatier and I am building the scikit scikit.bvp_solver
> which is for solving two-point boundary value problem.
>
> Anyway, I have two questions about how scikits work:
>
> 1) What is the best way to translate tests which are arranged so for nose to
> the configuration required for
> scikitshttp://www.scipy.org/scipy/scikits/wiki/ScikitsForDevelopers#Testing?
> Is there any easy way?

That information is old. numpy uses nose now itself. Just keep doing
what you are doing. However, I do recommend adding this to your
setup.py setup() call to enable "python setup.py test":

    tests_require = [
        'nose >= 0.10.3',
        ],
    test_suite = 'nose.collector',

> 2) How can I get an svn account for svn.scipy.org/svn/?

Email Jarrod Millman.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From d_l_goldsmith at yahoo.com  Thu Aug 20 12:27:45 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Thu, 20 Aug 2009 09:27:45 -0700 (PDT)
Subject: [SciPy-dev] Deprecate chararray [was  Plea for help]
Message-ID: <857977.74958.qm@web52106.mail.re2.yahoo.com>

--- On Thu, 8/20/09, Christopher Hanley <chanley at stsci.edu> wrote:

> I'd like to respectfully request that we move any
> discussion of what? 
> to do with the numpy.char module to the numpy list.

NP, done.

> I'm a little concerned about some of the assumptions that
> are being? 
> made about the number of users of the module.? I would
> also like to? 
> better understand the reasons for wanting to dump it.?

I think Ralf did a pretty good job of synopsizing the reasons for deprecation, but since we're moving the thread, I'll reprint them here:

0) "it gets very little use" (an assumption you presumably dispute);

1) "is pretty much undocumented" (less true than a week ago, but still true for several of the attributes, with another handful or so falling into the category of "poorly documented");

2) "probably more buggy than most other parts of NumPy" ("probably" being a euphemism, IMO);

3) "there is not a really good use-case for it" (a conjecture, but one that has yet to be challenged by counter-example); 

4) it's not the first time its presence in NumPy has been questioned ("as Stefan pointed out when asking this same question last year")

5) NumPy already has a (perhaps superior) alternative ("object arrays would do nicely if one needs this functionality");

to which I'll add:

6) it is, on its face, "counter to the spirit" of NumPy.

So far, IIRC, the only reason in favor of its continued inclusion is inertia.

> Let me be? 
> clear.? I'm not opposed to change.? However
> breaking other people's? 
> code just for the sake of change seems like a poor reason

So, I don't think we're proposing this "just for the sake of change"

> and a mean? 
> thing to do to our customers.

Apologies, but it is not proposed maliciously.

The only other things I would add by way of "review" from the scipy-dev thread:

a compromise proposal (made by Ralf):

"Put clearly in the docs that this module exists for backwards compatibility reasons, and is not recommended for new development"

and a clarification of deprecation process (provided by Robert):

"[asked by the present author] How has deprecation in Numpy worked in the past - by dictum, vote, or consensus?

[Robert's answer] Consensus or dictum without major objection. Voting is pointless except to inform one of those." 

Thanks for your time and consideration.

David Goldsmith

> Thank you for your time and help,
> Chris
> 
> 
> -- 
> Christopher Hanley
> Senior Systems Software Engineer
> Space Telescope Science Institute
> 3700 San Martin Drive
> Baltimore MD, 21218
> (410) 338-4338
> 
> On Aug 20, 2009, at 1:35 AM, Robert Kern wrote:
> 
> > On Wed, Aug 19, 2009 at 20:03, David? 
> > Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> >> I'm going to take it a step further: "breakage" is
> always the? 
> >> deterrent to change, and yet "change we must"
> (i.e., "adapt or? 
> >> die").? It's certainly not without precedent
> - even within Numpy, I? 
> >> believe - for things (though perhaps not whole
> namespaces) to be? 
> >> deemed "to-be-deprecated," have a warning to this
> effect? 
> >> established in one x.[even #].0 release, and then
> be removed by the? 
> >> x.[even # + 2 or + 4].0 release.? How has
> deprecation in Numpy? 
> >> worked in the past - by dictum, vote, or
> consensus?
> >
> > Consensus or dictum without major objection. Voting is
> pointless
> > except to inform one of those.
> >
> > -- 
> > Robert Kern
> >
> > "I have come to believe that the whole world is an
> enigma, a harmless
> > enigma that is made terrible by our own mad attempt to
> interpret it as
> > though it had an underlying truth."
> >? -- Umberto Eco
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>




      


From HAWRYLA at novachem.com  Thu Aug 20 13:08:04 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 20 Aug 2009 11:08:04 -0600
Subject: [SciPy-dev] Can I add an *.rst file via the Wiki?
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF287@CHMAILMBX01.novachem.com>

I have drafted an introduction to the NumPy User Guide. I could not see
a way to create a new *.rst file through the Wiki.
If I am correct that the Wiki is only designed to edit existing pages,
how do I create a file doc/source/user/introduction.rst?

Thanks,
Andrew
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090820/12de18dc/attachment.html>

From josef.pktd at gmail.com  Thu Aug 20 13:17:30 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 20 Aug 2009 13:17:30 -0400
Subject: [SciPy-dev] Can I add an *.rst file via the Wiki?
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF287@CHMAILMBX01.novachem.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF287@CHMAILMBX01.novachem.com>
Message-ID: <1cd32cbb0908201017l72e567f3p181debde2be6a90e@mail.gmail.com>

On Thu, Aug 20, 2009 at 1:08 PM, Andrew Hawryluk<HAWRYLA at novachem.com> wrote:
> I have drafted an introduction to the NumPy User Guide. I could not see a
> way to create a new *.rst file through the Wiki.
>
> If I am correct that the Wiki is only designed to edit existing pages, how
> do I create a file doc/source/user/introduction.rst?
>
> Thanks,
>
> Andrew
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>

some pages have "New item" add the bottom to create new files or
directories, e.g.

http://docs.scipy.org/numpy/docs/numpy-docs/user/

Josef


From HAWRYLA at novachem.com  Thu Aug 20 13:18:26 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 20 Aug 2009 11:18:26 -0600
Subject: [SciPy-dev] Can I add an *.rst file via the Wiki?
In-Reply-To: <1cd32cbb0908201017l72e567f3p181debde2be6a90e@mail.gmail.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF287@CHMAILMBX01.novachem.com>
	<1cd32cbb0908201017l72e567f3p181debde2be6a90e@mail.gmail.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF289@CHMAILMBX01.novachem.com>

Awesome. Thanks!

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of josef.pktd at gmail.com
> Sent: 20 Aug 2009 11:18 AM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] Can I add an *.rst file via the Wiki?
> 
> On Thu, Aug 20, 2009 at 1:08 PM, Andrew Hawryluk<HAWRYLA at novachem.com>
> wrote:
> > I have drafted an introduction to the NumPy User Guide. I could not
> > see a way to create a new *.rst file through the Wiki.
> >
> > If I am correct that the Wiki is only designed to edit existing
> pages,
> > how do I create a file doc/source/user/introduction.rst?
> >
> > Thanks,
> >
> > Andrew
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> >
> 
> some pages have "New item" add the bottom to create new files or
> directories, e.g.
> 
> http://docs.scipy.org/numpy/docs/numpy-docs/user/
> 
> Josef
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev


From josef.pktd at gmail.com  Thu Aug 20 13:25:22 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Thu, 20 Aug 2009 13:25:22 -0400
Subject: [SciPy-dev] Can I add an *.rst file via the Wiki?
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF289@CHMAILMBX01.novachem.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF287@CHMAILMBX01.novachem.com>
	<1cd32cbb0908201017l72e567f3p181debde2be6a90e@mail.gmail.com>
	<48C01AE7354EC240A26F19CEB995E943033AF289@CHMAILMBX01.novachem.com>
Message-ID: <1cd32cbb0908201025u46985d2an7ee4aa9946dfdf16@mail.gmail.com>

On Thu, Aug 20, 2009 at 1:18 PM, Andrew Hawryluk<HAWRYLA at novachem.com> wrote:
> Awesome. Thanks!
>
>> -----Original Message-----
>> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
>> On Behalf Of josef.pktd at gmail.com
>> Sent: 20 Aug 2009 11:18 AM
>> To: SciPy Developers List
>> Subject: Re: [SciPy-dev] Can I add an *.rst file via the Wiki?
>>
>> On Thu, Aug 20, 2009 at 1:08 PM, Andrew Hawryluk<HAWRYLA at novachem.com>
>> wrote:
>> > I have drafted an introduction to the NumPy User Guide. I could not
>> > see a way to create a new *.rst file through the Wiki.
>> >
>> > If I am correct that the Wiki is only designed to edit existing
>> pages,
>> > how do I create a file doc/source/user/introduction.rst?
>> >
>> > Thanks,
>> >
>> > Andrew
>> >
>> > _______________________________________________
>> > Scipy-dev mailing list
>> > Scipy-dev at scipy.org
>> > http://mail.scipy.org/mailman/listinfo/scipy-dev
>> >
>> >
>>
>> some pages have "New item" add the bottom to create new files or
>> directories, e.g.
>>
>> http://docs.scipy.org/numpy/docs/numpy-docs/user/
>>
>> Josef
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

(In case you look for this information)

If you add the name of the rstfile to the toctree in
http://docs.scipy.org/numpy/docs/numpy-docs/user/index.rst/edit/
it will be linked from the main user guide page.

Josef


From chanley at stsci.edu  Thu Aug 20 13:32:08 2009
From: chanley at stsci.edu (Christopher Hanley)
Date: Thu, 20 Aug 2009 13:32:08 -0400
Subject: [SciPy-dev] [Numpy-discussion] Deprecate chararray [was Plea
	for help]
In-Reply-To: <857977.74958.qm@web52106.mail.re2.yahoo.com>
References: <857977.74958.qm@web52106.mail.re2.yahoo.com>
Message-ID: <812BBECE-D1E8-4699-980A-BB8FB9657CB9@stsci.edu>

Here is what I know about the chararray usage at STScI since first  
looking into it this morning.

It is used in PyFITS and within the COS instrument calibration code.   
I have not heard back from the other projects yet given most of our  
developers are away at this time.

It appears that the COS code can be changed easily.  I am waiting to  
hear back from PyFITs.

Also, I do not know how many people use this particular feature.   
However I would point out that many people who use numpy are not also  
on the mailing lists.  Most of the STScI do not follow the numpy  
list.  I serve as our point of contact to the numpy community.  I'm  
trying to gather a list of projects that use this feature and specific  
use cases for you.  As I do not use this module myself I cannot  
counter your arguments at this time.   If we decide to deprecate this  
module would we reverse this decision if we then find out that the  
assumptions that went into the decision were in error?

Another concern is that we told people coming from numarray to use  
this module.  It is my opinion that at this point in the numpy release  
cycle that an API change needs a very strong justification.  Anecdotes  
about the number of users, a "change or die" philosophy, and an un- 
articulated notion of  "the spirit of numpy"  do not in my  
consideration meet that high bar.  If you would like us to provide  
additional documentation and tests that would be possible.  I'll do it  
myself if that is the only think keeping the module from remaining in  
numpy.

This also raises the question of what else is going to go?  Will  
recarray be removed?  What about the numarray c-api compatibility layer?

Like I said earlier, I'm not opposed to change.  I am just of the  
opinion that this isn't a simple, cut and dry decision.

For those at SciPy 2009 feel free to come yell at me and beat me with  
sticks.  I'm the fat guy in jeans and a blue shirt sitting towards the  
back middle on the left.

Cheers,
Chris



-- 
Christopher Hanley
Senior Systems Software Engineer
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4338

On Aug 20, 2009, at 12:27 PM, David Goldsmith wrote:

> --- On Thu, 8/20/09, Christopher Hanley <chanley at stsci.edu> wrote:
>
>> I'd like to respectfully request that we move any
>> discussion of what
>> to do with the numpy.char module to the numpy list.
>
> NP, done.
>
>> I'm a little concerned about some of the assumptions that
>> are being
>> made about the number of users of the module.  I would
>> also like to
>> better understand the reasons for wanting to dump it.
>
> I think Ralf did a pretty good job of synopsizing the reasons for  
> deprecation, but since we're moving the thread, I'll reprint them  
> here:
>
> 0) "it gets very little use" (an assumption you presumably dispute);
>
> 1) "is pretty much undocumented" (less true than a week ago, but  
> still true for several of the attributes, with another handful or so  
> falling into the category of "poorly documented");
>
> 2) "probably more buggy than most other parts of NumPy" ("probably"  
> being a euphemism, IMO);
>
> 3) "there is not a really good use-case for it" (a conjecture, but  
> one that has yet to be challenged by counter-example);
>
> 4) it's not the first time its presence in NumPy has been questioned  
> ("as Stefan pointed out when asking this same question last year")
>
> 5) NumPy already has a (perhaps superior) alternative ("object  
> arrays would do nicely if one needs this functionality");
>
> to which I'll add:
>
> 6) it is, on its face, "counter to the spirit" of NumPy.
>
> So far, IIRC, the only reason in favor of its continued inclusion is  
> inertia.
>
>> Let me be
>> clear.  I'm not opposed to change.  However
>> breaking other people's
>> code just for the sake of change seems like a poor reason
>
> So, I don't think we're proposing this "just for the sake of change"
>
>> and a mean
>> thing to do to our customers.
>
> Apologies, but it is not proposed maliciously.
>
> The only other things I would add by way of "review" from the scipy- 
> dev thread:
>
> a compromise proposal (made by Ralf):
>
> "Put clearly in the docs that this module exists for backwards  
> compatibility reasons, and is not recommended for new development"
>
> and a clarification of deprecation process (provided by Robert):
>
> "[asked by the present author] How has deprecation in Numpy worked  
> in the past - by dictum, vote, or consensus?
>
> [Robert's answer] Consensus or dictum without major objection.  
> Voting is pointless except to inform one of those."
>
> Thanks for your time and consideration.
>
> David Goldsmith
>
>> Thank you for your time and help,
>> Chris
>>
>>
>> -- 
>> Christopher Hanley
>> Senior Systems Software Engineer
>> Space Telescope Science Institute
>> 3700 San Martin Drive
>> Baltimore MD, 21218
>> (410) 338-4338
>>
>> On Aug 20, 2009, at 1:35 AM, Robert Kern wrote:
>>
>>> On Wed, Aug 19, 2009 at 20:03, David
>>> Goldsmith<d_l_goldsmith at yahoo.com>
>> wrote:
>>>> I'm going to take it a step further: "breakage" is
>> always the
>>>> deterrent to change, and yet "change we must"
>> (i.e., "adapt or
>>>> die").  It's certainly not without precedent
>> - even within Numpy, I
>>>> believe - for things (though perhaps not whole
>> namespaces) to be
>>>> deemed "to-be-deprecated," have a warning to this
>> effect
>>>> established in one x.[even #].0 release, and then
>> be removed by the
>>>> x.[even # + 2 or + 4].0 release.  How has
>> deprecation in Numpy
>>>> worked in the past - by dictum, vote, or
>> consensus?
>>>
>>> Consensus or dictum without major objection. Voting is
>> pointless
>>> except to inform one of those.
>>>
>>> -- 
>>> Robert Kern
>>>
>>> "I have come to believe that the whole world is an
>> enigma, a harmless
>>> enigma that is made terrible by our own mad attempt to
>> interpret it as
>>> though it had an underlying truth."
>>>   -- Umberto Eco
>>> _______________________________________________
>>> Scipy-dev mailing list
>>> Scipy-dev at scipy.org
>>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>> _______________________________________________
>> Scipy-dev mailing list
>> Scipy-dev at scipy.org
>> http://mail.scipy.org/mailman/listinfo/scipy-dev
>>
>
>
>
>
>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



From cournape at gmail.com  Thu Aug 20 15:04:10 2009
From: cournape at gmail.com (David Cournapeau)
Date: Thu, 20 Aug 2009 12:04:10 -0700
Subject: [SciPy-dev] [Numpy-discussion] Deprecate chararray [was Plea
	for help]
In-Reply-To: <812BBECE-D1E8-4699-980A-BB8FB9657CB9@stsci.edu>
References: <857977.74958.qm@web52106.mail.re2.yahoo.com>
	<812BBECE-D1E8-4699-980A-BB8FB9657CB9@stsci.edu>
Message-ID: <5b8d13220908201204s3c74cad1pabccdce47d3a13a1@mail.gmail.com>

On Thu, Aug 20, 2009 at 10:32 AM, Christopher Hanley<chanley at stsci.edu> wrote:

>
> Another concern is that we told people coming from numarray to use
> this module. ?It is my opinion that at this point in the numpy release
> cycle that an API change needs a very strong justification. ?Anecdotes
> about the number of users, a "change or die" philosophy, and an un-
> articulated notion of ?"the spirit of numpy" ?do not in my
> consideration meet that high bar.

I agree those are not strong reasons without more backing.  What
worries me the most, in both numpy and scipy, is code that nobody
knows about, without any test or documentation. When it breaks, we
can't fix it. That's unsustainable in the long term, because it takes
a lot of time that people could spend somewhere else more useful.
Especially when you have C code which does not work on some platforms,
with new version of python (python 3k port, for example).

I much prefer removing code to having code that barely works and
cannot be maintained. Old code that people are ready to maintain, I
have nothing against.

cheers,

David


From stefan at sun.ac.za  Thu Aug 20 15:13:11 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 20 Aug 2009 12:13:11 -0700
Subject: [SciPy-dev] Questions about scikits: 1) What is the best way to
	translate nosetests into numpy testing framework? 2) How can
	I get an svn account?
In-Reply-To: <3d375d730908200746of4a60acuaf35199304ac00e7@mail.gmail.com>
References: <113e17f20908200020q67278b89y18ef033ff184df7c@mail.gmail.com> 
	<3d375d730908200746of4a60acuaf35199304ac00e7@mail.gmail.com>
Message-ID: <9457e7c80908201213g2f7590fvcba0a7bf9dbc9030@mail.gmail.com>

2009/8/20 Robert Kern <robert.kern at gmail.com>:
>> 1) What is the best way to translate tests which are arranged so for nose to
>> the configuration required for
>> scikitshttp://www.scipy.org/scipy/scikits/wiki/ScikitsForDevelopers#Testing?
>> Is there any easy way?
>
> That information is old. numpy uses nose now itself. Just keep doing
> what you are doing. However, I do recommend adding this to your
> setup.py setup() call to enable "python setup.py test":
>
> ? ?tests_require = [
> ? ? ? ?'nose >= 0.10.3',
> ? ? ? ?],
> ? ?test_suite = 'nose.collector',

I updated the wiki.  Please add info as needed.

Cheers
St?fan


From stefan at sun.ac.za  Thu Aug 20 15:22:09 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 20 Aug 2009 12:22:09 -0700
Subject: [SciPy-dev] Please help!
In-Reply-To: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
Message-ID: <9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>

Hi Niall

2009/8/20 Niall Moran <niall.moran at gmail.com>:
> I am trying to find the lowest eigenvalues and eigenvectors of a
> sparse complex matrix using

I don't think anyone has done this using SciPy before. You may contact
Aric Hagberg or Neilen Marais (who wrapped ARPACK in
http://projects.scipy.org/scipy/ticket/231) for more clarity on the
situation.

Cheers
St?fan


From HAWRYLA at novachem.com  Thu Aug 20 18:02:03 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 20 Aug 2009 16:02:03 -0600
Subject: [SciPy-dev] draft: NumPy User Guide introduction
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>

I have posted a first draft of a NumPy User Guide introduction.
Hopefully it is both useful and factually correct, but please jump in
where you see room for improvement.

I am assuming familiarity with Python programming, but no prior
knowledge of NumPy.

http://docs.scipy.org/numpy/docs/numpy-docs/user/introduction.rst/


Andrew


From d_l_goldsmith at yahoo.com  Thu Aug 20 18:31:35 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Thu, 20 Aug 2009 15:31:35 -0700 (PDT)
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>
Message-ID: <221703.1925.qm@web52111.mail.re2.yahoo.com>

Thank you, Andrew, all contributions are very welcome.

DG

--- On Thu, 8/20/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:

> From: Andrew Hawryluk <HAWRYLA at novachem.com>
> Subject: [SciPy-dev] draft: NumPy User Guide introduction
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Thursday, August 20, 2009, 3:02 PM
> I have posted a first draft of a
> NumPy User Guide introduction.
> Hopefully it is both useful and factually correct, but
> please jump in
> where you see room for improvement.
> 
> I am assuming familiarity with Python programming, but no
> prior
> knowledge of NumPy.
> 
> http://docs.scipy.org/numpy/docs/numpy-docs/user/introduction.rst/
> 
> 
> Andrew
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From stefan at sun.ac.za  Thu Aug 20 18:33:48 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 20 Aug 2009 15:33:48 -0700
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>
Message-ID: <9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>

Hi Andrew

2009/8/20 Andrew Hawryluk <HAWRYLA at novachem.com>:
> I have posted a first draft of a NumPy User Guide introduction.
> Hopefully it is both useful and factually correct, but please jump in
> where you see room for improvement.

Thanks for taking the time to work on this!

Some comments:

The array multiplication example is a bit confusing, unless you read
ahead to figure out that you are talking about 1-dimension arrays or
vectors. On the Python side there is a small typo (len instead of
len(a)), and on the C++ side the size is not defined.

The term universal function should be explained: a function that
operates on a single element of an array at a time, applied to all
elements.

I think it's best to encourage people to use np.* functions, instead
of arr.* methods (maybe others differ with me on this point?).

This is a good start, and I think it can be expanded with some more
examples.  I wrote an intro while working on the docs last year:

http://mentat.za.net/numpy/intro/intro.html

Unfortunately, numpy arrays are not easy to teach in one page, so
these introductions tend to grow!

Cheers
St?fan


From robert.kern at gmail.com  Thu Aug 20 18:37:19 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Thu, 20 Aug 2009 15:37:19 -0700
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>
	<9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>
Message-ID: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>

2009/8/20 St?fan van der Walt <stefan at sun.ac.za>:

> I think it's best to encourage people to use np.* functions, instead
> of arr.* methods (maybe others differ with me on this point?).

>From an editor's perspective, I would be neutral. The author of an
example should use whatever they feel gets their point across best.
Editors should leave that choice alone.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From HAWRYLA at novachem.com  Thu Aug 20 18:53:34 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 20 Aug 2009 16:53:34 -0600
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com><9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>
	<3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF28B@CHMAILMBX01.novachem.com>

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of Robert Kern
> Sent: 20 Aug 2009 4:37 PM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] draft: NumPy User Guide introduction
> 
> 2009/8/20 St?fan van der Walt <stefan at sun.ac.za>:
> 
> > I think it's best to encourage people to use np.* functions, instead
> > of arr.* methods (maybe others differ with me on this point?).
> 
> From an editor's perspective, I would be neutral. The author of an
> example should use whatever they feel gets their point across best.
> Editors should leave that choice alone.

Maybe I'll put both versions in, so the new reader is aware that they both exist right away.
Travis' book indicates preference for array methods in section 4.4 (Functions redundant with methods):

"It is not recommended that these functions
be used in new programs, but there are no plans for removing them as in functional
form they work with arbitrary sequences which is sometimes desirable."

Andrew

From d_l_goldsmith at yahoo.com  Thu Aug 20 18:56:18 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Thu, 20 Aug 2009 15:56:18 -0700 (PDT)
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>
Message-ID: <308926.80222.qm@web52110.mail.re2.yahoo.com>

--- On Thu, 8/20/09, Robert Kern <robert.kern at gmail.com> wrote:

> > 2009/8/20 St?fan van der Walt <stefan at sun.ac.za> wrote:
> 
> > I think it's best to encourage people to use np.*
> functions, instead
> > of arr.* methods (maybe others differ with me on this
> point?).
> 
> From an editor's perspective, I would be neutral. The
> author of an
> example should use whatever they feel gets their point
> across best.
> Editors should leave that choice alone.
> 

However, I'm curious to hear Stefan's reasons/reasoning?

DG


      


From d_l_goldsmith at yahoo.com  Thu Aug 20 18:58:24 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Thu, 20 Aug 2009 15:58:24 -0700 (PDT)
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <48C01AE7354EC240A26F19CEB995E943033AF28B@CHMAILMBX01.novachem.com>
Message-ID: <789753.27634.qm@web52112.mail.re2.yahoo.com>

--- On Thu, 8/20/09, Andrew Hawryluk <HAWRYLA at novachem.com> wrote:
> Travis' book indicates preference for array methods in
> section 4.4 (Functions redundant with methods):
> 
> "It is not recommended that these functions
> be used in new programs, but there are no plans for
> removing them as in functional
> form they work with arbitrary sequences which is sometimes
> desirable."

Glad to see the author did his homework! :-)



      


From HAWRYLA at novachem.com  Thu Aug 20 19:09:55 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Thu, 20 Aug 2009 17:09:55 -0600
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>
References: <48C01AE7354EC240A26F19CEB995E943033AF28A@CHMAILMBX01.novachem.com>
	<9457e7c80908201533r61ca4182o7f0fd1c097b50ad1@mail.gmail.com>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF28C@CHMAILMBX01.novachem.com>

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of St?fan van der Walt
> Sent: 20 Aug 2009 4:34 PM
> To: SciPy Developers List
> Subject: Re: [SciPy-dev] draft: NumPy User Guide introduction
> 
> Hi Andrew
> 
> 2009/8/20 Andrew Hawryluk <HAWRYLA at novachem.com>:
> > I have posted a first draft of a NumPy User Guide introduction.
> > Hopefully it is both useful and factually correct, but please jump in
> > where you see room for improvement.
> 
> Thanks for taking the time to work on this!

You're welcome!

> Some comments:
> 
> The array multiplication example is a bit confusing, unless you read
> ahead to figure out that you are talking about 1-dimension arrays or
> vectors. On the Python side there is a small typo (len instead of
> len(a)), and on the C++ side the size is not defined.

Good point. I removed the length comparison for consistency and brevity.

> The term universal function should be explained: a function that
> operates on a single element of an array at a time, applied to all
> elements.

Done. Thanks!

> I think it's best to encourage people to use np.* functions, instead of
> arr.* methods (maybe others differ with me on this point?).
> 
> This is a good start, and I think it can be expanded with some more
> examples.  I wrote an intro while working on the docs last year:
> 
> http://mentat.za.net/numpy/intro/intro.html

That looks great. I will take a closer look.
 
> Unfortunately, numpy arrays are not easy to teach in one page, so these
> introductions tend to grow!

Indeed. A couple of brief examples would be very examples, but I think the bulk of the 'feature tour' should stay in the NumPy basics section, which is nicely organized and could be expanded as needed.

Thanks,
Andrew


From stefan at sun.ac.za  Thu Aug 20 19:22:17 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Thu, 20 Aug 2009 16:22:17 -0700
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <308926.80222.qm@web52110.mail.re2.yahoo.com>
References: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com> 
	<308926.80222.qm@web52110.mail.re2.yahoo.com>
Message-ID: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>

2009/8/20 David Goldsmith <d_l_goldsmith at yahoo.com>:
>> > of arr.* methods (maybe others differ with me on this
>> point?).
>>
>> From an editor's perspective, I would be neutral. The
>> author of an
>> example should use whatever they feel gets their point
>> across best.
>> Editors should leave that choice alone.
>>
>
> However, I'm curious to hear Stefan's reasons/reasoning?

I guess I should have said "others *will* differ with me on this point" :-)

I like the idea of an ndarray being a concise description of bytes in
memory.  At the moment, we have a couple of "special" methods attached
to it, but we could just as well have added fewer or more, and
personally I'd have preferred none.

Regards
St?fan


From chanley at stsci.edu  Thu Aug 20 19:43:27 2009
From: chanley at stsci.edu (Christopher Hanley)
Date: Thu, 20 Aug 2009 19:43:27 -0400
Subject: [SciPy-dev] Removing scipy.stsci was [Re: [Numpy-discussion]
	Deprecate chararray [was Plea for help]]
In-Reply-To: <5b8d13220908201204s3c74cad1pabccdce47d3a13a1@mail.gmail.com>
References: <857977.74958.qm@web52106.mail.re2.yahoo.com>
	<812BBECE-D1E8-4699-980A-BB8FB9657CB9@stsci.edu>
	<5b8d13220908201204s3c74cad1pabccdce47d3a13a1@mail.gmail.com>
Message-ID: <EA6326D8-062A-42C7-A689-91F5F0F053B7@stsci.edu>

I agree with David's comments.  In that theme I have removed  
scipy.stsci from scipy.  Users get it directly from us at STScI via  
STSCI_PYTHON.  It doesn't have any documentation in the doc system.   
It isn't by default in the scipy namespace.  And as a recent bug  
report indicates they can't import it anyway.

That should clean some code up.  If someday a generic image processing  
library is added to scipy we can consider incorporating our modules  
back into scipy.  Until that time I would rather remove the  
redundancy.  It also help scipy's maintainability and frees me from  
having to worry about a fork in the code developing.

Cheers,
Chris

-- 
Christopher Hanley
Senior Systems Software Engineer
Space Telescope Science Institute
3700 San Martin Drive
Baltimore MD, 21218
(410) 338-4338

On Aug 20, 2009, at 3:04 PM, David Cournapeau wrote:

> On Thu, Aug 20, 2009 at 10:32 AM, Christopher  
> Hanley<chanley at stsci.edu> wrote:
>
> I agree those are not strong reasons without more backing.  What
> worries me the most, in both numpy and scipy, is code that nobody
> knows about, without any test or documentation. When it breaks, we
> can't fix it. That's unsustainable in the long term, because it takes
> a lot of time that people could spend somewhere else more useful.
> Especially when you have C code which does not work on some platforms,
> with new version of python (python 3k port, for example).
>
> I much prefer removing code to having code that barely works and
> cannot be maintained. Old code that people are ready to maintain, I
> have nothing against.
>
> cheers,
>
> David
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion



From ndbecker2 at gmail.com  Fri Aug 21 09:09:52 2009
From: ndbecker2 at gmail.com (Neal Becker)
Date: Fri, 21 Aug 2009 09:09:52 -0400
Subject: [SciPy-dev] munkres (Hungarian algorithm or the Kuhn-Munkres
	algorithm)
Message-ID: <h6m6b0$kkm$1@ger.gmane.org>

I am trying to use munkres.  I found munkres on pypi.  It doesn't currently 
work with numpy.

I started trying to convert it, but so far only managed to break it 
(mysteriously).

Anyone have a version that works with numpy array?



From d_l_goldsmith at yahoo.com  Fri Aug 21 10:52:38 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 21 Aug 2009 07:52:38 -0700 (PDT)
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft: NumPy
	User Guide introduction]
In-Reply-To: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
Message-ID: <169303.42956.qm@web52102.mail.re2.yahoo.com>

--- On Thu, 8/20/09, St?fan van der Walt <stefan at sun.ac.za> wrote:

> I like the idea of an ndarray being a concise description
> of bytes in
> memory.? At the moment, we have a couple of "special"
> methods attached
> to it, but we could just as well have added fewer or more,
> and
> personally I'd have preferred none.

Interesting, I see the appeal from an aesthetics stand point - I don't know enough about such things as: do you think it would have performance implications?  If so, could this be tested by instantiating an ndarray and then assigning None to all of it's attributes (or would you shed only the public methods but not the data attributes, properties, and private accessors on which they rely)?  Just speculating idly at this point.

DG


      


From josef.pktd at gmail.com  Fri Aug 21 11:07:12 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 21 Aug 2009 11:07:12 -0400
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <169303.42956.qm@web52102.mail.re2.yahoo.com>
References: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
	<169303.42956.qm@web52102.mail.re2.yahoo.com>
Message-ID: <1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>

On Fri, Aug 21, 2009 at 10:52 AM, David
Goldsmith<d_l_goldsmith at yahoo.com> wrote:
> --- On Thu, 8/20/09, St?fan van der Walt <stefan at sun.ac.za> wrote:
>
>> I like the idea of an ndarray being a concise description
>> of bytes in
>> memory.? At the moment, we have a couple of "special"
>> methods attached
>> to it, but we could just as well have added fewer or more,
>> and
>> personally I'd have preferred none.
>
> Interesting, I see the appeal from an aesthetics stand point - I don't know enough about such things as: do you think it would have performance implications? ?If so, could this be tested by instantiating an ndarray and then assigning None to all of it's attributes (or would you shed only the public methods but not the data attributes, properties, and private accessors on which they rely)? ?Just speculating idly at this point.
>

Methods are useful for being overwritten by subclasses. And the last
time I checked method calls were a bit faster than functions.

eg. np.source(np.mean)

    try:
        mean = a.mean
    except AttributeError:
        return _wrapit(a, 'mean', axis, dtype, out)
    return mean(axis, dtype, out)


And, I like methods because they look very good in chaining
(especially with no trailing arguments like in german sentences)

I'm missing absolute value as a method.

mae = (x - a).abs().mean(0)  instead of  mae = np.mean(np.abs(x - a), 0)

Josef

> DG
>
>
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From gael.varoquaux at normalesup.org  Fri Aug 21 11:19:18 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 21 Aug 2009 17:19:18 +0200
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was
	draft:	NumPy User Guide introduction]
In-Reply-To: <1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>
References: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
	<169303.42956.qm@web52102.mail.re2.yahoo.com>
	<1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>
Message-ID: <20090821151918.GB17272@phare.normalesup.org>

On Fri, Aug 21, 2009 at 11:07:12AM -0400, josef.pktd at gmail.com wrote:
> Methods are useful for being overwritten by subclasses. And the last
> time I checked method calls were a bit faster than functions.

Fernando and I had a look recently, and as far as we could tell, all the
functions tried to call the methods, if any, sometimes after doing a bit
more magic.

Ga?l


From d_l_goldsmith at yahoo.com  Fri Aug 21 11:30:20 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 21 Aug 2009 08:30:20 -0700 (PDT)
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>
Message-ID: <962672.16322.qm@web52106.mail.re2.yahoo.com>

I wasn't thinking to replace ndarray w/ Stefan's idea; rather, I was thinking of the possibility of deriving ndarray from Stefan's idea, implemented as _ndarray (i.e., it'd be "private" so as to not disrupt the API and, hopefully, not confuse people too much).  But if there would be performance _loss_ in doing so...never mind. :-)

DG

--- On Fri, 8/21/09, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:

> From: josef.pktd at gmail.com <josef.pktd at gmail.com>
> Subject: Re: [SciPy-dev] Idle (?) speculation about ndarray [was draft: NumPy User Guide introduction]
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 21, 2009, 8:07 AM
> On Fri, Aug 21, 2009 at 10:52 AM,
> David
> Goldsmith<d_l_goldsmith at yahoo.com>
> wrote:
> > --- On Thu, 8/20/09, St?fan van der Walt <stefan at sun.ac.za>
> wrote:
> >
> >> I like the idea of an ndarray being a concise
> description
> >> of bytes in
> >> memory.? At the moment, we have a couple of
> "special"
> >> methods attached
> >> to it, but we could just as well have added fewer
> or more,
> >> and
> >> personally I'd have preferred none.
> >
> > Interesting, I see the appeal from an aesthetics stand
> point - I don't know enough about such things as: do you
> think it would have performance implications? ?If so, could
> this be tested by instantiating an ndarray and then
> assigning None to all of it's attributes (or would you shed
> only the public methods but not the data attributes,
> properties, and private accessors on which they rely)?
> ?Just speculating idly at this point.
> >
> 
> Methods are useful for being overwritten by subclasses. And
> the last
> time I checked method calls were a bit faster than
> functions.
> 
> eg. np.source(np.mean)
> 
> ? ? try:
> ? ? ? ? mean = a.mean
> ? ? except AttributeError:
> ? ? ? ? return _wrapit(a, 'mean', axis,
> dtype, out)
> ? ? return mean(axis, dtype, out)
> 
> 
> And, I like methods because they look very good in
> chaining
> (especially with no trailing arguments like in german
> sentences)
> 
> I'm missing absolute value as a method.
> 
> mae = (x - a).abs().mean(0)? instead of? mae =
> np.mean(np.abs(x - a), 0)
> 
> Josef
> 
> > DG
> >
> >
> >
> > _______________________________________________
> > Scipy-dev mailing list
> > Scipy-dev at scipy.org
> > http://mail.scipy.org/mailman/listinfo/scipy-dev
> >
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From josef.pktd at gmail.com  Fri Aug 21 11:32:07 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Fri, 21 Aug 2009 11:32:07 -0400
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <20090821151918.GB17272@phare.normalesup.org>
References: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
	<169303.42956.qm@web52102.mail.re2.yahoo.com>
	<1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>
	<20090821151918.GB17272@phare.normalesup.org>
Message-ID: <1cd32cbb0908210832r21e915c4t5b7979f714c34e6d@mail.gmail.com>

On Fri, Aug 21, 2009 at 11:19 AM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Fri, Aug 21, 2009 at 11:07:12AM -0400, josef.pktd at gmail.com wrote:
>> Methods are useful for being overwritten by subclasses. And the last
>> time I checked method calls were a bit faster than functions.
>
> Fernando and I had a look recently, and as far as we could tell, all the
> functions tried to call the methods, if any, sometimes after doing a bit
> more magic.
>
> Ga?l

depends on how you define "all the functions"
there was a discussion a while ago about np.dot

>>> a = np.ma.masked_invalid([1,2,3,np.nan,np.inf,5])
>>> a
masked_array(data = [1.0 2.0 3.0 -- -- 5.0],
             mask = [False False False  True  True False],
       fill_value = 1e+020)

>>> np.dot(a,a)
Traceback (most recent call last):
  File "<pyshell#54>", line 1, in <module>
    np.dot(a,a)
  File "C:\Programs\Python25\Lib\site-packages\numpy\ma\core.py", line
2447, in __array_finalize__
    self._mask.shape = self.shape
ValueError: total size of new array must be unchanged
>>> np.ma.dot(a,a)
masked_array(data = 39.0,
             mask = False,
       fill_value = 1e+020)

>>> np.source(np.dot)
Not available for this object.


From gael.varoquaux at normalesup.org  Fri Aug 21 11:33:58 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Fri, 21 Aug 2009 17:33:58 +0200
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was
	draft:	NumPy User Guide introduction]
In-Reply-To: <1cd32cbb0908210832r21e915c4t5b7979f714c34e6d@mail.gmail.com>
References: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
	<169303.42956.qm@web52102.mail.re2.yahoo.com>
	<1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com>
	<20090821151918.GB17272@phare.normalesup.org>
	<1cd32cbb0908210832r21e915c4t5b7979f714c34e6d@mail.gmail.com>
Message-ID: <20090821153358.GA22235@phare.normalesup.org>

On Fri, Aug 21, 2009 at 11:32:07AM -0400, josef.pktd at gmail.com wrote:
> depends on how you define "all the functions"
> there was a discussion a while ago about np.dot

Good point. I had excluded dot in my mind, because I knew it was special
(not a ufunc, calling out to blas), but you are right that this ,may be
confusing for the user.

Ga?l


From aric.hagberg at gmail.com  Fri Aug 21 12:50:13 2009
From: aric.hagberg at gmail.com (Aric Hagberg)
Date: Fri, 21 Aug 2009 10:50:13 -0600
Subject: [SciPy-dev] Please help!
In-Reply-To: <9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
Message-ID: <7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>

2009/8/20 St?fan van der Walt <stefan at sun.ac.za>:
> Hi Niall
>
> 2009/8/20 Niall Moran <niall.moran at gmail.com>:
>> I am trying to find the lowest eigenvalues and eigenvectors of a
>> sparse complex matrix using
>
> I don't think anyone has done this using SciPy before. You may contact
> Aric Hagberg or Neilen Marais (who wrapped ARPACK in
> http://projects.scipy.org/scipy/ticket/231) for more clarity on the
> situation.
>

It should work for complex double.  I just ran your example code and
it executed without failure for me.  It seems like it is giving the
correct answer (d and dreal, v and vreal are the same).

It may be that you have a miscompiled ARPACK in scipy or some part of
the complex double LAPACK/BLAS that ARPACK calls?

What platform are you using? I ran this test with stock Ubuntu
versions of LAPACK and BLAS and the dev versions of scipy and numpy.

Aric

$ uname -a
Linux ll 2.6.28-15-generic #48-Ubuntu SMP Wed Jul 29 08:53:35 UTC 2009
x86_64 GNU/Linux

$ python -c "import scipy;print scipy.__version__"
0.8.0.dev5895

$ python -c "import numpy;print numpy.__version__"
1.4.0.dev7303

$ gcc -v
Using built-in specs.
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu
4.3.3-5ubuntu4'
--with-bugurl=file:///usr/share/doc/gcc-4.3/README.Bugs
--enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr
--enable-shared --with-system-zlib --libexecdir=/usr/lib
--without-included-gettext --enable-threads=posix --enable-nls
--with-gxx-include-dir=/usr/include/c++/4.3 --program-suffix=-4.3
--enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc
--enable-mpfr --with-tune=generic --enable-checking=release
--build=x86_64-linux-gnu --host=x86_64-linux-gnu
--target=x86_64-linux-gnu
Thread model: posix
gcc version 4.3.3 (Ubuntu 4.3.3-5ubuntu4)


From neilcrighton at gmail.com  Fri Aug 21 14:35:51 2009
From: neilcrighton at gmail.com (Neil Crighton)
Date: Fri, 21 Aug 2009 19:35:51 +0100
Subject: [SciPy-dev] draft: NumPy User Guide introduction
Message-ID: <63751c30908211135x182b6e7q4d6e02a96035ac87@mail.gmail.com>

Andrew Hawryluk <HAWRYLA <at> novachem.com> writes:

>
> I have posted a first draft of a NumPy User Guide introduction.
> Hopefully it is both useful and factually correct, but please jump in
> where you see room for improvement.
>
> I am assuming familiarity with Python programming, but no prior
> knowledge of NumPy.
>
> http://docs.scipy.org/numpy/docs/numpy-docs/user/introduction.rst/
>
> Andrew
>

It looks good, thanks for putting it together!

A nitpick - say 'dictionaries' rather than 'dicts'.

I think it would help to explicitly give a and b lists (something like
[1, 2, 3, 4]). This will give an example of a possible data type
(integers), and make the following examples more clear.

If this is going to be the first stop for anyone wanting to know what
numpy can do, it would also be great to give an example of reading
columns of data from a text file into arrays with np.genfromtxt.

Neil


From stefan at sun.ac.za  Fri Aug 21 15:01:27 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 21 Aug 2009 12:01:27 -0700
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <20090821151918.GB17272@phare.normalesup.org>
References: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com> 
	<169303.42956.qm@web52102.mail.re2.yahoo.com>
	<1cd32cbb0908210807h6da90e8esd7143e453244e1fd@mail.gmail.com> 
	<20090821151918.GB17272@phare.normalesup.org>
Message-ID: <9457e7c80908211201u564fe9b1xdda6c983bb0a70b1@mail.gmail.com>

2009/8/21 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Fri, Aug 21, 2009 at 11:07:12AM -0400, josef.pktd at gmail.com wrote:
>> Methods are useful for being overwritten by subclasses. And the last
>> time I checked method calls were a bit faster than functions.
>
> Fernando and I had a look recently, and as far as we could tell, all the
> functions tried to call the methods, if any, sometimes after doing a bit
> more magic.

Just to be clear, I was simply expressing an opinion: there won't be
any changes to the ndarray object any time soon (not until 2.0 at
least).

The speed of functions vs. methods is an implementation detail that
can easily be changed.

Regards
St?fan


From pav+sp at iki.fi  Fri Aug 21 15:30:09 2009
From: pav+sp at iki.fi (Pauli Virtanen)
Date: Fri, 21 Aug 2009 19:30:09 +0000 (UTC)
Subject: [SciPy-dev] NumPy User Guide table of contents
References: <63751c30908211135x182b6e7q4d6e02a96035ac87@mail.gmail.com>
Message-ID: <h6msk1$jfn$1@ger.gmane.org>

Hi all,

It's great to see that people are interested in contributing
narrative documentation to Numpy! I think this effort will be
accelerated much if we have a table of contents for the final
manual we want, to guide the writing.

One suggestion is listed below. It's also on this wiki page:

	http://scipy.org/Developer_Zone/UG_Toc

It's not complete or final, but should already serve at least
as a starting point. I think it would be best if we agreed on
this first, fleshing out what should go where, and what each
chapter should contain, before spending too much time on
writing actual content.

So, please chip in: look at the suggestion below, and criticize
it and fill in additions and ideas.

Cheers,
Pauli

    ***

************
Numpy Manual
************

The aim here is to write narrative documentation that illustrates how
Numpy is best used in practice, demonstrating various features it
offers, with examples and enlightened discussion.

Below, a draft for the table of contents is listed. It is formed by
looking at

- `Numeric manual <http://numpy.scipy.org/numpydoc/numdoc.htm>`__
- `Guide to Numpy <http://www.tramy.us/>`__

and stealing whatever seemed good to include. It's a rough draft, and
probably needs tuning at various points. Also, I'm aware not all of
Numpy is there now, so additions should also be made.


#. Introduction to Numpy
   #. What it is
   #. What is there: a very high-level overview
   #. Conventions in this manual
#. Installing Numpy
   #. Instructions for getting binaries
   #. Instructions for building from source, on different platforms
#. Basics of arrays
  #. What is an array
  #. Storing data in arrays
     #. Arrays as literals
     #. Creating arrays: empty, zeros, ones, ...
     #. Saving/loading to a file: text, npz  -> point to other IO routines
  #. Extracting data from arrays
     #. Basic indexing and slicing
     #. Simple fancy indexing
     #. Finding items in arrays
        #. comparisons, logic operations, indexing based on them
        #. where, searchsorted
     #. Advanced indexing on arrays: ellipsis, newaxis
  #. Views and copies of arrays
     #. Demonstrate that slicing in general creates views
  #. Modifying contents of arrays
     #. Setting data in arrays via indexing
     #. add, multiply subtract
     #. in-place operations (+ the common indexing caveat!)
     #. sum, mean, min, max, ...
     #. remark on ufuncs: common methods
     #. Operating on an axis of an array
  #. Broadcasting
  #. Joining, splitting arrays and changing their shape
     #. *stack, *split
     #. reshape
     #. resize
  #. Working with different types of data: integers, floats, complex, strings...
     #. Basic creation of arrays with certain data types
     #. Building up data type objects
     #. Casting and converting array data, automatic casting, coercion
  #. Advanced data types and structured arrays
     #. Creating structured arrays
     #. Defining them: literals, loading from files
     #. Accessing data in them
#. Other topics
   #. Working with missing data
      #. NaNs as masking
      #. Masked arrays
   #. Linear algebra and matrices
   #. Working with polynomials
   #. Floating point issues: errors, error handling, inaccuracy, etc.
   #. Fourier transforms
   #. Generating random numbers
   #. Building and testing packages using Numpy
   #. Financial calculations with Numpy
#. Extending Numpy
   #. Subclassing numpy arrays
   #. Array interface
   #. Ctypes support in Numpy
   #. Cython? Pyrex? F2Py?
   #. Writing C extensions using Numpy
      #. Basics
      #. Iteration
      #. Ufuncs
      #. Data types
      #. Subclassing in C
#. Numpy internals
   #. Memory model
   #. Data type stuff
   #. Ufuncs
   #. etc?

#. Reference
   <insert our current "reference manual" here as-is and factor out
   any duplication later on>



From HAWRYLA at novachem.com  Fri Aug 21 16:37:14 2009
From: HAWRYLA at novachem.com (Andrew Hawryluk)
Date: Fri, 21 Aug 2009 14:37:14 -0600
Subject: [SciPy-dev] NumPy User Guide table of contents
In-Reply-To: <h6msk1$jfn$1@ger.gmane.org>
References: <63751c30908211135x182b6e7q4d6e02a96035ac87@mail.gmail.com>
	<h6msk1$jfn$1@ger.gmane.org>
Message-ID: <48C01AE7354EC240A26F19CEB995E943033AF295@CHMAILMBX01.novachem.com>

> -----Original Message-----
> From: scipy-dev-bounces at scipy.org [mailto:scipy-dev-bounces at scipy.org]
> On Behalf Of Pauli Virtanen
> Sent: 21 Aug 2009 1:30 PM
> To: scipy-dev at scipy.org
> Subject: [SciPy-dev] NumPy User Guide table of contents
> 
> Hi all,
> 
> It's great to see that people are interested in contributing narrative
> documentation to Numpy! I think this effort will be accelerated much
if
> we have a table of contents for the final manual we want, to guide the
> writing.
> 
> One suggestion is listed below. It's also on this wiki page:
> 
> 	http://scipy.org/Developer_Zone/UG_Toc


This is very helpful - thanks for putting it together. I only have a few
builds:

- section 3.5 "Modifying contents of arrays" may need a different name,
since the array methods (sum, min, max, ...) don't modify any array
contents and the ufuncs don't have to either. Maybe "Manipulating array
data" or "Calculations with array data" or "Number crunching". On second
thought, "Number crunching" is probably a poor choice for non-native
readers.

- should section 3.8 "Working with different types of data" be placed
earlier in the array basics? Could it be merged into 3.2 "Storing data
in arrays"?

Cheers,
Andrew


From d_l_goldsmith at yahoo.com  Fri Aug 21 18:39:32 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 21 Aug 2009 22:39:32 +0000 (GMT)
Subject: [SciPy-dev] NumPy User Guide table of contents
In-Reply-To: <h6msk1$jfn$1@ger.gmane.org>
Message-ID: <373983.55101.qm@web52108.mail.re2.yahoo.com>

--- On Fri, 8/21/09, Pauli Virtanen <pav+sp at iki.fi> wrote:

Thanks, Pauli!  I've made my suggestions "in-line", denoted by a '+'.

DG

> ************
> Numpy Manual - TOC
> ************

 + #. Acknowledgements 
 
> #. Introduction to Numpy
> ???#. What it is + what it isn't (e.g., a comprehensive numerical mathematics library, nor even a comprehensive linear algebra library)
> ???#. What is there: a very high-level
> overview
> ???#. Conventions in this manual
> #. Installing Numpy
> ???#. Instructions for getting binaries
> ???#. Instructions for building from source,
> on different platforms
> #. Basics of arrays
> ? #. What is an array (in Numpy, in general, or both?)
> ? #. Storing data in arrays
> ? ???#. Arrays as literals
> ? ???#. Creating arrays: empty, zeros,
> ones, ...
> ? ???#. Saving/loading to a file: text,
> npz? -> point to other IO routines
> ? #. Extracting data from arrays
> ? ???#. Basic indexing and slicing
> ? ???#. Simple fancy indexing
> ? ???#. Finding items in arrays
> ? ? ? ? #. comparisons, logic
> operations, indexing based on them
> ? ? ? ? #. where, searchsorted
> ? ???#. Advanced indexing on arrays:
> ellipsis, newaxis
> ? #. Views and copies of arrays
> ? ???#. Demonstrate that slicing in
> general creates views
> ? #. Modifying contents of arrays
> ? ???#. Setting data in arrays via
> indexing
   +   #. scalar-across-array broadcasting:
>     ? ???#. add, multiply subtract
> ? ???#. in-place operations (with the
> common indexing caveat!)
> ? ???#. sum, mean, min, max, ...
> ? ???#. remark on ufuncs: common
> methods
> ? ???#. Operating [al]on[g] an axis of an
> array
> ? #. + [Array-across-array] Broadcasting
> ? #. Joining, splitting arrays and changing their
> shape
> ? ???#. *stack, *split
> ? ???#. reshape
> ? ???#. resize
> ? #. Working with different types of data: integers,
> floats, complex, strings...
> ? ???#. Basic creation of arrays with
> certain data types
> ? ???#. Building up data type objects
> ? ???#. Casting and converting array
> data, automatic casting, coercion
> ? #. Advanced data types and structured arrays
> ? ???#. Creating structured arrays
> ? ???#. Defining them: literals,
> loading from files
> ? ???#. Accessing data in them
> #. Other topics
> ???#. Working with missing data
> ? ? ? #. NaNs as masking
> ? ? ? #. Masked arrays
> ???#. Linear algebra and matrices
> ???#. Working with polynomials
> ???#. Floating point issues: errors, error
> handling, inaccuracy, etc.
> ???#. Fourier transforms
> ???#. Generating random numbers
> ???#. Building and testing packages using
> Numpy
> ???#. Financial calculations with Numpy
> #. Extending Numpy
> ???#. Subclassing numpy arrays
> ???#. Array interface
> ???#. Ctypes support in Numpy
> ???#. Cython? Pyrex? F2Py?
> ???#. Writing C extensions using Numpy
> ? ? ? #. Basics
> ? ? ? #. Iteration
> ? ? ? #. Ufuncs
> ? ? ? #. Data types
> ? ? ? #. Subclassing in C
> #. Numpy internals
> ???#. Memory model
> ???#. Data type stuff
> ???#. Ufuncs
> ???#. etc?
> 
> #. Reference
> ???<insert our current "reference manual"
> here as-is and factor out
> ???any duplication later on>
> 
 + #. Online references/support

 + #. Citations

 + #. Index 

> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From pivanov314 at gmail.com  Fri Aug 21 20:15:15 2009
From: pivanov314 at gmail.com (Paul Ivanov)
Date: Fri, 21 Aug 2009 17:15:15 -0700
Subject: [SciPy-dev] I just do what Joe tell me to (Docs)
Message-ID: <20090822001515.GG8191@ykcyc>

Please give me permission to edit and proofread.

thank you,

      Paul Ivanov (ivanov)


From gael.varoquaux at normalesup.org  Fri Aug 21 20:17:05 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 22 Aug 2009 02:17:05 +0200
Subject: [SciPy-dev] I just do what Joe tell me to (Docs)
In-Reply-To: <20090822001515.GG8191@ykcyc>
References: <20090822001515.GG8191@ykcyc>
Message-ID: <20090822001705.GA31013@phare.normalesup.org>

On Fri, Aug 21, 2009 at 05:15:15PM -0700, Paul Ivanov wrote:
> Please give me permission to edit and proofread.

Done.

Ga?l


From stefan at sun.ac.za  Fri Aug 21 20:18:03 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 21 Aug 2009 17:18:03 -0700
Subject: [SciPy-dev] I just do what Joe tell me to (Docs)
In-Reply-To: <20090822001515.GG8191@ykcyc>
References: <20090822001515.GG8191@ykcyc>
Message-ID: <9457e7c80908211718p8cd3232y66c90a5453ea5b88@mail.gmail.com>

Hey Paul

2009/8/21 Paul Ivanov <pivanov314 at gmail.com>:
> Please give me permission to edit and proofread.

Can't argue with your logic, but you already have a username with
editing access!

Cheers
St?fan


From stefan at sun.ac.za  Fri Aug 21 20:19:05 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Fri, 21 Aug 2009 17:19:05 -0700
Subject: [SciPy-dev] I just do what Joe tell me to (Docs)
In-Reply-To: <20090822001705.GA31013@phare.normalesup.org>
References: <20090822001515.GG8191@ykcyc>
	<20090822001705.GA31013@phare.normalesup.org>
Message-ID: <9457e7c80908211719p45d2b1d9s4ee8acefe3be4bc5@mail.gmail.com>

2009/8/21 Gael Varoquaux <gael.varoquaux at normalesup.org>:
> On Fri, Aug 21, 2009 at 05:15:15PM -0700, Paul Ivanov wrote:
>> Please give me permission to edit and proofread.
>
> Done.

OK, so Gael beat me to it...

Have fun,
St?fan


From d_l_goldsmith at yahoo.com  Fri Aug 21 21:06:02 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 21 Aug 2009 18:06:02 -0700 (PDT)
Subject: [SciPy-dev] Sprints
Message-ID: <707675.38065.qm@web52111.mail.re2.yahoo.com>

Hi, folks.  So, I'm "signed-up" to participate in the Sprint(s), but I haven't been able to successfully install mayavi - given this, will I be able to be helpful in the Sprint(s)?

DG

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From jsseabold at gmail.com  Fri Aug 21 22:09:01 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Fri, 21 Aug 2009 22:09:01 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
Message-ID: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com>

Hello all,

I have been trying to package the models code using the scikits
template and have run into a problem.

On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
package (sudo python setup.py install), but when I do

from scikits import statsmodels
statsmodels.test()

I get

Running unit tests for scikits.statsmodels
NumPy version 1.4.0.dev7303
NumPy is installed in /usr/local/lib/python2.6/dist-packages/numpy
Python version 2.6.2 (release26-maint, Apr 19 2009, 01:58:18) [GCC 4.3.3]
nose version 0.11.1

----------------------------------------------------------------------
Ran 0 tests in 0.003s

OK
Out[2]: <nose.result.TextTestResult run=0 errors=0 failures=0>

Similarly if I try to use nosetests from the command line on the
installed tests folder, I get nothing

skipper at linux-desktop:~$ nosetests -v
/usr/local/lib/python2.6/dist-packages/scikits.statsmodels-0.1.1dev-py2.6.egg/scikits/statsmodels/tests/

----------------------------------------------------------------------
Ran 0 tests in 0.003s

OK

However, if I run the nosetests either from the source directory or if
I import the directory statsmodels the tests return errors (which I
expect at this point, due to incomplete refactoring).

skipper at linux-desktop:~$ nosetests -w
/home/skipper/nipy/statsmodels/scikits/statsmodels/tests/

<snip>
----------------------------------------------------------------------
Ran 6 tests in 0.188s

FAILED (errors=6)

Josef is able to install the package and run the tests
(statsmodels.test()) and return these errors on Windows with Python
2.5.

Is there anyone who would be willing to have a look at my setup.py and
see if they spot a problem, because I am at a loss.

My attempt is here.  Also I don't think the tester.py is correct,
though I've tried with and without (and don't think it would affect
this issue).

<http://bazaar.launchpad.net/~scipystats/%2Bjunk/statsmodels/files/1859>

Cheers,

Skipper


From robert.kern at gmail.com  Sat Aug 22 01:45:18 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Fri, 21 Aug 2009 22:45:18 -0700
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com>
Message-ID: <3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>

On Fri, Aug 21, 2009 at 19:09, Skipper Seabold<jsseabold at gmail.com> wrote:
> Hello all,
>
> I have been trying to package the models code using the scikits
> template and have run into a problem.
>
> On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
> package (sudo python setup.py install), but when I do
>
> from scikits import statsmodels
> statsmodels.test()

Known issue. setuptools installs files with the executable flag set
(for quite silly reasons), and nosetests refuses to load such files by
default. nose 0.11 has an option to disable that behavior.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From gael.varoquaux at normalesup.org  Sat Aug 22 02:11:39 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 22 Aug 2009 08:11:39 +0200
Subject: [SciPy-dev] Sprints
In-Reply-To: <707675.38065.qm@web52111.mail.re2.yahoo.com>
References: <707675.38065.qm@web52111.mail.re2.yahoo.com>
Message-ID: <20090822061139.GA32373@phare.normalesup.org>

On Fri, Aug 21, 2009 at 06:06:02PM -0700, David Goldsmith wrote:
> Hi, folks.  So, I'm "signed-up" to participate in the Sprint(s), but I haven't been able to successfully install mayavi - given this, will I be able to be helpful in the Sprint(s)?

Yes, there are plenty of things that need to be done that don't involve
mayavi.

Ga?l


From d_l_goldsmith at yahoo.com  Sat Aug 22 02:17:30 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Fri, 21 Aug 2009 23:17:30 -0700 (PDT)
Subject: [SciPy-dev] Sprints
In-Reply-To: <20090822061139.GA32373@phare.normalesup.org>
Message-ID: <223715.64354.qm@web52112.mail.re2.yahoo.com>

Okee-dokee; you really expect to start at 8am?

DG

--- On Fri, 8/21/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Sprints
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Friday, August 21, 2009, 11:11 PM
> On Fri, Aug 21, 2009 at 06:06:02PM
> -0700, David Goldsmith wrote:
> > Hi, folks.? So, I'm "signed-up" to participate in
> the Sprint(s), but I haven't been able to successfully
> install mayavi - given this, will I be able to be helpful in
> the Sprint(s)?
> 
> Yes, there are plenty of things that need to be done that
> don't involve
> mayavi.
> 
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From dwf at cs.toronto.edu  Sat Aug 22 07:05:33 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Sat, 22 Aug 2009 04:05:33 -0700
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <962672.16322.qm@web52106.mail.re2.yahoo.com>
References: <962672.16322.qm@web52106.mail.re2.yahoo.com>
Message-ID: <0A6A598A-8781-41D5-94D9-50F83AC47E48@cs.toronto.edu>


On 21-Aug-09, at 8:30 AM, David Goldsmith wrote:

> I wasn't thinking to replace ndarray w/ Stefan's idea; rather, I was  
> thinking of the possibility of deriving ndarray from Stefan's idea,  
> implemented as _ndarray (i.e., it'd be "private" so as to not  
> disrupt the API and, hopefully, not confuse people too much).  But  
> if there would be performance _loss_ in doing so...never mind. :-)

What Stefan describes is actually how Cython treats PEP3118 buffers. I  
don't think np.foo functions will work on such buffers at present but  
they might (and ideally they should, eventually).

David


From gael.varoquaux at normalesup.org  Sat Aug 22 09:34:45 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 22 Aug 2009 15:34:45 +0200
Subject: [SciPy-dev] Sprints
In-Reply-To: <223715.64354.qm@web52112.mail.re2.yahoo.com>
References: <20090822061139.GA32373@phare.normalesup.org>
	<223715.64354.qm@web52112.mail.re2.yahoo.com>
Message-ID: <20090822133445.GA23027@phare.normalesup.org>

On Fri, Aug 21, 2009 at 11:17:30PM -0700, David Goldsmith wrote:
> Okee-dokee; you really expect to start at 8am?

No.

Ga?l



From d_l_goldsmith at yahoo.com  Sat Aug 22 10:19:49 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 22 Aug 2009 07:19:49 -0700 (PDT)
Subject: [SciPy-dev] Idle (?) speculation about ndarray [was draft:
	NumPy User Guide introduction]
In-Reply-To: <0A6A598A-8781-41D5-94D9-50F83AC47E48@cs.toronto.edu>
Message-ID: <879140.40697.qm@web52110.mail.re2.yahoo.com>

--- On Sat, 8/22/09, David Warde-Farley <dwf at cs.toronto.edu> wrote:

> > I wasn't thinking to replace ndarray w/ Stefan's idea;
> rather, I was? 
> > thinking of the possibility of deriving ndarray from
> Stefan's idea,? 
> > implemented as _ndarray (i.e., it'd be "private" so as
> to not? 
> > disrupt the API and, hopefully, not confuse people too
> much).? But? 
> > if there would be performance _loss_ in doing
> so...never mind. :-)
> 
> What Stefan describes is actually how Cython treats PEP3118
> buffers. I? 
> don't think np.foo functions will work on such buffers at

(Don't know enough about Cython) not even if you "cast" the buffer on the fly: np.foo(np.array(PEP3118 buffer)) (assuming np.array(PEP3118 buffer) is supported)?  Regardless, good to know.

DG
> present but? 
> they might (and ideally they should, eventually).


> 
> David
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From d_l_goldsmith at yahoo.com  Sat Aug 22 10:20:59 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 22 Aug 2009 14:20:59 +0000 (GMT)
Subject: [SciPy-dev] Sprints
In-Reply-To: <20090822133445.GA23027@phare.normalesup.org>
Message-ID: <828526.39925.qm@web52108.mail.re2.yahoo.com>

OK, so you'll be able to assign people to jobs as they wander in.

DG

--- On Sat, 8/22/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Sprints
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Saturday, August 22, 2009, 6:34 AM
> On Fri, Aug 21, 2009 at 11:17:30PM
> -0700, David Goldsmith wrote:
> > Okee-dokee; you really expect to start at 8am?
> 
> No.
> 
> Ga?l
> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 


      


From jsseabold at gmail.com  Sat Aug 22 10:31:56 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 10:31:56 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
Message-ID: <c048da1c0908220731i2db39152l24822a8f6e7e8d7a@mail.gmail.com>

On Sat, Aug 22, 2009 at 1:45 AM, Robert Kern<robert.kern at gmail.com> wrote:
> On Fri, Aug 21, 2009 at 19:09, Skipper Seabold<jsseabold at gmail.com> wrote:
>> Hello all,
>>
>> I have been trying to package the models code using the scikits
>> template and have run into a problem.
>>
>> On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
>> package (sudo python setup.py install), but when I do
>>
>> from scikits import statsmodels
>> statsmodels.test()
>
> Known issue. setuptools installs files with the executable flag set
> (for quite silly reasons), and nosetests refuses to load such files by
> default. nose 0.11 has an option to disable that behavior.

Wonderful.  I was hoping it was something as simple as this.  I did
notice that the files were marked as executable and thought that odd.
I was adding shebangs to them all and then still pulling my hair
out...

Thanks.

Skipper


From gael.varoquaux at normalesup.org  Sat Aug 22 10:35:47 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 22 Aug 2009 16:35:47 +0200
Subject: [SciPy-dev] Sprints
In-Reply-To: <828526.39925.qm@web52108.mail.re2.yahoo.com>
References: <20090822133445.GA23027@phare.normalesup.org>
	<828526.39925.qm@web52108.mail.re2.yahoo.com>
Message-ID: <20090822143547.GC23027@phare.normalesup.org>

On Sat, Aug 22, 2009 at 02:20:59PM +0000, David Goldsmith wrote:
> OK, so you'll be able to assign people to jobs as they wander in.

People should talk to the core developers of the project that they are
interested in helping out. Also, Jarrod is championing a task of general
interest: a rewamp of the scipy.org website.

Ga?l


From d_l_goldsmith at yahoo.com  Sat Aug 22 11:34:36 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Sat, 22 Aug 2009 08:34:36 -0700 (PDT)
Subject: [SciPy-dev] Sprints
In-Reply-To: <20090822143547.GC23027@phare.normalesup.org>
Message-ID: <49623.44071.qm@web52105.mail.re2.yahoo.com>

Are there any besides Stefan's image processing SciKit and Jarrod's mentioned below (which hasn't made it to the sprints page yet, BTW)?

DG
--- On Sat, 8/22/09, Gael Varoquaux <gael.varoquaux at normalesup.org> wrote:

> From: Gael Varoquaux <gael.varoquaux at normalesup.org>
> Subject: Re: [SciPy-dev] Sprints
> To: "SciPy Developers List" <scipy-dev at scipy.org>
> Date: Saturday, August 22, 2009, 7:35 AM
> On Sat, Aug 22, 2009 at 02:20:59PM
> +0000, David Goldsmith wrote:
> > OK, so you'll be able to assign people to jobs as they
> wander in.
> 
> People should talk to the core developers of the project
> that they are
> interested in helping out. Also, Jarrod is championing a
> task of general
> interest: a rewamp of the scipy.org website.
> 
> Ga?l
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From stefan at sun.ac.za  Sat Aug 22 12:30:51 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 22 Aug 2009 09:30:51 -0700
Subject: [SciPy-dev] Sprints
In-Reply-To: <49623.44071.qm@web52105.mail.re2.yahoo.com>
References: <20090822143547.GC23027@phare.normalesup.org>
	<49623.44071.qm@web52105.mail.re2.yahoo.com>
Message-ID: <9457e7c80908220930r755e7c24h52d76290f22820b5@mail.gmail.com>

2009/8/22 David Goldsmith <d_l_goldsmith at yahoo.com>:
> Are there any besides Stefan's image processing SciKit and Jarrod's mentioned below (which hasn't made it to the sprints page yet, BTW)?

I'm sure there will be work done on IPython, Mayavi, SciPy, NumPy and
maybe Cython.  I'm not sure if any side-projects will be joining in.

People who are not at the conference: feel free to join in on #scipy
at irc.freenode.net.

I'll see you guys in a while -- still need to check out of my room etc.

Cheers
St?fan


From jsseabold at gmail.com  Sat Aug 22 13:06:36 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 13:06:36 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
Message-ID: <c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com>

On Sat, Aug 22, 2009 at 1:45 AM, Robert Kern<robert.kern at gmail.com> wrote:
> On Fri, Aug 21, 2009 at 19:09, Skipper Seabold<jsseabold at gmail.com> wrote:
>> Hello all,
>>
>> I have been trying to package the models code using the scikits
>> template and have run into a problem.
>>
>> On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
>> package (sudo python setup.py install), but when I do
>>
>> from scikits import statsmodels
>> statsmodels.test()
>
> Known issue. setuptools installs files with the executable flag set
> (for quite silly reasons), and nosetests refuses to load such files by
> default. nose 0.11 has an option to disable that behavior.
>

This is the case, of course.  I'm just not clear on what the
workaround is for the statsmodels tests (ie., not running nosetests on
the command line).  Do I need to change the tests to inherit from
np.testing.TestCase?  If so, then I think I need to move all of the
__init__ setup of the tests to a setup method.  I don't see any
special configuration options in any of the other scikits.

Skipper


From cournape at gmail.com  Sat Aug 22 13:09:58 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sat, 22 Aug 2009 10:09:58 -0700
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com>
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com>
Message-ID: <5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com>

On Sat, Aug 22, 2009 at 10:06 AM, Skipper Seabold<jsseabold at gmail.com> wrote:
> On Sat, Aug 22, 2009 at 1:45 AM, Robert Kern<robert.kern at gmail.com> wrote:
>> On Fri, Aug 21, 2009 at 19:09, Skipper Seabold<jsseabold at gmail.com> wrote:
>>> Hello all,
>>>
>>> I have been trying to package the models code using the scikits
>>> template and have run into a problem.
>>>
>>> On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
>>> package (sudo python setup.py install), but when I do
>>>
>>> from scikits import statsmodels
>>> statsmodels.test()
>>
>> Known issue. setuptools installs files with the executable flag set
>> (for quite silly reasons), and nosetests refuses to load such files by
>> default. nose 0.11 has an option to disable that behavior.
>>
>
> This is the case, of course. ?I'm just not clear on what the
> workaround is for the statsmodels tests (ie., not running nosetests on
> the command line). ?Do I need to change the tests to inherit from
> np.testing.TestCase? ?If so, then I think I need to move all of the
> __init__ setup of the tests to a setup method. ?I don't see any
> special configuration options in any of the other scikits.

I don't think there is any workaround, except for putting everything
back to no executable bit or using nose 0.11.

Hopefully, that's one of this stupid setuptools annoyance which will
be fixed in the distribute fork,

cheers,

David


From jsseabold at gmail.com  Sat Aug 22 13:28:29 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 13:28:29 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com>
Message-ID: <c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com>

On Sat, Aug 22, 2009 at 1:09 PM, David Cournapeau<cournape at gmail.com> wrote:
> On Sat, Aug 22, 2009 at 10:06 AM, Skipper Seabold<jsseabold at gmail.com> wrote:
>> On Sat, Aug 22, 2009 at 1:45 AM, Robert Kern<robert.kern at gmail.com> wrote:
>>> On Fri, Aug 21, 2009 at 19:09, Skipper Seabold<jsseabold at gmail.com> wrote:
>>>> Hello all,
>>>>
>>>> I have been trying to package the models code using the scikits
>>>> template and have run into a problem.
>>>>
>>>> On linux, both Kubuntu 64-bit and openSuSE 32-bit, I can install the
>>>> package (sudo python setup.py install), but when I do
>>>>
>>>> from scikits import statsmodels
>>>> statsmodels.test()
>>>
>>> Known issue. setuptools installs files with the executable flag set
>>> (for quite silly reasons), and nosetests refuses to load such files by
>>> default. nose 0.11 has an option to disable that behavior.
>>>
>>
>> This is the case, of course. ?I'm just not clear on what the
>> workaround is for the statsmodels tests (ie., not running nosetests on
>> the command line). ?Do I need to change the tests to inherit from
>> np.testing.TestCase? ?If so, then I think I need to move all of the
>> __init__ setup of the tests to a setup method. ?I don't see any
>> special configuration options in any of the other scikits.
>
> I don't think there is any workaround, except for putting everything
> back to no executable bit or using nose 0.11.
>
> Hopefully, that's one of this stupid setuptools annoyance which will
> be fixed in the distribute fork,
>

Whoa, that's odd.  Yeah it's the same with other scikits.  I assumed
that I was just missing something, as I'm still very much a neophyte.
Thanks.

Is it bad form to make package.test_linux() pass an argument to the
shell with the --exe option for nosetests, assuming there is a check
for the version of nosetests?  I suppose I could just add a note to
the README.

Skipper


From robert.kern at gmail.com  Sat Aug 22 13:34:01 2009
From: robert.kern at gmail.com (Robert Kern)
Date: Sat, 22 Aug 2009 10:34:01 -0700
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com>
Message-ID: <3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com>

On Sat, Aug 22, 2009 at 10:28, Skipper Seabold<jsseabold at gmail.com> wrote:
> Is it bad form to make package.test_linux() pass an argument to the
> shell with the --exe option for nosetests, assuming there is a check
> for the version of nosetests? ?I suppose I could just add a note to
> the README.

NoseTester.test() takes an extra_argv argument that can be used to
pass in --exe.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless
enigma that is made terrible by our own mad attempt to interpret it as
though it had an underlying truth."
  -- Umberto Eco


From jsseabold at gmail.com  Sat Aug 22 13:56:58 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 13:56:58 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com> 
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com>
Message-ID: <c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>

On Sat, Aug 22, 2009 at 1:34 PM, Robert Kern<robert.kern at gmail.com> wrote:
> On Sat, Aug 22, 2009 at 10:28, Skipper Seabold<jsseabold at gmail.com> wrote:
>> Is it bad form to make package.test_linux() pass an argument to the
>> shell with the --exe option for nosetests, assuming there is a check
>> for the version of nosetests? ?I suppose I could just add a note to
>> the README.
>
> NoseTester.test() takes an extra_argv argument that can be used to
> pass in --exe.

Is it passed as a string?  Judging by the code it should be, but I'm still not
having any luck.

in the package level __init__.py I have

from numpy.testing import Tester
test = Tester.test(extra_argv="--exe")

I then get OSerrors when I install and try to import the package about
there being no such modules '-', '-', 'e', ... in my home directory(?)

Skipper


From jsseabold at gmail.com  Sat Aug 22 13:59:07 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 13:59:07 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com> 
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com> 
	<c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>
Message-ID: <c048da1c0908221059h3d084b88u692f93c474239873@mail.gmail.com>

On Sat, Aug 22, 2009 at 1:56 PM, Skipper Seabold<jsseabold at gmail.com> wrote:
> On Sat, Aug 22, 2009 at 1:34 PM, Robert Kern<robert.kern at gmail.com> wrote:
>> On Sat, Aug 22, 2009 at 10:28, Skipper Seabold<jsseabold at gmail.com> wrote:
>>> Is it bad form to make package.test_linux() pass an argument to the
>>> shell with the --exe option for nosetests, assuming there is a check
>>> for the version of nosetests? ?I suppose I could just add a note to
>>> the README.
>>
>> NoseTester.test() takes an extra_argv argument that can be used to
>> pass in --exe.
>
> Is it passed as a string? ?Judging by the code it should be, but I'm still not
> having any luck.
>
> in the package level __init__.py I have
>
> from numpy.testing import Tester
> test = Tester.test(extra_argv="--exe")
>
> I then get OSerrors when I install and try to import the package about
> there being no such modules '-', '-', 'e', ... in my home directory(?)
>

Edit: in my working directory.

> Skipper
>


From stefan at sun.ac.za  Sat Aug 22 13:59:07 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 22 Aug 2009 10:59:07 -0700
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com> 
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com> 
	<c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>
Message-ID: <9457e7c80908221059q1cb9b592w1b0f6a040d5ab71@mail.gmail.com>

2009/8/22 Skipper Seabold <jsseabold at gmail.com>:
> from numpy.testing import Tester
> test = Tester.test(extra_argv="--exe")

I think it takes a list?

Cheers
St?fan


From jsseabold at gmail.com  Sat Aug 22 14:03:48 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sat, 22 Aug 2009 14:03:48 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <9457e7c80908221059q1cb9b592w1b0f6a040d5ab71@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com> 
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com> 
	<c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com> 
	<9457e7c80908221059q1cb9b592w1b0f6a040d5ab71@mail.gmail.com>
Message-ID: <c048da1c0908221103u20a08f7bh9fa294586c73fbdb@mail.gmail.com>

2009/8/22 St?fan van der Walt <stefan at sun.ac.za>:
> 2009/8/22 Skipper Seabold <jsseabold at gmail.com>:
>> from numpy.testing import Tester
>> test = Tester.test(extra_argv="--exe")
>
> I think it takes a list?

That makes a lot more senes...and works!  At least it gives import
errors from the tests due to incomplete refactoring on import of the
package, which is odd, but maybe expected?

*stops banging head against desk

Thanks,

Skipper


From gael.varoquaux at normalesup.org  Sat Aug 22 14:20:24 2009
From: gael.varoquaux at normalesup.org (Gael Varoquaux)
Date: Sat, 22 Aug 2009 20:20:24 +0200
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <c048da1c0908221103u20a08f7bh9fa294586c73fbdb@mail.gmail.com>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com>
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com>
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com>
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com>
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com>
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com>
	<c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com>
	<9457e7c80908221059q1cb9b592w1b0f6a040d5ab71@mail.gmail.com>
	<c048da1c0908221103u20a08f7bh9fa294586c73fbdb@mail.gmail.com>
Message-ID: <20090822182024.GB32326@phare.normalesup.org>

On Sat, Aug 22, 2009 at 02:03:48PM -0400, Skipper Seabold wrote:
> 2009/8/22 St?fan van der Walt <stefan at sun.ac.za>:
> > 2009/8/22 Skipper Seabold <jsseabold at gmail.com>:
> >> from numpy.testing import Tester
> >> test = Tester.test(extra_argv="--exe")

> > I think it takes a list?

> That makes a lot more senes...and works!

Maybe the docstring could be improved?

Ga?l 


From stefan at sun.ac.za  Sat Aug 22 16:19:51 2009
From: stefan at sun.ac.za (=?ISO-8859-1?Q?St=E9fan_van_der_Walt?=)
Date: Sat, 22 Aug 2009 13:19:51 -0700
Subject: [SciPy-dev] Sprinting at SciPy2009 today and tomorrow
Message-ID: <9457e7c80908221319peab26e8we224d5a7a6d3bebf@mail.gmail.com>

Hey everyone,

The SciPy2009 sprints are underway, and you are welcome to take part!
Topics include NumPy, SciPy, Mayavi, Traits, IPython, Documentation
and the image processing toolbox.

Join us on irc in channel #scipy, server irc.freenode.net.

The timezone here is GMT-7, and we'll be around both days from 10:00am
till late.

See you there,

St?fan


From charlesr.harris at gmail.com  Sun Aug 23 01:02:14 2009
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Sat, 22 Aug 2009 23:02:14 -0600
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
References: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>
	<308926.80222.qm@web52110.mail.re2.yahoo.com>
	<9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
Message-ID: <e06186140908222202n4bfb07a4md52a78100edeeb67@mail.gmail.com>

2009/8/20 St?fan van der Walt <stefan at sun.ac.za>

> 2009/8/20 David Goldsmith <d_l_goldsmith at yahoo.com>:
> >> > of arr.* methods (maybe others differ with me on this
> >> point?).
> >>
> >> From an editor's perspective, I would be neutral. The
> >> author of an
> >> example should use whatever they feel gets their point
> >> across best.
> >> Editors should leave that choice alone.
> >>
> >
> > However, I'm curious to hear Stefan's reasons/reasoning?
>
> I guess I should have said "others *will* differ with me on this point" :-)
>
> I like the idea of an ndarray being a concise description of bytes in
> memory.  At the moment, we have a couple of "special" methods attached
> to it, but we could just as well have added fewer or more, and
> personally I'd have preferred none.
>

Curiously, none was the choice made for the original numeric, although none
translated to about three (IIRC) in practice. I believe numarray introduced
the systematic use of methods in preference to functions and this was
carried over to numpy. Different folks have different preferences in that
regard. I'm basically indifferent to the choice at the user level, but for
maintainence and development I think that functions make it easier to add
functionality without bloating the ndarray object.  And figuring out what
all the desireable methods are up front is a difficult exercise in
prospective design.

Chuck
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090822/365ca421/attachment.html>

From fperez.net at gmail.com  Sun Aug 23 01:32:52 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 22 Aug 2009 22:32:52 -0700
Subject: [SciPy-dev] Please help!
In-Reply-To: <7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com> 
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com> 
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
Message-ID: <db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>

On Fri, Aug 21, 2009 at 9:50 AM, Aric Hagberg<aric.hagberg at gmail.com> wrote:

> It should work for complex double. ?I just ran your example code and
> it executed without failure for me. ?It seems like it is giving the
> correct answer (d and dreal, v and vreal are the same).
>
> It may be that you have a miscompiled ARPACK in scipy or some part of
> the complex double LAPACK/BLAS that ARPACK calls?
>
> What platform are you using? I ran this test with stock Ubuntu
> versions of LAPACK and BLAS and the dev versions of scipy and numpy.

Aric, I bet you're on Ubuntu 8.10 or prior, and  the OP is on 9.04.
It does fail on 9.04:

maqroll[Desktop]> python test_complex.py
[ -1.33333333e+00 +0.00000000e+00j  -1.33333333e+00 +0.00000000e+00j
   0.00000000e+00 +0.00000000e+00j  -1.23857134e-32 +0.00000000e+00j
  -6.49706555e-33 +2.10704681e-33j  -6.49706555e-33 -2.10704681e-33j
  -5.24286184e-34 +2.27743942e-33j  -5.24286184e-34 -2.27743942e-33j
   2.01220438e-33 +0.00000000e+00j   5.24878615e-32 +0.00000000e+00j]
Traceback (most recent call last):
  File "test_complex.py", line 98, in <module>
    (d,v) = eigen(A,k=10,which='SR')
  File "/home/fperez/usr/opt/lib/python2.6/site-packages/scipy/sparse/linalg/eigen/arpack/arpack.py",
line 220, in eigen
    raise RuntimeError("Error info=%d in arpack"%info)
RuntimeError: Error info=-9999 in arpack


And the problem is not your fault , it's a Debian/Ubuntu bug:

https://bugs.launchpad.net/ubuntu/+source/atlas/+bug/363510

The bad news is nothing much seems to be happening upstream.  The good
news is that  Matthew  Brett has  been pushing for ways to help with
this, and both David Cournapeau and Ondrej stepped up to fight it,
and  I think they've finally made progress (David told me on Friday
afternoon at the conference that they'd been able to crack some of the
problems related to the build).  I'm sure they'll post soon with more
details.

Cheers,

f


From aric.hagberg at gmail.com  Sun Aug 23 10:01:20 2009
From: aric.hagberg at gmail.com (Aric Hagberg)
Date: Sun, 23 Aug 2009 08:01:20 -0600
Subject: [SciPy-dev] Please help!
In-Reply-To: <db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
Message-ID: <7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>

On Sat, Aug 22, 2009 at 11:32 PM, Fernando Perez<fperez.net at gmail.com> wrote:
>
> Aric, I bet you're on Ubuntu 8.10 or prior, and ?the OP is on 9.04.
> It does fail on 9.04:

I am running 9.04 - but 64 bit.  So the 64 bit libraries seem to work
for this case.

Aric


From josef.pktd at gmail.com  Sun Aug 23 10:39:41 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sun, 23 Aug 2009 10:39:41 -0400
Subject: [SciPy-dev] setup.py sdist and bzr
Message-ID: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>

I had problems including data files in the source distribution
generated with setup.py sdist .

After spending some time with google, I figured out that
setuptools only includes files under svn version control by default,
but not under bzr

config.add_data_files(filename)
config.add_data_dir('scikits/statsmodels/tests')

worked with bdist_egg and bdist but not with sdist

I finally added a MANIFEST.in  file with almost  include *

Is the MANIFEST.in the only way or is there a better way similar to
the case with svn?

Josef


From nmb at wartburg.edu  Sun Aug 23 11:16:47 2009
From: nmb at wartburg.edu (Neil Martinsen-Burrell)
Date: Sun, 23 Aug 2009 09:16:47 -0600
Subject: [SciPy-dev] setup.py sdist and bzr
In-Reply-To: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
References: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
Message-ID: <4A915D5F.3000903@wartburg.edu>

On 2009-08-23 08:39 , josef.pktd at gmail.com wrote:
> I had problems including data files in the source distribution
> generated with setup.py sdist .
>
> After spending some time with google, I figured out that
> setuptools only includes files under svn version control by default,
> but not under bzr
>
> config.add_data_files(filename)
> config.add_data_dir('scikits/statsmodels/tests')
>
> worked with bdist_egg and bdist but not with sdist
>
> I finally added a MANIFEST.in  file with almost  include *
>
> Is the MANIFEST.in the only way or is there a better way similar to
> the case with svn?

I found this thread 
http://mail.python.org/pipermail/distutils-sig/2007-July/007914.html 
which pointed to this Launchpad project 
https://code.launchpad.net/~barry/setuptoolsbzr/trunk but I haven't used 
such a thing.  Post back to the list with your experiences if you would 
be so kind.  Peace,

-Neil


From cournape at gmail.com  Sun Aug 23 11:26:33 2009
From: cournape at gmail.com (David Cournapeau)
Date: Sun, 23 Aug 2009 08:26:33 -0700
Subject: [SciPy-dev] setup.py sdist and bzr
In-Reply-To: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
References: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
Message-ID: <5b8d13220908230826s7cb3b610mcd4bf57cc801de35@mail.gmail.com>

On Sun, Aug 23, 2009 at 7:39 AM, <josef.pktd at gmail.com> wrote:
> I had problems including data files in the source distribution
> generated with setup.py sdist .
>
> After spending some time with google, I figured out that
> setuptools only includes files under svn version control by default,
> but not under bzr
>
> config.add_data_files(filename)
> config.add_data_dir('scikits/statsmodels/tests')
>
> worked with bdist_egg and bdist but not with sdist
>
> I finally added a MANIFEST.in ?file with almost ?include *
>
> Is the MANIFEST.in the only way or is there a better way similar to
> the case with svn?

I strongly advise you not to use this "feature" of setuptools. Using
the source control system to decide which files to include brings a
lot of trouble (it makes reproducing the same tarball very difficult,
and strongly dependent on your environment).

cheers,

David


From niall.moran at gmail.com  Sun Aug 23 11:59:34 2009
From: niall.moran at gmail.com (Niall Moran)
Date: Sun, 23 Aug 2009 16:59:34 +0100
Subject: [SciPy-dev] Please help!
In-Reply-To: <7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
	<7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>
Message-ID: <dbcf15060908230859j2466f36ey11e1bb452a6632d9@mail.gmail.com>

Thanks for the replies. I am happy to hear that it is working for
someone. It is the 32bit ubuntu 9.04 that I am running indeed. Look
forward to hearing about a solution.

Thanks,

Niall.



On Sun, Aug 23, 2009 at 3:01 PM, Aric Hagberg<aric.hagberg at gmail.com> wrote:
> On Sat, Aug 22, 2009 at 11:32 PM, Fernando Perez<fperez.net at gmail.com> wrote:
>>
>> Aric, I bet you're on Ubuntu 8.10 or prior, and ?the OP is on 9.04.
>> It does fail on 9.04:
>
> I am running 9.04 - but 64 bit. ?So the 64 bit libraries seem to work
> for this case.
>
> Aric
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From josef.pktd at gmail.com  Sun Aug 23 13:15:50 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sun, 23 Aug 2009 13:15:50 -0400
Subject: [SciPy-dev] setup.py sdist and bzr
In-Reply-To: <5b8d13220908230826s7cb3b610mcd4bf57cc801de35@mail.gmail.com>
References: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
	<5b8d13220908230826s7cb3b610mcd4bf57cc801de35@mail.gmail.com>
Message-ID: <1cd32cbb0908231015w7dbea03av186f0e7c1bfa20df@mail.gmail.com>

On Sun, Aug 23, 2009 at 11:26 AM, David Cournapeau<cournape at gmail.com> wrote:
> On Sun, Aug 23, 2009 at 7:39 AM, <josef.pktd at gmail.com> wrote:
>> I had problems including data files in the source distribution
>> generated with setup.py sdist .
>>
>> After spending some time with google, I figured out that
>> setuptools only includes files under svn version control by default,
>> but not under bzr
>>
>> config.add_data_files(filename)
>> config.add_data_dir('scikits/statsmodels/tests')
>>
>> worked with bdist_egg and bdist but not with sdist
>>
>> I finally added a MANIFEST.in ?file with almost ?include *
>>
>> Is the MANIFEST.in the only way or is there a better way similar to
>> the case with svn?
>
> I strongly advise you not to use this "feature" of setuptools. Using
> the source control system to decide which files to include brings a
> lot of trouble (it makes reproducing the same tarball very difficult,
> and strongly dependent on your environment).
>
> cheers,
>
> David
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>

I assume you mean not use the automatic includes without MANIFEST.in

I'm still trying to figure out what the recommended way is, some
projects, scikits use the MANIFEST.in and some don't

from: http://www.scipy.org/scipy/scikits/wiki/ScikitsForDevelopers

Some more notes regarding setuptools ?

    * setuptools is largely backwards-compatible to distutils, the old
distutils ways can also be used although setuptools does offer
preferred ways for doing several things (e.g. a MANFIEST.in file can
be used; however setuptools will use it directly (in conjunction with
VC info) and completely ignore the contents of the MANIFEST file that
distutils generates from it; typically neither is required though as
in the absence of a MANIFEST.in file setuptools will automatically
include all files under version control when building a source
distribution), the old distutils ways can also be used


I would have thought a MANIFEST.in is not necessary, since the data
files are already specified with add_data_files, and there are no
problems with egg or binary distribution.

If the MANIFEST.in is the recommended and, for non-svn users, the
required way, then we could update the recommendations in the wiki and
the example scikits at
http://projects.scipy.org/scikits/browser/trunk/example

Thanks,

Josef


From josef.pktd at gmail.com  Sun Aug 23 13:30:21 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Sun, 23 Aug 2009 13:30:21 -0400
Subject: [SciPy-dev] setup.py sdist and bzr
In-Reply-To: <4A915D5F.3000903@wartburg.edu>
References: <1cd32cbb0908230739i53b46066j9ff4d66aa087cb52@mail.gmail.com>
	<4A915D5F.3000903@wartburg.edu>
Message-ID: <1cd32cbb0908231030v6b9233b4u91af2099278a099f@mail.gmail.com>

On Sun, Aug 23, 2009 at 11:16 AM, Neil
Martinsen-Burrell<nmb at wartburg.edu> wrote:
> On 2009-08-23 08:39 , josef.pktd at gmail.com wrote:
>> I had problems including data files in the source distribution
>> generated with setup.py sdist .
>>
>> After spending some time with google, I figured out that
>> setuptools only includes files under svn version control by default,
>> but not under bzr
>>
>> config.add_data_files(filename)
>> config.add_data_dir('scikits/statsmodels/tests')
>>
>> worked with bdist_egg and bdist but not with sdist
>>
>> I finally added a MANIFEST.in ?file with almost ?include *
>>
>> Is the MANIFEST.in the only way or is there a better way similar to
>> the case with svn?
>
> I found this thread
> http://mail.python.org/pipermail/distutils-sig/2007-July/007914.html
> which pointed to this Launchpad project
> https://code.launchpad.net/~barry/setuptoolsbzr/trunk but I haven't used
> such a thing. ?Post back to the list with your experiences if you would
> be so kind. ?Peace,

I hadn't found the earlier message but I looked briefly at
setuptoolsbzr, but I decided not to experiment with it and ask for
experience of others.

my impression from a quick look at setuptoolsbzr:

https://lists.launchpad.net/launchpad-users/msg04492.html  leaves it
unclear whether it actually works for this

it requires itself as an install dependency (not just a build dependency)

it hasn't been updated in a year

So I thought it's too risky to play with it, if there is any way of
avoiding the additional dependency.

Using MANIFEST.in seems to work and I even finally thought of looking
in the python docs for the format and command description.

Thanks, and thank you for the docstrings,

Josef

>
> -Neil
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From jsseabold at gmail.com  Sun Aug 23 14:48:23 2009
From: jsseabold at gmail.com (Skipper Seabold)
Date: Sun, 23 Aug 2009 14:48:23 -0400
Subject: [SciPy-dev] Problems packaging GSoC statsmodels code as scikit
In-Reply-To: <20090822182024.GB32326@phare.normalesup.org>
References: <c048da1c0908211909j4bc228fch9c39a684ec7da5f5@mail.gmail.com> 
	<3d375d730908212245k40687e5cyd356448900276728@mail.gmail.com> 
	<c048da1c0908221006i28cb0561s274066397a7d4821@mail.gmail.com> 
	<5b8d13220908221009v74dce193n6027700bdcd05dc7@mail.gmail.com> 
	<c048da1c0908221028g66e2b9d9o8f4d3a6e8112893a@mail.gmail.com> 
	<3d375d730908221034x13930745ic9618f5f0b3ea49b@mail.gmail.com> 
	<c048da1c0908221056s7e51f419hee358fc672164fe5@mail.gmail.com> 
	<9457e7c80908221059q1cb9b592w1b0f6a040d5ab71@mail.gmail.com> 
	<c048da1c0908221103u20a08f7bh9fa294586c73fbdb@mail.gmail.com> 
	<20090822182024.GB32326@phare.normalesup.org>
Message-ID: <c048da1c0908231148l1071c1ccw1c20e0efc87f866c@mail.gmail.com>

On Sat, Aug 22, 2009 at 2:20 PM, Gael
Varoquaux<gael.varoquaux at normalesup.org> wrote:
> On Sat, Aug 22, 2009 at 02:03:48PM -0400, Skipper Seabold wrote:
>> 2009/8/22 St?fan van der Walt <stefan at sun.ac.za>:
>> > 2009/8/22 Skipper Seabold <jsseabold at gmail.com>:
>> >> from numpy.testing import Tester
>> >> test = Tester.test(extra_argv="--exe")
>
>> > I think it takes a list?
>
>> That makes a lot more senes...and works!
>
> Maybe the docstring could be improved?
>

I wrote a patch that improves the docstring.  There is also a bug if
package is a string and then if it's a string and is installed
somewhere else other than site-packages I don't think it's picked up.

Patch is here with related ticket.

http://projects.scipy.org/numpy/ticket/1168

Skipper


From fperez.net at gmail.com  Sun Aug 23 17:07:25 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 23 Aug 2009 14:07:25 -0700
Subject: [SciPy-dev] Please help!
In-Reply-To: <7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com> 
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com> 
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com> 
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com> 
	<7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>
Message-ID: <db6b5ecc0908231407s7b6775a5lcd7205cde64fa6c@mail.gmail.com>

2009/8/23 Aric Hagberg <aric.hagberg at gmail.com>:
> On Sat, Aug 22, 2009 at 11:32 PM, Fernando Perez<fperez.net at gmail.com> wrote:
>>
>> Aric, I bet you're on Ubuntu 8.10 or prior, and ?the OP is on 9.04.
>> It does fail on 9.04:
>
> I am running 9.04 - but 64 bit. ?So the 64 bit libraries seem to work
> for this case.

Yes, sorry: I should have clarified the problem is on 32-bit 9.04 (and
A. Straw  confirmed that 9.10 also has it).

Cheers,

f


From dalcinl at gmail.com  Sun Aug 23 18:53:52 2009
From: dalcinl at gmail.com (Lisandro Dalcin)
Date: Sun, 23 Aug 2009 19:53:52 -0300
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <e06186140908222202n4bfb07a4md52a78100edeeb67@mail.gmail.com>
References: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>
	<308926.80222.qm@web52110.mail.re2.yahoo.com>
	<9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>
	<e06186140908222202n4bfb07a4md52a78100edeeb67@mail.gmail.com>
Message-ID: <e7ba66e40908231553i7c61cd60ud02fbdabf58c7bca@mail.gmail.com>

On Sun, Aug 23, 2009 at 2:02 AM, Charles R
Harris<charlesr.harris at gmail.com> wrote:
>
> but for
> maintainence and development I think that functions make it easier to add
> functionality without bloating the ndarray object.

Why do you say that adding methods bloat the ndarray object? Adding
functions bloats the numpy namespace,
adding methods bloat the numpy.ndarray namespace... In both cases you
are just populating a dict... What's the difference for you?

> And figuring out what
> all the desireable methods are up front is a difficult exercise in
> prospective design.
>

I think you are going to be faced to such difficulties when deciding
desirable functions for the numpy namespace.






>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>



-- 
Lisandro Dalc?n
---------------
Centro Internacional de M?todos Computacionales en Ingenier?a (CIMEC)
Instituto de Desarrollo Tecnol?gico para la Industria Qu?mica (INTEC)
Consejo Nacional de Investigaciones Cient?ficas y T?cnicas (CONICET)
PTLC - G?emes 3450, (3000) Santa Fe, Argentina
Tel/Fax: +54-(0)342-451.1594


From niall.moran at gmail.com  Mon Aug 24 08:59:43 2009
From: niall.moran at gmail.com (Niall Moran)
Date: Mon, 24 Aug 2009 13:59:43 +0100
Subject: [SciPy-dev] Please help!
In-Reply-To: <db6b5ecc0908231407s7b6775a5lcd7205cde64fa6c@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
	<7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com>
	<db6b5ecc0908231407s7b6775a5lcd7205cde64fa6c@mail.gmail.com>
Message-ID: <dbcf15060908240559y30db49c8yb1f7d888b1802f49@mail.gmail.com>

Hi I have tried rebuilding the latest versions of numpy and scipy and
the sparse complex eigen solver is now working perfeclty. I am not
sure if this is due to a fix someone applied (looked through svn
briefly but did not see anything that looked relevant) or some changes
I made to the packages installed possible the atlas ones.
The versions I built were

numpy.__version__
'1.4.0.dev7308'

 scipy.__version__
'0.8.0.dev5909'

I removed the atlas-sse2 blas and lapack library packages just leaving
the generic ones.
apt-show-versions | grep atlas
libatlas-base-dev/jaunty uptodate 3.6.0-22ubuntu2
libatlas-headers/jaunty uptodate 3.6.0-22ubuntu2
libatlas3gf-base/jaunty uptodate 3.6.0-22ubuntu2

Previously I had built my own umfpack and swig libraries. This time I
used the packaged versions from
libsuitesparse-3.2.0/jaunty uptodate 1:3.2.0-4ubuntu2
libsuitesparse-dev/jaunty uptodate 1:3.2.0-4ubuntu2
swig/jaunty uptodate 1.3.36-1ubuntu2

Commands used to build were simply

python setup.py build --fcompiler=gnu95 install

Many thanks,

Niall.


On Sun, Aug 23, 2009 at 10:07 PM, Fernando Perez<fperez.net at gmail.com> wrote:
> 2009/8/23 Aric Hagberg <aric.hagberg at gmail.com>:
>> On Sat, Aug 22, 2009 at 11:32 PM, Fernando Perez<fperez.net at gmail.com> wrote:
>>>
>>> Aric, I bet you're on Ubuntu 8.10 or prior, and ?the OP is on 9.04.
>>> It does fail on 9.04:
>>
>> I am running 9.04 - but 64 bit. ?So the 64 bit libraries seem to work
>> for this case.
>
> Yes, sorry: I should have clarified the problem is on 32-bit 9.04 (and
> A. Straw ?confirmed that 9.10 also has it).
>
> Cheers,
>
> f
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>


From fperez.net at gmail.com  Mon Aug 24 13:45:54 2009
From: fperez.net at gmail.com (Fernando Perez)
Date: Mon, 24 Aug 2009 10:45:54 -0700
Subject: [SciPy-dev] Please help!
In-Reply-To: <dbcf15060908240559y30db49c8yb1f7d888b1802f49@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com> 
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com> 
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com> 
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com> 
	<7ec4ad760908230701i1f09291amc37bcc10e4e6e423@mail.gmail.com> 
	<db6b5ecc0908231407s7b6775a5lcd7205cde64fa6c@mail.gmail.com> 
	<dbcf15060908240559y30db49c8yb1f7d888b1802f49@mail.gmail.com>
Message-ID: <db6b5ecc0908241045n37bdbc65w41f37a7961375f50@mail.gmail.com>

On Mon, Aug 24, 2009 at 5:59 AM, Niall Moran<niall.moran at gmail.com> wrote:
> I removed the atlas-sse2 blas and lapack library packages just leaving
> the generic ones.

This is what helped you.  The bug is in the atlas-sse2 package from
ubuntu 32-bit, as of 9.04 (and as far as we know, still in the
pre-9.10).

Cheers,

f


From matteo at naufraghi.net  Tue Aug 25 04:30:56 2009
From: matteo at naufraghi.net (Matteo Bertini)
Date: Tue, 25 Aug 2009 08:30:56 +0000 (UTC)
Subject: [SciPy-dev] [timeseries][patch] Accept Date(datetime) as
	Date(datetime=datetime)
Message-ID: <loom.20090825T103001-899@post.gmane.org>

Sometime I get lost in a nasty bug because I forget to use the named
argument Date(datetime=datetime).

I propose a small patch to make 'datetime' take the same way 'string'
is already following.

http://scipy.org/scipy/scikits/ticket/102

[---8<---]
    if (value %% (PyDateTime_Check(value) || PyDate_Check(value))) {
        if (!datetime) {
            datetime = value;
        }
        value = NULL;
    } // datetime = (datetime||value), value = NULL
[---8<---]

Ciao,
Matteo Bertini




From dagss at student.matnat.uio.no  Tue Aug 25 04:56:30 2009
From: dagss at student.matnat.uio.no (Dag Sverre Seljebotn)
Date: Tue, 25 Aug 2009 10:56:30 +0200
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <e7ba66e40908231553i7c61cd60ud02fbdabf58c7bca@mail.gmail.com>
References: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>	<308926.80222.qm@web52110.mail.re2.yahoo.com>	<9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>	<e06186140908222202n4bfb07a4md52a78100edeeb67@mail.gmail.com>
	<e7ba66e40908231553i7c61cd60ud02fbdabf58c7bca@mail.gmail.com>
Message-ID: <4A93A73E.50102@student.matnat.uio.no>

Lisandro Dalcin wrote:
> On Sun, Aug 23, 2009 at 2:02 AM, Charles R
> Harris<charlesr.harris at gmail.com> wrote:
>> but for
>> maintainence and development I think that functions make it easier to add
>> functionality without bloating the ndarray object.
> 
> Why do you say that adding methods bloat the ndarray object? Adding
> functions bloats the numpy namespace,
> adding methods bloat the numpy.ndarray namespace... In both cases you
> are just populating a dict... What's the difference for you?

I guess it is not directly related to "bloat", but:

  - Functions are closer to common mathematical notation
  - (As others have said,) functions will in the future work on any PEP 
3118 buffer, while methods need an ndarray object.

>> And figuring out what
>> all the desireable methods are up front is a difficult exercise in
>> prospective design.
>>
> 
> I think you are going to be faced to such difficulties when deciding
> desirable functions for the numpy namespace.

I disagree. It is quite straightforward that np.sin, np.mean and np.dot 
all are rightly included in the namespace -- however, why exactly is it that

arr.mean()

is OK, while arr.sin(), arr.cos() or arr.dot(otherarr) is not?

(It seems the rule is "array -> scalar" mappings are OK as method, 
others are not, but that seems arbitrary to me.)


-- 
Dag Sverre


From aisaac at american.edu  Tue Aug 25 08:49:16 2009
From: aisaac at american.edu (Alan G Isaac)
Date: Tue, 25 Aug 2009 08:49:16 -0400
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <4A93A73E.50102@student.matnat.uio.no>
References: <3d375d730908201537x2a3c3d5bncdf89b77cbf1ee01@mail.gmail.com>	<308926.80222.qm@web52110.mail.re2.yahoo.com>	<9457e7c80908201622i72df4465j33757dfd335d5268@mail.gmail.com>	<e06186140908222202n4bfb07a4md52a78100edeeb67@mail.gmail.com>	<e7ba66e40908231553i7c61cd60ud02fbdabf58c7bca@mail.gmail.com>
	<4A93A73E.50102@student.matnat.uio.no>
Message-ID: <4A93DDCC.7040801@american.edu>

On 8/25/2009 4:56 AM Dag Sverre Seljebotn apparently wrote:
> why exactly is it that
> 
> arr.mean()
> 
> is OK, while arr.sin(), arr.cos() or arr.dot(otherarr) is not?
> 
> (It seems the rule is "array -> scalar" mappings are OK as method, 
> others are not, but that seems arbitrary to me.)


I do not know the original justification,
but conceptually, the mean is a property
of the data in the array.

As a user, I'd actually like to see
`dot` implemented as a method so we
could do a1.dot(a2).  This would make
a lot of code cleaner to read.

Alan Isaac



From d_l_goldsmith at yahoo.com  Tue Aug 25 11:52:32 2009
From: d_l_goldsmith at yahoo.com (David Goldsmith)
Date: Tue, 25 Aug 2009 08:52:32 -0700 (PDT)
Subject: [SciPy-dev] draft: NumPy User Guide introduction
In-Reply-To: <4A93DDCC.7040801@american.edu>
Message-ID: <502991.91774.qm@web52106.mail.re2.yahoo.com>

--- On Tue, 8/25/09, Alan G Isaac <aisaac at american.edu> wrote:

> As a user, I'd actually like to see
> `dot` implemented as a method so we
> could do a1.dot(a2).? This would make
> a lot of code cleaner to read.
> 
> Alan Isaac

Submit a patch, see if it gets accepted:

    def dot(a2):
        if self.shape == a2.T.shape:
            return np.dot(self, a2)
        else:
            return self # (or whatever else is thought logical)

DG

> 
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
> 

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 


From dwf at cs.toronto.edu  Tue Aug 25 15:10:06 2009
From: dwf at cs.toronto.edu (David Warde-Farley)
Date: Tue, 25 Aug 2009 15:10:06 -0400
Subject: [SciPy-dev] botched PDF
Message-ID: <75BE8622-5AD7-44E8-BB9B-73997FA1AC28@cs.toronto.edu>

Someone brought it to my attention on IRC that the reference guide PDF  
is currently borked: http://docs.scipy.org/doc/scipy/scipy-ref.pdf

Can someone with requisite permissions rebuild it at your earliest  
convenience?

David


From josef.pktd at gmail.com  Tue Aug 25 15:19:58 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 25 Aug 2009 15:19:58 -0400
Subject: [SciPy-dev] botched PDF
In-Reply-To: <75BE8622-5AD7-44E8-BB9B-73997FA1AC28@cs.toronto.edu>
References: <75BE8622-5AD7-44E8-BB9B-73997FA1AC28@cs.toronto.edu>
Message-ID: <1cd32cbb0908251219l7c79b1fan15c081120eb710f6@mail.gmail.com>

On Tue, Aug 25, 2009 at 3:10 PM, David Warde-Farley<dwf at cs.toronto.edu> wrote:
> Someone brought it to my attention on IRC that the reference guide PDF
> is currently borked: http://docs.scipy.org/doc/scipy/scipy-ref.pdf
>
> Can someone with requisite permissions rebuild it at your earliest
> convenience?
>
> David

The pdf file at the link looks fine to me with Acrobat. What does
"borked" mean in this case?

Josef


From jamesbroadhead at gmail.com  Tue Aug 25 15:40:43 2009
From: jamesbroadhead at gmail.com (James Broadhead)
Date: Tue, 25 Aug 2009 20:40:43 +0100
Subject: [SciPy-dev] botched PDF
In-Reply-To: <1cd32cbb0908251219l7c79b1fan15c081120eb710f6@mail.gmail.com>
References: <75BE8622-5AD7-44E8-BB9B-73997FA1AC28@cs.toronto.edu>
	<1cd32cbb0908251219l7c79b1fan15c081120eb710f6@mail.gmail.com>
Message-ID: <3ae66f0f0908251240o7e492527s362212089cc5d7c5@mail.gmail.com>

2009/8/25  <josef.pktd at gmail.com>:
> On Tue, Aug 25, 2009 at 3:10 PM, David Warde-Farley<dwf at cs.toronto.edu> wrote:
>> Someone brought it to my attention on IRC that the reference guide PDF
>> is currently borked: http://docs.scipy.org/doc/scipy/scipy-ref.pdf
>>
>> Can someone with requisite permissions rebuild it at your earliest
>> convenience?
>>
>> David
>
> The pdf file at the link looks fine to me with Acrobat. What does
> "borked" mean in this case?
>
> Josef

It's been fixed by now.


JB


From josef.pktd at gmail.com  Tue Aug 25 16:39:23 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Tue, 25 Aug 2009 16:39:23 -0400
Subject: [SciPy-dev] help with plot directive in scipy.stats.tutorial
Message-ID: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>

Since the stats.tutorial has now been merged into svn, I would like to
clean up the plot directives. When I wrote this initially, I didn't
realize that the plot commands do not have access to previously
defined variables.

I would like to replace the current code with plot directives using
script files instead of the inline plot directive. Because, I'm
currently not able or setup to build the docs, I would like to know if
this is ok or whether someone can test this.

The attached files are my original scripts, I haven't really reviewed
it just checked that they work.

Thanks,

Josef

.. plot:: examples/normdiscr_plot1.py
   :align: center
   :include-source: 0

.. plot:: examples/normdiscr_plot2.py
   :align: center
   :include-source: 0
-------------- next part --------------
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

npoints = 20 # number of integer support points of the distribution minus 1
npointsh = npoints/2
npointsf = float(npoints)
nbound = 4 #bounds for the truncated normal
normbound = (1+1/npointsf) * nbound #actual bounds of truncated normal
grid = np.arange(-npointsh,npointsh+2,1) #integer grid
gridlimitsnorm = (grid-0.5) / npointsh * nbound #bin limits for the truncnorm
gridlimits = grid - 0.5
grid = grid[:-1]
probs = np.diff(stats.truncnorm.cdf(gridlimitsnorm, -normbound, normbound))
gridint = grid
normdiscrete = stats.rv_discrete(values = (gridint,np.round(probs,decimals=7)),
                        name='normdiscrete')
   
   
n_sample = 500
np.random.seed(87655678) #fix the seed for replicability
rvs = normdiscrete.rvs(size=n_sample)
rvsnd=rvs
f,l = np.histogram(rvs,bins=gridlimits)
sfreq = np.vstack([gridint,f,probs*n_sample]).T
fs = sfreq[:,1] / float(n_sample)
ft = sfreq[:,2] / float(n_sample)
nd_std = np.sqrt(normdiscrete.stats(moments =  'v'))

ind = gridint  # the x locations for the groups
width = 0.35       # the width of the bars

plt.subplot(111)
rects1 = plt.bar(ind, ft, width, color='b')
rects2 = plt.bar(ind+width, fs, width, color='r')
normline = plt.plot(ind+width/2.0, stats.norm.pdf(ind,scale=nd_std), color='b')

plt.ylabel('Frequency')
plt.title('Frequency and Probability of normdiscrete')
plt.xticks(ind+width, ind )
plt.legend( (rects1[0], rects2[0]), ('true', 'sample') )

plt.show()
-------------- next part --------------
import numpy as np
import matplotlib.pyplot as plt
from scipy import stats

npoints = 20 # number of integer support points of the distribution minus 1
npointsh = npoints/2
npointsf = float(npoints)
nbound = 4 #bounds for the truncated normal
normbound = (1+1/npointsf)*nbound #actual bounds of truncated normal
grid = np.arange(-npointsh,npointsh+2,1) #integer grid
gridlimitsnorm = (grid-0.5) / npointsh * nbound #bin limits for the truncnorm
gridlimits = grid - 0.5
grid = grid[:-1]
probs = np.diff(stats.truncnorm.cdf(gridlimitsnorm, -normbound, normbound))
gridint = grid
normdiscrete = stats.rv_discrete(values = (gridint,np.round(probs,decimals=7)),
                        name='normdiscrete')
   
n_sample = 500
np.random.seed(87655678) #fix the seed for replicability
rvs = normdiscrete.rvs(size=n_sample)
rvsnd=rvs
f,l = np.histogram(rvs,bins=gridlimits)
sfreq = np.vstack([gridint,f,probs*n_sample]).T
fs = sfreq[:,1]/float(n_sample)
ft = sfreq[:,2]/float(n_sample)
fs = sfreq[:,1].cumsum()/float(n_sample)
ft = sfreq[:,2].cumsum()/float(n_sample)
nd_std = np.sqrt(normdiscrete.stats(moments =  'v'))


ind = gridint  # the x locations for the groups
width = 0.35   # the width of the bars

plt.figure()
plt.subplot(111)
rects1 = plt.bar(ind, ft, width, color='b')
rects2 = plt.bar(ind+width, fs, width, color='r')
normline = plt.plot(ind+width/2.0, stats.norm.cdf(ind+0.5,scale=nd_std), color='b')

plt.ylabel('cdf')
plt.title('Cumulative Frequency and CDF of normdiscrete')
plt.xticks(ind+width, ind )
plt.legend( (rects1[0], rects2[0]), ('true', 'sample') )

plt.show()

From ralf.gommers at googlemail.com  Tue Aug 25 21:10:46 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Tue, 25 Aug 2009 21:10:46 -0400
Subject: [SciPy-dev] help with plot directive in scipy.stats.tutorial
In-Reply-To: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>
References: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>
Message-ID: <dde7764a0908251810m57be22aby384ecd018f44e06d@mail.gmail.com>

On Tue, Aug 25, 2009 at 4:39 PM, <josef.pktd at gmail.com> wrote:

> Since the stats.tutorial has now been merged into svn, I would like to
> clean up the plot directives. When I wrote this initially, I didn't
> realize that the plot commands do not have access to previously
> defined variables.


>
> I would like to replace the current code with plot directives using
> script files instead of the inline plot directive. Because, I'm
> currently not able or setup to build the docs, I would like to know if
> this is ok or whether someone can test this.


Hi Josef, I can't answer your question but if it works it seems like a clean
solution.

If you are cleaning up those plot directives would you mind using the
pep8.py script? I read the stats tutorial before and I like the content, but
it would be much more readable if the lines were <80 characters and the
whitespace issues were fixed. I tried it on one of your attached scripts and
it catches quite a few errors:

ralf at ralf-desktop:~/Desktop$ python pep8.py --repeat normdiscr_plot2.py
normdiscr_plot2.py:10:27: E231 missing whitespace after ','
normdiscr_plot2.py:16:51: E231 missing whitespace after ','
normdiscr_plot2.py:18:1: W291 trailing whitespace
normdiscr_plot2.py:23:2: E231 missing whitespace after ','
normdiscr_plot2.py:24:27: E231 missing whitespace after ','
normdiscr_plot2.py:25:13: E231 missing whitespace after ','
normdiscr_plot2.py:26:13: E231 missing whitespace after ','
normdiscr_plot2.py:27:13: E231 missing whitespace after ','
normdiscr_plot2.py:28:13: E231 missing whitespace after ','
normdiscr_plot2.py:29:45: E222 multiple spaces after operator
normdiscr_plot2.py:39:80: E501 line too long (83 characters)
normdiscr_plot2.py:39:58: E231 missing whitespace after ','
normdiscr_plot2.py:43:26: E202 whitespace before ')'
normdiscr_plot2.py:44:12: E201 whitespace after '('

There is more that the script does not catch, like whitespace issues around
'=' and operators, but it definitely helps.

Thanks,
Ralf


> The attached files are my original scripts, I haven't really reviewed
> it just checked that they work.


>
> Thanks,
>
> Josef
>
> .. plot:: examples/normdiscr_plot1.py
>   :align: center
>   :include-source: 0
>
> .. plot:: examples/normdiscr_plot2.py
>   :align: center
>   :include-source: 0
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090825/183c7f39/attachment.html>

From cournape at gmail.com  Wed Aug 26 09:38:57 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 26 Aug 2009 08:38:57 -0500
Subject: [SciPy-dev] Please help!
In-Reply-To: <db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
Message-ID: <5b8d13220908260638p2eb0d6c2oe1d6e730e875a819@mail.gmail.com>

On Sun, Aug 23, 2009 at 12:32 AM, Fernando Perez<fperez.net at gmail.com> wrote:

> The bad news is nothing much seems to be happening upstream. ?The good
> news is that ?Matthew ?Brett has ?been pushing for ways to help with
> this, and both David Cournapeau and Ondrej stepped up to fight it,
> and ?I think they've finally made progress (David told me on Friday
> afternoon at the conference that they'd been able to crack some of the
> problems related to the build). ?I'm sure they'll post soon with more
> details.

I have uploaded an experimental version of atlas 3.8.3 for i386 and amd64:

https://edge.launchpad.net/~david-ar/+archive/ppa

There are still some rough edges, but it solves the aforementioned
problem. Thanks to Ondrej and Pauli for the packaging help as well,

cheers,

David


From scott.sinclair.za at gmail.com  Wed Aug 26 10:27:04 2009
From: scott.sinclair.za at gmail.com (Scott Sinclair)
Date: Wed, 26 Aug 2009 16:27:04 +0200
Subject: [SciPy-dev] Please help!
In-Reply-To: <5b8d13220908260638p2eb0d6c2oe1d6e730e875a819@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
	<5b8d13220908260638p2eb0d6c2oe1d6e730e875a819@mail.gmail.com>
Message-ID: <6a17e9ee0908260727s17095626yf472a251e5a14606@mail.gmail.com>

> 2009/8/26 David Cournapeau <cournape at gmail.com>:
> On Sun, Aug 23, 2009 at 12:32 AM, Fernando Perez<fperez.net at gmail.com> wrote:
>
>> The bad news is nothing much seems to be happening upstream. ?The good
>> news is that ?Matthew ?Brett has ?been pushing for ways to help with
>> this, and both David Cournapeau and Ondrej stepped up to fight it,
>> and ?I think they've finally made progress (David told me on Friday
>> afternoon at the conference that they'd been able to crack some of the
>> problems related to the build). ?I'm sure they'll post soon with more
>> details.
>
> I have uploaded an experimental version of atlas 3.8.3 for i386 and amd64:
>
> https://edge.launchpad.net/~david-ar/+archive/ppa
>
> There are still some rough edges, but it solves the aforementioned
> problem. Thanks to Ondrej and Pauli for the packaging help as well,

The fixed ATLAS builds work perfectly on my 32bit Jaunty system.

After rebuilding NumPy...

scott at paragon:~$ python -c "import numpy; numpy.test()"
Running unit tests for numpy
NumPy version 1.4.0.dev
NumPy is installed in /usr/local/lib/python2.6/dist-packages/numpy
Python version 2.6.2 (release26-maint, Apr 19 2009, 01:56:41) [GCC 4.3.3]
nose version 0.11.1
.....
----------------------------------------------------------------------
Ran 2186 tests in 14.985s

OK (KNOWNFAIL=1, SKIP=11)

There used to be 16 failures related to complex math.

Thanks!
Scott


From cournape at gmail.com  Wed Aug 26 11:22:14 2009
From: cournape at gmail.com (David Cournapeau)
Date: Wed, 26 Aug 2009 10:22:14 -0500
Subject: [SciPy-dev] Please help!
In-Reply-To: <6a17e9ee0908260727s17095626yf472a251e5a14606@mail.gmail.com>
References: <dbcf15060908200252p7a4118cala459d7dfbc8e0051@mail.gmail.com>
	<9457e7c80908201222r5466894bpbd046106bb6b8c40@mail.gmail.com>
	<7ec4ad760908210950i491694f9q9dbbdd67ae141abe@mail.gmail.com>
	<db6b5ecc0908222232wbf2cf7se0cfc8a2d305aa91@mail.gmail.com>
	<5b8d13220908260638p2eb0d6c2oe1d6e730e875a819@mail.gmail.com>
	<6a17e9ee0908260727s17095626yf472a251e5a14606@mail.gmail.com>
Message-ID: <5b8d13220908260822x5f1411d2w514aee60ab83a563@mail.gmail.com>

On Wed, Aug 26, 2009 at 9:27 AM, Scott
Sinclair<scott.sinclair.za at gmail.com> wrote:
>> 2009/8/26 David Cournapeau <cournape at gmail.com>:
>> On Sun, Aug 23, 2009 at 12:32 AM, Fernando Perez<fperez.net at gmail.com> wrote:
>>
>>> The bad news is nothing much seems to be happening upstream. ?The good
>>> news is that ?Matthew ?Brett has ?been pushing for ways to help with
>>> this, and both David Cournapeau and Ondrej stepped up to fight it,
>>> and ?I think they've finally made progress (David told me on Friday
>>> afternoon at the conference that they'd been able to crack some of the
>>> problems related to the build). ?I'm sure they'll post soon with more
>>> details.
>>
>> I have uploaded an experimental version of atlas 3.8.3 for i386 and amd64:
>>
>> https://edge.launchpad.net/~david-ar/+archive/ppa
>>
>> There are still some rough edges, but it solves the aforementioned
>> problem. Thanks to Ondrej and Pauli for the packaging help as well,
>
> The fixed ATLAS builds work perfectly on my 32bit Jaunty system.
>
> After rebuilding NumPy...

In theory, you don't even need rebuild :)

cheers,

David


From josef.pktd at gmail.com  Wed Aug 26 15:21:44 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Wed, 26 Aug 2009 15:21:44 -0400
Subject: [SciPy-dev] help with plot directive in scipy.stats.tutorial
In-Reply-To: <dde7764a0908251810m57be22aby384ecd018f44e06d@mail.gmail.com>
References: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>
	<dde7764a0908251810m57be22aby384ecd018f44e06d@mail.gmail.com>
Message-ID: <1cd32cbb0908261221h757ee57ejb1dc42ef5b33427b@mail.gmail.com>

On Tue, Aug 25, 2009 at 9:10 PM, Ralf
Gommers<ralf.gommers at googlemail.com> wrote:
>
>
> On Tue, Aug 25, 2009 at 4:39 PM, <josef.pktd at gmail.com> wrote:
>>
>> Since the stats.tutorial has now been merged into svn, I would like to
>> clean up the plot directives. When I wrote this initially, I didn't
>> realize that the plot commands do not have access to previously
>> defined variables.
>>
>> I would like to replace the current code with plot directives using
>> script files instead of the inline plot directive. Because, I'm
>> currently not able or setup to build the docs, I would like to know if
>> this is ok or whether someone can test this.
>
> Hi Josef, I can't answer your question but if it works it seems like a clean
> solution.
>
> If you are cleaning up those plot directives would you mind using the
> pep8.py script? I read the stats tutorial before and I like the content, but
> it would be much more readable if the lines were <80 characters and the
> whitespace issues were fixed. I tried it on one of your attached scripts and
> it catches quite a few errors:
>
> ralf at ralf-desktop:~/Desktop$ python pep8.py --repeat normdiscr_plot2.py
> normdiscr_plot2.py:10:27: E231 missing whitespace after ','
> normdiscr_plot2.py:16:51: E231 missing whitespace after ','
> normdiscr_plot2.py:18:1: W291 trailing whitespace
> normdiscr_plot2.py:23:2: E231 missing whitespace after ','
> normdiscr_plot2.py:24:27: E231 missing whitespace after ','
> normdiscr_plot2.py:25:13: E231 missing whitespace after ','
> normdiscr_plot2.py:26:13: E231 missing whitespace after ','
> normdiscr_plot2.py:27:13: E231 missing whitespace after ','
> normdiscr_plot2.py:28:13: E231 missing whitespace after ','
> normdiscr_plot2.py:29:45: E222 multiple spaces after operator
> normdiscr_plot2.py:39:80: E501 line too long (83 characters)
> normdiscr_plot2.py:39:58: E231 missing whitespace after ','
> normdiscr_plot2.py:43:26: E202 whitespace before ')'
> normdiscr_plot2.py:44:12: E201 whitespace after '('
>
> There is more that the script does not catch, like whitespace issues around
> '=' and operators, but it definitely helps.
>
> Thanks,
> Ralf
>
>>
>> The attached files are my original scripts, I haven't really reviewed
>> it just checked that they work.
>>
>> Thanks,
>>
>> Josef
>>
>> .. plot:: examples/normdiscr_plot1.py
>> ? :align: center
>> ? :include-source: 0
>>
>> .. plot:: examples/normdiscr_plot2.py
>> ? :align: center
>> ? :include-source: 0
>>

I fixed some of the whitespace problems and checked it into svn in
revision 5910, and the stats.tutorial is open for editing, corrections
and review.

I finally tried to build the scipy docs again, and it works on Windows
without any problems for html and htmlhelp.

Two steps: I upgraded to the sphinx trunk (of last week), and copied a
generic make.bat (from statsmodels) into the doc directory
and "make html",  "make htmlhelp" and I have new docs.

Thanks to Pauli and whoever else helped getting the numpy/scipy build
process fully integrated with or supported by sphinx.

so revision 5910 is tested for correct build.

However, the only problem that I found so far, is that the two scripts
 examples/normdiscr_plot1.py and  examples/normdiscr_plot2.py  are not
copied to the build version. I don't know what the mechanism is to
copy script files or attachments into the build directories.

Josef

BTW: sphinx now also builds qthelp files, if anyone is interested.


From ralf.gommers at googlemail.com  Wed Aug 26 18:12:00 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Wed, 26 Aug 2009 18:12:00 -0400
Subject: [SciPy-dev] help with plot directive in scipy.stats.tutorial
In-Reply-To: <1cd32cbb0908261221h757ee57ejb1dc42ef5b33427b@mail.gmail.com>
References: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>
	<dde7764a0908251810m57be22aby384ecd018f44e06d@mail.gmail.com>
	<1cd32cbb0908261221h757ee57ejb1dc42ef5b33427b@mail.gmail.com>
Message-ID: <dde7764a0908261512r4fff4e6dg2ddb7bfb5cd2baa5@mail.gmail.com>

I fixed some of the whitespace problems and checked it into svn in
> revision 5910, and the stats.tutorial is open for editing, corrections
> and review.
>

Thanks Josef. I'll make some time to work on it this weekend.

Cheers,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090826/e149f7d2/attachment.html>

From jsalvati at u.washington.edu  Thu Aug 27 04:07:36 2009
From: jsalvati at u.washington.edu (John Salvatier)
Date: Thu, 27 Aug 2009 01:07:36 -0700
Subject: [SciPy-dev] Callback arguments in Sphinx Documentation
Message-ID: <113e17f20908270107ucc9785byf48eb53927f7bc31@mail.gmail.com>

Hello all,

I am writing the Documentation for scikits.bvp_solver using Sphinx.

bvp_solver uses a lot of relatively complex callback arguments, so I am
looking for a good way to document their format. Nesting ".. method::"
directives does not seem to work. Does anyone have any ideas about how to
document the signatures of callback arguments elegantly; has anyone dealt
with this before?

Best Regards,
John Salvatier
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090827/50c94f4f/attachment.html>

From pav+sp at iki.fi  Thu Aug 27 18:34:55 2009
From: pav+sp at iki.fi (Pauli Virtanen)
Date: Thu, 27 Aug 2009 22:34:55 +0000 (UTC)
Subject: [SciPy-dev] help with plot directive in scipy.stats.tutorial
References: <1cd32cbb0908251339o5b618af4o635c46f3bba974dd@mail.gmail.com>
	<dde7764a0908251810m57be22aby384ecd018f44e06d@mail.gmail.com>
	<1cd32cbb0908261221h757ee57ejb1dc42ef5b33427b@mail.gmail.com>
Message-ID: <h771mf$ovm$1@ger.gmane.org>

On 2009-08-26, josef.pktd at gmail.com <josef.pktd at gmail.com> wrote:
[clip]
> I fixed some of the whitespace problems and checked it into svn in
> revision 5910, and the stats.tutorial is open for editing, corrections
> and review.
>
> I finally tried to build the scipy docs again, and it works on Windows
> without any problems for html and htmlhelp.
>
> Two steps: I upgraded to the sphinx trunk (of last week), and copied a
> generic make.bat (from statsmodels) into the doc directory
> and "make html",  "make htmlhelp" and I have new docs.

Great! I suppose the main thing here was getting autosummary 
better integrated into Sphinx.

[clip]
> BTW: sphinx now also builds qthelp files, if anyone is interested.

It also builds Gnome Devhelp files. The problem with both of 
these is that they take some expertise to open -- it doesn't just 
work by

	qthelp filename.qth
	devhelp filename.zip

but you have to practise some more involved magic to open them. 
So I suppose they're not very useful to have at the moment.

-- 
Pauli Virtanen



From ralf.gommers at googlemail.com  Thu Aug 27 23:42:03 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Thu, 27 Aug 2009 23:42:03 -0400
Subject: [SciPy-dev] Callback arguments in Sphinx Documentation
In-Reply-To: <113e17f20908270107ucc9785byf48eb53927f7bc31@mail.gmail.com>
References: <113e17f20908270107ucc9785byf48eb53927f7bc31@mail.gmail.com>
Message-ID: <dde7764a0908272042q640a8d5cxfd19309b7f545923@mail.gmail.com>

On Thu, Aug 27, 2009 at 4:07 AM, John Salvatier
<jsalvati at u.washington.edu>wrote:

> Hello all,
>
> I am writing the Documentation for scikits.bvp_solver using Sphinx.
>
> bvp_solver uses a lot of relatively complex callback arguments, so I am
> looking for a good way to document their format. Nesting ".. method::"
> directives does not seem to work. Does anyone have any ideas about how to
> document the signatures of callback arguments elegantly; has anyone dealt
> with this before?


For numpy/scipy docs the signature is either contained in the type
description (like
http://docs.scipy.org/numpy/docs/numpy.lib.function_base.piecewise ) or just
given in words (like
http://docs.scipy.org/scipy/docs/scipy.integrate.quadpack.tplquad ).

It works because the signatures are usually simple, but there's certainly
room for improvement. Keep in mind though that your solution will have to
work for text terminals as well as with Sphinx, which is not so easy.
..method:: directives would look bad in a text terminal even if they did
work.

Cheers,
Ralf


>
>
> Best Regards,
> John Salvatier
>
> _______________________________________________
> Scipy-dev mailing list
> Scipy-dev at scipy.org
> http://mail.scipy.org/mailman/listinfo/scipy-dev
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090827/0213f75e/attachment.html>

From ilanschnell at gmail.com  Sat Aug 29 02:07:25 2009
From: ilanschnell at gmail.com (Ilan Schnell)
Date: Sat, 29 Aug 2009 01:07:25 -0500
Subject: [SciPy-dev] EPD 5.0 released
Message-ID: <2fbe16300908282307t618ccfa1y9dc049c522f362ea@mail.gmail.com>

Hello,

I am pleased to announce that EPD (Enthought Python Distribution) version
5.0.0 has been released.  You may find more information about EPD, as
well as download a 30 day free trial, here:

        http://www.enthought.com/products/epd.php


This release contains updates to a large number of packages.
You can find the release notes here:

        https://svn.enthought.com/epd/wiki/Py25/5.0.0/RelNotes


About EPD
---------
The Enthought Python Distribution (EPD) is a "kitchen-sink-included"
distribution of the Python Programming Language, including over 80
additional tools and libraries. The EPD bundle includes NumPy, SciPy,
IPython, 2D and 3D visualization, database adapters, and a lot of
other tools right out of the box.

        http://www.enthought.com/products/epdlibraries.php

It is currently available as a single-click installer for Windows XP (x86),
Mac OS X (a universal binary for OS X 10.4 and above),
RedHat 3, 4 and 5, as well as Solaris 10 (x86 and x86_64/amd64).

EPD is free for academic use.  An annual subscription including installation
support is available for individual and commercial use.  Additional
support options, including customization, bug fixes and training classes
are also available:

        http://www.enthought.com/products/support_level_table.php


- Ilan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090829/5f724584/attachment.html>

From ralf.gommers at googlemail.com  Sun Aug 30 18:05:45 2009
From: ralf.gommers at googlemail.com (Ralf Gommers)
Date: Sun, 30 Aug 2009 18:05:45 -0400
Subject: [SciPy-dev] SciPy Milestones doc editor link broken
Message-ID: <dde7764a0908301505v6e759841sd5558ef587234187@mail.gmail.com>

Hi,

The link to the SciPy Milestones (at the top of every page) in the doc wiki
is pointing to the NumPy version again. Can one of the wiki admins please
change the link back to http://docs.scipy.org/scipy/Milestones/ ?

Thanks,
Ralf
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-dev/attachments/20090830/89acb9a0/attachment.html>

From josef.pktd at gmail.com  Mon Aug 31 23:03:28 2009
From: josef.pktd at gmail.com (josef.pktd at gmail.com)
Date: Mon, 31 Aug 2009 23:03:28 -0400
Subject: [SciPy-dev] source repository in http://scikits.appspot.com/
Message-ID: <1cd32cbb0908312003g5b89b99cn8394dd9c3970f2e0@mail.gmail.com>

Is it possible to have a source repository that is not located in svn
on scipy.org?

I haven't seen an example for non svn repositories, e.g. Davids
audiolab and talkbox still link to the svn, other non-svn have the
source repository link missing.

For example, I would like to have "bzr branch lp:statsmodels"
instead of "svn checkout"

Thanks,

Josef