[Numpy-discussion] Maximum available dimensions in numpy

[Joe Harrington]

For reasons that were laid out when this topic came up a month or so
ago (perhaps on another of our infinitude of email lists), I'm opposed
to reducing the number of dimensions to even 10.  A sizeable chunk of
the community does use many dimensions.  I routinely use more than 5
dimensions, and I'm pretty good at reducing the number for performance
where it's practical.

There's an atmospheric sciences tool called vis5d (x,y,z,t,field
parameter vector), which shows just how commonly meteorologists use 5
dimensions.  I don't think of the atmospheric sciences as being a
particularly dimension-heavy group, but you can easily think of adding
a few more dimensions that let you choose model runs with varied
initial parameters.  Try tracking the variables of a complex medical
trial in just 5 dimensions.  IDL's limit is 7 dimensions, and many
don't use it as a result.  Having more in our default binary installs
is a selling point.

This is a pretty limiting change to be suggesting, so if there's a
change to be made, it should come only after implementing and
benchmarking to see what the actual performance benefits are, and then
polling the community.  If there is a really big improvement in having
4 or 8 dimensions max, and if "malloc and unlimit everything" isn't
fast, then it may be worth supporting low-dim and high-dim versions of
the binary installs.  But, let's wait until we have more volunteers
before doing that.

--jh--