[Numpy-discussion] Proposed Roadmap Overview

[Travis Oliphant]

Mark Wiebe and I have been discussing off and on (as well as talking with Charles) a good way forward to balance two competing desires: 

	* addition of new features that are needed in NumPy
	* improving the code-base generally and moving towards a more maintainable NumPy

I know there are load voices for just focusing on the second of these and avoiding the first until we have finished that.  I recognize the need to improve the code base, but I will also be pushing for improvements to the feature-set and user experience in the process. 

As a result, I am proposing a rough outline for releases over the next year: 

	* NumPy 1.7 to come out as soon as the serious bugs can be eliminated.  Bryan, Francesc, Mark, and I are able to help triage some of those.

	* NumPy 1.8 to come out in July which will have as many ABI-compatible feature enhancements as we can add while improving test coverage and code cleanup.   I will post to this list more details of what we plan to address with it later.    Included for possible inclusion are: 
	* resolving the NA/missing-data issues
	* finishing group-by
	* incorporating the start of label arrays
	* incorporating a meta-object 
	* a few new dtypes (variable-length string, varialbe-length unicode and an enum type)
	* adding ufunc support for flexible dtypes and possibly structured arrays
	* allowing generalized ufuncs to work on more kinds of arrays besides just contiguous
	* improving the ability for NumPy to receive JIT-generated function pointers for ufuncs and other calculation opportunities
	* adding "filters" to Input and Output
	* simple computed fields for dtypes
	* accepting a Data-Type specification as a class or JSON file
	* work towards improving the dtype-addition mechanism
	* re-factoring of code so that it can compile with a C++ compiler and be minimally dependent on Python data-structures.  

	* NumPy 2.0 to come out in January of 2013.   Mark Wiebe and I will post to this list a document that explains some of it's proposed features and enhancements.    I won't steal his thunder for some of the things he is working on. 

If there are code issues people would like to see addressed, it would be a great time to speak up and/or propose something that you would like to see. 

In general NumPy 1.8 will have new features that need to be explored in order that NumPy 2.0 has enough code "experience" in order to be as useful as possible.   I recognize that NumPy 1.8 has quite a few proposed features.   These have been building up and are the big reason I've committed so many resources to NumPy.   The feature-list did not just come out of my head.   They are the result of talking and interacting with many NumPy users and watching the code get used (and not used) in the real world.    This will be a faster pace of development.   But, all of this will be in the open.    If the NumPy 2.0 schedule is too aggressive, then we will have a NumPy 1.9 release in order to allow features to come out. 

Thanks,

-Travis