[Numpy-discussion] Accepting NEP 41: First step towards a new Datatype System

[Sebastian Berg]

Hi all,

I propose to officially accepting NEP 41:

"First step towards a new Datatype System"

If you have any concerns please let me know or discuss here within a
week. If there are no concerns voiced the NEP may be accepted.
I realize that there may be some who need time to think about this
individually and will of course wait, but at this time I hope that no
larger discussion on the mailing list will be necessary.

Again, the main immediate effect/design choice is that there will be
classes for each NumPy dtype:

    float64 = np.dtype("float64")  # Native byteorder float64
    Float64DType = type(float64)  # np.dtype[float64]
    issubclass(Float64DType, np.dtype)  # True
    isinstance(float64, np.dtype)  # True (as before)

And in the above `float64.newbyteorder()` will also be an instance of
the same `Float64DType` class.
As such the class `Float64DType` in the above represents what is
currently represented by the type number: `float64.num`

This does admittedly mean that `Float64DType` effectively is a class
with only a singleton instance in most cases, since non-native byte
order or metadata are rarely used. Multiple instances are mainly
necessary for datatypes such as current strings (with varying length)
or datetimes (with a unit).
There are probably alternatives and the boundaries between instances
and can be drawn at different places (even within this framework), but
I believe that it is the practical and intuitive approach to draw them
at the current type numbers.

Best,

Sebastian


On Wed, 2020-03-11 at 17:02 -0700, Sebastian Berg wrote:
> Hi all,
> 
> I am pleased to propose NEP 41: First step towards a new Datatype
> System https://numpy.org/neps/nep-0041-improved-dtype-support.html
> 
> This NEP motivates the larger restructure of the datatype machinery
> in
> NumPy and defines a few fundamental design aspects. The long term
> user
> impact will be allowing easier and more rich featured user defined
> datatypes.
> 
> As this is a large restructure, the NEP represents only the first
> steps
> with some additional information in further NEPs being drafted [1]
> (this may be helpful to look at depending on the level of detail you
> are interested in).
> The NEP itself does not propose to add significant new public API.
> Instead it proposes to move forward with an incremental internal
> refactor and lays the foundation for this process.
> 
> The main user facing change at this time is that datatypes will
> become
> classes (e.g. ``type(np.dtype("float64"))`` will be a float64
> specific
> class.
> For most users, the main impact should be many new datatypes in the
> long run (see the user impact section). However, for those interested
> in API design within NumPy or with respect to implementing new
> datatypes, this and the following NEPs are important decisions in the
> future roadmap for NumPy.
> 
> The current full text is reproduced below, although the above link is
> probably a better way to read it.
> 
> Cheers
> 
> Sebastian
> 
> 
> [1] NEP 40 gives some background information about the current
> systems
> and issues with it:
> https://github.com/numpy/numpy/blob/1248cf7a8765b7b53d883f9e7061173817533aac/doc/neps/nep-0040-legacy-datatype-impl.rst
> and NEP 42 being a first draft of how the new API may look like:
> 
> https://github.com/numpy/numpy/blob/f07e25cdff3967a19c4cc45c6e1a94a38f53cee3/doc/neps/nep-0042-new-dtypes.rst
> (links to current rendered versions, check 
> https://github.com/numpy/numpy/pull/15505 and 
> https://github.com/numpy/numpy/pull/15507 for updates)
> 
<snip>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 833 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/numpy-discussion/attachments/20200408/3c37aa6f/attachment.sig>