[Numpy-discussion] Proposal to accept NEP 49: Data allocation strategies

[Matti Picus]

On 6/5/21 2:07 pm, Eric Wieser wrote:
> The NEP looks good, but I worry the API isn't flexible enough. My two 
> main concerns are:
>
> ### Stateful allocators
>
> Consider an allocator that aligns to `N` bytes, where `N` is 
> configurable from a python call in someone else's extension module.
> ...
>
> ### Thread and async-local allocators
>
> For tracing purposes, I expect it to be valuable to be able to 
> configure the allocator within a single thread / coroutine.
> If we want to support this, we'd most likely want to work with the 
> PEP567 ContextVar API rather than a half-baked thread_local solution 
> that doesn't work for async code.
>
> This problem isn't as pressing as the statefulness problem.
> Fixing it would amount to extending the `PyDataMem_SetHandler` API, 
> and would be unlikely to break any code written against the current 
> version of the NEP; meaning it would be fine to leave as a follow-up.
> It might still be worth remarking upon as future work of some kind in 
> the NEP.
>
>
I would prefer to leave both of these to a future extension for the NEP. 
Setting the alignment from a python-level call seems to be asking for 
trouble, and I would need to be convinced that the extra layer of 
flexibility is worth it.


It might be worth mentioning that this NEP may be extended in the 
future, but truthfully I think that is the case for all NEPs.


Matti