[SciPy-Dev] Move some validations as asserts?

[Pamphile Roy]

Hi everyone,

FYI, I opened an issue to keep track of this discussion: https://github.com/scipy/scipy/issues/13761

Cheers,
Pamphile

> On 06.03.2021, at 21:28, Ralf Gommers <ralf.gommers at gmail.com> wrote:
> 
> 
> 
> On Wed, Mar 3, 2021 at 4:43 PM Pamphile Roy <roy.pamphile at gmail.com <mailto:roy.pamphile at gmail.com>> wrote:
> 
>> >
>> > A simple approach would be to systematically separate the logic in two: core function on one side, user interface on the other. This way the user could by-pass all the time the validation.
>> > Mock the validations. After some quick tests, it looks like the overhead is too big. I could be wrong.
>> 
>> I'm not sure what this means exactly.
> 
> I just did some quick tests like the following. (I just wanted to see how the API could look like.)
> 
> import time
> from unittest.mock import patch
> import numpy as np
> from scipy.spatial import distance
> 
> coords = np.random.random((100, 2))
> weights = [1, 2]
> 
> itime = time.time()
> for _ in range(50):
>     distance.cdist(coords, coords, metric='sqeuclidean', w=weights)
> 
> print(f"Time: {time.time() - itime}")
> 
> with patch('scipy.spatial.distance._validate_vector') as mock_requests:
>     mock_requests.side_effect = lambda x, *args, **kwargs: np.asarray(x)
> 
>     itime = time.time()
>     for _ in range(50):
>         distance.cdist(coords, coords, 'sqeuclidean', w=weights)
> 
>     print(f"Time: {time.time() - itime}")
> 
>> > Using assert for all validation.
>> 
>> One obvious issue is that `python -OO` will remove all asserts, and that's not what we want (we've had lots of issues with popular web deployment tools running with -OO by default). Hence the rule has always been "no plain asserts".
>> 
>> Also, error messages for plain asserts are bad usually.
> 
> Oh ok then it’s off the table.
> 
>> 
>> The first option sounds best to me.
>> 
>> Probably - if we do something, that's likely the best direction. The question though is how to provide a sensible UX for it, that's nontrivial. Splitting the whole API is a lot of work and new API surface. There may be alternatives, such as new keywords or fancier approaches (e.g., cython has a uniform way of turning off bounds checking and negative indexing with `cython.boundscheck`, `cython.wraparound`).
> 
> What I had in mind was more in line with what Evgini explains. We would not change the public API, just within the functions, separate the validation from the actual work.
> 
> That seems fine to me, and easy to do in most cases. We typically need that pattern anyway when we move code to Cython/Pythran.
> 
> 
> The decorator thing in Cython would probably be what I had in mind with the mocking. If we would manage to do something like this fast, this could be a great solution IMO.
> 
> This one would not be fast I'm afraid.
> 
> Cheers,
> Ralf
> 
> 
> 
> Cheers,
> Pamphile
> 
> 
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org <mailto:SciPy-Dev at python.org>
> https://mail.python.org/mailman/listinfo/scipy-dev <https://mail.python.org/mailman/listinfo/scipy-dev>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org <mailto:SciPy-Dev at python.org>
> https://mail.python.org/mailman/listinfo/scipy-dev <https://mail.python.org/mailman/listinfo/scipy-dev>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210331/6152d31b/attachment.html>