[SciPy-Dev] Hyper-dual numbers

Sat Feb 13 17:52:07 EST 2021

I will report it as such to the author

On Sat, Feb 13, 2021 at 4:37 PM Sebastian Berg <sebastian at sipsolutions.net>
wrote:

> On Sat, 2021-02-13 at 16:24 -0600, Andrea Cortis wrote:
> > I did try to port PyHD to python3 with 2to3
> > There are 38 warnings upon compiling (macos Catalina), but then when
> > I try
> > to run I get
> >
> > ----> 1 from hyperdual import numpy_hyperdual
> >
> > ValueError: Failed to register dtype for <class
> > 'hyperdual.hyperdual'>:
> > Legacy user dtypes using `NPY_ITEM_IS_POINTER` or `NPY_ITEM_REFCOUNT`
> > areunsupported.  It is possible to create such a dtype only if it is
> > a
> > structured dtype with names and fields hardcoded at registration
> > time.
> > Please contact the NumPy developers if this used to work but now
> > fails.
> >
>
> Sounds like a simple bug, the code fails to properly initialize the
> descriptor flags, which likely just means zero'ing them out.
> (NumPy was really designed with being passed a static version of the
> struct, although nothing wrong with the design here – it just will
> never be cleaned up).
>
> - Sebastian
>
>
> >
> > On Sat, Feb 13, 2021 at 2:09 PM Sebastian Berg <
> > sebastian at sipsolutions.net>
> > wrote:
> >
> > > On Sat, 2021-02-13 at 22:47 +0300, Evgeni Burovski wrote:
> > > > Sure, these are very different. The use of the quaternions
> > > > package
> > > > here is only that it implements the machinery. All I did was to
> > > > change
> > > > the multiplication table from the quaternion one to the dual
> > > > number
> > >
> > > It looks like someone has already done it:
> > > https://github.com/anandpratap/PyHD
> > >
> > > As to NumPy, I doubt it should be part of NumPy it seems far from
> > > "basic".  Should this be in SciPy?  My opinion is: maybe, but quite
> > > certainly not right now.
> > >
> > > The long story is, that I am doing a pretty big revamp of how NumPy
> > > does DTypes [1]. That will remove a lot of quirks and limitations.
> > > But
> > > those quirks should not be forbidding for a hyper-dual number dtype
> > > (and probably are not, I suspect that project above just works
> > > fairly
> > > well!).
> > >
> > > Should SciPy be in the business of providing new dtypes? Honestly,
> > > I
> > > hope that the answer may be "yes" at least after NumPy has this new
> > > API
> > > available and it has been proven/smoothened out.  If a dtype is
> > > useful
> > > enough in general SciPy terms of course.
> > > At this time, I think it is probably best to do in stand-alone
> > > packages
> > > for a while longer and hope that is not actually a limitation at
> > > all.
> > >
> > > Cheers,
> > >
> > > Sebastian
> > >
> > >
> > >
> > > [1] https://numpy.org/neps/nep-0041-improved-dtype-support.html
> > >
> > >
> > >
> > > > one. And it sort of works --- the branch above lets you define
> > > > arrays
> > > > with `dype=dual_number` and do basic arithmetics. Making it
> > > > usable is
> > > > reasonably straightforward, but I did not manage to find time
> > > > since
> > > > last November.
> > > >
> > > > Help's welcome --- even checking out the branch and trying corner
> > > > cases in arithmetics (there certainly are bugs) would be helpful.
> > > >
> > > > On Sat, Feb 13, 2021 at 10:36 PM Andrea Cortis <
> > > > andrea.cortis at gmail.com> wrote:
> > > > >
> > > > > I am no mathematician and cannot comment on the equivalence of
> > > > > quaternions vs hyper-dual numbers, even though they seem quite
> > > > > different to me at a first glance.
> > > > >
> > > > > For sure, I know that the algebra of dual-numbers is different
> > > > > from
> > > > > the algebra of complex numbers.
> > > > >
> > > > > It seems to me therefore that having a numpy `dtype=dual` would
> > > > > be
> > > > > extremely advantageous when looking to construct *exact* values
> > > > > of
> > > > > first (and n-th) derivatives.
> > > > > Say that one would want to replace automatic differentiation
> > > > > for
> > > > > backpropagation neural networks like in this blogpost
> > > > >
> > > > >
> > >
> https://blog.demofox.org/2017/03/13/neural-network-gradients-backpropagation-dual-numbers-finite-differences/
> > > > >
> > > > > then we would have a plethora of different implementations, say
> > > > > for
> > > > > pytorch, tensorflow, etc. and no unifying framework.
> > > > >
> > > > > Best,
> > > > >
> > > > > Andre
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > On Sat, Feb 13, 2021 at 1:21 PM Evgeni Burovski <
> > > > > evgeny.burovskiy at gmail.com> wrote:
> > > > > >
> > > > > > Ah, these. Suspected it, but wanted to make sure.
> > > > > >
> > > > > > IMO, these are best implemented as a numpy dtype. I'm biased
> > > > > > though
> > > > > > --- here's a branch which makes a start, based on Mike
> > > > > > Boyle's
> > > > > > version
> > > > > > of quaternion dtype (Mike and other authors, if you're
> > > > > > reading
> > > > > > this
> > > > > > --- thanks a ton!)
> > > > > >
> > > > > > https://github.com/ev-br/quaternion/tree/dual
> > > > > >
> > > > > > Now, I don't think scipy should carry around additional numpy
> > > > > > dtypes.
> > > > > > Cannot speak for the numpy project, but I strongly suspect
> > > > > > this
> > > > > > is
> > > > > > best implemented as a separate repository/project.
> > > > > >
> > > > > > Cheers,
> > > > > >
> > > > > > Evgeni
> > > > > >
> > > > > > On Sat, Feb 13, 2021 at 9:46 PM Joseph Fox-Rabinovitz
> > > > > > <jfoxrabinovitz at gmail.com> wrote:
> > > > > > >
> > > > > > > I directed Andrea here from Stack Overflow (
> > > > > > > https://stackoverflow.com/q/66179855/2988730). Based on the
> > > > > > > Wikipedia article
> > > > > > > (https://en.m.wikipedia.org/wiki/Dual_number), it seems
> > > > > > > like
> > > > > > > scipy is a much more likely place to look than numpy.
> > > > > > >
> > > > > > > - Joe
> > > > > > >
> > > > > > >
> > > > > > > On Sat, Feb 13, 2021, 13:19 Evgeni Burovski <
> > > > > > > evgeny.burovskiy at gmail.com> wrote:
> > > > > > > >
> > > > > > > > Hi Andrea, welcome!
> > > > > > > >
> > > > > > > > Since you're asking about numpy, you likely want the
> > > > > > > > numpy-
> > > > > > > > discussion
> > > > > > > > mailing list. (the overlap is non-zero, but
> > > > > > > > nevertheless).
> > > > > > > >
> > > > > > > > Out of curiosity, what's a hyper-dual type?
> > > > > > > >
> > > > > > > > Cheers,
> > > > > > > >
> > > > > > > > Evgeni
> > > > > > > >
> > > > > > > > On Sat, Feb 13, 2021 at 7:45 PM Andrea Cortis <
> > > > > > > > andrea.cortis at gmail.com> wrote:
> > > > > > > > >
> > > > > > > > > Hello, first time here. I was wondering if there are
> > > > > > > > > plans
> > > > > > > > > for the definition of a “hyper-dual” type in numpy. I
> > > > > > > > > think
> > > > > > > > > that would be most useful for neural nets training, and
> > > > > > > > > optimization in general.
> > > > > > > > >
> > > > > > > > > Andrea
> > > > > > > > >
> > > > > > > > > Sent from my iPad
> > > > > > > > > _______________________________________________
> > > > > > > > > SciPy-Dev mailing list
> > > > > > > > > SciPy-Dev at python.org
> > > > > > > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > > > > > > > _______________________________________________
> > > > > > > > SciPy-Dev mailing list
> > > > > > > > SciPy-Dev at python.org
> > > > > > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > > > > > >
> > > > > > > _______________________________________________
> > > > > > > SciPy-Dev mailing list
> > > > > > > SciPy-Dev at python.org
> > > > > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > > > > > _______________________________________________
> > > > > > SciPy-Dev mailing list
> > > > > > SciPy-Dev at python.org
> > > > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > > > >
> > > > > _______________________________________________
> > > > > SciPy-Dev mailing list
> > > > > SciPy-Dev at python.org
> > > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > > > _______________________________________________
> > > > SciPy-Dev mailing list
> > > > SciPy-Dev at python.org
> > > > https://mail.python.org/mailman/listinfo/scipy-dev
> > >
> > > _______________________________________________
> > > SciPy-Dev mailing list
> > > SciPy-Dev at python.org
> > > https://mail.python.org/mailman/listinfo/scipy-dev
> > >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
>
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210213/b0631f87/attachment-0001.html>