[SciPy-Dev] Hyper-dual numbers

[Andrea Cortis]

I am no mathematician and cannot comment on the equivalence of quaternions
vs hyper-dual numbers, even though they seem quite different to me at a
first glance.

For sure, I know that the algebra of dual-numbers is different from the
algebra of complex numbers.

It seems to me therefore that having a numpy `dtype=dual` would be
extremely advantageous when looking to construct *exact* values of first
(and n-th) derivatives.
Say that one would want to replace automatic differentiation for
backpropagation neural networks like in this blogpost

https://blog.demofox.org/2017/03/13/neural-network-gradients-backpropagation-dual-numbers-finite-differences/

then we would have a plethora of different implementations, say for
pytorch, tensorflow, etc. and no unifying framework.

Best,

Andre




On Sat, Feb 13, 2021 at 1:21 PM Evgeni Burovski <evgeny.burovskiy at gmail.com>
wrote:

> Ah, these. Suspected it, but wanted to make sure.
>
> IMO, these are best implemented as a numpy dtype. I'm biased though
> --- here's a branch which makes a start, based on Mike Boyle's version
> of quaternion dtype (Mike and other authors, if you're reading this
> --- thanks a ton!)
>
> https://github.com/ev-br/quaternion/tree/dual
>
> Now, I don't think scipy should carry around additional numpy dtypes.
> Cannot speak for the numpy project, but I strongly suspect this is
> best implemented as a separate repository/project.
>
> Cheers,
>
> Evgeni
>
> On Sat, Feb 13, 2021 at 9:46 PM Joseph Fox-Rabinovitz
> <jfoxrabinovitz at gmail.com> wrote:
> >
> > I directed Andrea here from Stack Overflow (
> https://stackoverflow.com/q/66179855/2988730). Based on the Wikipedia
> article (https://en.m.wikipedia.org/wiki/Dual_number), it seems like
> scipy is a much more likely place to look than numpy.
> >
> > - Joe
> >
> >
> > On Sat, Feb 13, 2021, 13:19 Evgeni Burovski <evgeny.burovskiy at gmail.com>
> wrote:
> >>
> >> Hi Andrea, welcome!
> >>
> >> Since you're asking about numpy, you likely want the numpy-discussion
> >> mailing list. (the overlap is non-zero, but nevertheless).
> >>
> >> Out of curiosity, what's a hyper-dual type?
> >>
> >> Cheers,
> >>
> >> Evgeni
> >>
> >> On Sat, Feb 13, 2021 at 7:45 PM Andrea Cortis <andrea.cortis at gmail.com>
> wrote:
> >> >
> >> > Hello, first time here. I was wondering if there are plans for the
> definition of a “hyper-dual” type in numpy. I think that would be most
> useful for neural nets training, and  optimization in general.
> >> >
> >> > Andrea
> >> >
> >> > Sent from my iPad
> >> > _______________________________________________
> >> > SciPy-Dev mailing list
> >> > SciPy-Dev at python.org
> >> > https://mail.python.org/mailman/listinfo/scipy-dev
> >> _______________________________________________
> >> SciPy-Dev mailing list
> >> SciPy-Dev at python.org
> >> https://mail.python.org/mailman/listinfo/scipy-dev
> >
> > _______________________________________________
> > SciPy-Dev mailing list
> > SciPy-Dev at python.org
> > https://mail.python.org/mailman/listinfo/scipy-dev
> _______________________________________________
> SciPy-Dev mailing list
> SciPy-Dev at python.org
> https://mail.python.org/mailman/listinfo/scipy-dev
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <https://mail.python.org/pipermail/scipy-dev/attachments/20210213/6d720896/attachment.html>