From jan-hendrik.berlin at cs.uni-dortmund.de  Mon Feb  4 20:07:35 2019
From: jan-hendrik.berlin at cs.uni-dortmund.de (Jan Hendrik Berlin)
Date: Tue, 5 Feb 2019 02:07:35 +0100
Subject: [SciPy-User] A priority of constrains for scipy.optimize.minimize
Message-ID: <3953ef2d-0660-8fe6-7e04-45b8f2c2735a@cs.uni-dortmund.de>

Hi,

I am solving a problem with some constrains. At first there is a 
constraint, that the sum of the percentages must be 1. The single 
percentage could be in the range from 0,0 to 1. And this is the main 
constraint. On the other side there are some constrains belonging to 
stuff of the calculation.  It is possible, that this constrains are to 
strong and the solver can't get a solution. In this Case I want to have 
an option to get a solution respecting the first constraint.

Has anybody an idea of a solution? I think there is no option for a 
priority of the constrains.

kind regards
Jan Hendrik Berlin

From vaggi.federico at gmail.com  Tue Feb  5 13:48:47 2019
From: vaggi.federico at gmail.com (federico vaggi)
Date: Tue, 5 Feb 2019 10:48:47 -0800
Subject: [SciPy-User] A priority of constrains for
 scipy.optimize.minimize
In-Reply-To: <mailman.63.1549386003.31000.scipy-user@python.org>
References: <mailman.63.1549386003.31000.scipy-user@python.org>
Message-ID: <CAGvd0=hfYnzMLq4Dw9Fm950_cpxAbM=Wf4BQYivBDqV5PiaX6g@mail.gmail.com>

Change your loss function to a penalized form.

IE:

Instead of minimizing L(x)

s.t.

f_1(x) (necessary constraint)
f_2(x) (nice but not necessary constraint)
....

Do this instead:

Minimize L(x) + \lambda * f_2(x)

s.t.

f_1(x)

where \lambda can be a hyper parameter you can tune to trade off how
important constraint f_2 is relative to loss function quality.

Alternatively: if you are doing minimization on a probability simplex, you
can probably re-parametrize your problem so the only viable solution
automatically satisfies those probabilities.  The most common way is to run
your output through a softmax (
https://en.wikipedia.org/wiki/Softmax_function)


On Tue, Feb 5, 2019 at 9:06 AM <scipy-user-request at python.org> wrote:

> Send SciPy-User mailing list submissions to
>         scipy-user at python.org
>
> To subscribe or unsubscribe via the World Wide Web, visit
>         https://mail.python.org/mailman/listinfo/scipy-user
> or, via email, send a message with subject or body 'help' to
>         scipy-user-request at python.org
>
> You can reach the person managing the list at
>         scipy-user-owner at python.org
>
> When replying, please edit your Subject line so it is more specific
> than "Re: Contents of SciPy-User digest..."
>
>
> Today's Topics:
>
>    1. A priority of constrains for scipy.optimize.minimize
>       (Jan Hendrik Berlin)
>
>
> ----------------------------------------------------------------------
>
> Message: 1
> Date: Tue, 5 Feb 2019 02:07:35 +0100
> From: Jan Hendrik Berlin <jan-hendrik.berlin at cs.uni-dortmund.de>
> To: SciPy-User <scipy-user at python.org>
> Subject: [SciPy-User] A priority of constrains for
>         scipy.optimize.minimize
> Message-ID: <3953ef2d-0660-8fe6-7e04-45b8f2c2735a at cs.uni-dortmund.de>
> Content-Type: text/plain; charset=utf-8; format=flowed
>
> Hi,
>
> I am solving a problem with some constrains. At first there is a
> constraint, that the sum of the percentages must be 1. The single
> percentage could be in the range from 0,0 to 1. And this is the main
> constraint. On the other side there are some constrains belonging to
> stuff of the calculation.  It is possible, that this constrains are to
> strong and the solver can't get a solution. In this Case I want to have
> an option to get a solution respecting the first constraint.
>
> Has anybody an idea of a solution? I think there is no option for a
> priority of the constrains.
>
> kind regards
> Jan Hendrik Berlin
>
>
> ------------------------------
>
> Subject: Digest Footer
>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
>
> ------------------------------
>
> End of SciPy-User Digest, Vol 186, Issue 2
> ******************************************
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190205/6aa81ca8/attachment.html>

From pi at berkeley.edu  Tue Feb  5 16:55:05 2019
From: pi at berkeley.edu (Paul Ivanov)
Date: Tue, 5 Feb 2019 13:55:05 -0800
Subject: [SciPy-User] SciPy 2019 Conference - 10 days left for submissions,
 registration now open
Message-ID: <CAKCGbV72_VMdN_t20d2wRsWb0aWPF4UthscbUDGoUnz8ZF-Cpw@mail.gmail.com>

SciPy 2019, the 18th annual Scientific Computing with Python conference,
will be held July 8-14, 2019 in Austin, Texas. The annual SciPy Conference
brings together over 800 participants from industry, academia, and
government to showcase their latest projects, learn from skilled users and
developers, and collaborate on code development. The call for abstracts for
SciPy 2019 for talks, posters and tutorials is now open. The original
deadline for submissions has been extended and the new deadline is February
15, 2019.

Conference Website: https://www.scipy2019.scipy.org/

Submission Website: https://easychair.org/conferences/?conf=scipy2019

*Talks and Posters (July 10-12, 2019)*
In addition to the general track, this year will have specialized tracks
focused on:


   - Data Driven Discoveries (including Machine Learning and Data Science)
   - Open Source Communities (Sustainability)


*Mini Symposia*

   - Science Communication through Visualization
   - Neuroscience and Cognitive Science
   - Image Processing
   - Earth, Ocean, Geo and Atmospheric Science


There will also be a SciPy Tools Plenary Session each day with 2 to 5
minute updates on tools and libraries.

*Tutorials (July 8-9, 2019)*

Tutorials should be focused on covering a well-defined topic in a hands-on
manner. We are looking for useful techniques or packages, helping new or
advanced Python programmers develop better or faster scientific
applications. We encourage submissions to be designed to allow at least 50%
of the time for hands-on exercises even if this means the subject matter
needs to be limited. Tutorials will be 4 hours in duration. In your
tutorial application, you can indicate what prerequisite skills and
knowledge will be needed for your tutorial, and the approximate expected
level of knowledge of your students (i.e., beginner, intermediate,
advanced). Instructors of accepted tutorials will receive a stipend.

-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7

-- 
                   _
                  / \
                A*   \^   -
             ,./   _.`\\ / \
            / ,--.S    \/   \
           /  `"~,_     \    \
     __o           ?
   _ \<,_         /:\
--(_)/-(_)----.../ | \
--------------.......J
Paul Ivanov
http://pirsquared.org | GPG/PGP key id: 0x0F3E28F7
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190205/a9dbe0ae/attachment.html>

From tmrsg11 at gmail.com  Wed Feb 13 17:50:39 2019
From: tmrsg11 at gmail.com (C W)
Date: Wed, 13 Feb 2019 17:50:39 -0500
Subject: [SciPy-User] Why slicing Pandas column and then subtract gives NaN?
Message-ID: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>

Dear list,

I have the following to Pandas Series: a, b. I want to slice and then
subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works
in Numpy.

Example 1: did not work
>>>a = pd.Series([85, 86, 87, 86])
>>>b = pd.Series([15, 72, 2, 3])
>>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
>>> type(a[1:4])
<class 'pandas.core.series.Series'>

Example 2: worked
If I use values() method, it's converted to a Numpy object. And it works!
>>> a.values[1:4]-b.values[0:3]
array([71, 15, 84])
>>> type(a.values[1:4])
<class 'numpy.ndarray'>

What's the reason that Pandas in example 1 did not work? Isn't Numpy built
on top of Pandas? So, why is everything ok in Numpy, but not in Pandas?

Thanks in advance!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190213/661c565c/attachment.html>

From pmhobson at gmail.com  Wed Feb 13 18:49:03 2019
From: pmhobson at gmail.com (Paul Hobson)
Date: Wed, 13 Feb 2019 15:49:03 -0800
Subject: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and
 then subtract gives NaN?
In-Reply-To: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
References: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
Message-ID: <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>

This is more a question for the pandas list, but since i'm here i'll take a
crack.


   - numpy aligns arrays by position.
   - pandas aligns by label.

So what you did in pandas is roughly equivalent to the following:

a = pandas.Series([85, 86, 87, 86], name='a').iloc[1:4].to_frame()
b = pandas.Series([15, 72, 2, 3], name='b').iloc[0:3].to_frame()
result = a.join(b,how='outer').assign(diff=lambda df: df['a'] - df['b'])
print(result)

      a     b  diff
0   NaN  15.0   NaN
1  86.0  72.0  14.0
2  87.0   2.0  85.0
3  86.0   NaN   NaN

So what I think you want would be the following:

a = pandas.Series([85, 86, 87, 86], name='a')
b = pandas.Series([15, 72, 2, 3], name='b')
result = a.subtract(b.shift()).dropna()
print(result)
1    71.0
2    15.0
3    84.0
dtype: float64



On Wed, Feb 13, 2019 at 2:51 PM C W <tmrsg11 at gmail.com> wrote:

> Dear list,
>
> I have the following to Pandas Series: a, b. I want to slice and then
> subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works
> in Numpy.
>
> Example 1: did not work
> >>>a = pd.Series([85, 86, 87, 86])
> >>>b = pd.Series([15, 72, 2, 3])
> >>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
> >>> type(a[1:4])
> <class 'pandas.core.series.Series'>
>
> Example 2: worked
> If I use values() method, it's converted to a Numpy object. And it works!
> >>> a.values[1:4]-b.values[0:3]
> array([71, 15, 84])
> >>> type(a.values[1:4])
> <class 'numpy.ndarray'>
>
> What's the reason that Pandas in example 1 did not work? Isn't Numpy built
> on top of Pandas? So, why is everything ok in Numpy, but not in Pandas?
>
> Thanks in advance!
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at python.org
> https://mail.python.org/mailman/listinfo/numpy-discussion
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190213/c83f9226/attachment.html>

From tmrsg11 at gmail.com  Thu Feb 14 15:20:20 2019
From: tmrsg11 at gmail.com (C W)
Date: Thu, 14 Feb 2019 15:20:20 -0500
Subject: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and
 then subtract gives NaN?
In-Reply-To: <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>
References: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
 <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>
Message-ID: <CAE2FW2krPn5z+vyqXSAe+beVdMUi-75_XHboZ4aQqJgp1fFwDA@mail.gmail.com>

Hi Paul,

Thanks for your response! I did not find a Pandas list for users, only for
developers. I'd love to be on there.

result = a.subtract(b.shift()).dropna()

This seems verbose, several layers of parenthesis follow by a dot method.
I'm new to Python, I thought Python code would be pity and short. Is this
what everyone will write?

Thank you!



On Wed, Feb 13, 2019 at 6:50 PM Paul Hobson <pmhobson at gmail.com> wrote:

> This is more a question for the pandas list, but since i'm here i'll take
> a crack.
>
>
>    - numpy aligns arrays by position.
>    - pandas aligns by label.
>
> So what you did in pandas is roughly equivalent to the following:
>
> a = pandas.Series([85, 86, 87, 86], name='a').iloc[1:4].to_frame()
> b = pandas.Series([15, 72, 2, 3], name='b').iloc[0:3].to_frame()
> result = a.join(b,how='outer').assign(diff=lambda df: df['a'] - df['b'])
> print(result)
>
>       a     b  diff
> 0   NaN  15.0   NaN
> 1  86.0  72.0  14.0
> 2  87.0   2.0  85.0
> 3  86.0   NaN   NaN
>
> So what I think you want would be the following:
>
> a = pandas.Series([85, 86, 87, 86], name='a')
> b = pandas.Series([15, 72, 2, 3], name='b')
> result = a.subtract(b.shift()).dropna()
> print(result)
> 1    71.0
> 2    15.0
> 3    84.0
> dtype: float64
>
>
>
> On Wed, Feb 13, 2019 at 2:51 PM C W <tmrsg11 at gmail.com> wrote:
>
>> Dear list,
>>
>> I have the following to Pandas Series: a, b. I want to slice and then
>> subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works
>> in Numpy.
>>
>> Example 1: did not work
>> >>>a = pd.Series([85, 86, 87, 86])
>> >>>b = pd.Series([15, 72, 2, 3])
>> >>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
>> >>> type(a[1:4])
>> <class 'pandas.core.series.Series'>
>>
>> Example 2: worked
>> If I use values() method, it's converted to a Numpy object. And it works!
>> >>> a.values[1:4]-b.values[0:3]
>> array([71, 15, 84])
>> >>> type(a.values[1:4])
>> <class 'numpy.ndarray'>
>>
>> What's the reason that Pandas in example 1 did not work? Isn't Numpy
>> built on top of Pandas? So, why is everything ok in Numpy, but not in
>> Pandas?
>>
>> Thanks in advance!
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at python.org
>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190214/ee2a4a40/attachment.html>

From takowl at gmail.com  Thu Feb 14 16:53:10 2019
From: takowl at gmail.com (Thomas Kluyver)
Date: Thu, 14 Feb 2019 21:53:10 +0000
Subject: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and
 then subtract gives NaN?
In-Reply-To: <CAE2FW2krPn5z+vyqXSAe+beVdMUi-75_XHboZ4aQqJgp1fFwDA@mail.gmail.com>
References: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
 <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>
 <CAE2FW2krPn5z+vyqXSAe+beVdMUi-75_XHboZ4aQqJgp1fFwDA@mail.gmail.com>
Message-ID: <CAOvn4qi1L4pV-=sxEJ0-qrmNeJZABeVGUf_NjUcvvOThDRZLQw@mail.gmail.com>

Maybe it's useful to look a bit more at what pandas is doing and why. The
'index' on a series or dataframe labels each row - e.g. if your series is
measuring total sales for each day, its index would be the dates. When you
combine (e.g. subtract) two series, pandas automatically lines up the
indices. So it will join up the numbers for February 14th, even if they're
not in the same position in the data.

In your example, you haven't specified an index, so pandas generates an
integer index which doesn't really mean anything, and aligning on it
doesn't do what you want.

What are you trying to do? If Numpy does exactly what you want, then the
answer might be to use Numpy.

> Isn't Numpy built on top of Pandas?

It's the other way round: pandas is built on Numpy. Pandas indices are an
extra layer of functionality on top of what Numpy does.

On Thu, 14 Feb 2019 at 20:22, C W <tmrsg11 at gmail.com> wrote:

> Hi Paul,
>
> Thanks for your response! I did not find a Pandas list for users, only for
> developers. I'd love to be on there.
>
> result = a.subtract(b.shift()).dropna()
>
> This seems verbose, several layers of parenthesis follow by a dot method.
> I'm new to Python, I thought Python code would be pity and short. Is this
> what everyone will write?
>
> Thank you!
>
>
>
> On Wed, Feb 13, 2019 at 6:50 PM Paul Hobson <pmhobson at gmail.com> wrote:
>
>> This is more a question for the pandas list, but since i'm here i'll take
>> a crack.
>>
>>
>>    - numpy aligns arrays by position.
>>    - pandas aligns by label.
>>
>> So what you did in pandas is roughly equivalent to the following:
>>
>> a = pandas.Series([85, 86, 87, 86], name='a').iloc[1:4].to_frame()
>> b = pandas.Series([15, 72, 2, 3], name='b').iloc[0:3].to_frame()
>> result = a.join(b,how='outer').assign(diff=lambda df: df['a'] - df['b'])
>> print(result)
>>
>>       a     b  diff
>> 0   NaN  15.0   NaN
>> 1  86.0  72.0  14.0
>> 2  87.0   2.0  85.0
>> 3  86.0   NaN   NaN
>>
>> So what I think you want would be the following:
>>
>> a = pandas.Series([85, 86, 87, 86], name='a')
>> b = pandas.Series([15, 72, 2, 3], name='b')
>> result = a.subtract(b.shift()).dropna()
>> print(result)
>> 1    71.0
>> 2    15.0
>> 3    84.0
>> dtype: float64
>>
>>
>>
>> On Wed, Feb 13, 2019 at 2:51 PM C W <tmrsg11 at gmail.com> wrote:
>>
>>> Dear list,
>>>
>>> I have the following to Pandas Series: a, b. I want to slice and then
>>> subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works
>>> in Numpy.
>>>
>>> Example 1: did not work
>>> >>>a = pd.Series([85, 86, 87, 86])
>>> >>>b = pd.Series([15, 72, 2, 3])
>>> >>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
>>> >>> type(a[1:4])
>>> <class 'pandas.core.series.Series'>
>>>
>>> Example 2: worked
>>> If I use values() method, it's converted to a Numpy object. And it works!
>>> >>> a.values[1:4]-b.values[0:3]
>>> array([71, 15, 84])
>>> >>> type(a.values[1:4])
>>> <class 'numpy.ndarray'>
>>>
>>> What's the reason that Pandas in example 1 did not work? Isn't Numpy
>>> built on top of Pandas? So, why is everything ok in Numpy, but not in
>>> Pandas?
>>>
>>> Thanks in advance!
>>> _______________________________________________
>>> NumPy-Discussion mailing list
>>> NumPy-Discussion at python.org
>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at python.org
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190214/16c613ac/attachment-0001.html>

From tmrsg11 at gmail.com  Fri Feb 15 01:02:01 2019
From: tmrsg11 at gmail.com (Mike C)
Date: Fri, 15 Feb 2019 06:02:01 +0000
Subject: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and
 then subtract gives NaN?
In-Reply-To: <CAOvn4qi1L4pV-=sxEJ0-qrmNeJZABeVGUf_NjUcvvOThDRZLQw@mail.gmail.com>
References: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
 <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>
 <CAE2FW2krPn5z+vyqXSAe+beVdMUi-75_XHboZ4aQqJgp1fFwDA@mail.gmail.com>,
 <CAOvn4qi1L4pV-=sxEJ0-qrmNeJZABeVGUf_NjUcvvOThDRZLQw@mail.gmail.com>
Message-ID: <DM5PR13MB17072A869C2D2CE55209D67DF5600@DM5PR13MB1707.namprd13.prod.outlook.com>

Thanks a lot, Thomas.

I don?t have index when I read in the data. I just want to slice two series to the same length, and subtract. That?s it!

I also don?t what numpy methods wrapped within methods. They work, but hard do understand.

How would you do it? In Matlab or R, it?s very simple, one line.




________________________________
From: SciPy-User <scipy-user-bounces+tmrsg11=gmail.com at python.org> on behalf of Thomas Kluyver <takowl at gmail.com>
Sent: Thursday, February 14, 2019 4:54 PM
To: SciPy Users List
Cc: Discussion of Numerical Python
Subject: Re: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and then subtract gives NaN?

Maybe it's useful to look a bit more at what pandas is doing and why. The 'index' on a series or dataframe labels each row - e.g. if your series is measuring total sales for each day, its index would be the dates. When you combine (e.g. subtract) two series, pandas automatically lines up the indices. So it will join up the numbers for February 14th, even if they're not in the same position in the data.

In your example, you haven't specified an index, so pandas generates an integer index which doesn't really mean anything, and aligning on it doesn't do what you want.

What are you trying to do? If Numpy does exactly what you want, then the answer might be to use Numpy.

> Isn't Numpy built on top of Pandas?

It's the other way round: pandas is built on Numpy. Pandas indices are an extra layer of functionality on top of what Numpy does.

On Thu, 14 Feb 2019 at 20:22, C W <tmrsg11 at gmail.com<mailto:tmrsg11 at gmail.com>> wrote:
Hi Paul,

Thanks for your response! I did not find a Pandas list for users, only for developers. I'd love to be on there.


result = a.subtract(b.shift()).dropna()

This seems verbose, several layers of parenthesis follow by a dot method. I'm new to Python, I thought Python code would be pity and short. Is this what everyone will write?

Thank you!


On Wed, Feb 13, 2019 at 6:50 PM Paul Hobson <pmhobson at gmail.com<mailto:pmhobson at gmail.com>> wrote:
This is more a question for the pandas list, but since i'm here i'll take a crack.


  *   numpy aligns arrays by position.
  *   pandas aligns by label.

So what you did in pandas is roughly equivalent to the following:

a = pandas.Series([85, 86, 87, 86], name='a').iloc[1:4].to_frame()
b = pandas.Series([15, 72, 2, 3], name='b').iloc[0:3].to_frame()
result = a.join(b,how='outer').assign(diff=lambda df: df['a'] - df['b'])
print(result)

      a     b  diff
0   NaN  15.0   NaN
1  86.0  72.0  14.0
2  87.0   2.0  85.0
3  86.0   NaN   NaN


So what I think you want would be the following:



a = pandas.Series([85, 86, 87, 86], name='a')
b = pandas.Series([15, 72, 2, 3], name='b')
result = a.subtract(b.shift()).dropna()
print(result)
1    71.0
2    15.0
3    84.0
dtype: float64



On Wed, Feb 13, 2019 at 2:51 PM C W <tmrsg11 at gmail.com<mailto:tmrsg11 at gmail.com>> wrote:
Dear list,

I have the following to Pandas Series: a, b. I want to slice and then subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works in Numpy.

Example 1: did not work
>>>a = pd.Series([85, 86, 87, 86])
>>>b = pd.Series([15, 72, 2, 3])
>>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
>>> type(a[1:4])
<class 'pandas.core.series.Series'>

Example 2: worked
If I use values() method, it's converted to a Numpy object. And it works!
>>> a.values[1:4]-b.values[0:3]
array([71, 15, 84])
>>> type(a.values[1:4])
<class 'numpy.ndarray'>

What's the reason that Pandas in example 1 did not work? Isn't Numpy built on top of Pandas? So, why is everything ok in Numpy, but not in Pandas?

Thanks in advance!
_______________________________________________
NumPy-Discussion mailing list
NumPy-Discussion at python.org<mailto:NumPy-Discussion at python.org>
https://mail.python.org/mailman/listinfo/numpy-discussion
_______________________________________________
SciPy-User mailing list
SciPy-User at python.org<mailto:SciPy-User at python.org>
https://mail.python.org/mailman/listinfo/scipy-user
_______________________________________________
SciPy-User mailing list
SciPy-User at python.org<mailto:SciPy-User at python.org>
https://mail.python.org/mailman/listinfo/scipy-user
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190215/cb57df8e/attachment.html>

From takowl at gmail.com  Fri Feb 15 04:10:55 2019
From: takowl at gmail.com (Thomas Kluyver)
Date: Fri, 15 Feb 2019 09:10:55 +0000
Subject: [SciPy-User] [Numpy-discussion] Why slicing Pandas column and
 then subtract gives NaN?
In-Reply-To: <DM5PR13MB17072A869C2D2CE55209D67DF5600@DM5PR13MB1707.namprd13.prod.outlook.com>
References: <CAE2FW2mARV2XRyxhMDtYiW45tc8nji8nrYLnQjNjPzc3DinejQ@mail.gmail.com>
 <CADT3MECKbwaA-Xgh1C=n_A-aoC3A8qhzVraVoJT_1zSqCzqGVQ@mail.gmail.com>
 <CAE2FW2krPn5z+vyqXSAe+beVdMUi-75_XHboZ4aQqJgp1fFwDA@mail.gmail.com>
 <CAOvn4qi1L4pV-=sxEJ0-qrmNeJZABeVGUf_NjUcvvOThDRZLQw@mail.gmail.com>
 <DM5PR13MB17072A869C2D2CE55209D67DF5600@DM5PR13MB1707.namprd13.prod.outlook.com>
Message-ID: <CAOvn4qjeWUpcN21-0RbUama_cCJ=QsbKvbXKXCXSsu8oj42n0A@mail.gmail.com>

> I don?t have index when I read in the data. I just want to slice two
series to the same length, and subtract. That?s it!

That sounds like you want Numpy. Pandas objects always have an index, even
if it's the default integer index. You've already found how to extract a
Numpy array from a pandas series.

On Fri, 15 Feb 2019 at 06:02, Mike C <tmrsg11 at gmail.com> wrote:

> Thanks a lot, Thomas.
>
> I don?t have index when I read in the data. I just want to slice two
> series to the same length, and subtract. That?s it!
>
> I also don?t what numpy methods wrapped within methods. They work, but
> hard do understand.
>
> How would you do it? In Matlab or R, it?s very simple, one line.
>
>
>
>
> ------------------------------
> *From:* SciPy-User <scipy-user-bounces+tmrsg11=gmail.com at python.org> on
> behalf of Thomas Kluyver <takowl at gmail.com>
> *Sent:* Thursday, February 14, 2019 4:54 PM
> *To:* SciPy Users List
> *Cc:* Discussion of Numerical Python
> *Subject:* Re: [SciPy-User] [Numpy-discussion] Why slicing Pandas column
> and then subtract gives NaN?
>
> Maybe it's useful to look a bit more at what pandas is doing and why. The
> 'index' on a series or dataframe labels each row - e.g. if your series is
> measuring total sales for each day, its index would be the dates. When you
> combine (e.g. subtract) two series, pandas automatically lines up the
> indices. So it will join up the numbers for February 14th, even if they're
> not in the same position in the data.
>
> In your example, you haven't specified an index, so pandas generates an
> integer index which doesn't really mean anything, and aligning on it
> doesn't do what you want.
>
> What are you trying to do? If Numpy does exactly what you want, then the
> answer might be to use Numpy.
>
> > Isn't Numpy built on top of Pandas?
>
> It's the other way round: pandas is built on Numpy. Pandas indices are an
> extra layer of functionality on top of what Numpy does.
>
> On Thu, 14 Feb 2019 at 20:22, C W <tmrsg11 at gmail.com> wrote:
>
>> Hi Paul,
>>
>> Thanks for your response! I did not find a Pandas list for users, only
>> for developers. I'd love to be on there.
>>
>> result = a.subtract(b.shift()).dropna()
>>
>> This seems verbose, several layers of parenthesis follow by a dot method.
>> I'm new to Python, I thought Python code would be pity and short. Is this
>> what everyone will write?
>>
>> Thank you!
>>
>>
>>
>> On Wed, Feb 13, 2019 at 6:50 PM Paul Hobson <pmhobson at gmail.com> wrote:
>>
>>> This is more a question for the pandas list, but since i'm here i'll
>>> take a crack.
>>>
>>>
>>>    - numpy aligns arrays by position.
>>>    - pandas aligns by label.
>>>
>>> So what you did in pandas is roughly equivalent to the following:
>>>
>>> a = pandas.Series([85, 86, 87, 86], name='a').iloc[1:4].to_frame()
>>> b = pandas.Series([15, 72, 2, 3], name='b').iloc[0:3].to_frame()
>>> result = a.join(b,how='outer').assign(diff=lambda df: df['a'] - df['b'])
>>> print(result)
>>>
>>>       a     b  diff
>>> 0   NaN  15.0   NaN
>>> 1  86.0  72.0  14.0
>>> 2  87.0   2.0  85.0
>>> 3  86.0   NaN   NaN
>>>
>>> So what I think you want would be the following:
>>>
>>> a = pandas.Series([85, 86, 87, 86], name='a')
>>> b = pandas.Series([15, 72, 2, 3], name='b')
>>> result = a.subtract(b.shift()).dropna()
>>> print(result)
>>> 1    71.0
>>> 2    15.0
>>> 3    84.0
>>> dtype: float64
>>>
>>>
>>>
>>> On Wed, Feb 13, 2019 at 2:51 PM C W <tmrsg11 at gmail.com> wrote:
>>>
>>>> Dear list,
>>>>
>>>> I have the following to Pandas Series: a, b. I want to slice and then
>>>> subtract. Like this: a[1:4] - b[0:3]. Why does it give me NaN? But it works
>>>> in Numpy.
>>>>
>>>> Example 1: did not work
>>>> >>>a = pd.Series([85, 86, 87, 86])
>>>> >>>b = pd.Series([15, 72, 2, 3])
>>>> >>> a[1:4]-b[0:3] 0   NaN 1   14.0 2   85.0 3   NaN
>>>> >>> type(a[1:4])
>>>> <class 'pandas.core.series.Series'>
>>>>
>>>> Example 2: worked
>>>> If I use values() method, it's converted to a Numpy object. And it
>>>> works!
>>>> >>> a.values[1:4]-b.values[0:3]
>>>> array([71, 15, 84])
>>>> >>> type(a.values[1:4])
>>>> <class 'numpy.ndarray'>
>>>>
>>>> What's the reason that Pandas in example 1 did not work? Isn't Numpy
>>>> built on top of Pandas? So, why is everything ok in Numpy, but not in
>>>> Pandas?
>>>>
>>>> Thanks in advance!
>>>> _______________________________________________
>>>> NumPy-Discussion mailing list
>>>> NumPy-Discussion at python.org
>>>> https://mail.python.org/mailman/listinfo/numpy-discussion
>>>>
>>> _______________________________________________
>>> SciPy-User mailing list
>>> SciPy-User at python.org
>>> https://mail.python.org/mailman/listinfo/scipy-user
>>>
>> _______________________________________________
>> SciPy-User mailing list
>> SciPy-User at python.org
>> https://mail.python.org/mailman/listinfo/scipy-user
>>
> _______________________________________________
> SciPy-User mailing list
> SciPy-User at python.org
> https://mail.python.org/mailman/listinfo/scipy-user
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190215/3a7e54df/attachment-0001.html>

From charlesr.harris at gmail.com  Tue Feb 26 15:00:17 2019
From: charlesr.harris at gmail.com (Charles R Harris)
Date: Tue, 26 Feb 2019 13:00:17 -0700
Subject: [SciPy-User] NumPy 1.16.2 released.
Message-ID: <CAB6mnx+UqoTCZ2=MT4he=i6RX7zAtQVjq_JpEqg8CYHXs7Q34A@mail.gmail.com>

Hi All,

On behalf of the NumPy team I am pleased to announce the release of NumPy
1.16.2. This is a quick release fixing several problems encountered on
Windows. The Python versions supported are 2.7 and 3.5-3.7. The Windows
problems addressed are:

   - DLL load problems for NumPy wheels on Windows,
   - distutils command line parsing on Windows.

There is also a regression fix correcting signed zeros produced by divmod,
see the release notes for details. Downstream developers building this
release should use Cython >= 0.29.2 and, if using OpenBLAS, OpenBLAS >
v0.3.4.

If you are installing using pip, you may encounter a problem with older
installed versions of NumPy that pip did not delete becoming mixed with the
current version, resulting in an ``ImportError``. That problem is
particularly
common on Debian derived distributions due to a modified pip.  The fix is
to make sure all previous NumPy versions installed by pip have been
removed. See #12736 <https://github.com/numpy/numpy/issues/12736> for
discussion of the issue.

Wheels for this release can be downloaded from PyPI
<https://pypi.org/project/numpy/1.16.2/>, source archives and release notes
are available from Github
<https://github.com/numpy/numpy/releases/tag/v1.16.2>.


*Contributors*

A total of 5 people contributed to this release.  People with a "+" by their
names contributed a patch for the first time.

* Charles Harris
* Eric Wieser
* Matti Picus
* Tyler Reddy
* Tony LaTorre +


*Pull requests merged*

A total of 7 pull requests were merged for this release.

* #12909: TST: fix vmImage dispatch in Azure
* #12923: MAINT: remove complicated test of multiarray import failure mode
* #13020: BUG: fix signed zero behavior in npy_divmod
* #13026: MAINT: Add functions to parse shell-strings in the
platform-native...
* #13028: BUG: Fix regression in parsing of F90 and F77 environment
variables
* #13038: BUG: parse shell escaping in extra_compile_args and
extra_link_args
* #13041: BLD: Windows absolute path DLL loading

Cheers,

Charles Harris
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/scipy-user/attachments/20190226/adcfbf13/attachment.html>

From opossumnano at gmail.com  Thu Feb 28 10:35:50 2019
From: opossumnano at gmail.com (Tiziano Zito)
Date: Thu, 28 Feb 2019 07:35:50 -0800 (PST)
Subject: [SciPy-User] =?utf-8?b?W0FOTl0gMTLhtZfKsCBBZHZhbmNlZCBTY2llbnRp?=
 =?utf-8?q?fic_Programming_in_Python_in_Camerino=2C_Italy=2C_2=E2=80=947_S?=
 =?utf-8?q?eptember=2C_2019?=
Message-ID: <5c77ffd6.1c69fb81.7cfce.8e9e@mx.google.com>

12?? Advanced Scientific Programming in Python
==============================================
a Summer School by the G-Node and the University of Camerino

https://python.g-node.org

Scientists spend more and more time writing, maintaining, and debugging
software. While techniques for doing this efficiently have evolved, only few
scientists have been trained to use them. As a result, instead of doing their
research, they spend far too much time writing deficient code and reinventing
the wheel. In this course we will present a selection of advanced programming
techniques and best practices which are standard in the industry, but especially
tailored to the needs of a programming scientist. Lectures are devised to be
interactive and to give the students enough time to acquire direct hands-on
experience with the materials. Students will work in pairs throughout the school
and will team up to practice the newly learned skills in a real programming
project ? an entertaining computer game.

We use the Python programming language for the entire course. Python works as
a simple programming language for beginners, but more importantly, it also works
great in scientific simulations and data analysis. We show how clean language
design, ease of extensibility, and the great wealth of open source libraries for
scientific computing and data visualization are driving Python to become
a standard tool for the programming scientist.

This school is targeted at Master or PhD students and Post-docs from all areas
of science. Competence in Python or in another language such as Java, C/C++,
MATLAB, or R is absolutely required. Basic knowledge of Python and of a version
control system such as git, subversion, mercurial, or bazaar is assumed.
Participants without any prior experience with Python and/or git should work
through the proposed introductory material before the course.

We are striving hard to get a pool of students which is international and
gender-balanced.

Date & Location
===============
2?7 September, 2019. Camerino, Italy.

Application
===========
You can apply online: https://python.g-node.org/wiki/applications

Application deadline: 23:59 UTC, 26 May, 2019.

There will be no deadline extension, so be sure to apply on time. Be sure to
read the FAQ before applying: https://python.g-node.org/wiki/faq

Participation is for free, i.e. no fee is charged! Participants however should
take care of travel, living, and accommodation expenses by themselves.

Program
=======

? Version control with git and how to contribute to open source projects with GitHub
? Tidy data analysis and visualization
? Testing and debugging scientific code
? Advanced NumPy
? Organizing, documenting, and distributing scientific code
? Advanced scientific Python: context managers and generators
? Writing parallel applications in Python
? Profiling and speeding up scientific code with Cython and numba
? Programming in teams

Faculty
=======

? Caterina Buizza, Personal Robotics Lab, Imperial College London, UK
? Jenni Rinker, Department of Wind Energy, Technical University of Denmark, Roskilde, Denmark
? Juan Nunez-Iglesias, Bioimage Analysis Research Fellow, Monash University, Australia
? Nelle Varoquaux, Department of Statistics, UC Berkeley, CA, USA
? Pamela Hathway, Neural Reckoning, Imperial College London, UK
? Pietro Berkes, NAGRA Kudelski, Lausanne, Switzerland
? Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universit?t zu Berlin, Germany
? St?fan van der Walt, Berkeley Institute for Data Science, UC Berkeley, CA, USA
? Tiziano Zito, Department of Psychology, Humboldt-Universit?t zu Berlin, Germany

Organizers
==========
For the German Neuroinformatics Node of the INCF (G-Node), Germany:

? Tiziano Zito, Department of Psychology, Humboldt-Universit?t zu Berlin, Germany
? Caterina Buizza, Personal Robotics Lab, Imperial College London, UK
? Zbigniew J?drzejewski-Szmek, Red Hat Inc., Warsaw, Poland
? Jakob Jordan, Department of Physiology, University of Bern, Switzerland

For the University of Camerino, Italy:

? Barbara Re, Computer Science Division, School of Science and Technology, University of Camerino Italy

Website: https://python.g-node.org
Contact: python-info at g-node.org