From banyal at iiap.res.in  Sun Apr  2 02:53:13 2017
From: banyal at iiap.res.in (banyal)
Date: Sun, 02 Apr 2017 12:23:13 +0530
Subject: [AstroPy] Astropy model fitting
Message-ID: <4de3d90b81512684e89eadbf197f49d5@mailnew.iiap.res.in>



-------- Original Message --------
Subject: Astropy model fitting
Date: 2017-04-02 12:19
 From: banyal <banyal at iiap.res.in>
To: Astronomical Python mailing list <astropy at scipy.org>

Hi,
I have a very general question related to model fitting in Astropy.  I 
want to use astropy.modeling to fit observational data to a compound 
model of type f*g, where * is a convolution operator,  f is high 
resolution stellar spectra (template) and g is instrument response (a 
gaussian form, with amplitude, mean and stddev as fitting parameters). 
The  objective is to use nonlinear LSF to extract the fitting? 
parameters of a gaussian function which best describe the instrument 
response. Essentially, the model has an integral form (f*g) that must 
fit the data in least square sense. After going through the 
astropy.modeling documentation, it seems the construction of a compound 
model in astropy is so far limited to combining  individual models with 
elementary arithmetic operations (convolution not included). Is there a 
way around this?? Or to put it another way, can I define a custom 
function of (f*g) and still use Astropy.modeling to do the least square 
fitting?? Many thanks will appreciate any thought on this.

With best wishes
Ravi



From jvmirca at gmail.com  Sun Apr  2 04:13:43 2017
From: jvmirca at gmail.com (=?utf-8?B?WsOpIFZpbsOtY2l1cw==?=)
Date: Sun, 2 Apr 2017 01:13:43 -0700
Subject: [AstroPy] Astropy model fitting
In-Reply-To: <4de3d90b81512684e89eadbf197f49d5@mailnew.iiap.res.in>
References: <4de3d90b81512684e89eadbf197f49d5@mailnew.iiap.res.in>
Message-ID: <54EECDA6-0B87-4AD2-B614-CDC6D803C074@gmail.com>

Hi Ravi,

As you mentioned, I also think this cannot be done with astropy compound models.
It would be really awesome if we could get compound models out of more complex operations (e.g. convolution).

However, I think you can do that with numpy and scipy, with something like that (?):

import numpy as np
from matplotlib import pyplot as plt
from scipy.optimize import curve_fit

def gaussian_convolved_with_ramp(x, alpha, beta):
    ramp = alpha * x
    gauss = np.exp(-0.5 * (x - beta) ** 2.)

    return np.convolve(ramp, gauss)

if __name__ == "__main__":
    xp = np.linspace(-7, 7, 100)
    g = gaussian_convolved(xp, 1, 0)

    
    toy_data = g + 10 * np.random.standard_normal(len(g))
    g_est, g_est_cov = curve_fit(gaussian_convolved_with_ramp, xp, toy_data)
    print(g_est)
    print(g_est_cov)

    x = np.linspace(-7, 7, len(g))
    plt.plot(x, toy_data, x, gaussian_convolved_with_ramp(xp, *g_est))
    plt.show()

Cheers,
Z?

> On Apr 1, 2017, at 11:53 PM, banyal <banyal at iiap.res.in> wrote:
> 
> 
> 
> -------- Original Message --------
> Subject: Astropy model fitting
> Date: 2017-04-02 12:19
> From: banyal <banyal at iiap.res.in>
> To: Astronomical Python mailing list <astropy at scipy.org>
> 
> Hi,
> I have a very general question related to model fitting in Astropy.  I want to use astropy.modeling to fit observational data to a compound model of type f*g, where * is a convolution operator,  f is high resolution stellar spectra (template) and g is instrument response (a gaussian form, with amplitude, mean and stddev as fitting parameters). The  objective is to use nonlinear LSF to extract the fitting? parameters of a gaussian function which best describe the instrument response. Essentially, the model has an integral form (f*g) that must fit the data in least square sense. After going through the astropy.modeling documentation, it seems the construction of a compound model in astropy is so far limited to combining  individual models with elementary arithmetic operations (convolution not included). Is there a way around this?? Or to put it another way, can I define a custom function of (f*g) and still use Astropy.modeling to do the least square fitting?? Many thanks will appreciate any thought on this.
> 
> With best wishes
> Ravi
> 
> 
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20170402/a95ff44f/attachment.html>

From axel.donath at mpi-hd.mpg.de  Wed Apr  5 03:41:17 2017
From: axel.donath at mpi-hd.mpg.de (Axel Donath)
Date: Wed, 05 Apr 2017 09:41:17 +0200
Subject: [AstroPy] Astropy model fitting
In-Reply-To: <54EECDA6-0B87-4AD2-B614-CDC6D803C074@gmail.com>
References: <4de3d90b81512684e89eadbf197f49d5@mailnew.iiap.res.in>
 <54EECDA6-0B87-4AD2-B614-CDC6D803C074@gmail.com>
Message-ID: <c210d91412701fb88a7296c007e2399e@mpi-hd.mpg.de>

Hi Ravi and Z?,

I remember that I've worked on this during the preparation for mentoring 
a GSoC
project 2015. Indeed it would not be that much work to include 
convolution as an
operation for compound models. Unfortunately I didn't find time to work 
on this
anymore. Here's a little gist with the proof of concept:

https://gist.github.com/adonath/4f6aef5f125b2cca4ae5edbd978ffc0b

I was suprised to see that it still works. Maybe someone is motivated to 
turn this
into a little PR for astropy core? The remaining task would be to 
integrate this
into the exiting modeling API, e.g. by introducing a helper function 
`convolve_models`,
'overloading' the existing `convolve_fft` method or introducing a new 
symbol
operator. I'd be happy to review the PR, but I don't have time to work 
on this myself.

Cheers, Axel



Am 2017-04-02 10:13, schrieb Z? Vin?cius:
> Hi Ravi,
> 
> As you mentioned, I also think this CANNOT be done with astropy
> compound models.
> It would be really awesome if we could get compound models out of more
> complex operations (e.g. convolution).
> 
> However, I think you can do that with numpy and scipy, with something
> like that (?):
> 
> import numpy as np
> from matplotlib import pyplot as plt
> from scipy.optimize import curve_fit
> 
> def gaussian_convolved_with_ramp(x, alpha, beta):
>     ramp = alpha * x
>     gauss = np.exp(-0.5 * (x - beta) ** 2.)
> 
>     return np.convolve(ramp, gauss)
> 
> if __name__ == "__main__":
>     xp = np.linspace(-7, 7, 100)
>     g = gaussian_convolved(xp, 1, 0)
> 
>     toy_data = g + 10 * np.random.standard_normal(len(g))
>     g_est, g_est_cov = curve_fit(gaussian_convolved_with_ramp, xp,
> toy_data)
>     print(g_est)
>     print(g_est_cov)
> 
>     x = np.linspace(-7, 7, len(g))
>     plt.plot(x, toy_data, x, gaussian_convolved_with_ramp(xp, *g_est))
>     plt.show()
> 
> Cheers,
> Z?
> 
>> On Apr 1, 2017, at 11:53 PM, banyal <banyal at iiap.res.in> wrote:
>> 
>> -------- Original Message --------
>> Subject: Astropy model fitting
>> Date: 2017-04-02 12:19
>> From: banyal <banyal at iiap.res.in>
>> To: Astronomical Python mailing list <astropy at scipy.org>
>> 
>> Hi,
>> I have a very general question related to model fitting in Astropy.
>> I want to use astropy.modeling to fit observational data to a
>> compound model of type f*g, where * is a convolution operator,  f is
>> high resolution stellar spectra (template) and g is instrument
>> response (a gaussian form, with amplitude, mean and stddev as
>> fitting parameters). The  objective is to use nonlinear LSF to
>> extract the fitting? parameters of a gaussian function which best
>> describe the instrument response. Essentially, the model has an
>> integral form (f*g) that must fit the data in least square sense.
>> After going through the astropy.modeling documentation, it seems the
>> construction of a compound model in astropy is so far limited to
>> combining  individual models with elementary arithmetic operations
>> (convolution not included). Is there a way around this?? Or to put
>> it another way, can I define a custom function of (f*g) and still
>> use Astropy.modeling to do the least square fitting?? Many thanks
>> will appreciate any thought on this.
>> 
>> With best wishes
>> Ravi
>> 
>> _______________________________________________
>> AstroPy mailing list
>> AstroPy at python.org
>> https://mail.python.org/mailman/listinfo/astropy
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy

From astropy at liska.ath.cx  Mon Apr 10 09:13:37 2017
From: astropy at liska.ath.cx (Ole Streicher)
Date: Mon, 10 Apr 2017 15:13:37 +0200
Subject: [AstroPy] [astropy.io.ascii] specify column type
Message-ID: <ytzmvboo16m.fsf@news.ole.ath.cx>

Hi,

I have a CSV table which has a number of columns that have integer
content but are of type str. Is there a way to specify the data type for
those so that they are not misinterpreted (and f.e. leading zeroes
removed)?

Best regards

Ole

From aldcroft at head.cfa.harvard.edu  Mon Apr 10 10:27:03 2017
From: aldcroft at head.cfa.harvard.edu (Aldcroft, Thomas)
Date: Mon, 10 Apr 2017 10:27:03 -0400
Subject: [AstroPy] [astropy.io.ascii] specify column type
In-Reply-To: <ytzmvboo16m.fsf@news.ole.ath.cx>
References: <ytzmvboo16m.fsf@news.ole.ath.cx>
Message-ID: <CAMtEP6xrDnDviLS8ZJMkQwsJ-V=2Esj4Ke6KNFKx_S5t9Ra5Hg@mail.gmail.com>

On Mon, Apr 10, 2017 at 9:13 AM, Ole Streicher <astropy at liska.ath.cx> wrote:

> Hi,
>
> I have a CSV table which has a number of columns that have integer
> content but are of type str. Is there a way to specify the data type for
> those so that they are not misinterpreted (and f.e. leading zeroes
> removed)?
>

Yes, this is possible.  The API is a little clunky, but see:

 http://astropy.readthedocs.io/en/latest/io/ascii/read.html#converters

Note also that this only works for the pure-Python (fast_reader=False)
reader.  There is an open issue to make this work for the fast C reader.

Cheers,
Tom



>
> Best regards
>
> Ole
> _______________________________________________
> AstroPy mailing list
> AstroPy at python.org
> https://mail.python.org/mailman/listinfo/astropy
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/astropy/attachments/20170410/989d8129/attachment.html>

From mcbeth at broggs.org  Thu Apr 13 07:52:13 2017
From: mcbeth at broggs.org (Jeffrey Brent McBeth)
Date: Thu, 13 Apr 2017 07:52:13 -0400
Subject: [AstroPy] Vector Quantities in QTable
Message-ID: <20170413115213.j7ltppc4mixjbgsa@broggs.org>

I'm trying to carry around state vectors for measurements I'm making.
Some of these measurements come in a large vector that I need to transform easily as a bulk later (calibration)
I'm struggling to get these measurements to load into QTable (which I assumed would be the right structure for this)

#+begin_src python
from astropy.table import QTable
import astropy.units as u
import numpy as np

# I've significantly reduced the number of objects in state1 to keep things simple
foo = QTable(names=('state1','mass'),dtype=('(3,1)f8','f8'))
foo['state1'].unit = u.K
foo['mass'].unit = u.mg

s0 = [235,240,245]*u.K
m0 = 5*u.g

try:
        foo.add_row((s0,m0)) # Incorrect length for column state1 after inserting [ 235.  240.  245.] K (expected 3,
got 1)
except Exception as e:
        print e

try:
        foo.add_row((s0[:,np.newaxis],m0)) # could not broadcast input array from shape (3,1) into shape (3)
except Exception as e:
        print e

print foo
#+end_src

-- 
"The man who does not read good books has no advantage over 
 the man who cannot read them."
 -- not Mark Twain, maybe a southen librarian in 1910


From walther at mpia.de  Thu Apr 13 14:25:39 2017
From: walther at mpia.de (Michael Walther)
Date: Thu, 13 Apr 2017 11:25:39 -0700
Subject: [AstroPy] Vector Quantities in QTable
In-Reply-To: <20170413115213.j7ltppc4mixjbgsa@broggs.org>
References: <20170413115213.j7ltppc4mixjbgsa@broggs.org>
Message-ID: <fa763132-d84a-6e7a-ea4a-a8200135acb5@mpia.de>

Hi Jeffrey,

Exchange:
foo = QTable(names=('state1','mass'),dtype=('(3,1)f8','f8'))
by:
foo = QTable(names=('state1','mass'),dtype=('(3,)f8','f8'))

Then it works (at least for this simplified case).

Cheers
Michael

On 13.04.2017 04:52, Jeffrey Brent McBeth wrote:
> I'm trying to carry around state vectors for measurements I'm making.
> Some of these measurements come in a large vector that I need to transform easily as a bulk later (calibration)
> I'm struggling to get these measurements to load into QTable (which I assumed would be the right structure for this)
> 
> #+begin_src python
> from astropy.table import QTable
> import astropy.units as u
> import numpy as np
> 
> # I've significantly reduced the number of objects in state1 to keep things simple
> foo = QTable(names=('state1','mass'),dtype=('(3,1)f8','f8'))
> foo['state1'].unit = u.K
> foo['mass'].unit = u.mg
> 
> s0 = [235,240,245]*u.K
> m0 = 5*u.g
> 
> try:
>         foo.add_row((s0,m0)) # Incorrect length for column state1 after inserting [ 235.  240.  245.] K (expected 3,
> got 1)
> except Exception as e:
>         print e
> 
> try:
>         foo.add_row((s0[:,np.newaxis],m0)) # could not broadcast input array from shape (3,1) into shape (3)
> except Exception as e:
>         print e
> 
> print foo
> #+end_src
> 


-- 
Michael Walther
Max-Planck-Institut f?r Astronomie
K?nigstuhl 17
69117 Heidelberg
Germany
Room 	E115
Phone	+49 6221 528 366

From mcbeth at broggs.org  Thu Apr 13 16:29:16 2017
From: mcbeth at broggs.org (Jeffrey Brent McBeth)
Date: Thu, 13 Apr 2017 16:29:16 -0400
Subject: [AstroPy] Vector Quantities in QTable
In-Reply-To: <fa763132-d84a-6e7a-ea4a-a8200135acb5@mpia.de>
References: <20170413115213.j7ltppc4mixjbgsa@broggs.org>
 <fa763132-d84a-6e7a-ea4a-a8200135acb5@mpia.de>
Message-ID: <20170413202916.aywx5adnlshdsnhl@broggs.org>

On Thu, Apr 13, 2017 at 11:25:39AM -0700, Michael Walther wrote:
> Hi Jeffrey,
> 
> Exchange:
> foo = QTable(names=('state1','mass'),dtype=('(3,1)f8','f8'))
> by:
> foo = QTable(names=('state1','mass'),dtype=('(3,)f8','f8'))
> 
> Then it works (at least for this simplified case).

I'm sorry, I cannot reproduce the code working with your substitution.
Both forms work fine (3,1), (3,) if I do not specify the units of the column
Well, I say fine, except the units are stripped off the data and are not preserved in the QTable

If I have the unit definitions before trying to add a row, then I'm back to where I was.

I have astropy 1.3.2 and I've tried on python 2.7.12 and 3.5.2.

Thanks,
 Jeffrey
-- 
"The man who does not read good books has no advantage over 
 the man who cannot read them."
 -- not Mark Twain, maybe a southen librarian in 1910