From gideon.simpson at gmail.com Sun Apr 2 17:41:44 2017 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Sun, 2 Apr 2017 17:41:44 -0400 Subject: [SciPy-User] extracting diagonals efficiently from DIA matrix Message-ID: If I have a matrix that was constructed using spdiags is there a routine that will efficiently extract the diagonals? -gideon -------------- next part -------------- An HTML attachment was scrubbed... URL: From jni.soma at gmail.com Mon Apr 3 00:17:14 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Mon, 3 Apr 2017 00:17:14 -0400 Subject: [SciPy-User] extracting diagonals efficiently from DIA matrix In-Reply-To: References: Message-ID: <33131cde-ab99-450b-b6a9-4d79af32fc9e@Spark> Hey Gideon, The diagonals are in the .data attribute, and the offsets in the .offsets attribute. hth, Juan. On 2 Apr 2017, 5:42 PM -0400, Gideon Simpson , wrote: > If I have a matrix that was constructed using spdiags is there a routine that will efficiently extract the diagonals? > > -gideon > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From gideon.simpson at gmail.com Mon Apr 3 13:04:53 2017 From: gideon.simpson at gmail.com (Gideon Simpson) Date: Mon, 3 Apr 2017 13:04:53 -0400 Subject: [SciPy-User] preserving sparse matrix type under operations Message-ID: <9B3BB074-6977-4891-ACC8-AB547D3BE15F@gmail.com> I was wondering if there?s a way to (cheaply) retain sparsity type of matrices which are of the same sparsity pattern? For instance, if I have two DIA formatted matrices with the same banding, and I want to compute A + s * B, where s is a scalar, it seems to convert everything over to CSR. Is there a way to avoid this? -gideon -------------- next part -------------- An HTML attachment was scrubbed... URL: From clj72 at cornell.edu Mon Apr 3 14:09:32 2017 From: clj72 at cornell.edu (Colin Jermain) Date: Mon, 3 Apr 2017 14:09:32 -0400 Subject: [SciPy-User] ANN: PyMeasure version 0.4.3 released Message-ID: <35a9d62d-1ce4-1dec-0a4a-3a6cbd3b1cef@cornell.edu> Hey Scipy users, We've recently released a new version of PyMeasure , a library that makes it easy to build measurement programs with live-plotting. In under 150 lines of code you get produce a fully-fledged program for running scientific measurements. PyMeasure is accessible to beginner programmers, and programs are easy to maintain. The library also supports 22 scientific instruments out of the box. There are number of great examples and a full set of documentation if you are interested. We would love to hear your feedback and suggestions for improvement. Feel free to email me, or strike up an issue on GitHub . Happy coding, Colin GitHub: https://github.com/ralph-group/pymeasure Documentation: http://pymeasure.readthedocs.io/en/latest/ Examples: https://github.com/ralph-group/pymeasure/tree/master/examples -------------- next part -------------- An HTML attachment was scrubbed... URL: From jni.soma at gmail.com Mon Apr 3 16:00:31 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Mon, 3 Apr 2017 16:00:31 -0400 Subject: [SciPy-User] preserving sparse matrix type under operations In-Reply-To: <9B3BB074-6977-4891-ACC8-AB547D3BE15F@gmail.com> References: <9B3BB074-6977-4891-ACC8-AB547D3BE15F@gmail.com> Message-ID: <77545aef-d2f1-419a-85cb-cc4e08998103@Spark> Hey Gideon, I believe this behaviour exists to prevent having to implement different adding operations for all possible combinations of sparse matrices. You can see in the source code that it?s doing exactly what you suspect: In [24]: type(A) Out[24]: scipy.sparse.dia.dia_matrix In [25]: %psource A.__add__ ? ? def __add__(self, other): ? # self + other ? ? ? ? return self.tocsr().__add__(other) If the banding is indeed the same, you can just add the `.data` attributes together: In [16]: A = sparse.spdiags([[0, 1, 2], [0, 1, 2.]], [0, 1], 3, 3) In [17]: B = sparse.spdiags([[9, 2, 1], [5, 5, 5.]], [0, 1], 3, 3) In [18]: s = 0.5 In [19]: C = A.copy() In [20]: C.data += s * B.data In [21]: C.A Out[21]: array([[ 4.5, ?3.5, ?0. ], ? ? ? ?[ 0. , ?2. , ?4.5], ? ? ? ?[ 0. , ?0. , ?2.5]]) If you are going to be doing a lot of this and want to preserve nice linear algebra syntax, you can monkey-patch as follows In [28]: def add_diag(self, other): ? ? ...: ? ? if (isinstance(other, sparse.dia.dia_matrix) and ? ? ...: ? ? ? ? ? ? np.all(self.offsets == other.offsets)): ? ? ...: ? ? ? ? result = self.copy() ? ? ...: ? ? ? ? result.data += other.data ? ? ...: ? ? ? ? return result ? ? ...: ? ? else: ? ? ...: ? ? ? ? return self.tocsr().__add__(other) ? ? ...: In [29]: sparse.dia.dia_matrix.__add__ = add_diag In [30]: type(A + s * B) Out[30]: scipy.sparse.dia.dia_matrix Hope that helps! Juan. On 3 Apr 2017, 1:05 PM -0400, Gideon Simpson , wrote: > I was wondering if there?s a way to (cheaply) retain sparsity type of matrices which are of the same sparsity pattern? ?For instance, if I have two DIA formatted matrices with the same banding, and I want to compute > > A + s * B, where s is a scalar, it seems to convert everything over to CSR. ?Is there a way to avoid this? > > -gideon > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergio_r at mail.com Tue Apr 4 20:00:57 2017 From: sergio_r at mail.com (Sergio Rojas) Date: Wed, 5 Apr 2017 02:00:57 +0200 Subject: [SciPy-User] A contribution for enhancing your SciPy skills Message-ID: Hello Guys, I am just very happy to have finished my video project with Pack on a brief introduction to Machine Learning via SciPy : https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video Previously, as you might know, we finished this one: https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition https://github.com/rojassergio/Learning-Scipy Hope you can spread the word. Salut, Sergio From arsh840 at gmail.com Tue Apr 4 23:45:41 2017 From: arsh840 at gmail.com (A.S. Khangura) Date: Wed, 5 Apr 2017 09:15:41 +0530 Subject: [SciPy-User] A contribution for enhancing your SciPy skills In-Reply-To: References: Message-ID: On Wed, Apr 5, 2017 at 5:30 AM, Sergio Rojas wrote: > https://github.com/rojassergio/Learning-Scipy I was really looking for something like this. > Hope you can spread the word. Sure, Thanks. -- Thanks Arshpreet Singh Python Developer Web Development/Data Science/Systems Integration Mobile: (91)987 6458 387 https://www.linkedin.com/in/arsh840 Life is a journey, not a destination. From lieemlieem at gmail.com Wed Apr 5 09:16:22 2017 From: lieemlieem at gmail.com (Otto Ngeke) Date: Wed, 5 Apr 2017 15:16:22 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy Message-ID: Hi ALl, I am trying to a three-term gaussian with a specific form to fit a data set. The fit isn't working out as I nicely. Could you help me out? Here attached is my code and the data, along with the fit I obtain: import numpy as np #from scipy.optimize import curve_fit import scipy.optimize as optimize import matplotlib.pyplot as plt #r=np.linspace(0.0e-15,4e-15, 100) data = np.loadtxt('V_lambda_n.dat') r = data[:, 0] V = data[:, 1] std_dev=np.std(data) def func(x, ps1, ps2, ps3, ps4): return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + ps3*np.exp(-(x/ps4)**2) popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], maxfev=10000) #params = optimize.curve_fit(func, ps1, ps2, ps3, ps4) #[ps1, ps2, ps2, ps4] = params[0] p1=plt.plot(r, V, 'bo', label='data') p2=plt.plot(r, func(r, *popt), 'r-', label='fit') plt.xticks(np.linspace(0, 4, 9, endpoint=True)) plt.yticks(np.linspace(-50, 150, 9, endpoint=True)) plt.show() -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: V_lambda_n.dat Type: application/octet-stream Size: 7680 bytes Desc: not available URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gaussian_fit.png Type: image/png Size: 23436 bytes Desc: not available URL: From jni.soma at gmail.com Wed Apr 5 10:45:11 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Wed, 5 Apr 2017 10:45:11 -0400 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: Message-ID: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Your func should get parameters as a vector/list, not as separate parameters. I?m surprised the code doesn?t just fail. So, try: def func(x, params): ? ? ps1, ps2, ps3, ps4 = params ? ? ? I think that should work, but don?t have time to check right now? Good luck! Juan. On 5 Apr 2017, 9:16 AM -0400, Otto Ngeke , wrote: > Hi ALl, > > I am trying to a three-term gaussian with a specific form to fit a data set. The fit isn't working out as I nicely. Could you help me out? Here attached is my code and the data, along with the fit I obtain: > > import numpy as np > > #from scipy.optimize import curve_fit > import scipy.optimize as optimize > > import matplotlib.pyplot as plt > > #r=np.linspace(0.0e-15,4e-15, 100) > > data = np.loadtxt('V_lambda_n.dat') > r = data[:, 0] > V = data[:, 1] > > std_dev=np.std(data) > > def func(x, ps1, ps2, ps3, ps4): > ? ? return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + ps3*np.exp(-(x/ps4)**2) > > popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], maxfev=10000) > > #params = optimize.curve_fit(func, ps1, ps2, ps3, ps4) > > #[ps1, ps2, ps2, ps4] = params[0] > > p1=plt.plot(r, V, 'bo', label='data') > p2=plt.plot(r, func(r, *popt), 'r-', label='fit') > > plt.xticks(np.linspace(0, 4, 9, endpoint=True)) > plt.yticks(np.linspace(-50, 150, 9, endpoint=True)) > plt.show() > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From lieemlieem at gmail.com Wed Apr 5 11:24:57 2017 From: lieemlieem at gmail.com (Otto Ngeke) Date: Wed, 5 Apr 2017 17:24:57 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> References: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Message-ID: Hi Juan, Here is my new function as you suggested: def func(x, *params): for i in range(0, len(params), 2): ps1=params[i] ps2=params[i+1] return ps1*np.exp(-(x/ps2)**2) I still get the same fit as before. However, I am worried because my Gaussians are such that the amplitude of the second Gaussian term is the standard deviation of the first: some kind of coupling between the terms. I don't know if this new function definition actually represents that property. On Wed, Apr 5, 2017 at 4:45 PM, Juan Nunez-Iglesias wrote: > Your func should get parameters as a vector/list, not as separate > parameters. I?m surprised the code doesn?t just fail. So, try: > > def func(x, params): > ps1, ps2, ps3, ps4 = params > ? > > I think that should work, but don?t have time to check right now? Good > luck! > > Juan. > > On 5 Apr 2017, 9:16 AM -0400, Otto Ngeke , wrote: > > Hi ALl, > > I am trying to a three-term gaussian with a specific form to fit a data > set. The fit isn't working out as I nicely. Could you help me out? Here > attached is my code and the data, along with the fit I obtain: > > import numpy as np > > #from scipy.optimize import curve_fit > import scipy.optimize as optimize > > import matplotlib.pyplot as plt > > #r=np.linspace(0.0e-15,4e-15, 100) > > data = np.loadtxt('V_lambda_n.dat') > r = data[:, 0] > V = data[:, 1] > > std_dev=np.std(data) > > def func(x, ps1, ps2, ps3, ps4): > return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + > ps3*np.exp(-(x/ps4)**2) > > popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], > maxfev=10000) > > #params = optimize.curve_fit(func, ps1, ps2, ps3, ps4) > > #[ps1, ps2, ps2, ps4] = params[0] > > p1=plt.plot(r, V, 'bo', label='data') > p2=plt.plot(r, func(r, *popt), 'r-', label='fit') > > plt.xticks(np.linspace(0, 4, 9, endpoint=True)) > plt.yticks(np.linspace(-50, 150, 9, endpoint=True)) > plt.show() > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From davidmenhur at gmail.com Wed Apr 5 11:41:05 2017 From: davidmenhur at gmail.com (=?UTF-8?B?RGHPgGlk?=) Date: Wed, 5 Apr 2017 17:41:05 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Message-ID: On 5 April 2017 at 17:24, Otto Ngeke wrote: > I still get the same fit as before. However, I am worried because my > Gaussians are such that the amplitude of the second Gaussian term is the > standard deviation of the first: some kind of coupling between the terms. I > don't know if this new function definition actually represents that > property. It should be: def func(x, params): ps1, ps2, ps3, ps4 = params return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + ps3*np.exp(-(x/ps4)**2) Note that in your initialisation you have a symmetry between the first and the third Gaussian: popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], maxfev=10000) So unless curve_fit adds some random noise, the two are going to be always the same because you cannot break the symmetry. Instead, give them different values, for example: [50, 1.2 * std_dev, 30, std_dev / 1.2] (I have no idea if this particular values make sense, but they do will allow you to get two independent Gaussians, not too far from your initial guess.) For hard fits (and arbitrary fits are surprisingly hard!), iminuit is a quite powerful option. /David. -------------- next part -------------- An HTML attachment was scrubbed... URL: From newville at cars.uchicago.edu Wed Apr 5 11:41:00 2017 From: newville at cars.uchicago.edu (Matt Newville) Date: Wed, 5 Apr 2017 10:41:00 -0500 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: Message-ID: Otto, I think the simplest conclusion is that your model function (three Gaussians, each centered at x=0 and with the amplitude of one constrained to be the width of the next) does not easily match your data. If it is, you must need at least one of the Gaussians to have a negative amplitude or else you'll never get values below zero. Are you sure this is a good model for your data? On Wed, Apr 5, 2017 at 8:16 AM, Otto Ngeke wrote: > Hi ALl, > > I am trying to a three-term gaussian with a specific form to fit a data > set. The fit isn't working out as I nicely. Could you help me out? Here > attached is my code and the data, along with the fit I obtain: > > import numpy as np > > #from scipy.optimize import curve_fit > import scipy.optimize as optimize > > import matplotlib.pyplot as plt > > #r=np.linspace(0.0e-15,4e-15, 100) > > data = np.loadtxt('V_lambda_n.dat') > r = data[:, 0] > V = data[:, 1] > > std_dev=np.std(data) > > def func(x, ps1, ps2, ps3, ps4): > return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + > ps3*np.exp(-(x/ps4)**2) > > popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], > maxfev=10000) > > #params = optimize.curve_fit(func, ps1, ps2, ps3, ps4) > > #[ps1, ps2, ps2, ps4] = params[0] > > p1=plt.plot(r, V, 'bo', label='data') > p2=plt.plot(r, func(r, *popt), 'r-', label='fit') > > plt.xticks(np.linspace(0, 4, 9, endpoint=True)) > plt.yticks(np.linspace(-50, 150, 9, endpoint=True)) > plt.show() > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From newville at cars.uchicago.edu Wed Apr 5 12:39:40 2017 From: newville at cars.uchicago.edu (Matt Newville) Date: Wed, 5 Apr 2017 11:39:40 -0500 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Message-ID: On Wed, Apr 5, 2017 at 10:41 AM, Da?id wrote: > > On 5 April 2017 at 17:24, Otto Ngeke wrote: > >> I still get the same fit as before. However, I am worried because my >> Gaussians are such that the amplitude of the second Gaussian term is the >> standard deviation of the first: some kind of coupling between the terms. I >> don't know if this new function definition actually represents that >> property. > > > It should be: > > def func(x, params): > ps1, ps2, ps3, ps4 = params > return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + > ps3*np.exp(-(x/ps4)**2) > > > Note that in your initialisation you have a symmetry between the first and > the third Gaussian: > > popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, std_dev], > maxfev=10000) > > So unless curve_fit adds some random noise, the two are going to be always > the same because you cannot break the symmetry. > That is not correct. Setting multiple parameters to the same initial values does not make them always have the same value. Each parameter here is separate, and there is no symmetry here at all. Instead, give them different values, for example: > > [50, 1.2 * std_dev, 30, std_dev / 1.2] > > (I have no idea if this particular values make sense, but they do will > allow you to get two independent Gaussians, not too far from your initial > guess.) > > Certainly giving initial values that are close as possible is a good idea, but they do not need to have values that are unique. > For hard fits (and arbitrary fits are surprisingly hard!), iminuit is a > quite powerful option. > > Lmfit might also be useful here. But the basic problem is that the model of multiple Gaussians with the same center will have a very hard time describing the data shown. --Matt -------------- next part -------------- An HTML attachment was scrubbed... URL: From rain.private at gmail.com Wed Apr 5 22:03:10 2017 From: rain.private at gmail.com (Matthew Shawn) Date: Thu, 6 Apr 2017 10:03:10 +0800 Subject: [SciPy-User] Hi everyone , I wanna consult one question about linear calculate Message-ID: I have little basic knowledge about python, can build CMS with Django ,install,upgrade,uninstall python modules. I occur one question in fact work, and I fee Scipy can fix the question ,but I can't understand how to use Scipy and using which module or function even I have read two books about Numpy and Scipy in two weeks. So I have to seek help form somebody. Here is the question's description :: We have 50 numbers [A1,A2,A3,A4,A5,A6,A7,A8,A9...A50] and we also know the average of these numbers , for example the average is 3219, we will group the 50 numbers into 11 groups,So the average number 3218 and the sum of each group also will have 11 subtractions , and in the sum of 11 subtract abs value , the minimize one is our target. we wanna find the minimize sum of the 11 subtractions, and the group style of the 50 numbers. 1. We have 50 numbers [A1, A2, A3, ..., A49, A50] and the avg of these numbers: 3219; 2. Group these numbers in 11 groups : S1 = x1_1*A1 + x2_1*A2 + x3_1*A3 + ... + x49_1*A49 + x50_1*A50; S2 = x1_2*A1 + x2_2*A2 + x3_2*A3 + ... + x49_2*A49 + x50_2*A50; S3 = x1_3*A1 + x2_3*A2 + x3_3*A3 + ... + x49_3*A49 + x50_3*A50; ..... S11 = x1_11*A1 + x2_11*A2 + x3_11*A3 + ... + x49_11*A49 + x50_11*A50; 3. F(X) = abs(S1-3219) + abs(S2-3219) + abs(S3-3219) + ... +abs(S11-3219) 4. LIMIT each one of the 50 numbers only in one group: x1_1 + x1_2 + x1_3+ x1_4 + x1_5 + x1_6 + x1_7 + x1_8 + x1_9 + x1_10 + x1_11 = 1 x2_1 + x2_2 + x2_3+ x2_4 + x2_5 + x2_6 + x2_7 + x2_8 + x2_9 + x2_10 + x2_11 = 1 x3_1 + ...... + x2_11 = 1 ... x50_1 + ...... + x50_11 = 1 5. LIMIT each one of the x variables only can be 0 or 1; x1_1 = {0,1} x1_2 = {0,1} ..... x50_11 = {0,1} 6.solve the minimize function: minimize F(X) Will anyone teach me how to solve this? I will be very thankful for your reply. -------------- next part -------------- An HTML attachment was scrubbed... URL: From lieemlieem at gmail.com Thu Apr 6 02:56:56 2017 From: lieemlieem at gmail.com (Otto Ngeke) Date: Thu, 6 Apr 2017 08:56:56 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Message-ID: I tried to pay more attention to the initial guess as suggested, and found a very nearly close fit! The new initial guess I used is p0=[V.max(), std_dev, V.max(), 2] See attached for new fit. I then used the resulting values for my parameters ps1, ps2,ps3, ps4 as a new initial guess, but surprisingly, there is no improvement in the fit. It just remains the same, even after repeatedly using new parameters as initial guess for like five successive times. Any way I can improve my guess? On Wed, Apr 5, 2017 at 6:39 PM, Matt Newville wrote: > > > On Wed, Apr 5, 2017 at 10:41 AM, Da?id wrote: > >> >> On 5 April 2017 at 17:24, Otto Ngeke wrote: >> >>> I still get the same fit as before. However, I am worried because my >>> Gaussians are such that the amplitude of the second Gaussian term is the >>> standard deviation of the first: some kind of coupling between the terms. I >>> don't know if this new function definition actually represents that >>> property. >> >> >> It should be: >> >> def func(x, params): >> ps1, ps2, ps3, ps4 = params >> return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + >> ps3*np.exp(-(x/ps4)**2) >> >> >> Note that in your initialisation you have a symmetry between the first >> and the third Gaussian: >> >> popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, >> std_dev], maxfev=10000) >> >> So unless curve_fit adds some random noise, the two are going to be >> always the same because you cannot break the symmetry. >> > > That is not correct. Setting multiple parameters to the same initial > values does not make them always have the same value. Each parameter here > is separate, and there is no symmetry here at all. > > Instead, give them different values, for example: >> >> [50, 1.2 * std_dev, 30, std_dev / 1.2] >> >> (I have no idea if this particular values make sense, but they do will >> allow you to get two independent Gaussians, not too far from your initial >> guess.) >> >> > Certainly giving initial values that are close as possible is a good idea, > but they do not need to have values that are unique. > > >> For hard fits (and arbitrary fits are surprisingly hard!), iminuit is a >> quite powerful option. >> >> > Lmfit might also be useful here. But the basic problem is that the model > of multiple Gaussians with the same center will have a very hard time > describing the data shown. > > --Matt > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- A non-text attachment was scrubbed... Name: gaussian_fit2.png Type: image/png Size: 24856 bytes Desc: not available URL: From andyfaff at gmail.com Thu Apr 6 03:39:51 2017 From: andyfaff at gmail.com (Andrew Nelson) Date: Thu, 6 Apr 2017 17:39:51 +1000 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy In-Reply-To: References: <32ed2045-bbbd-4f07-9a60-40d20c3429b7@Spark> Message-ID: If you are using the output parameters from a curve fit as the input to a new curve fit then you won't see any improvement because you will already be in a local/ global minimum in chi2 space. On 6 Apr 2017 4:58 pm, "Otto Ngeke" wrote: I tried to pay more attention to the initial guess as suggested, and found a very nearly close fit! The new initial guess I used is p0=[V.max(), std_dev, V.max(), 2] See attached for new fit. I then used the resulting values for my parameters ps1, ps2,ps3, ps4 as a new initial guess, but surprisingly, there is no improvement in the fit. It just remains the same, even after repeatedly using new parameters as initial guess for like five successive times. Any way I can improve my guess? On Wed, Apr 5, 2017 at 6:39 PM, Matt Newville wrote: > > > On Wed, Apr 5, 2017 at 10:41 AM, Da?id wrote: > >> >> On 5 April 2017 at 17:24, Otto Ngeke wrote: >> >>> I still get the same fit as before. However, I am worried because my >>> Gaussians are such that the amplitude of the second Gaussian term is the >>> standard deviation of the first: some kind of coupling between the terms. I >>> don't know if this new function definition actually represents that >>> property. >> >> >> It should be: >> >> def func(x, params): >> ps1, ps2, ps3, ps4 = params >> return ps1*np.exp(-(x/ps2)**2) + ps2*np.exp(-(x/ps3)**2) + >> ps3*np.exp(-(x/ps4)**2) >> >> >> Note that in your initialisation you have a symmetry between the first >> and the third Gaussian: >> >> popt, pcov = optimize.curve_fit(func, r, V, p0=[50, std_dev, 50, >> std_dev], maxfev=10000) >> >> So unless curve_fit adds some random noise, the two are going to be >> always the same because you cannot break the symmetry. >> > > That is not correct. Setting multiple parameters to the same initial > values does not make them always have the same value. Each parameter here > is separate, and there is no symmetry here at all. > > Instead, give them different values, for example: >> >> [50, 1.2 * std_dev, 30, std_dev / 1.2] >> >> (I have no idea if this particular values make sense, but they do will >> allow you to get two independent Gaussians, not too far from your initial >> guess.) >> >> > Certainly giving initial values that are close as possible is a good idea, > but they do not need to have values that are unique. > > >> For hard fits (and arbitrary fits are surprisingly hard!), iminuit is a >> quite powerful option. >> >> > Lmfit might also be useful here. But the basic problem is that the model > of multiple Gaussians with the same center will have a very hard time > describing the data shown. > > --Matt > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > _______________________________________________ SciPy-User mailing list SciPy-User at python.org https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergio_r at mail.com Thu Apr 6 17:46:47 2017 From: sergio_r at mail.com (Sergio Rojas) Date: Thu, 6 Apr 2017 23:46:47 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy (Otto Ngeke) In-Reply-To: References: Message-ID: ------------------------------ Message: 3 Date: Thu, 6 Apr 2017 08:56:56 +0200 From: Otto Ngeke To: SciPy Users List Subject: Re: [SciPy-User] Three-term gaussian fit to gaussian data using scipy Message-ID: Content-Type: text/plain; charset="utf-8" I tried to pay more attention to the initial guess as suggested, and found a very nearly close fit! The new initial guess I used is p0=[V.max(), std_dev, V.max(), 2] See attached for new fit. ... ... ------------------- In case a set of gaussians aren't a constraint to your fit, you might want to play a bit with something like, return ps1*np.exp(-(x/ps2)**2)*np.sin(x)**2 + ps2*np.exp(-(x/ps3)**2) + ps3*np.exp(-(x/ps4)**2) and the set of parameters: p0=[V.max(), std_dev, V.min(), 2] Reagrds, Sergio Non-linear fitting stuff: https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video -------------- next part -------------- A non-text attachment was scrubbed... Name: out.png Type: image/png Size: 23027 bytes Desc: not available URL: From lieemlieem at gmail.com Fri Apr 7 02:57:15 2017 From: lieemlieem at gmail.com (Otto Ngeke) Date: Fri, 7 Apr 2017 08:57:15 +0200 Subject: [SciPy-User] Three-term gaussian fit to gaussian data using scipy (Otto Ngeke) In-Reply-To: References: Message-ID: HI Sergio, Thanks for the new fit. I am in a bit of a difficulty because the set of Gaussians I have chosen is to go as input to another software, and that software only accepts the Gaussians in that specific form. If I modify as you suggested, then I can no longer use the software. On Thu, Apr 6, 2017 at 11:46 PM, Sergio Rojas wrote: > > ------------------------------ > > Message: 3 > Date: Thu, 6 Apr 2017 08:56:56 +0200 > From: Otto Ngeke > To: SciPy Users List > Subject: Re: [SciPy-User] Three-term gaussian fit to gaussian data > using scipy > Message-ID: > > Content-Type: text/plain; charset="utf-8" > > I tried to pay more attention to the initial guess as suggested, and found > a very nearly close fit! The new initial guess I used is > > p0=[V.max(), std_dev, V.max(), 2] > > See attached for new fit. > > ... > ... > ------------------- > > In case a set of gaussians aren't a constraint to your fit, > > you might want to play a bit with something like, > > return ps1*np.exp(-(x/ps2)**2)*np.sin(x)**2 + ps2*np.exp(-(x/ps3)**2) + > ps3*np.exp(-(x/ps4)**2) > > and the set of parameters: > > p0=[V.max(), std_dev, V.min(), 2] > > Reagrds, > > Sergio > Non-linear fitting stuff: > https://www.packtpub.com/big-data-and-business-intelligence/numerical-and- > scientific-computing-scipy-video > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From gdmcbain at protonmail.com Fri Apr 7 16:33:44 2017 From: gdmcbain at protonmail.com (G. D. McBain) Date: Fri, 07 Apr 2017 16:33:44 -0400 Subject: [SciPy-User] Hi everyone , I wanna consult one question about linear calculate In-Reply-To: References: Message-ID: This isn't a SciPy-User question, it would be better asked on a mathematical list. But now that you're here. I won't answer the question here as it's off-topic, but to help guide you to a more appropriate forum I'll note that this is not a problem of linear calculation since your cost function is nonlinear due to the use of abs. Moreover the constraint that the coefficients be boolean rather than real put this outside the realm of numerical linear algebra. Instead it's a problem of "combinatorial optimization", a "partition problem", like the "knapsack problem". Those terms might help you search for a solution in the literature or for a suitable forum. For a link which is on a more suitable forum and also contains answers to a closely related problem which you might find useful, try: http://stackoverflow.com/questions/17497164/split-data-into-groups-of-equal-means -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.molnar at sbcglobal.net Sat Apr 15 06:29:14 2017 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Sat, 15 Apr 2017 06:29:14 -0400 Subject: [SciPy-User] Molecular Index Calculation Message-ID: <58F1F5FA.5030102@sbcglobal.net> I am a newcomer to Python and am attempting to translate a FORTRAN program that I wrote about 20 years ago into Python, specifically v-3.6. Let me say that I am not asking someone to write the program for me, but only to point me is the right direction. So far, I have bumbled my way to the point that I can get all of the input data resulting from a quantum mechanical calculation of a very simple organic molecule in to a Python program, but am encountering problems with processing the data. The equation that I a want to evaluate (the one that I programmed in FORTRAN) is equation (7) in the attached scan of one of the pages of the following document. Stephen P. Molnar and James W. King, Theory and Applications of the Integrated Molecular Transform and the Normalized Molecular Moment Structure Descriptors: QSAR and QSPR Paradigms, Int. J Quantum Chem., 85, 662 (2001). the equation is I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * sin s*r_sub_ij ----------------- Equation (7) s*r_sub_ij What I have managed to do so far is for a simple organic molecule containing 5 atoms (CHFClBr): import numpy as np import numpy, pandas, scipy.spatial.distance as dist r=[] #distances between pairs of atoms z = [] #atomic numbers of atoms Z_dist = [] #products of successive atomic numbers I = [] #vector calculated in equation (7) start=1 finish=31 points=300 s = np.linspace(start,finish,points) name = input("Enter Molecule ID: ") name = str(name) name_in = name+'.dat' df = pandas.read_table(name_in, skiprows=2, sep=" ", skipinitialspace=True) z = numpy.array(df["ZA"]) N = numpy.ma.size(z) ## of atoms in molecule a = numpy.array(df[["X", "Y", "Z"]]) dist.squareform(dist.pdist(a, "euclidean")) anrows, ancols = np.shape(a) a_new = a.reshape(anrows, 1, ancols) diff = a_new - a r = (diff ** 2).sum(2) r = np.sqrt(r) for j in range(0,N-1): for k in range(j+1,N): Z_diff = (z[j]*z[k]) For the test molecule I get the following: MASS = [ 12.011 1.008 79.9 35.453 18.998] (Atomic weights of atoms) r: [ 0. 2.059801 3.60937686 3.32591826 2.81569212] [ 2.059801 0. 4.71452879 4.45776445 4.00467382] [ 3.60937686 4.71452879 0. 5.66500917 5.26602175] [ 3.32591826 4.45776445 5.66500917 0. 5.02324896] [ 2.81569212 4.00467382 5.26602175 5.02324896 0. ] z: 6.0 210.0 102.0 54.0 35.0 17.0 9.0 595.0 315.0 153.0 I have checked these calculations with a spreadsheet calculation, consequently I'm correct as far as I have gotten. However, my problem is calculating the value of I for the 300 x-coordinate values required for the molecular index calculation. The problem that I am currently having is calculating the product of the pre-sine function and the sine term for the 300 values of s. Unfortunately, the wealth of function in Python makes it difficult for me to ascertain just how to proceed. It seems that I have the problem of seeing the trees because of the size of the forest. At this point Goggling only makes my confusion deeper. Any pointers in the correct direction will be greatly appreciated. Thanks in advance. -- Stephen P. Molnar, Ph.D. Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1 -------------- next part -------------- A non-text attachment was scrubbed... Name: I_Formula.pdf Type: application/pdf Size: 94038 bytes Desc: not available URL: From hturesson at gmail.com Sat Apr 15 16:07:40 2017 From: hturesson at gmail.com (Hjalmar Turesson) Date: Sat, 15 Apr 2017 16:07:40 -0400 Subject: [SciPy-User] Molecular Index Calculation In-Reply-To: <58F1F5FA.5030102@sbcglobal.net> References: <58F1F5FA.5030102@sbcglobal.net> Message-ID: Hi Stephen, I see that N = 10 (ie. the length of Z), and that r is indexed by i and j, which run up to N. However, r is only a 5x5 array. Is this correct? Otherwise, something like this might work: import numpy as np start=1 finish=31 points=300 s = np.linspace(start, finish, points) MASS = np.array([12.011, 1.008, 79.9, 35.453, 18.998]) r = np.array([[0., 2.059801, 3.60937686, 3.32591826, 2.81569212], [2.059801, 0., 4.71452879, 4.45776445, 4.00467382], [3.60937686, 4.71452879, 0., 5.66500917, 5.26602175], [3.32591826, 4.45776445, 5.66500917, 0., 5.02324896], [2.81569212, 4.00467382, 5.26602175, 5.02324896, 0.]]) r[r == 0.] = 0.0001 Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, 153.0]) N = Z.size I = np.zeros(s.shape) for i in range(1, N): for j in range(j): I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) Do you have an example of the correct output? Best, Hjalmar On Sat, Apr 15, 2017 at 6:29 AM, Stephen P. Molnar wrote: > I am a newcomer to Python and am attempting to translate a FORTRAN program > that I wrote about 20 years ago into Python, specifically v-3.6. > > Let me say that I am not asking someone to write the program for me, but > only to point me is the right direction. > > So far, I have bumbled my way to the point that I can get all of the input > data resulting from a quantum mechanical calculation of a very simple > organic molecule in to a Python program, but am encountering problems with > processing the data. > > The equation that I a want to evaluate (the one that I programmed in > FORTRAN) is equation (7) in the attached scan of one of the pages of the > following document. > > Stephen P. Molnar and James W. King, Theory and Applications of the > Integrated Molecular Transform and the Normalized Molecular Moment > Structure Descriptors: QSAR and QSPR Paradigms, Int. J Quantum Chem., 85, > 662 (2001). > > the equation is > > I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * > > sin s*r_sub_ij > ----------------- Equation (7) > s*r_sub_ij > > What I have managed to do so far is for a simple organic molecule > containing 5 atoms (CHFClBr): > > import numpy as np > import numpy, pandas, scipy.spatial.distance as dist > > r=[] #distances between pairs of atoms > z = [] #atomic numbers of atoms > Z_dist = [] #products of successive atomic numbers > I = [] #vector calculated in equation (7) > > start=1 > finish=31 > points=300 > s = np.linspace(start,finish,points) > > > name = input("Enter Molecule ID: ") > name = str(name) > name_in = name+'.dat' > > df = pandas.read_table(name_in, skiprows=2, sep=" ", skipinitialspace=True) > z = numpy.array(df["ZA"]) > N = numpy.ma.size(z) ## of atoms in molecule > a = numpy.array(df[["X", "Y", "Z"]]) > dist.squareform(dist.pdist(a, "euclidean")) > anrows, ancols = np.shape(a) > a_new = a.reshape(anrows, 1, ancols) > diff = a_new - a > > r = (diff ** 2).sum(2) > r = np.sqrt(r) > > for j in range(0,N-1): > for k in range(j+1,N): > Z_diff = (z[j]*z[k]) > > > For the test molecule I get the following: > > MASS = [ 12.011 1.008 79.9 35.453 18.998] (Atomic weights of > atoms) > > r: [ 0. 2.059801 3.60937686 3.32591826 2.81569212] > [ 2.059801 0. 4.71452879 4.45776445 4.00467382] > [ 3.60937686 4.71452879 0. 5.66500917 5.26602175] > [ 3.32591826 4.45776445 5.66500917 0. 5.02324896] > [ 2.81569212 4.00467382 5.26602175 5.02324896 0. ] > > z: > > 6.0 > 210.0 > 102.0 > 54.0 > 35.0 > 17.0 > 9.0 > 595.0 > 315.0 > 153.0 > > I have checked these calculations with a spreadsheet calculation, > consequently I'm correct as far as I have gotten. > > However, my problem is calculating the value of I for the 300 x-coordinate > values required for the molecular index calculation. > > The problem that I am currently having is calculating the product of the > pre-sine function and the sine term for the 300 values of s. > > Unfortunately, the wealth of function in Python makes it difficult for me > to ascertain just how to proceed. It seems that I have the problem of > seeing the trees because of the size of the forest. At this point Goggling > only makes my confusion deeper. Any pointers in the correct direction will > be greatly appreciated. > > Thanks in advance. > > -- > Stephen P. Molnar, Ph.D. Life is a fuzzy set > www.molecular-modeling.net Stochastic and multivariate > (614)312-7528 (c) > Skype: smolnar1 > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.molnar at sbcglobal.net Sun Apr 16 08:25:47 2017 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Sun, 16 Apr 2017 08:25:47 -0400 Subject: [SciPy-User] Molecular Index Calculation In-Reply-To: References: <58F1F5FA.5030102@sbcglobal.net> Message-ID: <58F362CB.8020005@sbcglobal.net> On 04/15/2017 04:07 PM, Hjalmar Turesson wrote: > Hi Stephen, > > > I see that N = 10 (ie. the length of Z), and that r is indexed by i and > j, which run up to N. However, r is only a 5x5 array. Is this correct? > > Otherwise, something like this might work: > > import numpy as np > > start=1 > finish=31 > points=300 > s = np.linspace(start, finish, points) > > MASS = np.array([12.011, 1.008, 79.9, 35.453, 18.998]) > > r = np.array([[0., 2.059801, 3.60937686, 3.32591826, 2.81569212], > [2.059801, 0., 4.71452879, 4.45776445, 4.00467382], > [3.60937686, 4.71452879, 0., 5.66500917, 5.26602175], > [3.32591826, 4.45776445, 5.66500917, 0., 5.02324896], > [2.81569212, 4.00467382, 5.26602175, 5.02324896, 0.]]) > > r[r == 0.] = 0.0001 > > Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, > 153.0]) > > N = Z.size > > I = np.zeros(s.shape) > > for i in range(1, N): > for j in range(j): > I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) > > > Do you have an example of the correct output? > > Best, > Hjalmar > > On Sat, Apr 15, 2017 at 6:29 AM, Stephen P. Molnar > > wrote: > > I am a newcomer to Python and am attempting to translate a FORTRAN > program that I wrote about 20 years ago into Python, specifically v-3.6. > > Let me say that I am not asking someone to write the program for me, > but only to point me is the right direction. > > So far, I have bumbled my way to the point that I can get all of the > input data resulting from a quantum mechanical calculation of a very > simple organic molecule in to a Python program, but am encountering > problems with processing the data. > > The equation that I a want to evaluate (the one that I programmed in > FORTRAN) is equation (7) in the attached scan of one of the pages of > the following document. > > Stephen P. Molnar and James W. King, Theory and Applications of the > Integrated Molecular Transform and the Normalized Molecular Moment > Structure Descriptors: QSAR and QSPR Paradigms, Int. J Quantum > Chem., 85, 662 (2001). > > the equation is > > I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * > > sin s*r_sub_ij > ----------------- Equation (7) > s*r_sub_ij > > What I have managed to do so far is for a simple organic molecule > containing 5 atoms (CHFClBr): > > import numpy as np > import numpy, pandas, scipy.spatial.distance as dist > > r=[] #distances between pairs of atoms > z = [] #atomic numbers of atoms > Z_dist = [] #products of successive atomic numbers > I = [] #vector calculated in equation (7) > > start=1 > finish=31 > points=300 > s = np.linspace(start,finish,points) > > > name = input("Enter Molecule ID: ") > name = str(name) > name_in = name+'.dat' > > df = pandas.read_table(name_in, skiprows=2, sep=" ", > skipinitialspace=True) > z = numpy.array(df["ZA"]) > N = numpy.ma.size(z) ## of atoms in molecule > a = numpy.array(df[["X", "Y", "Z"]]) > dist.squareform(dist.pdist(a, "euclidean")) > anrows, ancols = np.shape(a) > a_new = a.reshape(anrows, 1, ancols) > diff = a_new - a > > r = (diff ** 2).sum(2) > r = np.sqrt(r) > > for j in range(0,N-1): > for k in range(j+1,N): > Z_diff = (z[j]*z[k]) > > > For the test molecule I get the following: > > MASS = [ 12.011 1.008 79.9 35.453 18.998] (Atomic weights > of atoms) > > r: [ 0. 2.059801 3.60937686 3.32591826 2.81569212] > [ 2.059801 0. 4.71452879 4.45776445 4.00467382] > [ 3.60937686 4.71452879 0. 5.66500917 5.26602175] > [ 3.32591826 4.45776445 5.66500917 0. 5.02324896] > [ 2.81569212 4.00467382 5.26602175 5.02324896 0 > . ] > > z: > > 6.0 > 210.0 > 102.0 > 54.0 > 35.0 > 17.0 > 9.0 > 595.0 > 315.0 > 153.0 > > I have checked these calculations with a spreadsheet calculation, > consequently I'm correct as far as I have gotten. > > However, my problem is calculating the value of I for the 300 > x-coordinate values required for the molecular index calculation. > > The problem that I am currently having is calculating the product of > the pre-sine function and the sine term for the 300 values of s. > > Unfortunately, the wealth of function in Python makes it difficult > for me to ascertain just how to proceed. It seems that I have the > problem of seeing the trees because of the size of the forest. At > this point Goggling only makes my confusion deeper. Any pointers in > the correct direction will be greatly appreciated. > > Thanks in advance. > > -- > Stephen P. Molnar, Ph.D. Life is a fuzzy set > www.molecular-modeling.net > Stochastic and multivariate > (614)312-7528 (c) > Skype: smolnar1 > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > Hjalmar Thanks for your response. I'm sorry that my original request wasn't a bit clearer. I presume that the paper I attached to the request help was deleted before the email was posted to the list. Here is how I incorporated your suggestion in the code. import numpy as np import numpy, pandas, scipy.spatial.distance as dist r = [] s=[] z = [] Z = [] I = [] start=1 finish=31 points=300 s = np.linspace(start,finish,points) np.savetxt('Number of Points',s,delimiter=' ') name = input("Enter Molecule ID: ") name = str(name) name_in = name+'.dat' df = pandas.read_table(name_in, skiprows=2, sep=" ", skipinitialspace=True) Z = numpy.array(df["ZA"]) print('z = ',z) N = numpy.ma.size(z) a = numpy.array(df[["X", "Y", "Z"]]) dist.squareform(dist.pdist(a, "euclidean")) anrows, ancols = np.shape(a) a_new = a.reshape(anrows, 1, ancols) diff = a_new - a r = (diff ** 2).sum(2) r = np.sqrt(z) r[r == 0.] = 0.0001 N = Z.size I = np.zeros(s.shape) for i in range(1, N): for j in range(j): I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) print('I: ',I) np.savetxt('I',I,delimiter=' ') The for j in range(j): has a marginal note (I'm using the Spyder IDE) "undefined name 'y' " and I is a vector with three hundred 0.0 terms. also, please note that I removed 'Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, 153.0])' from your suggestion as that is the result of the per-sine term 'Z[i] * Z[j]' Regards, Steve -- Stephen P. Molnar, Ph.D. Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1 From david.mikolas1 at gmail.com Sun Apr 16 11:56:17 2017 From: david.mikolas1 at gmail.com (David Mikolas) Date: Sun, 16 Apr 2017 23:56:17 +0800 Subject: [SciPy-User] Molecular Index Calculation In-Reply-To: <58F362CB.8020005@sbcglobal.net> References: <58F1F5FA.5030102@sbcglobal.net> <58F362CB.8020005@sbcglobal.net> Message-ID: I escaped a cluttered Fortran/C/Pascal/Basic/Perl jumble by "living" in stackoverflow. Mostly by reading questions and answers, and learning how not to get my questions closed in milliseconds. Break your program into small pieces, and look for how each piece might be done differently. If you try to find ways to do things without ever manually checking the sizes of arrays, you'll naturally be guided towards "pythonic" methods and techniques. There is almost always a better way. Computers are so fast these days, don't worry about ultimate speed until you really hit a brick wall. Letting python, numpy, or scipy methods handle housekeeping for you means you are usually defaulting to very well written compiled routines. So for example, let python do the thinking... r = np.sqrt((diff ** 2).sum(2)) r[r < 0.0001] = 0.0001 # or just r = np.maximum(np.sqrt((diff ** 2).sum(2)), 0.0001) # and Zij = Z * Z[:, None] triangle = np.triu(np.ones_like(Zij), k=1) I = Zij*trangle.sum() # or just I = np.triu(Z * Z[:, None], k=1).sum() On Sun, Apr 16, 2017 at 8:25 PM, Stephen P. Molnar wrote: > On 04/15/2017 04:07 PM, Hjalmar Turesson wrote: > >> Hi Stephen, >> >> >> I see that N = 10 (ie. the length of Z), and that r is indexed by i and >> j, which run up to N. However, r is only a 5x5 array. Is this correct? >> >> Otherwise, something like this might work: >> >> import numpy as np >> >> start=1 >> finish=31 >> points=300 >> s = np.linspace(start, finish, points) >> >> MASS = np.array([12.011, 1.008, 79.9, 35.453, 18.998]) >> >> r = np.array([[0., 2.059801, 3.60937686, 3.32591826, 2.81569212], >> [2.059801, 0., 4.71452879, 4.45776445, 4.00467382], >> [3.60937686, 4.71452879, 0., 5.66500917, 5.26602175], >> [3.32591826, 4.45776445, 5.66500917, 0., 5.02324896], >> [2.81569212, 4.00467382, 5.26602175, 5.02324896, 0.]]) >> >> r[r == 0.] = 0.0001 >> >> Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, >> 153.0]) >> >> N = Z.size >> >> I = np.zeros(s.shape) >> >> for i in range(1, N): >> for j in range(j): >> I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) >> >> >> Do you have an example of the correct output? >> >> Best, >> Hjalmar >> >> On Sat, Apr 15, 2017 at 6:29 AM, Stephen P. Molnar >> > wrote: >> >> I am a newcomer to Python and am attempting to translate a FORTRAN >> program that I wrote about 20 years ago into Python, specifically >> v-3.6. >> >> Let me say that I am not asking someone to write the program for me, >> but only to point me is the right direction. >> >> So far, I have bumbled my way to the point that I can get all of the >> input data resulting from a quantum mechanical calculation of a very >> simple organic molecule in to a Python program, but am encountering >> problems with processing the data. >> >> The equation that I a want to evaluate (the one that I programmed in >> FORTRAN) is equation (7) in the attached scan of one of the pages of >> the following document. >> >> Stephen P. Molnar and James W. King, Theory and Applications of the >> Integrated Molecular Transform and the Normalized Molecular Moment >> Structure Descriptors: QSAR and QSPR Paradigms, Int. J Quantum >> Chem., 85, 662 (2001). >> >> the equation is >> >> I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * >> >> sin s*r_sub_ij >> ----------------- Equation (7) >> s*r_sub_ij >> >> What I have managed to do so far is for a simple organic molecule >> containing 5 atoms (CHFClBr): >> >> import numpy as np >> import numpy, pandas, scipy.spatial.distance as dist >> >> r=[] #distances between pairs of atoms >> z = [] #atomic numbers of atoms >> Z_dist = [] #products of successive atomic numbers >> I = [] #vector calculated in equation (7) >> >> start=1 >> finish=31 >> points=300 >> s = np.linspace(start,finish,points) >> >> >> name = input("Enter Molecule ID: ") >> name = str(name) >> name_in = name+'.dat' >> >> df = pandas.read_table(name_in, skiprows=2, sep=" ", >> skipinitialspace=True) >> z = numpy.array(df["ZA"]) >> N = numpy.ma.size(z) ## of atoms in molecule >> a = numpy.array(df[["X", "Y", "Z"]]) >> dist.squareform(dist.pdist(a, "euclidean")) >> anrows, ancols = np.shape(a) >> a_new = a.reshape(anrows, 1, ancols) >> diff = a_new - a >> >> r = (diff ** 2).sum(2) >> r = np.sqrt(r) >> >> for j in range(0,N-1): >> for k in range(j+1,N): >> Z_diff = (z[j]*z[k]) >> >> >> For the test molecule I get the following: >> >> MASS = [ 12.011 1.008 79.9 35.453 18.998] (Atomic weights >> of atoms) >> >> r: [ 0. 2.059801 3.60937686 3.32591826 2.81569212] >> [ 2.059801 0. 4.71452879 4.45776445 4.00467382] >> [ 3.60937686 4.71452879 0. 5.66500917 5.26602175] >> [ 3.32591826 4.45776445 5.66500917 0. 5.02324896] >> [ 2.81569212 4.00467382 5.26602175 5.02324896 0 >> . ] >> >> >> z: >> >> 6.0 >> 210.0 >> 102.0 >> 54.0 >> 35.0 >> 17.0 >> 9.0 >> 595.0 >> 315.0 >> 153.0 >> >> I have checked these calculations with a spreadsheet calculation, >> consequently I'm correct as far as I have gotten. >> >> However, my problem is calculating the value of I for the 300 >> x-coordinate values required for the molecular index calculation. >> >> The problem that I am currently having is calculating the product of >> the pre-sine function and the sine term for the 300 values of s. >> >> Unfortunately, the wealth of function in Python makes it difficult >> for me to ascertain just how to proceed. It seems that I have the >> problem of seeing the trees because of the size of the forest. At >> this point Goggling only makes my confusion deeper. Any pointers in >> the correct direction will be greatly appreciated. >> >> Thanks in advance. >> >> -- >> Stephen P. Molnar, Ph.D. Life is a fuzzy set >> www.molecular-modeling.net >> Stochastic and multivariate >> (614)312-7528 (c) >> Skype: smolnar1 >> >> >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> >> >> >> >> >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> >> Hjalmar > > Thanks for your response. > > I'm sorry that my original request wasn't a bit clearer. I presume that > the paper I attached to the request help was deleted before the email was > posted to the list. > > Here is how I incorporated your suggestion in the code. > > import numpy as np > import numpy, pandas, scipy.spatial.distance as dist > > r = [] > s=[] > z = [] > Z = [] > I = [] > > start=1 > finish=31 > points=300 > s = np.linspace(start,finish,points) > np.savetxt('Number of Points',s,delimiter=' ') > > name = input("Enter Molecule ID: ") > name = str(name) > name_in = name+'.dat' > > df = pandas.read_table(name_in, skiprows=2, sep=" ", skipinitialspace=True) > Z = numpy.array(df["ZA"]) > print('z = ',z) > N = numpy.ma.size(z) > a = numpy.array(df[["X", "Y", "Z"]]) > dist.squareform(dist.pdist(a, "euclidean")) > anrows, ancols = np.shape(a) > a_new = a.reshape(anrows, 1, ancols) > diff = a_new - a > > r = (diff ** 2).sum(2) > r = np.sqrt(z) > > r[r == 0.] = 0.0001 > > N = Z.size > > I = np.zeros(s.shape) > > for i in range(1, N): > for j in range(j): > I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) > print('I: ',I) > > np.savetxt('I',I,delimiter=' ') > > The for j in range(j): has a marginal note (I'm using the Spyder IDE) > "undefined name 'y' " and I is a vector with three hundred 0.0 terms. > > also, please note that I removed 'Z = np.array([6.0, 210.0, 102.0, 54.0, > 35.0, 17.0, 9.0, 595.0, 315.0, 153.0])' from your suggestion as that is the > result of the per-sine term 'Z[i] * Z[j]' > > Regards, > > Steve > > > -- > Stephen P. Molnar, Ph.D. Life is a fuzzy set > www.molecular-modeling.net Stochastic and multivariate > (614)312-7528 (c) > Skype: smolnar1 > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From david.mikolas1 at gmail.com Sun Apr 16 11:58:02 2017 From: david.mikolas1 at gmail.com (David Mikolas) Date: Sun, 16 Apr 2017 23:58:02 +0800 Subject: [SciPy-User] Molecular Index Calculation In-Reply-To: References: <58F1F5FA.5030102@sbcglobal.net> <58F362CB.8020005@sbcglobal.net> Message-ID: wrong: I = Zij*trangle.sum() better: I = (Zij*trangle).sum() On Sun, Apr 16, 2017 at 11:56 PM, David Mikolas wrote: > I escaped a cluttered Fortran/C/Pascal/Basic/Perl jumble by "living" in > stackoverflow. Mostly by reading questions and answers, and learning how > not to get my questions closed in milliseconds. Break your program into > small pieces, and look for how each piece might be done differently. > > If you try to find ways to do things without ever manually checking the > sizes of arrays, you'll naturally be guided towards "pythonic" methods and > techniques. There is almost always a better way. > > Computers are so fast these days, don't worry about ultimate speed until > you really hit a brick wall. Letting python, numpy, or scipy methods handle > housekeeping for you means you are usually defaulting to very well written > compiled routines. > > So for example, let python do the thinking... > > r = np.sqrt((diff ** 2).sum(2)) > r[r < 0.0001] = 0.0001 > # or just > r = np.maximum(np.sqrt((diff ** 2).sum(2)), 0.0001) > > # and > > Zij = Z * Z[:, None] > triangle = np.triu(np.ones_like(Zij), k=1) > I = Zij*trangle.sum() > > # or just > > I = np.triu(Z * Z[:, None], k=1).sum() > > > On Sun, Apr 16, 2017 at 8:25 PM, Stephen P. Molnar > wrote: > >> On 04/15/2017 04:07 PM, Hjalmar Turesson wrote: >> >>> Hi Stephen, >>> >>> >>> I see that N = 10 (ie. the length of Z), and that r is indexed by i and >>> j, which run up to N. However, r is only a 5x5 array. Is this correct? >>> >>> Otherwise, something like this might work: >>> >>> import numpy as np >>> >>> start=1 >>> finish=31 >>> points=300 >>> s = np.linspace(start, finish, points) >>> >>> MASS = np.array([12.011, 1.008, 79.9, 35.453, 18.998]) >>> >>> r = np.array([[0., 2.059801, 3.60937686, 3.32591826, 2.81569212], >>> [2.059801, 0., 4.71452879, 4.45776445, 4.00467382], >>> [3.60937686, 4.71452879, 0., 5.66500917, 5.26602175], >>> [3.32591826, 4.45776445, 5.66500917, 0., 5.02324896], >>> [2.81569212, 4.00467382, 5.26602175, 5.02324896, 0.]]) >>> >>> r[r == 0.] = 0.0001 >>> >>> Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, >>> 153.0]) >>> >>> N = Z.size >>> >>> I = np.zeros(s.shape) >>> >>> for i in range(1, N): >>> for j in range(j): >>> I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) >>> >>> >>> Do you have an example of the correct output? >>> >>> Best, >>> Hjalmar >>> >>> On Sat, Apr 15, 2017 at 6:29 AM, Stephen P. Molnar >>> > wrote: >>> >>> I am a newcomer to Python and am attempting to translate a FORTRAN >>> program that I wrote about 20 years ago into Python, specifically >>> v-3.6. >>> >>> Let me say that I am not asking someone to write the program for me, >>> but only to point me is the right direction. >>> >>> So far, I have bumbled my way to the point that I can get all of the >>> input data resulting from a quantum mechanical calculation of a very >>> simple organic molecule in to a Python program, but am encountering >>> problems with processing the data. >>> >>> The equation that I a want to evaluate (the one that I programmed in >>> FORTRAN) is equation (7) in the attached scan of one of the pages of >>> the following document. >>> >>> Stephen P. Molnar and James W. King, Theory and Applications of the >>> Integrated Molecular Transform and the Normalized Molecular Moment >>> Structure Descriptors: QSAR and QSPR Paradigms, Int. J Quantum >>> Chem., 85, 662 (2001). >>> >>> the equation is >>> >>> I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * >>> >>> sin s*r_sub_ij >>> ----------------- Equation (7) >>> s*r_sub_ij >>> >>> What I have managed to do so far is for a simple organic molecule >>> containing 5 atoms (CHFClBr): >>> >>> import numpy as np >>> import numpy, pandas, scipy.spatial.distance as dist >>> >>> r=[] #distances between pairs of atoms >>> z = [] #atomic numbers of atoms >>> Z_dist = [] #products of successive atomic numbers >>> I = [] #vector calculated in equation (7) >>> >>> start=1 >>> finish=31 >>> points=300 >>> s = np.linspace(start,finish,points) >>> >>> >>> name = input("Enter Molecule ID: ") >>> name = str(name) >>> name_in = name+'.dat' >>> >>> df = pandas.read_table(name_in, skiprows=2, sep=" ", >>> skipinitialspace=True) >>> z = numpy.array(df["ZA"]) >>> N = numpy.ma.size(z) ## of atoms in molecule >>> a = numpy.array(df[["X", "Y", "Z"]]) >>> dist.squareform(dist.pdist(a, "euclidean")) >>> anrows, ancols = np.shape(a) >>> a_new = a.reshape(anrows, 1, ancols) >>> diff = a_new - a >>> >>> r = (diff ** 2).sum(2) >>> r = np.sqrt(r) >>> >>> for j in range(0,N-1): >>> for k in range(j+1,N): >>> Z_diff = (z[j]*z[k]) >>> >>> >>> For the test molecule I get the following: >>> >>> MASS = [ 12.011 1.008 79.9 35.453 18.998] (Atomic weights >>> of atoms) >>> >>> r: [ 0. 2.059801 3.60937686 3.32591826 2.81569212] >>> [ 2.059801 0. 4.71452879 4.45776445 4.00467382] >>> [ 3.60937686 4.71452879 0. 5.66500917 5.26602175] >>> [ 3.32591826 4.45776445 5.66500917 0. 5.02324896] >>> [ 2.81569212 4.00467382 5.26602175 5.02324896 0 >>> . ] >>> >>> >>> z: >>> >>> 6.0 >>> 210.0 >>> 102.0 >>> 54.0 >>> 35.0 >>> 17.0 >>> 9.0 >>> 595.0 >>> 315.0 >>> 153.0 >>> >>> I have checked these calculations with a spreadsheet calculation, >>> consequently I'm correct as far as I have gotten. >>> >>> However, my problem is calculating the value of I for the 300 >>> x-coordinate values required for the molecular index calculation. >>> >>> The problem that I am currently having is calculating the product of >>> the pre-sine function and the sine term for the 300 values of s. >>> >>> Unfortunately, the wealth of function in Python makes it difficult >>> for me to ascertain just how to proceed. It seems that I have the >>> problem of seeing the trees because of the size of the forest. At >>> this point Goggling only makes my confusion deeper. Any pointers in >>> the correct direction will be greatly appreciated. >>> >>> Thanks in advance. >>> >>> -- >>> Stephen P. Molnar, Ph.D. Life is a fuzzy set >>> www.molecular-modeling.net >>> Stochastic and multivariate >>> (614)312-7528 (c) >>> Skype: smolnar1 >>> >>> >>> _______________________________________________ >>> SciPy-User mailing list >>> SciPy-User at python.org >>> https://mail.python.org/mailman/listinfo/scipy-user >>> >>> >>> >>> >>> >>> _______________________________________________ >>> SciPy-User mailing list >>> SciPy-User at python.org >>> https://mail.python.org/mailman/listinfo/scipy-user >>> >>> Hjalmar >> >> Thanks for your response. >> >> I'm sorry that my original request wasn't a bit clearer. I presume that >> the paper I attached to the request help was deleted before the email was >> posted to the list. >> >> Here is how I incorporated your suggestion in the code. >> >> import numpy as np >> import numpy, pandas, scipy.spatial.distance as dist >> >> r = [] >> s=[] >> z = [] >> Z = [] >> I = [] >> >> start=1 >> finish=31 >> points=300 >> s = np.linspace(start,finish,points) >> np.savetxt('Number of Points',s,delimiter=' ') >> >> name = input("Enter Molecule ID: ") >> name = str(name) >> name_in = name+'.dat' >> >> df = pandas.read_table(name_in, skiprows=2, sep=" ", >> skipinitialspace=True) >> Z = numpy.array(df["ZA"]) >> print('z = ',z) >> N = numpy.ma.size(z) >> a = numpy.array(df[["X", "Y", "Z"]]) >> dist.squareform(dist.pdist(a, "euclidean")) >> anrows, ancols = np.shape(a) >> a_new = a.reshape(anrows, 1, ancols) >> diff = a_new - a >> >> r = (diff ** 2).sum(2) >> r = np.sqrt(z) >> >> r[r == 0.] = 0.0001 >> >> N = Z.size >> >> I = np.zeros(s.shape) >> >> for i in range(1, N): >> for j in range(j): >> I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) >> print('I: ',I) >> >> np.savetxt('I',I,delimiter=' ') >> >> The for j in range(j): has a marginal note (I'm using the Spyder IDE) >> "undefined name 'y' " and I is a vector with three hundred 0.0 terms. >> >> also, please note that I removed 'Z = np.array([6.0, 210.0, 102.0, 54.0, >> 35.0, 17.0, 9.0, 595.0, 315.0, 153.0])' from your suggestion as that is the >> result of the per-sine term 'Z[i] * Z[j]' >> >> Regards, >> >> Steve >> >> >> -- >> Stephen P. Molnar, Ph.D. Life is a fuzzy set >> www.molecular-modeling.net Stochastic and multivariate >> (614)312-7528 (c) >> Skype: smolnar1 >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From s.molnar at sbcglobal.net Sun Apr 16 12:58:01 2017 From: s.molnar at sbcglobal.net (Stephen P. Molnar) Date: Sun, 16 Apr 2017 12:58:01 -0400 Subject: [SciPy-User] Molecular Index Calculation In-Reply-To: References: <58F1F5FA.5030102@sbcglobal.net> <58F362CB.8020005@sbcglobal.net> Message-ID: <58F3A299.5010409@sbcglobal.net> On 04/16/2017 11:58 AM, David Mikolas wrote: > wrong: I = Zij*trangle.sum() > > better: I = (Zij*trangle).sum() > > > On Sun, Apr 16, 2017 at 11:56 PM, David Mikolas > > wrote: > > I escaped a cluttered Fortran/C/Pascal/Basic/Perl jumble by "living" > in stackoverflow. Mostly by reading questions and answers, and > learning how not to get my questions closed in milliseconds. Break > your program into small pieces, and look for how each piece might be > done differently. > > If you try to find ways to do things without ever manually checking > the sizes of arrays, you'll naturally be guided towards "pythonic" > methods and techniques. There is almost always a better way. > > Computers are so fast these days, don't worry about ultimate speed > until you really hit a brick wall. Letting python, numpy, or scipy > methods handle housekeeping for you means you are usually defaulting > to very well written compiled routines. > > So for example, let python do the thinking... > > r = np.sqrt((diff ** 2).sum(2)) > r[r < 0.0001] = 0.0001 > # or just > r = np.maximum(np.sqrt((diff ** 2).sum(2)), 0.0001) > > # and > > Zij = Z * Z[:, None] > triangle = np.triu(np.ones_like(Zij), k=1) > I = Zij*trangle.sum() > > # or just > > I = np.triu(Z * Z[:, None], k=1).sum() > > > On Sun, Apr 16, 2017 at 8:25 PM, Stephen P. Molnar > > wrote: > > On 04/15/2017 04:07 PM, Hjalmar Turesson wrote: > > Hi Stephen, > > > I see that N = 10 (ie. the length of Z), and that r is > indexed by i and > j, which run up to N. However, r is only a 5x5 array. Is > this correct? > > Otherwise, something like this might work: > > import numpy as np > > start=1 > finish=31 > points=300 > s = np.linspace(start, finish, points) > > MASS = np.array([12.011, 1.008, 79.9, 35.453, 18.998]) > > r = np.array([[0., 2.059801, 3.60937686, 3.32591826, > 2.81569212], > [2.059801, 0., 4.71452879, 4.45776445, > 4.00467382], > [3.60937686, 4.71452879, 0., 5.66500917, > 5.26602175], > [3.32591826, 4.45776445, 5.66500917, 0., > 5.02324896], > [2.81569212, 4.00467382, 5.26602175, > 5.02324896, 0.]]) > > r[r == 0.] = 0.0001 > > Z = np.array([6.0, 210.0, 102.0, 54.0, 35.0, 17.0, 9.0, > 595.0, 315.0, > 153.0]) > > N = Z.size > > I = np.zeros(s.shape) > > for i in range(1, N): > for j in range(j): > I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) > > > Do you have an example of the correct output? > > Best, > Hjalmar > > On Sat, Apr 15, 2017 at 6:29 AM, Stephen P. Molnar > > >> wrote: > > I am a newcomer to Python and am attempting to > translate a FORTRAN > program that I wrote about 20 years ago into Python, > specifically v-3.6. > > Let me say that I am not asking someone to write the > program for me, > but only to point me is the right direction. > > So far, I have bumbled my way to the point that I can > get all of the > input data resulting from a quantum mechanical > calculation of a very > simple organic molecule in to a Python program, but am > encountering > problems with processing the data. > > The equation that I a want to evaluate (the one that I > programmed in > FORTRAN) is equation (7) in the attached scan of one of > the pages of > the following document. > > Stephen P. Molnar and James W. King, Theory and > Applications of the > Integrated Molecular Transform and the Normalized > Molecular Moment > Structure Descriptors: QSAR and QSPR Paradigms, Int. J > Quantum > Chem., 85, 662 (2001). > > the equation is > > I(s) = sum from i=2toN sum from j=1toi-1 Z_sub_i*Z_sub_j * > > sin s*r_sub_ij > ----------------- Equation (7) > s*r_sub_ij > > What I have managed to do so far is for a simple > organic molecule > containing 5 atoms (CHFClBr): > > import numpy as np > import numpy, pandas, scipy.spatial.distance as dist > > r=[] #distances between pairs of atoms > z = [] #atomic numbers of atoms > Z_dist = [] #products of successive atomic numbers > I = [] #vector calculated in equation (7) > > start=1 > finish=31 > points=300 > s = np.linspace(start,finish,points) > > > name = input("Enter Molecule ID: ") > name = str(name) > name_in = name+'.dat' > > df = pandas.read_table(name_in, skiprows=2, sep=" ", > skipinitialspace=True) > z = numpy.array(df["ZA"]) > N = numpy.ma.size(z) ## of atoms in molecule > a = numpy.array(df[["X", "Y", "Z"]]) > dist.squareform(dist.pdist(a, "euclidean")) > anrows, ancols = np.shape(a) > a_new = a.reshape(anrows, 1, ancols) > diff = a_new - a > > r = (diff ** 2).sum(2) > r = np.sqrt(r) > > for j in range(0,N-1): > for k in range(j+1,N): > Z_diff = (z[j]*z[k]) > > > For the test molecule I get the following: > > MASS = [ 12.011 1.008 79.9 35.453 18.998] > (Atomic weights > of atoms) > > r: [ 0. 2.059801 3.60937686 3.32591826 > 2.81569212] > [ 2.059801 0. 4.71452879 4.45776445 > 4.00467382] > [ 3.60937686 4.71452879 0. 5.66500917 > 5.26602175] > [ 3.32591826 4.45776445 5.66500917 0. > 5.02324896] > [ 2.81569212 4.00467382 5.26602175 5.02324896 0 > . ] > > > z: > > 6.0 > 210.0 > 102.0 > 54.0 > 35.0 > 17.0 > 9.0 > 595.0 > 315.0 > 153.0 > > I have checked these calculations with a spreadsheet > calculation, > consequently I'm correct as far as I have gotten. > > However, my problem is calculating the value of I for > the 300 > x-coordinate values required for the molecular index > calculation. > > The problem that I am currently having is calculating > the product of > the pre-sine function and the sine term for the 300 > values of s. > > Unfortunately, the wealth of function in Python makes > it difficult > for me to ascertain just how to proceed. It seems that > I have the > problem of seeing the trees because of the size of the > forest. At > this point Goggling only makes my confusion deeper. > Any pointers in > the correct direction will be greatly appreciated. > > Thanks in advance. > > -- > Stephen P. Molnar, Ph.D. Life is a fuzzy set > www.molecular-modeling.net > > > > Stochastic and multivariate > (614)312-7528 (c) > Skype: smolnar1 > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > > > https://mail.python.org/mailman/listinfo/scipy-user > > > > > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > Hjalmar > > Thanks for your response. > > I'm sorry that my original request wasn't a bit clearer. I > presume that the paper I attached to the request help was > deleted before the email was posted to the list. > > Here is how I incorporated your suggestion in the code. > > import numpy as np > import numpy, pandas, scipy.spatial.distance as dist > > r = [] > s=[] > z = [] > Z = [] > I = [] > > start=1 > finish=31 > points=300 > s = np.linspace(start,finish,points) > np.savetxt('Number of Points',s,delimiter=' ') > > name = input("Enter Molecule ID: ") > name = str(name) > name_in = name+'.dat' > > df = pandas.read_table(name_in, skiprows=2, sep=" ", > skipinitialspace=True) > Z = numpy.array(df["ZA"]) > print('z = ',z) > N = numpy.ma.size(z) > a = numpy.array(df[["X", "Y", "Z"]]) > dist.squareform(dist.pdist(a, "euclidean")) > anrows, ancols = np.shape(a) > a_new = a.reshape(anrows, 1, ancols) > diff = a_new - a > > r = (diff ** 2).sum(2) > r = np.sqrt(z) > > r[r == 0.] = 0.0001 > > N = Z.size > > I = np.zeros(s.shape) > > for i in range(1, N): > for j in range(j): > I += Z[i] * Z[j] * np.sin(s * r[i, j])/(s * r[i, j]) > print('I: ',I) > > np.savetxt('I',I,delimiter=' ') > > The for j in range(j): has a marginal note (I'm using the Spyder > IDE) "undefined name 'y' " and I is a vector with three hundred > 0.0 terms. > > also, please note that I removed 'Z = np.array([6.0, 210.0, > 102.0, 54.0, 35.0, 17.0, 9.0, 595.0, 315.0, 153.0])' from your > suggestion as that is the result of the per-sine term 'Z[i] * Z[j]' > > Regards, > > Steve > > > -- > Stephen P. Molnar, Ph.D. Life is a fuzzy set > www.molecular-modeling.net > Stochastic and multivariate > (614)312-7528 (c) > Skype: smolnar1 > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > Great suggestions! Many thanks. -- Stephen P. Molnar, Ph.D. Life is a fuzzy set www.molecular-modeling.net Stochastic and multivariate (614)312-7528 (c) Skype: smolnar1 From daniele at grinta.net Tue Apr 18 13:17:26 2017 From: daniele at grinta.net (Daniele Nicolodi) Date: Tue, 18 Apr 2017 11:17:26 -0600 Subject: [SciPy-User] Multivariate linear (bilinear) fit Message-ID: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Hello, before I write my own, does SciPy come with a function for multivariate (often also called bilinear, in the case of two variables) linear fits? I would like to fit a function if the form: y = c + p1 * x1 + p2 * x2 + ... + pn * xn which is a simple least squares problem, but I would like to avoid to write the code :-) Thanks. Cheers, Daniele From pawel.kw at gmail.com Wed Apr 19 01:46:47 2017 From: pawel.kw at gmail.com (=?UTF-8?Q?Pawe=C5=82_Kwa=C5=9Bniewski?=) Date: Wed, 19 Apr 2017 07:46:47 +0200 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: Hi Daniele, Do you know lmfit (https://lmfit.github.io/lmfit-py/)? It's really simple to use and very well documented. Cheers, Pawel Pawe? 2017-04-18 19:17 GMT+02:00 Daniele Nicolodi : > Hello, > > before I write my own, does SciPy come with a function for multivariate > (often also called bilinear, in the case of two variables) linear fits? > > I would like to fit a function if the form: > > y = c + p1 * x1 + p2 * x2 + ... + pn * xn > > which is a simple least squares problem, but I would like to avoid to > write the code :-) > > Thanks. Cheers, > Daniele > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From sergio_r at mail.com Wed Apr 19 16:54:50 2017 From: sergio_r at mail.com (Sergio Rojas) Date: Wed, 19 Apr 2017 22:54:50 +0200 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: References: Message-ID: ---------------------------------------------------------------------- Message: 1 Date: Tue, 18 Apr 2017 11:17:26 -0600 From: Daniele Nicolodi To: scipy-user at python.org Subject: [SciPy-User] Multivariate linear (bilinear) fit Message-ID: <7728d109-52a4-50f5-1289-e75961a7a5d6 at grinta.net> Content-Type: text/plain; charset=utf-8 Hello, before I write my own, does SciPy come with a function for multivariate (often also called bilinear, in the case of two variables) linear fits? I would like to fit a function if the form: y = c + p1 * x1 + p2 * x2 + ... + pn * xn which is a simple least squares problem, but I would like to avoid to write the code :-) Thanks. Cheers, Daniele ===================== Take a look at the optimization tools from SciPy http://www.scipy-lectures.org/advanced/mathematical_optimization/index.html Sergio Enhance your #MachineLearning and #BigData skills via #Python #SciPy: 1) https://www.packtpub.com/big-data-and-business-intelligence/numerical-and-scientific-computing-scipy-video 2) https://www.packtpub.com/big-data-and-business-intelligence/learning-scipy-numerical-and-scientific-computing-second-edition From njs at pobox.com Wed Apr 19 17:17:01 2017 From: njs at pobox.com (Nathaniel Smith) Date: Wed, 19 Apr 2017 14:17:01 -0700 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi wrote: > Hello, > > before I write my own, does SciPy come with a function for multivariate > (often also called bilinear, in the case of two variables) linear fits? > > I would like to fit a function if the form: > > y = c + p1 * x1 + p2 * x2 + ... + pn * xn > > which is a simple least squares problem, but I would like to avoid to > write the code :-) np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and multivariate problems. They're pretty bare-bones, though, and they also compute the sum of squared residuals which can make them slow if you don't need it and are solving lots of problems in a tight loop. -n -- Nathaniel J. Smith -- https://vorpus.org From joferkington at gmail.com Wed Apr 19 18:38:16 2017 From: joferkington at gmail.com (Joe Kington) Date: Wed, 19 Apr 2017 17:38:16 -0500 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: They're relatively recent additions, but numpy.polynomial.polyvander2d and numpy.polynomial.polyval2d should also do what you want, unless I'm misunderstanding the problem. https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d You can also do things like (you could generalize this to N-dimensions, as well): def polyfit2d(x, y, z, order=3): ncols = (order + 1)**2 G = np.zeros((x.size, ncols)) ij = itertools.product(range(order+1), range(order+1)) for k, (i,j) in enumerate(ij): G[:,k] = x**i * y**j m, _, _, _ = np.linalg.lstsq(G, z) return m def polyval2d(x, y, m): order = int(np.sqrt(len(m))) - 1 ij = itertools.product(range(order+1), range(order+1)) z = np.zeros_like(x) for a, (i,j) in zip(m, ij): z += a * x**i * y**j return z On Wed, Apr 19, 2017 at 4:17 PM, Nathaniel Smith wrote: > On Tue, Apr 18, 2017 at 10:17 AM, Daniele Nicolodi > wrote: > > Hello, > > > > before I write my own, does SciPy come with a function for multivariate > > (often also called bilinear, in the case of two variables) linear fits? > > > > I would like to fit a function if the form: > > > > y = c + p1 * x1 + p2 * x2 + ... + pn * xn > > > > which is a simple least squares problem, but I would like to avoid to > > write the code :-) > > np.linalg.lstsq and scipy.linalg.lstsq handle both univariate and > multivariate problems. They're pretty bare-bones, though, and they > also compute the sum of squared residuals which can make them slow if > you don't need it and are solving lots of problems in a tight loop. > > -n > > -- > Nathaniel J. Smith -- https://vorpus.org > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From charlesr.harris at gmail.com Wed Apr 19 20:12:49 2017 From: charlesr.harris at gmail.com (Charles R Harris) Date: Wed, 19 Apr 2017 18:12:49 -0600 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington wrote: > They're relatively recent additions, but numpy.polynomial.polyvander2d and > numpy.polynomial.polyval2d should also do what you want, unless I'm > misunderstanding the problem. > > https://docs.scipy.org/doc/numpy/reference/generated/ > numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial. > polyvander2d > https://docs.scipy.org/doc/numpy/reference/generated/ > numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial. > polyval2d > > You can also do things like (you could generalize this to N-dimensions, as > well): > > def polyfit2d(x, y, z, order=3): > ncols = (order + 1)**2 > G = np.zeros((x.size, ncols)) > ij = itertools.product(range(order+1), range(order+1)) > for k, (i,j) in enumerate(ij): > G[:,k] = x**i * y**j > m, _, _, _ = np.linalg.lstsq(G, z) > return m > def polyval2d(x, y, m): > order = int(np.sqrt(len(m))) - 1 > ij = itertools.product(range(order+1), range(order+1)) > z = np.zeros_like(x) > for a, (i,j) in zip(m, ij): > z += a * x**i * y**j > return z > > > I think the "bilinear" is a mistake, as bilinear usually means terms of degree two. AFAICT, this question is just about multivariate linear fits only Chuck > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From josef.pktd at gmail.com Wed Apr 19 20:46:45 2017 From: josef.pktd at gmail.com (josef.pktd at gmail.com) Date: Wed, 19 Apr 2017 20:46:45 -0400 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris wrote: > > > On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington wrote: >> >> They're relatively recent additions, but numpy.polynomial.polyvander2d and >> numpy.polynomial.polyval2d should also do what you want, unless I'm >> misunderstanding the problem. >> >> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d >> >> https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d >> >> You can also do things like (you could generalize this to N-dimensions, as >> well): >> >> def polyfit2d(x, y, z, order=3): >> ncols = (order + 1)**2 >> G = np.zeros((x.size, ncols)) >> ij = itertools.product(range(order+1), range(order+1)) >> for k, (i,j) in enumerate(ij): >> G[:,k] = x**i * y**j >> m, _, _, _ = np.linalg.lstsq(G, z) >> return m >> >> def polyval2d(x, y, m): >> order = int(np.sqrt(len(m))) - 1 >> ij = itertools.product(range(order+1), range(order+1)) >> z = np.zeros_like(x) >> for a, (i,j) in zip(m, ij): >> z += a * x**i * y**j >> return z >> >> > > I think the "bilinear" is a mistake, as bilinear usually means terms of > degree two. AFAICT, this question is just about multivariate linear fits > only like statsmodels OLS Josef > > > > Chuck >> >> >> > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > From denis.akhiyarov at gmail.com Wed Apr 19 22:05:03 2017 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Thu, 20 Apr 2017 02:05:03 +0000 Subject: [SciPy-User] Multivariate linear (bilinear) fit In-Reply-To: References: <7728d109-52a4-50f5-1289-e75961a7a5d6@grinta.net> Message-ID: I would also recommend looking at robust regression (RLM), before you settle on OLS. Especially if the data is noisy or has outliers. SciPy has a new regression method with bounds on the coefficients, if you get unrealistic factors from the OLS regression results, but it requires some "tuning". Whatever you pick, looking at results in parity plots and error histograms may help as well. http://scikit-learn.org/stable/auto_examples/linear_model/plot_ransac.html https://docs.scipy.org/doc/scipy-0.18.1/reference/generated/scipy.optimize.lsq_linear.html On Wed, Apr 19, 2017, 7:47 PM wrote: > On Wed, Apr 19, 2017 at 8:12 PM, Charles R Harris > wrote: > > > > > > On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington > wrote: > >> > >> They're relatively recent additions, but numpy.polynomial.polyvander2d > and > >> numpy.polynomial.polyval2d should also do what you want, unless I'm > >> misunderstanding the problem. > >> > >> > >> > https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial.polyvander2d > >> > >> > https://docs.scipy.org/doc/numpy/reference/generated/numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial.polyval2d > >> > >> You can also do things like (you could generalize this to N-dimensions, > as > >> well): > >> > >> def polyfit2d(x, y, z, order=3): > >> ncols = (order + 1)**2 > >> G = np.zeros((x.size, ncols)) > >> ij = itertools.product(range(order+1), range(order+1)) > >> for k, (i,j) in enumerate(ij): > >> G[:,k] = x**i * y**j > >> m, _, _, _ = np.linalg.lstsq(G, z) > >> return m > >> > >> def polyval2d(x, y, m): > >> order = int(np.sqrt(len(m))) - 1 > >> ij = itertools.product(range(order+1), range(order+1)) > >> z = np.zeros_like(x) > >> for a, (i,j) in zip(m, ij): > >> z += a * x**i * y**j > >> return z > >> > >> > > > > I think the "bilinear" is a mistake, as bilinear usually means terms of > > degree two. AFAICT, this question is just about multivariate linear fits > > only > > like statsmodels OLS > > Josef > > > > > > > > > > Chuck > >> > >> > >> > > > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jschwabedal at gmail.com Thu Apr 20 02:19:10 2017 From: jschwabedal at gmail.com (Justus Schwabedal) Date: Thu, 20 Apr 2017 08:19:10 +0200 Subject: [SciPy-User] Multivariate linear (bilinear) fit (Charles R Harris) In-Reply-To: References: Message-ID: Indeed, in statistics it's a linear model. Try r2py. R's really good in linear models and will give you additional information on significance tests. Best, Jus On Apr 20, 2017 02:14, wrote: Send SciPy-User mailing list submissions to scipy-user at python.org To subscribe or unsubscribe via the World Wide Web, visit https://mail.python.org/mailman/listinfo/scipy-user or, via email, send a message with subject or body 'help' to scipy-user-request at python.org You can reach the person managing the list at scipy-user-owner at python.org When replying, please edit your Subject line so it is more specific than "Re: Contents of SciPy-User digest..." Today's Topics: 1. Re: Multivariate linear (bilinear) fit (Charles R Harris) ---------------------------------------------------------------------- Message: 1 Date: Wed, 19 Apr 2017 18:12:49 -0600 From: Charles R Harris To: SciPy Users List Subject: Re: [SciPy-User] Multivariate linear (bilinear) fit Message-ID: Content-Type: text/plain; charset="utf-8" On Wed, Apr 19, 2017 at 4:38 PM, Joe Kington wrote: > They're relatively recent additions, but numpy.polynomial.polyvander2d and > numpy.polynomial.polyval2d should also do what you want, unless I'm > misunderstanding the problem. > > https://docs.scipy.org/doc/numpy/reference/generated/ > numpy.polynomial.polynomial.polyvander2d.html#numpy.polynomial.polynomial. > polyvander2d > https://docs.scipy.org/doc/numpy/reference/generated/ > numpy.polynomial.polynomial.polyval2d.html#numpy.polynomial.polynomial. > polyval2d > > You can also do things like (you could generalize this to N-dimensions, as > well): > > def polyfit2d(x, y, z, order=3): > ncols = (order + 1)**2 > G = np.zeros((x.size, ncols)) > ij = itertools.product(range(order+1), range(order+1)) > for k, (i,j) in enumerate(ij): > G[:,k] = x**i * y**j > m, _, _, _ = np.linalg.lstsq(G, z) > return m > def polyval2d(x, y, m): > order = int(np.sqrt(len(m))) - 1 > ij = itertools.product(range(order+1), range(order+1)) > z = np.zeros_like(x) > for a, (i,j) in zip(m, ij): > z += a * x**i * y**j > return z > > > I think the "bilinear" is a mistake, as bilinear usually means terms of degree two. AFAICT, this question is just about multivariate linear fits only Chuck > > > -------------- next part -------------- An HTML attachment was scrubbed... URL: ------------------------------ Subject: Digest Footer _______________________________________________ SciPy-User mailing list SciPy-User at python.org https://mail.python.org/mailman/listinfo/scipy-user ------------------------------ End of SciPy-User Digest, Vol 164, Issue 16 ******************************************* -------------- next part -------------- An HTML attachment was scrubbed... URL: From hiluming at gmail.com Tue Apr 25 12:09:34 2017 From: hiluming at gmail.com (Ming Lu) Date: Wed, 26 Apr 2017 00:09:34 +0800 Subject: [SciPy-User] why scipy.special.mathieu_cem is not periodic? Message-ID: Dear all: I have a question regarding the special function scipy.special.mathieu_cem, why this function not periodic? I thought this function should have period pi when m is even, and 2*pi when m is odd, however this is not the case: x = np.linspace(0,10*np.pi,100) plt.plot(x,sp.mathieu_cem(3,3,x)[0]) plt.show() compare with mathematica, it clearly shows a period graph. To see the graph, I have the detailed post on stackoverflow. http://stackoverflow.com/questions/43615531/why-mathieu-function-in-scipy-not-periodic Best regards, Ming From eulergaussriemann at gmail.com Tue Apr 25 12:36:35 2017 From: eulergaussriemann at gmail.com (David Goldsmith) Date: Tue, 25 Apr 2017 16:36:35 +0000 Subject: [SciPy-User] why scipy.special.mathieu_cem is not periodic? In-Reply-To: References: Message-ID: Did you see the answer on stackoverflow? DLG On Tue, Apr 25, 2017 at 9:10 AM Ming Lu wrote: > Dear all: > > I have a question regarding the special function > scipy.special.mathieu_cem, why this function > not periodic? I thought this function should have period pi when m is > even, and 2*pi when m > is odd, however this is not the case: > > x = np.linspace(0,10*np.pi,100) > plt.plot(x,sp.mathieu_cem(3,3,x)[0]) > plt.show() > > compare with mathematica, it clearly shows a period graph. > > To see the graph, I have the detailed post on stackoverflow. > > > http://stackoverflow.com/questions/43615531/why-mathieu-function-in-scipy-not-periodic > > Best regards, > Ming > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From hiluming at gmail.com Tue Apr 25 20:22:24 2017 From: hiluming at gmail.com (Ming Lu) Date: Wed, 26 Apr 2017 08:22:24 +0800 Subject: [SciPy-User] why scipy.special.mathieu_cem is not periodic? In-Reply-To: References: Message-ID: The period seems more in scipy than in mathematica, even after converting to degrees. On 4/26/17, David Goldsmith wrote: > Did you see the answer on stackoverflow? > > DLG > > On Tue, Apr 25, 2017 at 9:10 AM Ming Lu wrote: > >> Dear all: >> >> I have a question regarding the special function >> scipy.special.mathieu_cem, why this function >> not periodic? I thought this function should have period pi when m is >> even, and 2*pi when m >> is odd, however this is not the case: >> >> x = np.linspace(0,10*np.pi,100) >> plt.plot(x,sp.mathieu_cem(3,3,x)[0]) >> plt.show() >> >> compare with mathematica, it clearly shows a period graph. >> >> To see the graph, I have the detailed post on stackoverflow. >> >> >> http://stackoverflow.com/questions/43615531/why-mathieu-function-in-scipy-not-periodic >> >> Best regards, >> Ming >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> > -- Regards, Ming Lu ---- Ming Lu, graduate student of second year, ICQM, Peking University From eulergaussriemann at gmail.com Tue Apr 25 20:49:57 2017 From: eulergaussriemann at gmail.com (David Goldsmith) Date: Wed, 26 Apr 2017 00:49:57 +0000 Subject: [SciPy-User] why scipy.special.mathieu_cem is not periodic? In-Reply-To: References: Message-ID: But you are now seeing periodicity? Visual comparison can be "tricky": output numerical results from each at common abscissae to, e.g., CSV files, and then import them into a spreadsheet and then look at the differences that way. (I imagine such detailed verification of a problem will be required before a ticket would be accepted, unless someone else is concerned enough to independently verify the problem. In other words, people are probably going to want to see that you've done "due diligence" before pursuing this. But i could be wrong...) DLG On Tue, Apr 25, 2017 at 5:23 PM Ming Lu wrote: > The period seems more in scipy than in mathematica, even after > converting to degrees. > > On 4/26/17, David Goldsmith wrote: > > Did you see the answer on stackoverflow? > > > > DLG > > > > On Tue, Apr 25, 2017 at 9:10 AM Ming Lu wrote: > > > >> Dear all: > >> > >> I have a question regarding the special function > >> scipy.special.mathieu_cem, why this function > >> not periodic? I thought this function should have period pi when m is > >> even, and 2*pi when m > >> is odd, however this is not the case: > >> > >> x = np.linspace(0,10*np.pi,100) > >> plt.plot(x,sp.mathieu_cem(3,3,x)[0]) > >> plt.show() > >> > >> compare with mathematica, it clearly shows a period graph. > >> > >> To see the graph, I have the detailed post on stackoverflow. > >> > >> > >> > http://stackoverflow.com/questions/43615531/why-mathieu-function-in-scipy-not-periodic > >> > >> Best regards, > >> Ming > >> _______________________________________________ > >> SciPy-User mailing list > >> SciPy-User at python.org > >> https://mail.python.org/mailman/listinfo/scipy-user > >> > > > > > -- > Regards, > Ming Lu > > ---- > Ming Lu, graduate student of second year, > ICQM, Peking University > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From noflaco at gmail.com Wed Apr 26 23:20:45 2017 From: noflaco at gmail.com (Carlton Banks) Date: Thu, 27 Apr 2017 05:20:45 +0200 Subject: [SciPy-User] Convert data to image and then image to numpy.ndarray Message-ID: <17360C92-108C-4A8E-B7B4-8AC84CCEFDC8@gmail.com> Is it possible to convert a data set stored as a numpy.ndarray -> plot the data with a colormap , possibly cm.jet. -> and then store the image pixels to a numpy.ndarray. The data array of the image should have the same shape of the data array used to make the array. The image conversion is only used for getting a visual representation of the data, and normalise it using the cm.jet. So I actually just interested in the RGB values of the colormap normalised input data. IS this possible? From robert.kern at gmail.com Thu Apr 27 00:55:54 2017 From: robert.kern at gmail.com (Robert Kern) Date: Wed, 26 Apr 2017 21:55:54 -0700 Subject: [SciPy-User] Convert data to image and then image to numpy.ndarray In-Reply-To: <17360C92-108C-4A8E-B7B4-8AC84CCEFDC8@gmail.com> References: <17360C92-108C-4A8E-B7B4-8AC84CCEFDC8@gmail.com> Message-ID: On Wed, Apr 26, 2017 at 8:20 PM, Carlton Banks wrote: > > Is it possible to convert a data set stored as a numpy.ndarray -> plot the data with a colormap , possibly cm.jet. -> and then store the image pixels to a numpy.ndarray. > > The data array of the image should have the same shape of the data array used to make the array. > The image conversion is only used for getting a visual representation of the data, and normalise it using the cm.jet. > So I actually just interested in the RGB values of the colormap normalised input data. > > IS this possible? Sure, you can use matplotlib's colormap objects as functions to convert arrays with values in the interval [0.0, 1.0] to arrays of colors. [~] |6> cm.viridis(np.linspace(0.0, 1.0, 5)) array([[ 0.267004, 0.004874, 0.329415, 1. ], [ 0.229739, 0.322361, 0.545706, 1. ], [ 0.127568, 0.566949, 0.550556, 1. ], [ 0.369214, 0.788888, 0.382914, 1. ], [ 0.993248, 0.906157, 0.143936, 1. ]]) [~] |7> cm.viridis? ... Call signature: cm.viridis(X, alpha=None, bytes=False) Call docstring: Parameters ---------- X : scalar, ndarray The data value(s) to convert to RGBA. For floats, X should be in the interval ``[0.0, 1.0]`` to return the RGBA values ``X*100`` percent along the Colormap line. For integers, X should be in the interval ``[0, Colormap.N)`` to return RGBA values *indexed* from the Colormap with index ``X``. alpha : float, None Alpha must be a scalar between 0 and 1, or None. bytes : bool If False (default), the returned RGBA values will be floats in the interval ``[0, 1]`` otherwise they will be uint8s in the interval ``[0, 255]``. Returns ------- Tuple of RGBA values if X is scalar, othewise an array of RGBA values with a shape of ``X.shape + (4, )``. -- Robert Kern -------------- next part -------------- An HTML attachment was scrubbed... URL: From nitish.ragoomundun at gmail.com Fri Apr 28 01:24:25 2017 From: nitish.ragoomundun at gmail.com (Nitish Ragoomundun) Date: Fri, 28 Apr 2017 09:24:25 +0400 Subject: [SciPy-User] SciPy special function pro_rad1( ) Message-ID: Hello all, I need to compute values for the prolate spheroidal function and compare values between different orders. From the reference guide, the function is described as *scipy.special.pro_rad1(m, n, c, x) = * *Prolate spheroidal radial function of the first kind and its derivative* *Computes the prolate spheroidal radial function of the first kind and its derivative (with respect to x) for mode parameters m>=0 and n>=m, spheroidal parameter c and* *|x| < 1.0.* This tells me that x is the independent variable and that it is between -1.0 and +1.0. But, this is not the case. When I put x between 0.0 and 1.0 for example, the output is NaN. Also, I would like to know what are m and n. It is not clearly defined. I need to know which one is the order. Thanks. -------------- next part -------------- An HTML attachment was scrubbed... URL: From opossumnano at gmail.com Fri Apr 28 05:29:39 2017 From: opossumnano at gmail.com (Tiziano Zito) Date: Fri, 28 Apr 2017 02:29:39 -0700 (PDT) Subject: [SciPy-User] =?utf-8?b?W0FOTl0gMTDhtYDhtLQgQWR2YW5jZWQgU2NpZW50?= =?utf-8?q?ific_Programming_in_Python_in_Nikiti=2C_Greece=2C_August_28?= =?utf-8?q?=E2=80=94September_2=2C_2017?= Message-ID: <59030b83.ea85df0a.38a89.24ce@mx.google.com> 10?? Advanced Scientific Programming in Python ============================================== a Summer School by the G-Node and the Municipality of Sithonia Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists have been trained to use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques and best practices which are standard in the industry, but especially tailored to the needs of a programming scientist. Lectures are devised to be interactive and to give the students enough time to acquire direct hands-on experience with the materials. Students will work in pairs throughout the school and will team up to practice the newly learned skills in a real programming project ? an entertaining computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python and of a version control system such as git, subversion, mercurial, or bazaar is assumed. Participants without any prior experience with Python and/or git should work through the proposed introductory material before the course. We are striving hard to get a pool of students which is international and gender-balanced. You can apply online: https://python.g-node.org Application deadline: 23:59 UTC, May 31, 2017. There will be no deadline extension, so be sure to apply on time ;-) Be sure to read the FAQ before applying. Participation is for free, i.e. no fee is charged! Participants however should take care of travel, living, and accommodation expenses by themselves. Date & Location =============== August 28?September 2, 2017. Nikiti, Sithonia, Halkidiki, Greece Program ======= ? Best Programming Practices ? Best practices for scientific programming ? Version control with git and how to contribute to open source projects with GitHub ? Best practices in data visualization ? Software Carpentry ? Test-driven development ? Debugging with a debuggger ? Profiling code ? Scientific Tools for Python ? Advanced NumPy ? Advanced Python ? Decorators ? Context managers ? Generators ? The Quest for Speed ? Writing parallel applications ? Interfacing to C with Cython ? Memory-bound problems and memory profiling ? Data containers: storage and fast access to large data ? Practical Software Development ? Group project Preliminary Faculty =================== ? Francesc Alted, freelance consultant, author of Blosc, Castell? de la Plana, Spain ? Pietro Berkes, NAGRA Kudelski, Lausanne, Switzerland ? Zbigniew J?drzejewski-Szmek, Krasnow Institute, George Mason University, Fairfax, VA USA ? Eilif Muller, Blue Brain Project, ?cole Polytechnique F?d?rale de Lausanne Switzerland ? Juan Nunez-Iglesias, Victorian Life Sciences Computation Initiative, University of Melbourne, Australia ? Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universit?t zu Berlin, Germany ? Nicolas P. Rougier, Inria Bordeaux Sud-Ouest, Institute of Neurodegenerative Disease, University of Bordeaux, France ? Bartosz Tele?czuk, European Institute for Theoretical Neuroscience, CNRS, Paris, France ? St?fan van der Walt, Berkeley Institute for Data Science, UC Berkeley, CA USA ? Nelle Varoquaux, Berkeley Institute for Data Science, UC Berkeley, CA USA ? Tiziano Zito, freelance consultant, Berlin, Germany Organizers ========== For the German Neuroinformatics Node of the INCF (G-Node) Germany: ? Tiziano Zito, freelance consultant, Berlin, Germany ? Zbigniew J?drzejewski-Szmek, Krasnow Institute, George Mason University, Fairfax, USA ? Jakob Jordan, Institute of Neuroscience and Medicine (INM-6), Forschungszentrum J?lich GmbH, Germany ? Etienne Roesch, Centre for Integrative Neuroscience and Neurodynamics, University of Reading, UK Website: https://python.g-node.org Contact: python-info at g-node.org From david at drhagen.com Sat Apr 29 21:56:48 2017 From: david at drhagen.com (David Hagen) Date: Sat, 29 Apr 2017 21:56:48 -0400 Subject: [SciPy-User] Install Scipy with Anaconda's MKL libraries Message-ID: Anaconda now comes with free MKL libraries. I am trying to install the development branch of Scipy into an Anaconda environment, which means running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK in order to compile, so the built in MKL seems to make sense. However, it seems that Scipy can't find the MKL libraries because it gives the "no lapack/blas resources found" error during setup.py when running a pip install. Is there an environment variable I need to set in order to tell Scipy's setup where my LAPACK library lives? Here is exactly what I am running: conda create --name scipy_master python=3.6 numpy cython activate scipy_master pip install https://github.com/scipy/scipy/tarball/master I asked this on Stack Overflow last week but didn't get an answer. -------------- next part -------------- An HTML attachment was scrubbed... URL: From denis.akhiyarov at gmail.com Sun Apr 30 00:18:38 2017 From: denis.akhiyarov at gmail.com (Denis Akhiyarov) Date: Sat, 29 Apr 2017 23:18:38 -0500 Subject: [SciPy-User] Install Scipy with Anaconda's MKL libraries In-Reply-To: References: Message-ID: Welcome to the world of pain with building scientific packages from source on Windows! You need Fortran and C/C++ compilers on Windows to build scipy from source (and also enough disk space): https://www.scipy.org/scipylib/building/windows.html The mingwpy project mentioned in the wiki does not support Python 3.5+ due to UCRT. mkl binaries linked with numpy and scipy and provided in Anaconda are not enough for building scipy. If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy. But I had no luck with this, looks like this still requires dealing with include paths for blas and lapack. On Sat, Apr 29, 2017 at 8:56 PM, David Hagen wrote: > Anaconda now comes with free MKL libraries. I am trying to install the > development branch of Scipy into an Anaconda environment, which means > running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK > in order to compile, so the built in MKL seems to make sense. However, it > seems that Scipy can't find the MKL libraries because it gives the "no > lapack/blas resources found" error during setup.py when running a pip > install. Is there an environment variable I need to set in order to tell > Scipy's setup where my LAPACK library lives? > > Here is exactly what I am running: > > conda create --name scipy_master python=3.6 numpy cython > activate scipy_master > pip install https://github.com/scipy/scipy/tarball/master > > I asked this on Stack Overflow > last > week but didn't get an answer. > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From laughingrice at gmail.com Sun Apr 30 00:45:51 2017 From: laughingrice at gmail.com (Micha F) Date: Sun, 30 Apr 2017 00:45:51 -0400 Subject: [SciPy-User] Install Scipy with Anaconda's MKL libraries In-Reply-To: References: Message-ID: <0F40FE50-6E31-4FDA-8006-74FE29F775AE@gmail.com> I?m pretty sure that anaconda does not come with the development files for MKL, only the runtime files. I just looked under the mac couldn?t find any of the header files. That would make sense if Intel has not changed the licensing for MKL in the last few years. This means that you cannot link new code to the MKL shipped with Anaconda but would need to get your own development license for MKL. > On Apr 29, 2017, at 21:56, David Hagen wrote: > > Anaconda now comes with free MKL libraries. I am trying to install the development branch of Scipy into an Anaconda environment, which means running the full Scipy install, not just the wheel. Scipy needs BLAS/LAPACK in order to compile, so the built in MKL seems to make sense. However, it seems that Scipy can't find the MKL libraries because it gives the "no lapack/blas resources found" error during setup.py when running a pip install. Is there an environment variable I need to set in order to tell Scipy's setup where my LAPACK library lives? > > Here is exactly what I am running: > > conda create --name scipy_master python=3.6 numpy cython > activate scipy_master > pip install https://github.com/scipy/scipy/tarball/master > > I asked this on Stack Overflow last week but didn't get an answer. > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From david at drhagen.com Sun Apr 30 17:41:05 2017 From: david at drhagen.com (David Hagen) Date: Sun, 30 Apr 2017 17:41:05 -0400 Subject: [SciPy-User] Install Scipy with Anaconda's MKL libraries In-Reply-To: References: Message-ID: > Welcome to the world of pain with building scientific packages from source on Windows! I am beginning to feel it. > You need Fortran and C/C++ compilers on Windows to build scipy from source I have MinGW-w64 installed, which seems to be the recommended method. > I?m pretty sure that anaconda does not come with the development files for MKL, only the runtime files. I understand now. It looks like MKL is not the way to go unless I want to pay Intel. > If you don't need mkl and lapack/blas is good enough, then m2w64-toolchain from conda should have all necessary dependencies for building scipy. My only goal is to install and use Scipy master somewhere where it won't break my stable installation. I thought Anaconda would be a good place to start because once I activate an Anaconda environment, I should be able to treat like a normal Python installation and follow the normal install-from-source instructions. I went ahead and installed that m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I should change my question to: how do I install Scipy on Windows from source? Though when I search for this specifically on the web, the answer seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by Scipy. Do you know of a conda/pip package that provides ATLAS for building Scipy or another more suitable BLAS/LAPACK? -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthieu.brucher at gmail.com Sun Apr 30 18:22:03 2017 From: matthieu.brucher at gmail.com (Matthieu Brucher) Date: Sun, 30 Apr 2017 23:22:03 +0100 Subject: [SciPy-User] Install Scipy with Anaconda's MKL libraries In-Reply-To: References: Message-ID: Why do you want to pay Intel? You can install the MKL and develop with it, no sweat. 2017-04-30 22:41 GMT+01:00 David Hagen : > > Welcome to the world of pain with building scientific packages from > source on Windows! > > I am beginning to feel it. > > > You need Fortran and C/C++ compilers on Windows to build scipy from > source > > I have MinGW-w64 installed, which seems to be the recommended method. > > > I?m pretty sure that anaconda does not come with the development files > for MKL, only the runtime files. > > I understand now. It looks like MKL is not the way to go unless I want to > pay Intel. > > > If you don't need mkl and lapack/blas is good enough, then > m2w64-toolchain from conda should have all necessary dependencies for > building scipy. > > My only goal is to install and use Scipy master somewhere where it won't > break my stable installation. I thought Anaconda would be a good place to > start because once I activate an Anaconda environment, I should be able to > treat like a normal Python installation and follow the normal > install-from-source instructions. I went ahead and installed that > m2w64-toolchain package, but it still doesn't find any BLAS/LAPACK. Maybe I > should change my question to: how do I install Scipy on Windows from > source? Though when I search for this specifically on the web, the answer > seems to be "Don't.". It seems that MinGW-w64 and ATLAS are recommended by > Scipy. Do you know of a conda/pip package that provides ATLAS for building > Scipy or another more suitable BLAS/LAPACK? > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -- Information System Engineer, Ph.D. Blog: http://blog.audio-tk.com/ LinkedIn: http://www.linkedin.com/in/matthieubrucher -------------- next part -------------- An HTML attachment was scrubbed... URL: