From robert.pollak at jku.at Tue Feb 7 10:28:01 2017 From: robert.pollak at jku.at (Robert Pollak) Date: Tue, 7 Feb 2017 16:28:01 +0100 Subject: [SciPy-User] EuroSciPy 2017 Message-ID: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> Dear EuroSciPy team, I have noticed that EuroSciPy 2017 in Erlangen is announced as "upcoming" on https://conference.scipy.org/ . Is there already a date fixed? Best regards, Robert From sylvain.corlay at gmail.com Wed Feb 8 09:41:11 2017 From: sylvain.corlay at gmail.com (Sylvain Corlay) Date: Wed, 8 Feb 2017 15:41:11 +0100 Subject: [SciPy-User] EuroSciPy 2017 In-Reply-To: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> References: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> Message-ID: Also, the date for Scipy Austin 2017 is in 2016. :) Sylvain On Tue, Feb 7, 2017 at 4:28 PM, Robert Pollak wrote: > Dear EuroSciPy team, > > I have noticed that EuroSciPy 2017 in Erlangen is announced as > "upcoming" on https://conference.scipy.org/ . Is there already a date > fixed? > > Best regards, > Robert > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -------------- next part -------------- An HTML attachment was scrubbed... URL: From pierre.debuyl at kuleuven.be Wed Feb 8 13:02:03 2017 From: pierre.debuyl at kuleuven.be (Pierre de Buyl) Date: Wed, 8 Feb 2017 19:02:03 +0100 Subject: [SciPy-User] EuroSciPy 2017 In-Reply-To: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> References: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> Message-ID: <20170208180203.GJ14363@pi-x230> Dear Robert, The official contact information for EuroSciPy is euroscipy-org at python.org or https://twitter.com/EuroSciPy The dates are not set yet as we are trying to arrange for the venue (the previous venue could not be taken at our typical dates). The official announcement will be posted on our Twitter account and also on the NumPy and SciPy lists. Best regards, Pierre On Tue, Feb 07, 2017 at 04:28:01PM +0100, Robert Pollak wrote: > Dear EuroSciPy team, > > I have noticed that EuroSciPy 2017 in Erlangen is announced as > "upcoming" on https://conference.scipy.org/ . Is there already a date fixed? > > Best regards, > Robert From robert.pollak at jku.at Thu Feb 9 02:25:19 2017 From: robert.pollak at jku.at (Robert Pollak) Date: Thu, 9 Feb 2017 08:25:19 +0100 Subject: [SciPy-User] EuroSciPy 2017 In-Reply-To: <20170208180203.GJ14363@pi-x230> References: <0a08235d-eb73-56ee-b937-df840d4b4176@jku.at> <20170208180203.GJ14363@pi-x230> Message-ID: <7de7c916-8e57-dd85-b126-df46403cb8cf@jku.at> On 2017-02-08 19:02, Pierre de Buyl wrote: > The dates [for EuroSciPy 2017] are not set yet as we are trying to arrange for the venue Thank you for the update! I am looking forward to the announcement. > The official contact information for EuroSciPy is > euroscipy-org at python.org or https://twitter.com/EuroSciPy Ah - I just found this email address hidden in the HTML source of https://www.euroscipy.org/. Could you perhaps make it more visible there? Best regards, Robert From npropadovic at gmail.com Wed Feb 22 02:41:06 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Wed, 22 Feb 2017 08:41:06 +0100 Subject: [SciPy-User] Hello and first basic question Message-ID: Hello everybody, I'd like to be sure: is this list the right place to ask detailed questions about SciPy- (and maybe Numpy-) usage? When I was trying to find answers to my questions some two weeks ago I was rather confused with the original list basically ending around November last year ... Yeah, yesterday I digged in again, and saw the post announcing the switch to this location and now I'm here... But still, all posts of some depth related to the questions I have seem to be at stackoverflow... And I even saw a post somewhere recommending to use stackoverflow for SciPy-related questions. So may I ask what's the intention? Is stackoverflow actually the better place? And related to that, is there a BD (as in BDFL) here? Or some group of BD-s? I find it amazing that while math-applications of Python are all the rage nowdays, this group, which would naturally be the center of it all, is very quiet. Anyway, for my next (technical) question, I'll stick around here for the moment. Thanx in advance, Nenad -------------- next part -------------- An HTML attachment was scrubbed... URL: From jni.soma at gmail.com Wed Feb 22 04:04:56 2017 From: jni.soma at gmail.com (Juan Nunez-Iglesias) Date: Wed, 22 Feb 2017 20:04:56 +1100 Subject: [SciPy-User] Hello and first basic question In-Reply-To: References: Message-ID: <977893b5-4573-4a4f-a373-cd3c8aebb433@Spark> Hey Nenad, I'm not sure about BDs or email volume, but I follow this list and have found it informative. I don't generally monitor SO but obviously Google often directs me there. In short, I'd recommend subscribing to this list. The low volume might be an advantage ? and perhaps can be interpreted as high signal-to-noise. (Present thread excepted. =P) Juan. On 22 Feb 2017, 6:47 PM +1100, Propadovic Nenad , wrote: > Hello everybody, > > I'd like to be sure: is this list the right place to ask detailed questions about SciPy- (and maybe Numpy-) usage? > > When I was trying to find answers to my questions some two weeks ago I was rather confused with the original list basically ending around November last year ... Yeah, yesterday I digged in again, and saw the post announcing the switch to this location and now I'm here... But still, all posts of some depth related to the questions I have seem to be at stackoverflow... And I even saw a post somewhere recommending to use stackoverflow for SciPy-related questions. So may I ask what's the intention? Is stackoverflow actually the better place? > > And related to that, is there a BD (as in BDFL) here? Or some group of BD-s? > I find it amazing that while math-applications of Python are all the rage nowdays, this group, which would naturally be the center of it all, is very quiet. > > Anyway, for my next (technical) question, I'll stick around here for the moment. > > Thanx in advance, > Nenad > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From npropadovic at gmail.com Wed Feb 22 06:02:44 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Wed, 22 Feb 2017 12:02:44 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat Message-ID: Hello, bear with me for the long post that follows: it took me more than a week to get this far, and I tried to compress all the relevant information into the post. There seems to be a bug in scipy.io.loadmat; I'll present it by a short piece of code and it's output. I create file x.mat with the following: import scipy.io d = {'CanData': { 'msg': { 'RPDO2': { 'timest': [0.0, 0.0039679999899817631, 0.0079779999941820279], 'sig': { 'VelAct': { 'Values': [-0.050000000000000003, -0.10000000000000001, 0.29999999999999999, ], 'Name': 'VelAct' }, 'PosAct': { 'Values': [61.960000000000001, 61.960000000000001, 61.960000000000001, ], 'Name': 'PosAct' } } } } } } scipy.io.savemat("x.mat", d) Matlab is happy with the file and handles it the way I expect. When I read in the data stored in the file and print it out: import scipy.io y = scipy.io.loadmat("x.mat") # print y cd = y['CanData'] msg = cd['msg'] print msg print msg.dtype print msg.dtype.names The output is: >C:\Anaconda2\pythonw -u "test1.py" [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), array([u'PosAct'], dtype='C:\Anaconda2\pythonw -u "test1.py" ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')), dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),) object None >Exit code: 0 All well, but the name 'RPDO2' disapeared from the data! Now I need this information; in future I won't handle what's put into x.mat, so I need a way to access through the data all the way down (and handle the variations that will come). I have found a workaround at: http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/ The problem is, the workaround uses struct_as_record=False in loadmat, and which boils down to using scipy.io.matlab.mio5_params.mat_struct, and when you read the docstring of class mat_struct, it says: ''' ... We deprecate this method of holding struct information, and will soon remove it, in favor of the recarray method (see loadmat docstring) ''' So my questions: 1) Did I miss something? Is there a way to access the data in 'RPDO2' by using this name, without using parameter struct_as_record=False in loadmat? 2) If not, where do I file a bug? The workaround is five years old, so the issue seems to be in scipy for ages... (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but this does not seem to matter). Thanks a lot for the answers, in advance. Nenad -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregor.thalhammer at gmail.com Wed Feb 22 11:02:53 2017 From: gregor.thalhammer at gmail.com (Gregor Thalhammer) Date: Wed, 22 Feb 2017 17:02:53 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: Message-ID: <905AD1C7-BA68-47AE-B728-F88CC4F89CAD@gmail.com> > Am 22.02.2017 um 12:02 schrieb Propadovic Nenad : > > Hello, > > bear with me for the long post that follows: it took me more than a week to get this far, and I tried to compress all the relevant information into the post. > > There seems to be a bug in scipy.io.loadmat; I'll present it by a short piece of code and it's output. > > I create file x.mat with the following: > > import scipy.io > > d = {'CanData': > { > 'msg': { > 'RPDO2': { > 'timest': [0.0, 0.0039679999899817631, 0.0079779999941820279], > 'sig': { > 'VelAct': { > 'Values': [-0.050000000000000003, -0.10000000000000001, 0.29999999999999999, ], > 'Name': 'VelAct' > }, > 'PosAct': { > 'Values': [61.960000000000001, 61.960000000000001, 61.960000000000001, ], > 'Name': 'PosAct' > } > } > } > } > } > } > scipy.io.savemat("x.mat", d) > > Matlab is happy with the file and handles it the way I expect. > > When I read in the data stored in the file and print it out: > > import scipy.io > y = scipy.io.loadmat("x.mat") > # print y > cd = y['CanData'] > msg = cd['msg'] > print msg > print msg.dtype > print msg.dtype.names > > The output is: > >C:\Anaconda2\pythonw -u "test1.py" > [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), array([u'PosAct'], > dtype=' dtype=' dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , 0.003968, 0.007978]]))]],)]], > dtype=[('RPDO2', 'O')])]] > object > None > > Now I've read the manual, and as I see it I have no way for me to access the deeper layers of data I just put in the file x.mat, although they are obviously right there in the data read in. Access via msg['RPDO2'] gives: > IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis (`None`) and integer or boolean arrays are valid indices. For historic reasons, in Matlab everything is at least a 2D array, even scalars. By sprinkling some [0,0] in your code you should get what you want, e.g. msg[0,0]['RPDO2'][0,0]['timest'][0,0] array([[ 0. , 0.003968, 0.007978]]) Gregor > > If I use parameter squeeze_me=True: > > scipy.io.savemat("x.mat", d) > y = scipy.io.loadmat("x.mat", squeeze_me=True) > # print y > cd = y['CanData'] > msg = cd['msg'] > print msg > print msg.dtype > print msg.dtype.names > > I get output: > >C:\Anaconda2\pythonw -u "test1.py" > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')), > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),) > object > None > >Exit code: 0 > > All well, but the name 'RPDO2' disapeared from the data! > > Now I need this information; in future I won't handle what's put into x.mat, so I need a way to access through the data all the way down (and handle the variations that will come). > > I have found a workaround at: > http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/ > > The problem is, the workaround uses struct_as_record=False in loadmat, and which boils down to using scipy.io.matlab.mio5_params.mat_struct, and when you read the docstring of class mat_struct, it says: > > ''' > ... > We deprecate this method of holding struct information, and will > soon remove it, in favor of the recarray method (see loadmat > docstring) > ''' > So my questions: > 1) Did I miss something? Is there a way to access the data in 'RPDO2' by using this name, without using parameter struct_as_record=False in loadmat? > 2) If not, where do I file a bug? The workaround is five years old, so the issue seems to be in scipy for ages... > > (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but this does not seem to matter). > > Thanks a lot for the answers, in advance. > > Nenad > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From jmsachs at gmail.com Wed Feb 22 11:11:16 2017 From: jmsachs at gmail.com (Jason Sachs) Date: Wed, 22 Feb 2017 09:11:16 -0700 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: <905AD1C7-BA68-47AE-B728-F88CC4F89CAD@gmail.com> References: <905AD1C7-BA68-47AE-B728-F88CC4F89CAD@gmail.com> Message-ID: This looks familiar, I ran into this a few years ago, and if I recall correctly, there is an option to loadmat to reduce array dimensions appropriately. There is a "squeeze_me" option (unfortunately named... should probably be deprecated in favor of "squeeze") which I think does this. https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < gregor.thalhammer at gmail.com> wrote: > > Am 22.02.2017 um 12:02 schrieb Propadovic Nenad : > > Hello, > > bear with me for the long post that follows: it took me more than a week > to get this far, and I tried to compress all the relevant information into > the post. > > There seems to be a bug in scipy.io.loadmat; I'll present it by a short > piece of code and it's output. > > I create file x.mat with the following: > > import scipy.io > > d = {'CanData': > { > 'msg': { > 'RPDO2': { > 'timest': [0.0, 0.0039679999899817631, > 0.0079779999941820279], > 'sig': { > 'VelAct': { > 'Values': [-0.050000000000000003, > -0.10000000000000001, 0.29999999999999999, ], > 'Name': 'VelAct' > }, > 'PosAct': { > 'Values': [61.960000000000001, 61.960000000000001, > 61.960000000000001, ], > 'Name': 'PosAct' > } > } > } > } > } > } > scipy.io.savemat("x.mat", d) > > Matlab is happy with the file and handles it the way I expect. > > When I read in the data stored in the file and print it out: > > import scipy.io > y = scipy.io.loadmat("x.mat") > # print y > cd = y['CanData'] > msg = cd['msg'] > print msg > print msg.dtype > print msg.dtype.names > > The output is: > >C:\Anaconda2\pythonw -u "test1.py" > [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > array([u'PosAct'], > dtype=' array([u'VelAct'], > dtype=' dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > 0.003968, 0.007978]]))]],)]], > dtype=[('RPDO2', 'O')])]] > object > None > > Now I've read the manual, and as I see it I have no way for me to access > the deeper layers of data I just put in the file x.mat, although they are > obviously right there in the data read in. Access via msg['RPDO2'] gives: > IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > (`None`) and integer or boolean arrays are valid indices. > > > For historic reasons, in Matlab everything is at least a 2D array, even > scalars. By sprinkling some [0,0] in your code you should get what you > want, e.g. > > msg[0,0]['RPDO2'][0,0]['timest'][0,0] > > array([[ 0. , 0.003968, 0.007978]]) > > > Gregor > > > > If I use parameter squeeze_me=True: > > scipy.io.savemat("x.mat", d) > y = scipy.io.loadmat("x.mat", squeeze_me=True) > # print y > cd = y['CanData'] > msg = cd['msg'] > print msg > print msg.dtype > print msg.dtype.names > > I get output: > >C:\Anaconda2\pythonw -u "test1.py" > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 > , 0.3 ]), u'VelAct')), > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > 0.003968, 0.007978])),) > object > None > >Exit code: 0 > > All well, but the name 'RPDO2' disapeared from the data! > > Now I need this information; in future I won't handle what's put into > x.mat, so I need a way to access through the data all the way down (and > handle the variations that will come). > > I have found a workaround at: > http://stackoverflow.com/questions/7008608/scipy-io- > loadmat-nested-structures-i-e-dictionaries/ > > The problem is, the workaround uses struct_as_record=False in loadmat, and > which boils down to using scipy.io.matlab.mio5_params.mat_struct, and > when you read the docstring of class mat_struct, it says: > > ''' > ... > We deprecate this method of holding struct information, and will > soon remove it, in favor of the recarray method (see loadmat > docstring) > ''' > So my questions: > 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > using this name, without using parameter struct_as_record=False in loadmat? > 2) If not, where do I file a bug? The workaround is five years old, so the > issue seems to be in scipy for ages... > > (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > this does not seem to matter). > > Thanks a lot for the answers, in advance. > > Nenad > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- An HTML attachment was scrubbed... URL: From jmsachs at gmail.com Wed Feb 22 11:12:26 2017 From: jmsachs at gmail.com (Jason Sachs) Date: Wed, 22 Feb 2017 09:12:26 -0700 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: <905AD1C7-BA68-47AE-B728-F88CC4F89CAD@gmail.com> Message-ID: ah, yes, here it is: https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html ---- So, in MATLAB, the struct array must be at least 2D, and we replicate that when we read into Scipy. If you want all length 1 dimensions squeezed out, try this: >>> >>> mat_contents = sio.loadmat('octave_struct.mat', squeeze_me=True)>>> oct_struct = mat_contents['my_struct']>>> oct_struct.shape() On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs wrote: > This looks familiar, I ran into this a few years ago, and if I recall > correctly, there is an option to loadmat to reduce array dimensions > appropriately. There is a "squeeze_me" option (unfortunately named... > should probably be deprecated in favor of "squeeze") which I think does > this. > > https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > gregor.thalhammer at gmail.com> wrote: > >> >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad : >> >> Hello, >> >> bear with me for the long post that follows: it took me more than a week >> to get this far, and I tried to compress all the relevant information into >> the post. >> >> There seems to be a bug in scipy.io.loadmat; I'll present it by a short >> piece of code and it's output. >> >> I create file x.mat with the following: >> >> import scipy.io >> >> d = {'CanData': >> { >> 'msg': { >> 'RPDO2': { >> 'timest': [0.0, 0.0039679999899817631, >> 0.0079779999941820279], >> 'sig': { >> 'VelAct': { >> 'Values': [-0.050000000000000003, >> -0.10000000000000001, 0.29999999999999999, ], >> 'Name': 'VelAct' >> }, >> 'PosAct': { >> 'Values': [61.960000000000001, >> 61.960000000000001, 61.960000000000001, ], >> 'Name': 'PosAct' >> } >> } >> } >> } >> } >> } >> scipy.io.savemat("x.mat", d) >> >> Matlab is happy with the file and handles it the way I expect. >> >> When I read in the data stored in the file and print it out: >> >> import scipy.io >> y = scipy.io.loadmat("x.mat") >> # print y >> cd = y['CanData'] >> msg = cd['msg'] >> print msg >> print msg.dtype >> print msg.dtype.names >> >> The output is: >> >C:\Anaconda2\pythonw -u "test1.py" >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), >> array([u'PosAct'], >> dtype='> array([u'VelAct'], >> dtype='> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , >> 0.003968, 0.007978]]))]],)]], >> dtype=[('RPDO2', 'O')])]] >> object >> None >> >> Now I've read the manual, and as I see it I have no way for me to access >> the deeper layers of data I just put in the file x.mat, although they are >> obviously right there in the data read in. Access via msg['RPDO2'] gives: >> IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis >> (`None`) and integer or boolean arrays are valid indices. >> >> >> For historic reasons, in Matlab everything is at least a 2D array, even >> scalars. By sprinkling some [0,0] in your code you should get what you >> want, e.g. >> >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] >> >> array([[ 0. , 0.003968, 0.007978]]) >> >> >> Gregor >> >> >> >> If I use parameter squeeze_me=True: >> >> scipy.io.savemat("x.mat", d) >> y = scipy.io.loadmat("x.mat", squeeze_me=True) >> # print y >> cd = y['CanData'] >> msg = cd['msg'] >> print msg >> print msg.dtype >> print msg.dtype.names >> >> I get output: >> >C:\Anaconda2\pythonw -u "test1.py" >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, >> -0.1 , 0.3 ]), u'VelAct')), >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , >> 0.003968, 0.007978])),) >> object >> None >> >Exit code: 0 >> >> All well, but the name 'RPDO2' disapeared from the data! >> >> Now I need this information; in future I won't handle what's put into >> x.mat, so I need a way to access through the data all the way down (and >> handle the variations that will come). >> >> I have found a workaround at: >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- >> nested-structures-i-e-dictionaries/ >> >> The problem is, the workaround uses struct_as_record=False in loadmat, >> and which boils down to using scipy.io.matlab.mio5_params.mat_struct, >> and when you read the docstring of class mat_struct, it says: >> >> ''' >> ... >> We deprecate this method of holding struct information, and will >> soon remove it, in favor of the recarray method (see loadmat >> docstring) >> ''' >> So my questions: >> 1) Did I miss something? Is there a way to access the data in 'RPDO2' by >> using this name, without using parameter struct_as_record=False in loadmat? >> 2) If not, where do I file a bug? The workaround is five years old, so >> the issue seems to be in scipy for ages... >> >> (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but >> this does not seem to matter). >> >> Thanks a lot for the answers, in advance. >> >> Nenad >> >> >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> >> >> >> _______________________________________________ >> SciPy-User mailing list >> SciPy-User at python.org >> https://mail.python.org/mailman/listinfo/scipy-user >> >> > -------------- next part -------------- An HTML attachment was scrubbed... URL: From npropadovic at gmail.com Thu Feb 23 07:50:16 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Thu, 23 Feb 2017 13:50:16 +0100 Subject: [SciPy-User] SciPy-User Digest, Vol 162, Issue 5 In-Reply-To: References: Message-ID: Hello Gregor, thank you for the answer. I was aware of the need to access "Matlab scalars" in scipy by "sprinkling in [0,0]" in the access path. This, however, does not solve my problem, as I might get parts of the access path with some kind of wildcard. For example, I might get the 'RPDO2'-part in the form of '*DO2'. As I have no way to inspect the names of the sub-structures from the imported Matlab data, I can't even try to build all the possible names with '*DO2', as their number is infinite. So, as I said, if I don't know exactly, that 'RPDO2' will be part of my path, this does not save me. (I can assure that the requirement to get parts of the access path in such a vague form makes sense in the context of the environment of the client I work for). Regards, Nenad 2017-02-22 17:11 GMT+01:00 : > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Gregor Thalhammer) > 2. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Jason Sachs) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 22 Feb 2017 17:02:53 +0100 > From: Gregor Thalhammer > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: <905AD1C7-BA68-47AE-B728-F88CC4F89CAD at gmail.com> > Content-Type: text/plain; charset="us-ascii" > > > > Am 22.02.2017 um 12:02 schrieb Propadovic Nenad : > > > > Hello, > > > > bear with me for the long post that follows: it took me more than a week > to get this far, and I tried to compress all the relevant information into > the post. > > > > There seems to be a bug in scipy.io.loadmat; I'll present it by a short > piece of code and it's output. > > > > I create file x.mat with the following: > > > > import scipy.io > > > > d = {'CanData': > > { > > 'msg': { > > 'RPDO2': { > > 'timest': [0.0, 0.0039679999899817631, > 0.0079779999941820279], > > 'sig': { > > 'VelAct': { > > 'Values': [-0.050000000000000003, > -0.10000000000000001, 0.29999999999999999, ], > > 'Name': 'VelAct' > > }, > > 'PosAct': { > > 'Values': [61.960000000000001, > 61.960000000000001, 61.960000000000001, ], > > 'Name': 'PosAct' > > } > > } > > } > > } > > } > > } > > scipy.io.savemat("x.mat", d) > > > > Matlab is happy with the file and handles it the way I expect. > > > > When I read in the data stored in the file and print it out: > > > > import scipy.io > > y = scipy.io.loadmat("x.mat") > > # print y > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > print msg.dtype > > print msg.dtype.names > > > > The output is: > > >C:\Anaconda2\pythonw -u "test1.py" > > [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > array([u'PosAct'], > > dtype=' array([u'VelAct'], > > dtype=' > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > 0.003968, 0.007978]]))]],)]], > > dtype=[('RPDO2', 'O')])]] > > object > > None > > > > Now I've read the manual, and as I see it I have no way for me to > access the deeper layers of data I just put in the file x.mat, although > they are obviously right there in the data read in. Access via msg['RPDO2'] > gives: > > IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > (`None`) and integer or boolean arrays are valid indices. > > For historic reasons, in Matlab everything is at least a 2D array, even > scalars. By sprinkling some [0,0] in your code you should get what you > want, e.g. > > msg[0,0]['RPDO2'][0,0]['timest'][0,0] > array([[ 0. , 0.003968, 0.007978]]) > > Gregor > > > > > > If I use parameter squeeze_me=True: > > > > scipy.io.savemat("x.mat", d) > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > # print y > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > print msg.dtype > > print msg.dtype.names > > > > I get output: > > >C:\Anaconda2\pythonw -u "test1.py" > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 , 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > 0.003968, 0.007978])),) > > object > > None > > >Exit code: 0 > > > > All well, but the name 'RPDO2' disapeared from the data! > > > > Now I need this information; in future I won't handle what's put into > x.mat, so I need a way to access through the data all the way down (and > handle the variations that will come). > > > > I have found a workaround at: > > http://stackoverflow.com/questions/7008608/scipy-io- > loadmat-nested-structures-i-e-dictionaries/ questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/> > > > > The problem is, the workaround uses struct_as_record=False in loadmat, > and which boils down to using scipy.io.matlab.mio5_params.mat_struct, and > when you read the docstring of class mat_struct, it says: > > > > ''' > > ... > > We deprecate this method of holding struct information, and will > > soon remove it, in favor of the recarray method (see loadmat > > docstring) > > ''' > > So my questions: > > 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > using this name, without using parameter struct_as_record=False in loadmat? > > 2) If not, where do I file a bug? The workaround is five years old, so > the issue seems to be in scipy for ages... > > > > (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > this does not seem to matter). > > > > Thanks a lot for the answers, in advance. > > > > Nenad > > > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: attachments/20170222/8be8fa5c/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Wed, 22 Feb 2017 09:11:16 -0700 > From: Jason Sachs > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: > gmail.com> > Content-Type: text/plain; charset="utf-8" > > This looks familiar, I ran into this a few years ago, and if I recall > correctly, there is an option to loadmat to reduce array dimensions > appropriately. There is a "squeeze_me" option (unfortunately named... > should probably be deprecated in favor of "squeeze") which I think does > this. > > https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > gregor.thalhammer at gmail.com> wrote: > > > > > Am 22.02.2017 um 12:02 schrieb Propadovic Nenad : > > > > Hello, > > > > bear with me for the long post that follows: it took me more than a week > > to get this far, and I tried to compress all the relevant information > into > > the post. > > > > There seems to be a bug in scipy.io.loadmat; I'll present it by a short > > piece of code and it's output. > > > > I create file x.mat with the following: > > > > import scipy.io > > > > d = {'CanData': > > { > > 'msg': { > > 'RPDO2': { > > 'timest': [0.0, 0.0039679999899817631, > > 0.0079779999941820279], > > 'sig': { > > 'VelAct': { > > 'Values': [-0.050000000000000003, > > -0.10000000000000001, 0.29999999999999999, ], > > 'Name': 'VelAct' > > }, > > 'PosAct': { > > 'Values': [61.960000000000001, > 61.960000000000001, > > 61.960000000000001, ], > > 'Name': 'PosAct' > > } > > } > > } > > } > > } > > } > > scipy.io.savemat("x.mat", d) > > > > Matlab is happy with the file and handles it the way I expect. > > > > When I read in the data stored in the file and print it out: > > > > import scipy.io > > y = scipy.io.loadmat("x.mat") > > # print y > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > print msg.dtype > > print msg.dtype.names > > > > The output is: > > >C:\Anaconda2\pythonw -u "test1.py" > > [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > > array([u'PosAct'], > > dtype=' > array([u'VelAct'], > > dtype=' > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > > 0.003968, 0.007978]]))]],)]], > > dtype=[('RPDO2', 'O')])]] > > object > > None > > > > Now I've read the manual, and as I see it I have no way for me to access > > the deeper layers of data I just put in the file x.mat, although they are > > obviously right there in the data read in. Access via msg['RPDO2'] gives: > > IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > > (`None`) and integer or boolean arrays are valid indices. > > > > > > For historic reasons, in Matlab everything is at least a 2D array, even > > scalars. By sprinkling some [0,0] in your code you should get what you > > want, e.g. > > > > msg[0,0]['RPDO2'][0,0]['timest'][0,0] > > > > array([[ 0. , 0.003968, 0.007978]]) > > > > > > Gregor > > > > > > > > If I use parameter squeeze_me=True: > > > > scipy.io.savemat("x.mat", d) > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > # print y > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > print msg.dtype > > print msg.dtype.names > > > > I get output: > > >C:\Anaconda2\pythonw -u "test1.py" > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 > > , 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > 0.003968, 0.007978])),) > > object > > None > > >Exit code: 0 > > > > All well, but the name 'RPDO2' disapeared from the data! > > > > Now I need this information; in future I won't handle what's put into > > x.mat, so I need a way to access through the data all the way down (and > > handle the variations that will come). > > > > I have found a workaround at: > > http://stackoverflow.com/questions/7008608/scipy-io- > > loadmat-nested-structures-i-e-dictionaries/ > > > > The problem is, the workaround uses struct_as_record=False in loadmat, > and > > which boils down to using scipy.io.matlab.mio5_params.mat_struct, and > > when you read the docstring of class mat_struct, it says: > > > > ''' > > ... > > We deprecate this method of holding struct information, and will > > soon remove it, in favor of the recarray method (see loadmat > > docstring) > > ''' > > So my questions: > > 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > > using this name, without using parameter struct_as_record=False in > loadmat? > > 2) If not, where do I file a bug? The workaround is five years old, so > the > > issue seems to be in scipy for ages... > > > > (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > > this does not seem to matter). > > > > Thanks a lot for the answers, in advance. > > > > Nenad > > > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > > > > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: attachments/20170222/a4113f89/attachment.html> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 5 > ****************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From npropadovic at gmail.com Thu Feb 23 08:12:06 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Thu, 23 Feb 2017 14:12:06 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat Message-ID: Hello Jason, than you a lot for the answer to my post. I was aware of the squeeze_me=True option, I think I mentioned it in my initial question post. However, as I stated in the answer to Gregors kind answer, I actually need a way to inspect the parts of the access path in the data structure I import form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' disappear completely: import scipy.io y = scipy.io.loadmat("x.mat", squeeze_me=True) cd = y['CanData'] msg = cd['msg'] print msg Output: ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')), dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),) And I really need to be able to find it by some kind of inspection, so that I don't return parts of the structure that don't correspond to the intention of the person searching. Thanks once again! Nenad 2017-02-22 18:00 GMT+01:00 : > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Jason Sachs) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 22 Feb 2017 09:12:26 -0700 > From: Jason Sachs > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: > gmail.com> > Content-Type: text/plain; charset="utf-8" > > ah, yes, here it is: > > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html > > ---- > > So, in MATLAB, the struct array must be at least 2D, and we replicate that > when we read into Scipy. If you want all length 1 dimensions squeezed out, > try this: > >>> > > >>> mat_contents = sio.loadmat('octave_struct.mat', squeeze_me=True)>>> > oct_struct = mat_contents['my_struct']>>> oct_struct.shape() > > > On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs wrote: > > > This looks familiar, I ran into this a few years ago, and if I recall > > correctly, there is an option to loadmat to reduce array dimensions > > appropriately. There is a "squeeze_me" option (unfortunately named... > > should probably be deprecated in favor of "squeeze") which I think does > > this. > > > > https://docs.scipy.org/doc/scipy/reference/generated/ > scipy.io.loadmat.html > > > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > > gregor.thalhammer at gmail.com> wrote: > > > >> > >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad >: > >> > >> Hello, > >> > >> bear with me for the long post that follows: it took me more than a week > >> to get this far, and I tried to compress all the relevant information > into > >> the post. > >> > >> There seems to be a bug in scipy.io.loadmat; I'll present it by a short > >> piece of code and it's output. > >> > >> I create file x.mat with the following: > >> > >> import scipy.io > >> > >> d = {'CanData': > >> { > >> 'msg': { > >> 'RPDO2': { > >> 'timest': [0.0, 0.0039679999899817631, > >> 0.0079779999941820279], > >> 'sig': { > >> 'VelAct': { > >> 'Values': [-0.050000000000000003, > >> -0.10000000000000001, 0.29999999999999999, ], > >> 'Name': 'VelAct' > >> }, > >> 'PosAct': { > >> 'Values': [61.960000000000001, > >> 61.960000000000001, 61.960000000000001, ], > >> 'Name': 'PosAct' > >> } > >> } > >> } > >> } > >> } > >> } > >> scipy.io.savemat("x.mat", d) > >> > >> Matlab is happy with the file and handles it the way I expect. > >> > >> When I read in the data stored in the file and print it out: > >> > >> import scipy.io > >> y = scipy.io.loadmat("x.mat") > >> # print y > >> cd = y['CanData'] > >> msg = cd['msg'] > >> print msg > >> print msg.dtype > >> print msg.dtype.names > >> > >> The output is: > >> >C:\Anaconda2\pythonw -u "test1.py" > >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > >> array([u'PosAct'], > >> dtype=' >> array([u'VelAct'], > >> dtype=' >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > >> 0.003968, 0.007978]]))]],)]], > >> dtype=[('RPDO2', 'O')])]] > >> object > >> None > >> > >> Now I've read the manual, and as I see it I have no way for me to > access > >> the deeper layers of data I just put in the file x.mat, although they > are > >> obviously right there in the data read in. Access via msg['RPDO2'] > gives: > >> IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > >> (`None`) and integer or boolean arrays are valid indices. > >> > >> > >> For historic reasons, in Matlab everything is at least a 2D array, even > >> scalars. By sprinkling some [0,0] in your code you should get what you > >> want, e.g. > >> > >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] > >> > >> array([[ 0. , 0.003968, 0.007978]]) > >> > >> > >> Gregor > >> > >> > >> > >> If I use parameter squeeze_me=True: > >> > >> scipy.io.savemat("x.mat", d) > >> y = scipy.io.loadmat("x.mat", squeeze_me=True) > >> # print y > >> cd = y['CanData'] > >> msg = cd['msg'] > >> print msg > >> print msg.dtype > >> print msg.dtype.names > >> > >> I get output: > >> >C:\Anaconda2\pythonw -u "test1.py" > >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > >> -0.1 , 0.3 ]), u'VelAct')), > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > >> 0.003968, 0.007978])),) > >> object > >> None > >> >Exit code: 0 > >> > >> All well, but the name 'RPDO2' disapeared from the data! > >> > >> Now I need this information; in future I won't handle what's put into > >> x.mat, so I need a way to access through the data all the way down (and > >> handle the variations that will come). > >> > >> I have found a workaround at: > >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- > >> nested-structures-i-e-dictionaries/ > >> > >> The problem is, the workaround uses struct_as_record=False in loadmat, > >> and which boils down to using scipy.io.matlab.mio5_params.mat_struct, > >> and when you read the docstring of class mat_struct, it says: > >> > >> ''' > >> ... > >> We deprecate this method of holding struct information, and will > >> soon remove it, in favor of the recarray method (see loadmat > >> docstring) > >> ''' > >> So my questions: > >> 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > >> using this name, without using parameter struct_as_record=False in > loadmat? > >> 2) If not, where do I file a bug? The workaround is five years old, so > >> the issue seems to be in scipy for ages... > >> > >> (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > >> this does not seem to matter). > >> > >> Thanks a lot for the answers, in advance. > >> > >> Nenad > >> > >> > >> _______________________________________________ > >> SciPy-User mailing list > >> SciPy-User at python.org > >> https://mail.python.org/mailman/listinfo/scipy-user > >> > >> > >> > >> _______________________________________________ > >> SciPy-User mailing list > >> SciPy-User at python.org > >> https://mail.python.org/mailman/listinfo/scipy-user > >> > >> > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: attachments/20170222/e992b074/attachment-0001.html> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 6 > ****************************************** > -------------- next part -------------- An HTML attachment was scrubbed... URL: From fausto_barbuto at yahoo.ca Thu Feb 23 08:19:36 2017 From: fausto_barbuto at yahoo.ca (Fausto Arinos de A. Barbuto) Date: Thu, 23 Feb 2017 13:19:36 +0000 (UTC) Subject: [SciPy-User] Hello and first basic question References: <799566841.3209435.1487855976254.ref@mail.yahoo.com> Message-ID: <799566841.3209435.1487855976254@mail.yahoo.com> Hello Propadovic, This is a rather quiet and infrequent list. I joined it because I had a simplenevertheless important (to me) question about SciPy installation. Never gotany replies. C'est la vie. Maybe the question wasn't that simple after all.Now that you mentioned that people resort to StackOverflow to get answers totheir SciPy-related questions I'm pondering whether I shouldn't give SO a shot.Who knows, eh? Good luck! Fausto -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Thu Feb 23 08:34:40 2017 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Thu, 23 Feb 2017 16:34:40 +0300 Subject: [SciPy-User] Hello and first basic question In-Reply-To: References: Message-ID: > I'd like to be sure: is this list the right place to ask detailed questions > about SciPy- (and maybe Numpy-) usage? Yes. > When I was trying to find answers to my questions some two weeks ago I was > rather confused with the original list basically ending around November last > year ... Yeah, yesterday I digged in again, and saw the post announcing the > switch to this location and now I'm here... But still, all posts of some > depth related to the questions I have seem to be at stackoverflow... And I > even saw a post somewhere recommending to use stackoverflow for > SciPy-related questions. So may I ask what's the intention? Is stackoverflow > actually the better place? Yeah, the move from an old list to this one was rather recent, and the we are still in transition. I for one, sent an email to an old list just a couple of days ago and had to stare a bit at an automated reply saying I'm not subscribed. Hopefully this list will pick it up --- we'll see if current users find Stackoverflow a more convenient medium. > And related to that, is there a BD (as in BDFL) here? Or some group of BD-s? Here's the relevant email thread from scipy-dev list: https://mail.scipy.org/pipermail/scipy-dev/2016-September/021519.html > I find it amazing that while math-applications of Python are all the rage > nowdays, this group, which would naturally be the center of it all, is very > quiet. > > Anyway, for my next (technical) question, I'll stick around here for the > moment. Sure! Like I said, this list is active if people use it, so ask away! Cheers, Evgeni From npropadovic at gmail.com Thu Feb 23 09:21:10 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Thu, 23 Feb 2017 15:21:10 +0100 Subject: [SciPy-User] Hello and first basic question Message-ID: Thanks, Evgeni. I think that people will choose whatever they perceive as THE place where scipy questions are answered. And from where I stand it seems that StackOveflow is that place. That's at least what google suggests. I've another question though, one that I already sked in the other thread: where do I report something I consider a bug? Regards, Nenad -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Thu Feb 23 09:34:42 2017 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Thu, 23 Feb 2017 17:34:42 +0300 Subject: [SciPy-User] Hello and first basic question In-Reply-To: References: Message-ID: > I think that people will choose whatever they perceive as THE place where > scipy questions are answered. And from where I stand it seems that > StackOveflow is that place. That's at least what google suggests. That's a self-organized system :-). > I've another question though, one that I already sked in the other thread: > where do I report something I consider a bug? Bug reports are welcome on the issue tracker, https://github.com/scipy/scipy Ideally, you file an issue with a reproducible example. If the decision is that it's not a bug, the issue might be closed --- nothing personal! Usage questions are best asked on this list. If this is a bug, you might be asked to file an issue on the tracker so that it's not lost. There is also the scipy-dev mailing list (the one I linked to up thread). The line between -dev and -user lists can be somewhat blurry at times, so questions can be directed over to that list as well. Cheers, Evgeni From matthew.brett at gmail.com Thu Feb 23 20:41:34 2017 From: matthew.brett at gmail.com (Matthew Brett) Date: Thu, 23 Feb 2017 17:41:34 -0800 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: Message-ID: Hi, On Thu, Feb 23, 2017 at 5:12 AM, Propadovic Nenad wrote: > Hello Jason, than you a lot for the answer to my post. I was aware of the > squeeze_me=True option, I think I mentioned it in my initial question post. > > However, as I stated in the answer to Gregors kind answer, I actually need a > way to inspect the parts of the access path in the data structure I import > form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' > disappear completely: > > > import scipy.io > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > cd = y['CanData'] > msg = cd['msg'] > print msg > > Output: > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , > 0.3 ]), u'VelAct')), > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > 0.003968, 0.007978])),) > > And I really need to be able to find it by some kind of inspection, so that > I don't return parts of the structure that don't correspond to the intention > of the person searching. Sorry if I'm not following, but, does this help? In [36]: y = scipy.io.loadmat("x.mat") In [37]: y['CanData'][0, 0]['msg'][0, 0]['RPDO2'] Out[37]: array([[ (array([[ 0. , 0.003968, 0.007978]]), array([[ (array([[(array([[ 61.96, 61.96, 61.96]]), array(['PosAct'], dtype=' Best, Matthew From s.atasever at gmail.com Fri Feb 24 01:59:04 2017 From: s.atasever at gmail.com (Sema Atasever) Date: Fri, 24 Feb 2017 09:59:04 +0300 Subject: [SciPy-User] Use Distance Matrix in scipy.cluster.hierarchy.linkage()? Message-ID: Dear SciPy list member, I want to ask you about clustering usign scipy.cluster.hierarchy. I have a *distance matrix* n*n M where M_ij is the distance between object_i and object_j. You can see file format in the attachment --> (dm.csv) I want to cluster these n objects with hierarchical clustering. For this purpose i am usign this python code that you can see in the attachment (scipy_code.py) I want to ask that how can get clusters values in pdf format or text format and how many clusters did i get and what clusters includes what members? Thanks in Advance, Best regards. -------------- next part -------------- An HTML attachment was scrubbed... URL: -------------- next part -------------- import scipy import pylab import scipy.cluster.hierarchy as sch import pandas as pd import numpy as np D=np.loadtxt(open("C:\dm.csv", "rb"), delimiter="\t", usecols=range(1,11)) print (D) print (D.shape) # Compute and plot dendrogram. fig = pylab.figure() axdendro = fig.add_axes([0.09,0.1,0.2,0.8]) Y = sch.linkage(D, method='single') Z = sch.dendrogram(Y, orientation='right') axdendro.set_xticks([]) axdendro.set_yticks([]) # Plot distance matrix. axmatrix = fig.add_axes([0.3,0.1,0.6,0.8]) index = Z['leaves'] D = D[index,:] D = D[:,index] im = axmatrix.matshow(D, aspect='auto', origin='lower') axmatrix.set_xticks([]) axmatrix.set_yticks([]) # Plot colorbar. axcolor = fig.add_axes([0.91,0.1,0.02,0.8]) pylab.colorbar(im, cax=axcolor) # Display and save figure. fig.show() fig.savefig('dendrogram.png') -------------- next part -------------- A non-text attachment was scrubbed... Name: dm.csv Type: text/csv Size: 680 bytes Desc: not available URL: From gregor.thalhammer at gmail.com Fri Feb 24 06:15:18 2017 From: gregor.thalhammer at gmail.com (Gregor Thalhammer) Date: Fri, 24 Feb 2017 12:15:18 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: Message-ID: <7CCED80F-462D-4BED-9540-21E262206886@gmail.com> > Am 23.02.2017 um 14:12 schrieb Propadovic Nenad : > > Hello Jason, than you a lot for the answer to my post. I was aware of the squeeze_me=True option, I think I mentioned it in my initial question post. > > However, as I stated in the answer to Gregors kind answer, I actually need a way to inspect the parts of the access path in the data structure I import form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' disappear completely: > The substruct names are somewhat hidden, try this: y = scipy.io.loadmat('x.mat', squeeze_me=True, struct_as_record=False) y['CanData'].msg._fieldnames ['RPDO2'] Gregor PS: for introspection of objects, just take a look at the __dict__ attribute. > > import scipy.io > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > cd = y['CanData'] > msg = cd['msg'] > print msg > > Output: > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')), > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),) > > And I really need to be able to find it by some kind of inspection, so that I don't return parts of the structure that don't correspond to the intention of the person searching. > > Thanks once again! > > Nenad > > > > 2017-02-22 18:00 GMT+01:00 >: > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Jason Sachs) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Wed, 22 Feb 2017 09:12:26 -0700 > From: Jason Sachs > > To: SciPy Users List > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: > > > Content-Type: text/plain; charset="utf-8" > > ah, yes, here it is: > > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html > > ---- > > So, in MATLAB, the struct array must be at least 2D, and we replicate that > when we read into Scipy. If you want all length 1 dimensions squeezed out, > try this: > >>> > > >>> mat_contents = sio.loadmat('octave_struct.mat', squeeze_me=True)>>> oct_struct = mat_contents['my_struct']>>> oct_struct.shape() > > > On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs > wrote: > > > This looks familiar, I ran into this a few years ago, and if I recall > > correctly, there is an option to loadmat to reduce array dimensions > > appropriately. There is a "squeeze_me" option (unfortunately named... > > should probably be deprecated in favor of "squeeze") which I think does > > this. > > > > https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html > > > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > > gregor.thalhammer at gmail.com > wrote: > > > >> > >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad >: > >> > >> Hello, > >> > >> bear with me for the long post that follows: it took me more than a week > >> to get this far, and I tried to compress all the relevant information into > >> the post. > >> > >> There seems to be a bug in scipy.io.loadmat; I'll present it by a short > >> piece of code and it's output. > >> > >> I create file x.mat with the following: > >> > >> import scipy.io > >> > >> d = {'CanData': > >> { > >> 'msg': { > >> 'RPDO2': { > >> 'timest': [0.0, 0.0039679999899817631, > >> 0.0079779999941820279], > >> 'sig': { > >> 'VelAct': { > >> 'Values': [-0.050000000000000003, > >> -0.10000000000000001, 0.29999999999999999, ], > >> 'Name': 'VelAct' > >> }, > >> 'PosAct': { > >> 'Values': [61.960000000000001, > >> 61.960000000000001, 61.960000000000001, ], > >> 'Name': 'PosAct' > >> } > >> } > >> } > >> } > >> } > >> } > >> scipy.io.savemat("x.mat", d) > >> > >> Matlab is happy with the file and handles it the way I expect. > >> > >> When I read in the data stored in the file and print it out: > >> > >> import scipy.io > >> y = scipy.io.loadmat("x.mat") > >> # print y > >> cd = y['CanData'] > >> msg = cd['msg'] > >> print msg > >> print msg.dtype > >> print msg.dtype.names > >> > >> The output is: > >> >C:\Anaconda2\pythonw -u "test1.py" > >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > >> array([u'PosAct'], > >> dtype=' >> array([u'VelAct'], > >> dtype=' >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > >> 0.003968, 0.007978]]))]],)]], > >> dtype=[('RPDO2', 'O')])]] > >> object > >> None > >> > >> Now I've read the manual, and as I see it I have no way for me to access > >> the deeper layers of data I just put in the file x.mat, although they are > >> obviously right there in the data read in. Access via msg['RPDO2'] gives: > >> IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > >> (`None`) and integer or boolean arrays are valid indices. > >> > >> > >> For historic reasons, in Matlab everything is at least a 2D array, even > >> scalars. By sprinkling some [0,0] in your code you should get what you > >> want, e.g. > >> > >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] > >> > >> array([[ 0. , 0.003968, 0.007978]]) > >> > >> > >> Gregor > >> > >> > >> > >> If I use parameter squeeze_me=True: > >> > >> scipy.io.savemat("x.mat", d) > >> y = scipy.io.loadmat("x.mat", squeeze_me=True) > >> # print y > >> cd = y['CanData'] > >> msg = cd['msg'] > >> print msg > >> print msg.dtype > >> print msg.dtype.names > >> > >> I get output: > >> >C:\Anaconda2\pythonw -u "test1.py" > >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > >> -0.1 , 0.3 ]), u'VelAct')), > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > >> 0.003968, 0.007978])),) > >> object > >> None > >> >Exit code: 0 > >> > >> All well, but the name 'RPDO2' disapeared from the data! > >> > >> Now I need this information; in future I won't handle what's put into > >> x.mat, so I need a way to access through the data all the way down (and > >> handle the variations that will come). > >> > >> I have found a workaround at: > >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- > >> nested-structures-i-e-dictionaries/ > >> > >> The problem is, the workaround uses struct_as_record=False in loadmat, > >> and which boils down to using scipy.io.matlab.mio5_params.mat_struct, > >> and when you read the docstring of class mat_struct, it says: > >> > >> ''' > >> ... > >> We deprecate this method of holding struct information, and will > >> soon remove it, in favor of the recarray method (see loadmat > >> docstring) > >> ''' > >> So my questions: > >> 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > >> using this name, without using parameter struct_as_record=False in loadmat? > >> 2) If not, where do I file a bug? The workaround is five years old, so > >> the issue seems to be in scipy for ages... > >> > >> (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > >> this does not seem to matter). > >> > >> Thanks a lot for the answers, in advance. > >> > >> Nenad > >> > >> > >> _______________________________________________ > >> SciPy-User mailing list > >> SciPy-User at python.org > >> https://mail.python.org/mailman/listinfo/scipy-user > >> > >> > >> > >> _______________________________________________ > >> SciPy-User mailing list > >> SciPy-User at python.org > >> https://mail.python.org/mailman/listinfo/scipy-user > >> > >> > > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 6 > ****************************************** > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From npropadovic at gmail.com Fri Feb 24 07:36:53 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Fri, 24 Feb 2017 13:36:53 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat Message-ID: Hello Matthew, hello Gregor, thank you for your answers. Yes, values struct_as_record=False and squeeze_me=True are what the workaround I mentioned in the initial post at: http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/ is based on. I will use that workaround, it gives nice access to everything I need. Why I hesitated initially, after finding it was that setting struct_as_record=False boils down to using the class scipy.io.matlab.mio5_params.mat_struct. When you look at the docstring of that class it says: "We deprecate this method of holding struct information, and will soon remove it, in favor of the recarray method". So the way I understand this, I'm building my code upon something that won't be around for long. An uncomfortable situation, I don't know if you agree? I'm a consultant, the code will be around the clients place longer than myself. So this was the reason I started this thread in the first place. I really tried to explain that in the initial post, with all the necessary detail, but it seems that the post was too confusing. Sorry for bothering everybody for so long. I'm, however, still not sure if I should file this as a bug, or some kind of feature request, as there seem to be very few people who bother (I counted maybe 2-3 more related posts, plus the post containing the workaround, at stack overflow). Regards, Nenad 2017-02-24 12:15 GMT+01:00 : > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Matthew Brett) > 2. Use Distance Matrix in scipy.cluster.hierarchy.linkage()? > (Sema Atasever) > 3. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Gregor Thalhammer) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 23 Feb 2017 17:41:34 -0800 > From: Matthew Brett > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: > ail.com> > Content-Type: text/plain; charset=UTF-8 > > Hi, > > On Thu, Feb 23, 2017 at 5:12 AM, Propadovic Nenad > wrote: > > Hello Jason, than you a lot for the answer to my post. I was aware of the > > squeeze_me=True option, I think I mentioned it in my initial question > post. > > > > However, as I stated in the answer to Gregors kind answer, I actually > need a > > way to inspect the parts of the access path in the data structure I > import > > form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' > > disappear completely: > > > > > > import scipy.io > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > > > Output: > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 , > > 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > 0.003968, 0.007978])),) > > > > And I really need to be able to find it by some kind of inspection, so > that > > I don't return parts of the structure that don't correspond to the > intention > > of the person searching. > > Sorry if I'm not following, but, does this help? > > In [36]: y = scipy.io.loadmat("x.mat") > > In [37]: y['CanData'][0, 0]['msg'][0, 0]['RPDO2'] > Out[37]: > array([[ (array([[ 0. , 0.003968, 0.007978]]), array([[ > (array([[(array([[ 61.96, 61.96, 61.96]]), array(['PosAct'], > dtype=' dtype=[('Values', 'O'), ('Name', 'O')]), array([[(array([[-0.05, > -0.1 , 0.3 ]]), array(['VelAct'], > dtype=' dtype=[('Values', 'O'), ('Name', 'O')]))]], > dtype=[('PosAct', 'O'), ('VelAct', 'O')]))]], > dtype=[('timest', 'O'), ('sig', 'O')]) > > In [54]: y2 = scipy.io.loadmat('x.mat', squeeze_me=True, > struct_as_record=False) > > In [55]: y2['CanData'].msg.RPDO2 > Out[55]: > > Best, > > Matthew > > > ------------------------------ > > Message: 2 > Date: Fri, 24 Feb 2017 09:59:04 +0300 > From: Sema Atasever > To: scipy-user at python.org > Subject: [SciPy-User] Use Distance Matrix in > scipy.cluster.hierarchy.linkage()? > Message-ID: > ail.com> > Content-Type: text/plain; charset="utf-8" > > Dear SciPy list member, > > I want to ask you about clustering usign scipy.cluster.hierarchy. > > I have a *distance matrix* n*n M where M_ij is the distance between > object_i and object_j. You can see file format in the attachment --> > (dm.csv) > > I want to cluster these n objects with hierarchical clustering. > > For this purpose i am usign this python code that you can see in the > attachment (scipy_code.py) > > I want to ask that how can get clusters values in pdf format or text format > and how many clusters did i get and what clusters includes what members? > > Thanks in Advance, Best regards. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: 20170224/dbac91b0/attachment-0001.html> > -------------- next part -------------- > import scipy > import pylab > import scipy.cluster.hierarchy as sch > import pandas as pd > import numpy as np > > D=np.loadtxt(open("C:\dm.csv", "rb"), delimiter="\t", usecols=range(1,11)) > > print (D) > print (D.shape) > > # Compute and plot dendrogram. > fig = pylab.figure() > axdendro = fig.add_axes([0.09,0.1,0.2,0.8]) > Y = sch.linkage(D, method='single') > Z = sch.dendrogram(Y, orientation='right') > axdendro.set_xticks([]) > axdendro.set_yticks([]) > > # Plot distance matrix. > axmatrix = fig.add_axes([0.3,0.1,0.6,0.8]) > index = Z['leaves'] > D = D[index,:] > D = D[:,index] > im = axmatrix.matshow(D, aspect='auto', origin='lower') > axmatrix.set_xticks([]) > axmatrix.set_yticks([]) > > # Plot colorbar. > axcolor = fig.add_axes([0.91,0.1,0.02,0.8]) > pylab.colorbar(im, cax=axcolor) > > # Display and save figure. > fig.show() > fig.savefig('dendrogram.png') > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: dm.csv > Type: text/csv > Size: 680 bytes > Desc: not available > URL: 20170224/dbac91b0/attachment-0001.csv> > > ------------------------------ > > Message: 3 > Date: Fri, 24 Feb 2017 12:15:18 +0100 > From: Gregor Thalhammer > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: <7CCED80F-462D-4BED-9540-21E262206886 at gmail.com> > Content-Type: text/plain; charset="us-ascii" > > > > Am 23.02.2017 um 14:12 schrieb Propadovic Nenad : > > > > Hello Jason, than you a lot for the answer to my post. I was aware of > the squeeze_me=True option, I think I mentioned it in my initial question > post. > > > > However, as I stated in the answer to Gregors kind answer, I actually > need a way to inspect the parts of the access path in the data structure I > import form the x.mat-file, and if I use squeeze_me=True, parts like > 'RPDO2' disappear completely: > > > > The substruct names are somewhat hidden, try this: > > y = scipy.io.loadmat('x.mat', squeeze_me=True, struct_as_record=False) > y['CanData'].msg._fieldnames > > ['RPDO2'] > > Gregor > > PS: for introspection of objects, just take a look at the __dict__ > attribute. > > > > > import scipy.io > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > > > Output: > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 , 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > 0.003968, 0.007978])),) > > > > And I really need to be able to find it by some kind of inspection, so > that I don't return parts of the structure that don't correspond to the > intention of the person searching. > > > > Thanks once again! > > > > Nenad > > > > > > > > 2017-02-22 18:00 GMT+01:00 scipy-user-request at python.org>>: > > Send SciPy-User mailing list submissions to > > scipy-user at python.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > or, via email, send a message with subject or body 'help' to > > scipy-user-request at python.org hon.org> > > > > You can reach the person managing the list at > > scipy-user-owner at python.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of SciPy-User digest..." > > > > > > Today's Topics: > > > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > > the data inaccessible after loadmat (Jason Sachs) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Wed, 22 Feb 2017 09:12:26 -0700 > > From: Jason Sachs > > > To: SciPy Users List scipy-user at python.org>> > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > > bug: parts of the data inaccessible after loadmat > > Message-ID: > > mail.com w at mail.gmail.com>> > > Content-Type: text/plain; charset="utf-8" > > > > ah, yes, here it is: > > > > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html < > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html> > > > > ---- > > > > So, in MATLAB, the struct array must be at least 2D, and we replicate > that > > when we read into Scipy. If you want all length 1 dimensions squeezed > out, > > try this: > > >>> > > > > >>> mat_contents = sio.loadmat('octave_struct.mat', squeeze_me=True)>>> > oct_struct = mat_contents['my_struct']>>> oct_struct.shape() > > > > > > On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs jmsachs at gmail.com>> wrote: > > > > > This looks familiar, I ran into this a few years ago, and if I recall > > > correctly, there is an option to loadmat to reduce array dimensions > > > appropriately. There is a "squeeze_me" option (unfortunately named... > > > should probably be deprecated in favor of "squeeze") which I think > does > > > this. > > > > > > https://docs.scipy.org/doc/scipy/reference/generated/scipy. > io.loadmat.html ipy/reference/generated/scipy.io.loadmat.html> > > > > > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > > > gregor.thalhammer at gmail.com > > wrote: > > > > > >> > > >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad < > npropadovic at gmail.com >: > > >> > > >> Hello, > > >> > > >> bear with me for the long post that follows: it took me more than a > week > > >> to get this far, and I tried to compress all the relevant information > into > > >> the post. > > >> > > >> There seems to be a bug in scipy.io.loadmat; I'll present it by a > short > > >> piece of code and it's output. > > >> > > >> I create file x.mat with the following: > > >> > > >> import scipy.io > > >> > > >> d = {'CanData': > > >> { > > >> 'msg': { > > >> 'RPDO2': { > > >> 'timest': [0.0, 0.0039679999899817631, > > >> 0.0079779999941820279], > > >> 'sig': { > > >> 'VelAct': { > > >> 'Values': [-0.050000000000000003, > > >> -0.10000000000000001, 0.29999999999999999, ], > > >> 'Name': 'VelAct' > > >> }, > > >> 'PosAct': { > > >> 'Values': [61.960000000000001, > > >> 61.960000000000001, 61.960000000000001, ], > > >> 'Name': 'PosAct' > > >> } > > >> } > > >> } > > >> } > > >> } > > >> } > > >> scipy.io.savemat("x.mat", d) > > >> > > >> Matlab is happy with the file and handles it the way I expect. > > >> > > >> When I read in the data stored in the file and print it out: > > >> > > >> import scipy.io > > >> y = scipy.io.loadmat("x.mat") > > >> # print y > > >> cd = y['CanData'] > > >> msg = cd['msg'] > > >> print msg > > >> print msg.dtype > > >> print msg.dtype.names > > >> > > >> The output is: > > >> >C:\Anaconda2\pythonw -u "test1.py" > > >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > > >> array([u'PosAct'], > > >> dtype=' > >> array([u'VelAct'], > > >> dtype=' > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > > >> 0.003968, 0.007978]]))]],)]], > > >> dtype=[('RPDO2', 'O')])]] > > >> object > > >> None > > >> > > >> Now I've read the manual, and as I see it I have no way for me to > access > > >> the deeper layers of data I just put in the file x.mat, although they > are > > >> obviously right there in the data read in. Access via msg['RPDO2'] > gives: > > >> IndexError: only integers, slices (`:`), ellipsis (`...`), > numpy.newaxis > > >> (`None`) and integer or boolean arrays are valid indices. > > >> > > >> > > >> For historic reasons, in Matlab everything is at least a 2D array, > even > > >> scalars. By sprinkling some [0,0] in your code you should get what you > > >> want, e.g. > > >> > > >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] > > >> > > >> array([[ 0. , 0.003968, 0.007978]]) > > >> > > >> > > >> Gregor > > >> > > >> > > >> > > >> If I use parameter squeeze_me=True: > > >> > > >> scipy.io.savemat("x.mat", d) > > >> y = scipy.io.loadmat("x.mat", squeeze_me=True) > > >> # print y > > >> cd = y['CanData'] > > >> msg = cd['msg'] > > >> print msg > > >> print msg.dtype > > >> print msg.dtype.names > > >> > > >> I get output: > > >> >C:\Anaconda2\pythonw -u "test1.py" > > >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > > >> -0.1 , 0.3 ]), u'VelAct')), > > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > >> 0.003968, 0.007978])),) > > >> object > > >> None > > >> >Exit code: 0 > > >> > > >> All well, but the name 'RPDO2' disapeared from the data! > > >> > > >> Now I need this information; in future I won't handle what's put into > > >> x.mat, so I need a way to access through the data all the way down > (and > > >> handle the variations that will come). > > >> > > >> I have found a workaround at: > > >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- < > http://stackoverflow.com/questions/7008608/scipy-io-loadmat-> > > >> nested-structures-i-e-dictionaries/ > > >> > > >> The problem is, the workaround uses struct_as_record=False in loadmat, > > >> and which boils down to using scipy.io.matlab.mio5_params.mat_struct, > > >> and when you read the docstring of class mat_struct, it says: > > >> > > >> ''' > > >> ... > > >> We deprecate this method of holding struct information, and will > > >> soon remove it, in favor of the recarray method (see loadmat > > >> docstring) > > >> ''' > > >> So my questions: > > >> 1) Did I miss something? Is there a way to access the data in 'RPDO2' > by > > >> using this name, without using parameter struct_as_record=False in > loadmat? > > >> 2) If not, where do I file a bug? The workaround is five years old, so > > >> the issue seems to be in scipy for ages... > > >> > > >> (For the records, I use scipy within Anaconda2 1.4.1, under Windows, > but > > >> this does not seem to matter). > > >> > > >> Thanks a lot for the answers, in advance. > > >> > > >> Nenad > > >> > > >> > > >> _______________________________________________ > > >> SciPy-User mailing list > > >> SciPy-User at python.org > > >> https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > >> > > >> > > >> > > >> _______________________________________________ > > >> SciPy-User mailing list > > >> SciPy-User at python.org > > >> https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > >> > > >> > > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: 20170222/e992b074/attachment-0001.html ail/scipy-user/attachments/20170222/e992b074/attachment-0001.html>> > > > > ------------------------------ > > > > Subject: Digest Footer > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > > > > > ------------------------------ > > > > End of SciPy-User Digest, Vol 162, Issue 6 > > ****************************************** > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: 20170224/46a54c05/attachment.html> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 10 > ******************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Fri Feb 24 09:36:51 2017 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Fri, 24 Feb 2017 17:36:51 +0300 Subject: [SciPy-User] ANN: Scipy 0.19.0 release candidate 2 Message-ID: Hi, I'm pleased to announce the availability of the second release candidate for Scipy 0.19.0. It contains contributions from 121 people over the course of seven months. The main difference to the rc1 is several Windows specific issues that were fixed (special thanks to Christoph Gohlke). Please try this release and report any issues on Github tracker, https://github.com/scipy/scipy, or scipy-dev mailing list. Source tarballs and release notes are available from Github releases, https://github.com/scipy/scipy/releases/tag/v0.19.0rc2 We appreciate if you could both run self-tests on your hardware, and test your code against this release. If no issues are reported, this will graduate to Scipy 0.19.0 final on the 9th March 2017. Thanks to everyone who contributed to this release! Cheers, Evgeni -----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 ========================== SciPy 0.19.0 Release Notes ========================== .. note:: Scipy 0.19.0 is not released yet! .. contents:: SciPy 0.19.0 is the culmination of X months of hard work. It contains many new features, numerous bug-fixes, improved test coverage and better documentation. There have been a number of deprecations and API changes in this release, which are documented below. All users are encouraged to upgrade to this release, as there are a large number of bug-fixes and optimizations. Moreover, our development attention will now shift to bug-fix releases on the 0.19.x branch, and on adding new features on the master branch. This release requires Python 2.7 or 3.4-3.6 and NumPy 1.8.2 or greater. Highlights of this release include: - - A unified foreign function interface layer, `scipy.LowLevelCallable`. - - Cython API for scalar, typed versions of the universal functions from the `scipy.special` module, via `cimport scipy.special.cython_special`. New features ============ Foreign function interface improvements - --------------------------------------- `scipy.LowLevelCallable` provides a new unified interface for wrapping low-level compiled callback functions in the Python space. It supports Cython imported "api" functions, ctypes function pointers, CFFI function pointers, ``PyCapsules``, Numba jitted functions and more. See `gh-6509 `_ for details. `scipy.linalg` improvements - --------------------------- The function `scipy.linalg.solve` obtained two more keywords ``assume_a`` and ``transposed``. The underlying LAPACK routines are replaced with "expert" versions and now can also be used to solve symmetric, hermitian and positive definite coefficient matrices. Moreover, ill-conditioned matrices now cause a warning to be emitted with the estimated condition number information. Old ``sym_pos`` keyword is kept for backwards compatibility reasons however it is identical to using ``assume_a='pos'``. Moreover, the ``debug`` keyword, which had no function but only printing the ``overwrite_`` values, is deprecated. The function `scipy.linalg.matrix_balance` was added to perform the so-called matrix balancing using the LAPACK xGEBAL routine family. This can be used to approximately equate the row and column norms through diagonal similarity transformations. The functions `scipy.linalg.solve_continuous_are` and `scipy.linalg.solve_discrete_are` have numerically more stable algorithms. These functions can also solve generalized algebraic matrix Riccati equations. Moreover, both gained a ``balanced`` keyword to turn balancing on and off. `scipy.spatial` improvements - ---------------------------- `scipy.spatial.SphericalVoronoi.sort_vertices_of_regions` has been re-written in Cython to improve performance. `scipy.spatial.SphericalVoronoi` can handle > 200 k points (at least 10 million) and has improved performance. The function `scipy.spatial.distance.directed_hausdorff` was added to calculate the directed Hausdorff distance. ``count_neighbors`` method of `scipy.spatial.cKDTree` gained an ability to perform weighted pair counting via the new keywords ``weights`` and ``cumulative``. See `gh-5647 `_ for details. `scipy.spatial.distance.pdist` and `scipy.spatial.distance.cdist` now support non-double custom metrics. `scipy.ndimage` improvements - ---------------------------- The callback function C API supports PyCapsules in Python 2.7 Multidimensional filters now allow having different extrapolation modes for different axes. `scipy.optimize` improvements - ----------------------------- The `scipy.optimize.basinhopping` global minimizer obtained a new keyword, `seed`, which can be used to seed the random number generator and obtain repeatable minimizations. The keyword `sigma` in `scipy.optimize.curve_fit` was overloaded to also accept the covariance matrix of errors in the data. `scipy.signal` improvements - --------------------------- The function `scipy.signal.correlate` and `scipy.signal.convolve` have a new optional parameter `method`. The default value of `auto` estimates the fastest of two computation methods, the direct approach and the Fourier transform approach. A new function has been added to choose the convolution/correlation method, `scipy.signal.choose_conv_method` which may be appropriate if convolutions or correlations are performed on many arrays of the same size. New functions have been added to calculate complex short time fourier transforms of an input signal, and to invert the transform to recover the original signal: `scipy.signal.stft` and `scipy.signal.istft`. This implementation also fixes the previously incorrect ouput of `scipy.signal.spectrogram` when complex output data were requested. The function `scipy.signal.sosfreqz` was added to compute the frequency response from second-order sections. The function `scipy.signal.unit_impulse` was added to conveniently generate an impulse function. The function `scipy.signal.iirnotch` was added to design second-order IIR notch filters that can be used to remove a frequency component from a signal. The dual function `scipy.signal.iirpeak` was added to compute the coefficients of a second-order IIR peak (resonant) filter. The function `scipy.signal.minimum_phase` was added to convert linear-phase FIR filters to minimum phase. The functions `scipy.signal.upfirdn` and `scipy.signal.resample_poly` are now substantially faster when operating on some n-dimensional arrays when n > 1. The largest reduction in computation time is realized in cases where the size of the array is small (<1k samples or so) along the axis to be filtered. `scipy.fftpack` improvements - ---------------------------- Fast Fourier transform routines now accept `np.float16` inputs and upcast them to `np.float32`. Previously, they would raise an error. `scipy.cluster` improvements - ---------------------------- Methods ``"centroid"`` and ``"median"`` of `scipy.cluster.hierarchy.linkage` have been significantly sped up. Long-standing issues with using ``linkage`` on large input data (over 16 GB) have been resolved. `scipy.sparse` improvements - --------------------------- The functions `scipy.sparse.save_npz` and `scipy.sparse.load_npz` were added, providing simple serialization for some sparse formats. The `prune` method of classes `bsr_matrix`, `csc_matrix`, and `csr_matrix` was updated to reallocate backing arrays under certain conditions, reducing memory usage. The methods `argmin` and `argmax` were added to classes `coo_matrix`, `csc_matrix`, `csr_matrix`, and `bsr_matrix`. New function `scipy.sparse.csgraph.structural_rank` computes the structural rank of a graph with a given sparsity pattern. New function `scipy.sparse.linalg.spsolve_triangular` solves a sparse linear system with a triangular left hand side matrix. `scipy.special` improvements - ---------------------------- Scalar, typed versions of universal functions from `scipy.special` are available in the Cython space via ``cimport`` from the new module `scipy.special.cython_special`. These scalar functions can be expected to be significantly faster then the universal functions for scalar arguments. See the `scipy.special` tutorial for details. Better control over special-function errors is offered by the functions `scipy.special.geterr` and `scipy.special.seterr` and the context manager `scipy.special.errstate`. The names of orthogonal polynomial root functions have been changed to be consistent with other functions relating to orthogonal polynomials. For example, `scipy.special.j_roots` has been renamed `scipy.special.roots_jacobi` for consistency with the related functions `scipy.special.jacobi` and `scipy.special.eval_jacobi`. To preserve back-compatibility the old names have been left as aliases. Wright Omega function is implemented as `scipy.special.wrightomega`. `scipy.stats` improvements - -------------------------- The function `scipy.stats.weightedtau` was added. It provides a weighted version of Kendall's tau. New class `scipy.stats.multinomial` implements the multinomial distribution. New class `scipy.stats.rv_histogram` constructs a continuous univariate distribution with a piecewise linear CDF from a binned data sample. New class `scipy.stats.argus` implements the Argus distribution. `scipy.interpolate` improvements - -------------------------------- New class `scipy.interpolate.BSpline` represents splines. ``BSpline`` objects contain knots and coefficients and can evaluate the spline. The format is consistent with FITPACK, so that one can do, for example:: >>> t, c, k = splrep(x, y, s=0) >>> spl = BSpline(t, c, k) >>> np.allclose(spl(x), y) ``spl*`` functions, `scipy.interpolate.splev`, `scipy.interpolate.splint`, `scipy.interpolate.splder` and `scipy.interpolate.splantider`, accept both ``BSpline`` objects and ``(t, c, k)`` tuples for backwards compatibility. For multidimensional splines, ``c.ndim > 1``, ``BSpline`` objects are consistent with piecewise polynomials, `scipy.interpolate.PPoly`. This means that ``BSpline`` objects are not immediately consistent with `scipy.interpolate.splprep`, and one *cannot* do ``>>> BSpline(*splprep([x, y])[0])``. Consult the `scipy.interpolate` test suite for examples of the precise equivalence. In new code, prefer using ``scipy.interpolate.BSpline`` objects instead of manipulating ``(t, c, k)`` tuples directly. New function `scipy.interpolate.make_interp_spline` constructs an interpolating spline given data points and boundary conditions. New function `scipy.interpolate.make_lsq_spline` constructs a least-squares spline approximation given data points. `scipy.integrate` improvements - ------------------------------ Now `scipy.integrate.fixed_quad` supports vector-valued functions. Deprecated features =================== `scipy.interpolate.splmake`, `scipy.interpolate.spleval` and `scipy.interpolate.spline` are deprecated. The format used by `splmake/spleval` was inconsistent with `splrep/splev` which was confusing to users. `scipy.special.errprint` is deprecated. Improved functionality is available in `scipy.special.seterr`. calling `scipy.spatial.distance.pdist` or `scipy.spatial.distance.cdist` with arguments not needed by the chosen metric is deprecated. Also, metrics `"old_cosine"` and `"old_cos"` are deprecated. Backwards incompatible changes ============================== The deprecated ``scipy.weave`` submodule was removed. `scipy.spatial.distance.squareform` now returns arrays of the same dtype as the input, instead of always float64. `scipy.special.errprint` now returns a boolean. The function `scipy.signal.find_peaks_cwt` now returns an array instead of a list. `scipy.stats.kendalltau` now computes the correct p-value in case the input contains ties. The p-value is also identical to that computed by `scipy.stats.mstats.kendalltau` and by R. If the input does not contain ties there is no change w.r.t. the previous implementation. The function `scipy.linalg.block_diag` will not ignore zero-sized matrices anymore. Instead it will insert rows or columns of zeros of the appropriate size. See gh-4908 for more details. Other changes ============= SciPy wheels will now report their dependency on ``numpy`` on all platforms. This change was made because Numpy wheels are available, and because the pip upgrade behavior is finally changing for the better (use ``--upgrade-strategy=only-if-needed`` for ``pip >= 8.2``; that behavior will become the default in the next major version of ``pip``). Numerical values returned by `scipy.interpolate.interp1d` with ``kind="cubic"`` and ``"quadratic"`` may change relative to previous scipy versions. If your code depended on specific numeric values (i.e., on implementation details of the interpolators), you may want to double-check your results. Authors ======= * @endolith * Max Argus + * Herv? Audren * Alessandro Pietro Bardelli + * Michael Benfield + * Felix Berkenkamp * Matthew Brett * Per Brodtkorb * Evgeni Burovski * Pierre de Buyl * CJ Carey * Brandon Carter + * Tim Cera * Klesk Chonkin * Christian H?ggstr?m + * Luca Citi * Peadar Coyle + * Daniel da Silva + * Greg Dooper + * John Draper + * drlvk + * David Ellis + * Yu Feng * Baptiste Fontaine + * Jed Frey + * Siddhartha Gandhi + * GiggleLiu + * Wim Glenn + * Akash Goel + * Christoph Gohlke * Ralf Gommers * Alexander Goncearenco + * Richard Gowers + * Alex Griffing * Radoslaw Guzinski + * Charles Harris * Callum Jacob Hays + * Ian Henriksen * Randy Heydon + * Lindsey Hiltner + * Gerrit Holl + * Hiroki IKEDA + * jfinkels + * Mher Kazandjian + * Thomas Keck + * keuj6 + * Kornel Kielczewski + * Sergey B Kirpichev + * Vasily Kokorev + * Eric Larson * Denis Laxalde * Gregory R. Lee * Josh Lefler + * Julien Lhermitte + * Evan Limanto + * Nikolay Mayorov * Geordie McBain + * Josue Melka + * Matthieu Melot * michaelvmartin15 + * Surhud More + * Brett M. Morris + * Chris Mutel + * Paul Nation * Andrew Nelson * David Nicholson + * Aaron Nielsen + * Joel Nothman * nrnrk + * Juan Nunez-Iglesias * Mikhail Pak + * Gavin Parnaby + * Thomas Pingel + * Ilhan Polat + * Aman Pratik + * Sebastian Pucilowski * Ted Pudlik * puenka + * Eric Quintero * Tyler Reddy * Joscha Reimer * Antonio Horta Ribeiro + * Edward Richards + * Roman Ring + * Rafael Rossi + * Colm Ryan + * Sami Salonen + * Alvaro Sanchez-Gonzalez + * Johannes Schmitz * Kari Schoonbee * Yurii Shevchuk + * Jonathan Siebert + * Jonathan Tammo Siebert + * Scott Sievert + * Sourav Singh * Byron Smith + * Srikiran + * Samuel St-Jean + * Yoni Teitelbaum + * Bhavika Tekwani * Martin Thoma * timbalam + * Svend Vanderveken + * Sebastiano Vigna + * Aditya Vijaykumar + * Santi Villalba + * Ze Vinicius * Pauli Virtanen * Matteo Visconti * Yusuke Watanabe + * Warren Weckesser * Phillip Weinberg + * Nils Werner * Jakub Wilk * Josh Wilson * wirew0rm + * David Wolever + * Nathan Woods * ybeltukov + * G Young * Evgeny Zhurko + A total of 121 people contributed to this release. People with a "+" by their names contributed a patch for the first time. This list of names is automatically generated, and may not be fully complete. Issues closed for 0.19.0 - ------------------------ - - `#1767 `__: Function definitions in __fitpack.h should be moved. (Trac #1240) - - `#1774 `__: _kmeans chokes on large thresholds (Trac #1247) - - `#2089 `__: Integer overflows cause segfault in linkage function with large... - - `#2190 `__: Are odd-length window functions supposed to be always symmetrical?... - - `#2251 `__: solve_discrete_are in scipy.linalg does (sometimes) not solve... - - `#2580 `__: scipy.interpolate.UnivariateSpline (or a new superclass of it)... - - `#2592 `__: scipy.stats.anderson assumes gumbel_l - - `#3054 `__: scipy.linalg.eig does not handle infinite eigenvalues - - `#3160 `__: multinomial pmf / logpmf - - `#3904 `__: scipy.special.ellipj dn wrong values at quarter period - - `#4044 `__: Inconsistent code book initialization in kmeans - - `#4234 `__: scipy.signal.flattop documentation doesn't list a source for... - - `#4831 `__: Bugs in C code in __quadpack.h - - `#4908 `__: bug: unnessesary validity check for block dimension in scipy.sparse.block_diag - - `#4917 `__: BUG: indexing error for sparse matrix with ix_ - - `#4938 `__: Docs on extending ndimage need to be updated. - - `#5056 `__: sparse matrix element-wise multiplying dense matrix returns dense... - - `#5337 `__: Formula in documentation for correlate is wrong - - `#5537 `__: use OrderedDict in io.netcdf - - `#5750 `__: [doc] missing data index value in KDTree, cKDTree - - `#5755 `__: p-value computation in scipy.stats.kendalltau() in broken in... - - `#5757 `__: BUG: Incorrect complex output of signal.spectrogram - - `#5964 `__: ENH: expose scalar versions of scipy.special functions to cython - - `#6107 `__: scipy.cluster.hierarchy.single segmentation fault with 2**16... - - `#6278 `__: optimize.basinhopping should take a RandomState object - - `#6296 `__: InterpolatedUnivariateSpline: check_finite fails when w is unspecified - - `#6306 `__: Anderson-Darling bad results - - `#6314 `__: scipy.stats.kendaltau() p value not in agreement with R, SPSS... - - `#6340 `__: Curve_fit bounds and maxfev - - `#6377 `__: expm_multiply, complex matrices not working using start,stop,ect... - - `#6382 `__: optimize.differential_evolution stopping criterion has unintuitive... - - `#6391 `__: Global Benchmarking times out at 600s. - - `#6397 `__: mmwrite errors with large (but still 64-bit) integers - - `#6413 `__: scipy.stats.dirichlet computes multivariate gaussian differential... - - `#6428 `__: scipy.stats.mstats.mode modifies input - - `#6440 `__: Figure out ABI break policy for scipy.special Cython API - - `#6441 `__: Using Qhull for halfspace intersection : segfault - - `#6442 `__: scipy.spatial : In incremental mode volume is not recomputed - - `#6451 `__: Documentation for scipy.cluster.hierarchy.to_tree is confusing... - - `#6490 `__: interp1d (kind=zero) returns wrong value for rightmost interpolation... - - `#6521 `__: scipy.stats.entropy does *not* calculate the KL divergence - - `#6530 `__: scipy.stats.spearmanr unexpected NaN handling - - `#6541 `__: Test runner does not run scipy._lib/tests? - - `#6552 `__: BUG: misc.bytescale returns unexpected results when using cmin/cmax... - - `#6556 `__: RectSphereBivariateSpline(u, v, r) fails if min(v) >= pi - - `#6559 `__: Differential_evolution maxiter causing memory overflow - - `#6565 `__: Coverage of spectral functions could be improved - - `#6628 `__: Incorrect parameter name in binomial documentation - - `#6634 `__: Expose LAPACK's xGESVX family for linalg.solve ill-conditioned... - - `#6657 `__: Confusing documentation for `scipy.special.sph_harm` - - `#6676 `__: optimize: Incorrect size of Jacobian returned by `minimize(...,... - - `#6681 `__: add a new context manager to wrap `scipy.special.seterr` - - `#6700 `__: BUG: scipy.io.wavfile.read stays in infinite loop, warns on wav... - - `#6721 `__: scipy.special.chebyt(N) throw a 'TypeError' when N > 64 - - `#6727 `__: Documentation for scipy.stats.norm.fit is incorrect - - `#6764 `__: Documentation for scipy.spatial.Delaunay is partially incorrect - - `#6811 `__: scipy.spatial.SphericalVoronoi fails for large number of points - - `#6841 `__: spearmanr fails when nan_policy='omit' is set - - `#6869 `__: Currently in gaussian_kde, the logpdf function is calculated... - - `#6875 `__: SLSQP inconsistent handling of invalid bounds - - `#6876 `__: Python stopped working (Segfault?) with minimum/maximum filter... - - `#6889 `__: dblquad gives different results under scipy 0.17.1 and 0.18.1 - - `#6898 `__: BUG: dblquad ignores error tolerances - - `#6901 `__: Solving sparse linear systems in CSR format with complex values - - `#6903 `__: issue in spatial.distance.pdist docstring - - `#6917 `__: Problem in passing drop_rule to scipy.sparse.linalg.spilu - - `#6926 `__: signature mismatches for LowLevelCallable - - `#6961 `__: Scipy contains shebang pointing to /usr/bin/python and /bin/bash... - - `#6972 `__: BUG: special: `generate_ufuncs.py` is broken - - `#6984 `__: Assert raises test failure for test_ill_condition_warning - - `#6990 `__: BUG: sparse: Bad documentation of the `k` argument in `sparse.linalg.eigs` - - `#6991 `__: Division by zero in linregress() - - `#7011 `__: possible speed improvment in rv_continuous.fit() - - `#7015 `__: Test failure with Python 3.5 and numpy master Pull requests for 0.19.0 - ------------------------ - - `#2908 `__: Scipy 1.0 Roadmap - - `#3174 `__: add b-splines - - `#4606 `__: ENH: Add a unit impulse waveform function - - `#5608 `__: Adds keyword argument to choose faster convolution method - - `#5647 `__: ENH: Faster count_neighour in cKDTree / + weighted input data - - `#6021 `__: Netcdf append - - `#6058 `__: ENH: scipy.signal - Add stft and istft - - `#6059 `__: ENH: More accurate signal.freqresp for zpk systems - - `#6195 `__: ENH: Cython interface for special - - `#6234 `__: DOC: Fixed a typo in ward() help - - `#6261 `__: ENH: add docstring and clean up code for signal.normalize - - `#6270 `__: MAINT: special: add tests for cdflib - - `#6271 `__: Fix for scipy.cluster.hierarchy.is_isomorphic - - `#6273 `__: optimize: rewrite while loops as for loops - - `#6279 `__: MAINT: Bessel tweaks - - `#6291 `__: Fixes gh-6219: remove runtime warning from genextreme distribution - - `#6294 `__: STY: Some PEP8 and cleaning up imports in stats/_continuous_distns.py - - `#6297 `__: Clarify docs in misc/__init__.py - - `#6300 `__: ENH: sparse: Loosen input validation for `diags` with empty inputs - - `#6301 `__: BUG: standardizes check_finite behavior re optional weights,... - - `#6303 `__: Fixing example in _lazyselect docstring. - - `#6307 `__: MAINT: more improvements to gammainc/gammaincc - - `#6308 `__: Clarified documentation of hypergeometric distribution. - - `#6309 `__: BUG: stats: Improve calculation of the Anderson-Darling statistic. - - `#6315 `__: ENH: Descending order of x in PPoly - - `#6317 `__: ENH: stats: Add support for nan_policy to stats.median_test - - `#6321 `__: TST: fix a typo in test name - - `#6328 `__: ENH: sosfreqz - - `#6335 `__: Define LinregressResult outside of linregress - - `#6337 `__: In anderson test, added support for right skewed gumbel distribution. - - `#6341 `__: Accept several spellings for the curve_fit max number of function... - - `#6342 `__: DOC: cluster: clarify hierarchy.linkage usage - - `#6352 `__: DOC: removed brentq from its own 'see also' - - `#6362 `__: ENH: stats: Use explicit formulas for sf, logsf, etc in weibull... - - `#6369 `__: MAINT: special: add a comment to hyp0f1_complex - - `#6375 `__: Added the multinomial distribution. - - `#6387 `__: MAINT: special: improve accuracy of ellipj's `dn` at quarter... - - `#6388 `__: BenchmarkGlobal - getting it to work in Python3 - - `#6394 `__: ENH: scipy.sparse: add save and load functions for sparse matrices - - `#6400 `__: MAINT: moves global benchmark run from setup_cache to track_all - - `#6403 `__: ENH: seed kwd for basinhopping. Closes #6278 - - `#6404 `__: ENH: signal: added irrnotch and iirpeak functions. - - `#6406 `__: ENH: special: extend `sici`/`shichi` to complex arguments - - `#6407 `__: ENH: Window functions should not accept non-integer or negative... - - `#6408 `__: MAINT: _differentialevolution now uses _lib._util.check_random_state - - `#6427 `__: MAINT: Fix gmpy build & test that mpmath uses gmpy - - `#6439 `__: MAINT: ndimage: update callback function c api - - `#6443 `__: BUG: Fix volume computation in incremental mode - - `#6447 `__: Fixes issue #6413 - Minor documentation fix in the entropy function... - - `#6448 `__: ENH: Add halfspace mode to Qhull - - `#6449 `__: ENH: rtol and atol for differential_evolution termination fixes... - - `#6453 `__: DOC: Add some See Also links between similar functions - - `#6454 `__: DOC: linalg: clarify callable signature in `ordqz` - - `#6457 `__: ENH: spatial: enable non-double dtypes in squareform - - `#6459 `__: BUG: Complex matrices not handled correctly by expm_multiply... - - `#6465 `__: TST DOC Window docs, tests, etc. - - `#6469 `__: ENH: linalg: better handling of infinite eigenvalues in `eig`/`eigvals` - - `#6475 `__: DOC: calling interp1d/interp2d with NaNs is undefined - - `#6477 `__: Document magic numbers in optimize.py - - `#6481 `__: TST: Supress some warnings from test_windows - - `#6485 `__: DOC: spatial: correct typo in procrustes - - `#6487 `__: Fix Bray-Curtis formula in pdist docstring - - `#6493 `__: ENH: Add covariance functionality to scipy.optimize.curve_fit - - `#6494 `__: ENH: stats: Use log1p() to improve some calculations. - - `#6495 `__: BUG: Use MST algorithm instead of SLINK for single linkage clustering - - `#6497 `__: MRG: Add minimum_phase filter function - - `#6505 `__: reset scipy.signal.resample window shape to 1-D - - `#6507 `__: BUG: linkage: Raise exception if y contains non-finite elements - - `#6509 `__: ENH: _lib: add common machinery for low-level callback functions - - `#6520 `__: scipy.sparse.base.__mul__ non-numpy/scipy objects with 'shape'... - - `#6522 `__: Replace kl_div by rel_entr in entropy - - `#6524 `__: DOC: add next_fast_len to list of functions - - `#6527 `__: DOC: Release notes to reflect the new covariance feature in optimize.curve_fit - - `#6532 `__: ENH: Simplify _cos_win, document it, add symmetric/periodic arg - - `#6535 `__: MAINT: sparse.csgraph: updating old cython loops - - `#6540 `__: DOC: add to documentation of orthogonal polynomials - - `#6544 `__: TST: Ensure tests for scipy._lib are run by scipy.test() - - `#6546 `__: updated docstring of stats.linregress - - `#6553 `__: commited changes that I originally submitted for scipy.signal.cspline? - - `#6561 `__: BUG: modify signal.find_peaks_cwt() to return array and accept... - - `#6562 `__: DOC: Negative binomial distribution clarification - - `#6563 `__: MAINT: be more liberal in requiring numpy - - `#6567 `__: MAINT: use xrange for iteration in differential_evolution fixes... - - `#6572 `__: BUG: "sp.linalg.solve_discrete_are" fails for random data - - `#6578 `__: BUG: misc: allow both cmin/cmax and low/high params in bytescale - - `#6581 `__: Fix some unfortunate typos - - `#6582 `__: MAINT: linalg: make handling of infinite eigenvalues in `ordqz`... - - `#6585 `__: DOC: interpolate: correct seealso links to ndimage - - `#6588 `__: Update docstring of scipy.spatial.distance_matrix - - `#6592 `__: DOC: Replace 'first' by 'smallest' in mode - - `#6593 `__: MAINT: remove scipy.weave submodule - - `#6594 `__: DOC: distance.squareform: fix html docs, add note about dtype... - - `#6598 `__: [DOC] Fix incorrect error message in medfilt2d - - `#6599 `__: MAINT: linalg: turn a `solve_discrete_are` test back on - - `#6600 `__: DOC: Add SOS goals to roadmap - - `#6601 `__: DEP: Raise minimum numpy version to 1.8.2 - - `#6605 `__: MAINT: 'new' module is deprecated, don't use it - - `#6607 `__: DOC: add note on change in wheel dependency on numpy and pip... - - `#6609 `__: Fixes #6602 - Typo in docs - - `#6616 `__: ENH: generalization of continuous and discrete Riccati solvers... - - `#6621 `__: DOC: improve cluster.hierarchy docstrings. - - `#6623 `__: CS matrix prune method should copy data from large unpruned arrays - - `#6625 `__: DOC: special: complete documentation of `eval_*` functions - - `#6626 `__: TST: special: silence some deprecation warnings - - `#6631 `__: fix parameter name doc for discrete distributions - - `#6632 `__: MAINT: stats: change some instances of `special` to `sc` - - `#6633 `__: MAINT: refguide: py2k long integers are equal to py3k integers - - `#6638 `__: MAINT: change type declaration in cluster.linkage, prevent overflow - - `#6640 `__: BUG: fix issue with duplicate values used in cluster.vq.kmeans - - `#6641 `__: BUG: fix corner case in cluster.vq.kmeans for large thresholds - - `#6643 `__: MAINT: clean up truncation modes of dendrogram - - `#6645 `__: MAINT: special: rename `*_roots` functions - - `#6646 `__: MAINT: clean up mpmath imports - - `#6647 `__: DOC: add sqrt to Mahalanobis description for pdist - - `#6648 `__: DOC: special: add a section on `cython_special` to the tutorial - - `#6649 `__: ENH: Added scipy.spatial.distance.directed_hausdorff - - `#6650 `__: DOC: add Sphinx roles for DOI and arXiv links - - `#6651 `__: BUG: mstats: make sure mode(..., None) does not modify its input - - `#6652 `__: DOC: special: add section to tutorial on functions not in special - - `#6653 `__: ENH: special: add the Wright Omega function - - `#6656 `__: ENH: don't coerce input to double with custom metric in cdist... - - `#6658 `__: Faster/shorter code for computation of discordances - - `#6659 `__: DOC: special: make __init__ summaries and html summaries match - - `#6661 `__: general.rst: Fix a typo - - `#6664 `__: TST: Spectral functions' window correction factor - - `#6665 `__: [DOC] Conditions on v in RectSphereBivariateSpline - - `#6668 `__: DOC: Mention negative masses for center of mass - - `#6675 `__: MAINT: special: remove outdated README - - `#6677 `__: BUG: Fixes computation of p-values. - - `#6679 `__: BUG: optimize: return correct Jacobian for method 'SLSQP' in... - - `#6680 `__: ENH: Add structural rank to sparse.csgraph - - `#6686 `__: TST: Added Airspeed Velocity benchmarks for SphericalVoronoi - - `#6687 `__: DOC: add section "deciding on new features" to developer guide. - - `#6691 `__: ENH: Clearer error when fmin_slsqp obj doesn't return scalar - - `#6702 `__: TST: Added airspeed velocity benchmarks for scipy.spatial.distance.cdist - - `#6707 `__: TST: interpolate: test fitpack wrappers, not _impl - - `#6709 `__: TST: fix a number of test failures on 32-bit systems - - `#6711 `__: MAINT: move function definitions from __fitpack.h to _fitpackmodule.c - - `#6712 `__: MAINT: clean up wishlist in stats.morestats, and copyright statement. - - `#6715 `__: DOC: update the release notes with BSpline et al. - - `#6716 `__: MAINT: scipy.io.wavfile: No infinite loop when trying to read... - - `#6717 `__: some style cleanup - - `#6723 `__: BUG: special: cast to float before in-place multiplication in... - - `#6726 `__: address performance regressions in interp1d - - `#6728 `__: DOC: made code examples in `integrate` tutorial copy-pasteable - - `#6731 `__: DOC: scipy.optimize: Added an example for wrapping complex-valued... - - `#6732 `__: MAINT: cython_special: remove `errprint` - - `#6733 `__: MAINT: special: fix some pyflakes warnings - - `#6734 `__: DOC: sparse.linalg: fixed matrix description in `bicgstab` doc - - `#6737 `__: BLD: update `cythonize.py` to detect changes in pxi files - - `#6740 `__: DOC: special: some small fixes to docstrings - - `#6741 `__: MAINT: remove dead code in interpolate.py - - `#6742 `__: BUG: fix ``linalg.block_diag`` to support zero-sized matrices. - - `#6744 `__: ENH: interpolate: make PPoly.from_spline accept BSpline objects - - `#6746 `__: DOC: special: clarify use of Condon-Shortley phase in `sph_harm`/`lpmv` - - `#6750 `__: ENH: sparse: avoid densification on broadcasted elem-wise mult - - `#6751 `__: sinm doc explained cosm - - `#6753 `__: ENH: special: allow for more fine-tuned error handling - - `#6759 `__: Move logsumexp and pade from scipy.misc to scipy.special and... - - `#6761 `__: ENH: argmax and argmin methods for sparse matrices - - `#6762 `__: DOC: Improve docstrings of sparse matrices - - `#6763 `__: ENH: Weighted tau - - `#6768 `__: ENH: cythonized spherical Voronoi region polygon vertex sorting - - `#6770 `__: Correction of Delaunay class' documentation - - `#6775 `__: ENH: Integrating LAPACK "expert" routines with conditioning warnings... - - `#6776 `__: MAINT: Removing the trivial f2py warnings - - `#6777 `__: DOC: Update rv_continuous.fit doc. - - `#6778 `__: MAINT: cluster.hierarchy: Improved wording of error msgs - - `#6786 `__: BLD: increase minimum Cython version to 0.23.4 - - `#6787 `__: DOC: expand on ``linalg.block_diag`` changes in 0.19.0 release... - - `#6789 `__: ENH: Add further documentation for norm.fit - - `#6790 `__: MAINT: Fix a potential problem in nn_chain linkage algorithm - - `#6791 `__: DOC: Add examples to scipy.ndimage.fourier - - `#6792 `__: DOC: fix some numpydoc / Sphinx issues. - - `#6793 `__: MAINT: fix circular import after moving functions out of misc - - `#6796 `__: TST: test importing each submodule. Regression test for gh-6793. - - `#6799 `__: ENH: stats: Argus distribution - - `#6801 `__: ENH: stats: Histogram distribution - - `#6803 `__: TST: make sure tests for ``_build_utils`` are run. - - `#6804 `__: MAINT: more fixes in `loggamma` - - `#6806 `__: ENH: Faster linkage for 'centroid' and 'median' methods - - `#6810 `__: ENH: speed up upfirdn and resample_poly for n-dimensional arrays - - `#6812 `__: TST: Added ConvexHull asv benchmark code - - `#6814 `__: ENH: Different extrapolation modes for different dimensions in... - - `#6826 `__: Signal spectral window default fix - - `#6828 `__: BUG: SphericalVoronoi Space Complexity (Fixes #6811) - - `#6830 `__: RealData docstring correction - - `#6834 `__: DOC: Added reference for skewtest function. See #6829 - - `#6836 `__: DOC: Added mode='mirror' in the docstring for the functions accepting... - - `#6838 `__: MAINT: sparse: start removing old BSR methods - - `#6844 `__: handle incompatible dimensions when input is not an ndarray in... - - `#6847 `__: Added maxiter to golden search. - - `#6850 `__: BUG: added check for optional param scipy.stats.spearmanr - - `#6858 `__: MAINT: Removing redundant tests - - `#6861 `__: DEP: Fix escape sequences deprecated in Python 3.6. - - `#6862 `__: DOC: dx should be float, not int - - `#6863 `__: updated documentation curve_fit - - `#6866 `__: DOC : added some documentation to j1 referring to spherical_jn - - `#6867 `__: DOC: cdist move long examples list into Notes section - - `#6868 `__: BUG: Make stats.mode return a ModeResult namedtuple on empty... - - `#6871 `__: Corrected documentation. - - `#6874 `__: ENH: gaussian_kde.logpdf based on logsumexp - - `#6877 `__: BUG: ndimage: guard against footprints of all zeros - - `#6881 `__: python 3.6 - - `#6885 `__: Vectorized integrate.fixed_quad - - `#6886 `__: fixed typo - - `#6891 `__: TST: fix failures for linalg.dare/care due to tightened test... - - `#6892 `__: DOC: fix a bunch of Sphinx errors. - - `#6894 `__: TST: Added asv benchmarks for scipy.spatial.Voronoi - - `#6908 `__: BUG: Fix return dtype for complex input in spsolve - - `#6909 `__: ENH: fftpack: use float32 routines for float16 inputs. - - `#6911 `__: added min/max support to binned_statistic - - `#6913 `__: Fix 6875: SLSQP raise ValueError for all invalid bounds. - - `#6914 `__: DOCS: GH6903 updating docs of Spatial.distance.pdist - - `#6916 `__: MAINT: fix some issues for 32-bit Python - - `#6924 `__: BLD: update Bento build for scipy.LowLevelCallable - - `#6932 `__: ENH: Use OrderedDict in io.netcdf. Closes gh-5537 - - `#6933 `__: BUG: fix LowLevelCallable issue on 32-bit Python. - - `#6936 `__: BUG: sparse: handle size-1 2D indexes correctly - - `#6938 `__: TST: fix test failures in special on 32-bit Python. - - `#6939 `__: Added attributes list to cKDTree docstring - - `#6940 `__: improve efficiency of dok_matrix.tocoo - - `#6942 `__: DOC: add link to liac-arff package in the io.arff docstring. - - `#6943 `__: MAINT: Docstring fixes and an additional test for linalg.solve - - `#6944 `__: DOC: Add example of odeint with a banded Jacobian to the integrate... - - `#6946 `__: ENH: hypergeom.logpmf in terms of betaln - - `#6947 `__: TST: speedup distance tests - - `#6948 `__: DEP: Deprecate the keyword "debug" from linalg.solve - - `#6950 `__: BUG: Correctly treat large integers in MMIO (fixes #6397) - - `#6952 `__: ENH: Minor user-friendliness cleanup in LowLevelCallable - - `#6956 `__: DOC: improve description of 'output' keyword for convolve - - `#6957 `__: ENH more informative error in sparse.bmat - - `#6962 `__: Shebang fixes - - `#6964 `__: DOC: note argmin/argmax addition - - `#6965 `__: BUG: Fix issues passing error tolerances in dblquad and tplquad. - - `#6971 `__: fix the docstring of signaltools.correlate - - `#6973 `__: Silence expected numpy warnings in scipy.ndimage.interpolation.zoom() - - `#6975 `__: BUG: special: fix regex in `generate_ufuncs.py` - - `#6976 `__: Update docstring for griddata - - `#6978 `__: Avoid division by zero in zoom factor calculation - - `#6979 `__: BUG: ARE solvers did not check the generalized case carefully - - `#6985 `__: ENH: sparse: add scipy.sparse.linalg.spsolve_triangular - - `#6994 `__: MAINT: spatial: updates to plotting utils - - `#6995 `__: DOC: Bad documentation of k in sparse.linalg.eigs See #6990 - - `#6997 `__: TST: Changed the test with a less singular example - - `#7000 `__: DOC: clarify interp1d 'zero' argument - - `#7007 `__: BUG: Fix division by zero in linregress() for 2 data points - - `#7009 `__: BUG: Fix problem in passing drop_rule to scipy.sparse.linalg.spilu - - `#7012 `__: speed improvment in _distn_infrastructure.py - - `#7014 `__: Fix Typo: add a single quotation mark to fix a slight typo - - `#7021 `__: MAINT: stats: use machine constants from np.finfo, not machar - - `#7026 `__: MAINT: update .mailmap - - `#7032 `__: Fix layout of rv_histogram docs - - `#7035 `__: DOC: update 0.19.0 release notes - - `#7036 `__: ENH: Add more boundary options to signal.stft - - `#7040 `__: TST: stats: skip too slow tests - - `#7042 `__: MAINT: sparse: speed up setdiag tests - - `#7043 `__: MAINT: refactory and code cleaning Xdist - - `#7053 `__: Fix msvc 9 and 10 compile errors - - `#7060 `__: DOC: updated release notes with #7043 and #6656 - - `#7062 `__: MAINT: Change defaut STFT boundary kwarg to "zeros" - - `#7064 `__: Fix ValueError: path is on mount 'X:', start on mount 'D:' on... - - `#7067 `__: TST: Fix PermissionError: [Errno 13] Permission denied on Windows - - `#7068 `__: TST: Fix UnboundLocalError: local variable 'data' referenced... - - `#7069 `__: Fix OverflowError: Python int too large to convert to C long... - - `#7071 `__: TST: silence RuntimeWarning for nan test of stats.spearmanr - - `#7072 `__: Fix OverflowError: Python int too large to convert to C long... - - `#7084 `__: TST: linalg: bump tolerance in test_falker Checksums ========= MD5 ~~~ a1d4a08cb0408fda554787e502a99d89 scipy-0.19.0rc2.tar.gz e5531359c06c6cccb544e0fcb728d832 scipy-0.19.0rc2.tar.xz aad1a08d3eee196a6d3f850b61d8aec5 scipy-0.19.0rc2.zip SHA256 ~~~~~~ a9f978f8cc569138d16f0b115476da8355fd149cbacefed7f283682d52540d05 scipy-0.19.0rc2.tar.gz 643d47551d16d965efe4f16613ee71ac6396481df649df3ad3a63848776cc084 scipy-0.19.0rc2.tar.xz e8f80a26a1089b35b7f410509fa703598a4cd74dd42636f22aa1b65085beaf9f scipy-0.19.0rc2.zip -----BEGIN PGP SIGNATURE----- Version: GnuPG v1.4.11 (GNU/Linux) iQEcBAEBCAAGBQJYsEDoAAoJEIp0pQ0zQcu+UjAH/1gErT0trZxgNqtxEtIbQ0RP V9EIzQ3kj1QJHx7KKgheswHT4Ee6LYhA5MXzMAIDRx2TIMzE3ByBktK/So6+8U96 jCkceIHMwHgVnzxA8lkoB6AILHWMaaN/jNKk0u9Y6IKwUY4S1kQkYysAlMOw3gaG g1fOJzw1Bn4Yq0dEXw+tAAFRVPSAYKohJgc7U6AvW435VI3QmOqsCCbwh2ua1oN6 egaIBD0QBOGVYb7vlhOMiKPKKkIhEY2DVXnA7EVCiJMykq/dc+pReIL8YNZ+xI6F WUNv1KUAmVfKXU5MfSwTcxLLH89HJ7WfE8asCmVpgavhqdk0i7D4zVzN20uGe18= =f4kz -----END PGP SIGNATURE----- From kevin.gullikson at gmail.com Fri Feb 24 09:50:23 2017 From: kevin.gullikson at gmail.com (Kevin Gullikson) Date: Fri, 24 Feb 2017 14:50:23 +0000 Subject: [SciPy-User] Use Distance Matrix in scipy.cluster.hierarchy.linkage()? In-Reply-To: References: Message-ID: Sema, It is a two step process (at least). Here is an example: from scipy.cluster import hierarchy Z = hierarchy.linkage(dm, metric='precomputed') labels = hierarchy.fcluster(Z, n_clusters, criterion='maxclust') The labels array will have length n, and will look like [0, 0, 1, 2, 0, ...]. Typically, you have to choose the number of clusters (or some other criterion, see fcluster docs). You can do that by looking at the dendrogram plot with this command, which will make a matplotlib plot: hierarchy.dendrogram(Z) # This has a bunch of useful keyword arguments too. Happy clustering! Kevin On Fri, Feb 24, 2017 at 12:59 AM Sema Atasever wrote: > Dear SciPy list member, > > I want to ask you about clustering usign scipy.cluster.hierarchy. > > I have a *distance matrix* n*n M where M_ij is the distance between > object_i and object_j. You can see file format in the attachment --> > (dm.csv) > > I want to cluster these n objects with hierarchical clustering. > > For this purpose i am usign this python code that you can see in the > attachment (scipy_code.py) > > I want to ask that how can get clusters values in pdf format or text > format and how many clusters did i get and what clusters includes what > members? > > Thanks in Advance, Best regards. > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > -- Kevin Gullikson Data Scientist Spark Cognition -------------- next part -------------- An HTML attachment was scrubbed... URL: From gregor.thalhammer at gmail.com Fri Feb 24 11:13:22 2017 From: gregor.thalhammer at gmail.com (Gregor Thalhammer) Date: Fri, 24 Feb 2017 17:13:22 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: Message-ID: <589B0B68-10F3-42D6-AE35-DB7D1824B558@gmail.com> > Am 24.02.2017 um 13:36 schrieb Propadovic Nenad : > > Hello Matthew, hello Gregor, > > thank you for your answers. > > Yes, values struct_as_record=False and squeeze_me=True are what the workaround I mentioned in the initial post at: > http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/ is based on. > > I will use that workaround, it gives nice access to everything I need. > > Why I hesitated initially, after finding it was that setting struct_as_record=False boils down to using the class scipy.io.matlab.mio5_params.mat_struct. > > When you look at the docstring of that class it says: "We deprecate this method of holding struct information, and will soon remove it, in favor of the recarray method?. This comment has been added 7 years ago, but the mat_struct is still there (but it is not used anymore with default settings). So I think it cannot be taken seriously. Perhaps you might file an issue for removing or altering this remark. Anyhow, the field names are accessible by y = scipy.io.loadmat("x.mat") y['CanData'][0, 0]['msg'].dtype.names ('RPDO2',) or y = scipy.io.loadmat("x.mat", squeeze_me=True) y['CanData']['msg'].item().dtype.names > > So the way I understand this, I'm building my code upon something that won't be around for long. An uncomfortable situation, I don't know if you agree? I'm a consultant, the code will be around the clients place longer than myself. So this was the reason I started this thread in the first place. I really tried to explain that in the initial post, with all the necessary detail, but it seems that the post was too confusing. Sorry for bothering everybody for so long. I'm, however, still not sure if I should file this as a bug, or some kind of feature request, as there seem to be very few people who bother (I counted maybe 2-3 more related posts, plus the post containing the workaround, at stack overflow). I would not consider this as a bug, more a documentation issue, which is resolved by this discussion. You might add a comment on stack overflow with your findings. Gregor > > Regards, > > Nenad > > > 2017-02-24 12:15 GMT+01:00 >: > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Matthew Brett) > 2. Use Distance Matrix in scipy.cluster.hierarchy.linkage()? > (Sema Atasever) > 3. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Gregor Thalhammer) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Thu, 23 Feb 2017 17:41:34 -0800 > From: Matthew Brett > > To: SciPy Users List > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: > > > Content-Type: text/plain; charset=UTF-8 > > Hi, > > On Thu, Feb 23, 2017 at 5:12 AM, Propadovic Nenad > wrote: > > Hello Jason, than you a lot for the answer to my post. I was aware of the > > squeeze_me=True option, I think I mentioned it in my initial question post. > > > > However, as I stated in the answer to Gregors kind answer, I actually need a > > way to inspect the parts of the access path in the data structure I import > > form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' > > disappear completely: > > > > > > import scipy.io > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > > > Output: > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , > > 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > 0.003968, 0.007978])),) > > > > And I really need to be able to find it by some kind of inspection, so that > > I don't return parts of the structure that don't correspond to the intention > > of the person searching. > > Sorry if I'm not following, but, does this help? > > In [36]: y = scipy.io.loadmat("x.mat") > > In [37]: y['CanData'][0, 0]['msg'][0, 0]['RPDO2'] > Out[37]: > array([[ (array([[ 0. , 0.003968, 0.007978]]), array([[ > (array([[(array([[ 61.96, 61.96, 61.96]]), array(['PosAct'], > dtype=' dtype=[('Values', 'O'), ('Name', 'O')]), array([[(array([[-0.05, > -0.1 , 0.3 ]]), array(['VelAct'], > dtype=' dtype=[('Values', 'O'), ('Name', 'O')]))]], > dtype=[('PosAct', 'O'), ('VelAct', 'O')]))]], > dtype=[('timest', 'O'), ('sig', 'O')]) > > In [54]: y2 = scipy.io.loadmat('x.mat', squeeze_me=True, struct_as_record=False) > > In [55]: y2['CanData'].msg.RPDO2 > Out[55]: > > Best, > > Matthew > > > ------------------------------ > > Message: 2 > Date: Fri, 24 Feb 2017 09:59:04 +0300 > From: Sema Atasever > > To: scipy-user at python.org > Subject: [SciPy-User] Use Distance Matrix in > scipy.cluster.hierarchy.linkage()? > Message-ID: > > > Content-Type: text/plain; charset="utf-8" > > Dear SciPy list member, > > I want to ask you about clustering usign scipy.cluster.hierarchy. > > I have a *distance matrix* n*n M where M_ij is the distance between > object_i and object_j. You can see file format in the attachment --> > (dm.csv) > > I want to cluster these n objects with hierarchical clustering. > > For this purpose i am usign this python code that you can see in the > attachment (scipy_code.py) > > I want to ask that how can get clusters values in pdf format or text format > and how many clusters did i get and what clusters includes what members? > > Thanks in Advance, Best regards. > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > -------------- next part -------------- > import scipy > import pylab > import scipy.cluster.hierarchy as sch > import pandas as pd > import numpy as np > > D=np.loadtxt(open("C:\dm.csv", "rb"), delimiter="\t", usecols=range(1,11)) > > print (D) > print (D.shape) > > # Compute and plot dendrogram. > fig = pylab.figure() > axdendro = fig.add_axes([0.09,0.1,0.2,0.8]) > Y = sch.linkage(D, method='single') > Z = sch.dendrogram(Y, orientation='right') > axdendro.set_xticks([]) > axdendro.set_yticks([]) > > # Plot distance matrix. > axmatrix = fig.add_axes([0.3,0.1,0.6,0.8]) > index = Z['leaves'] > D = D[index,:] > D = D[:,index] > im = axmatrix.matshow(D, aspect='auto', origin='lower') > axmatrix.set_xticks([]) > axmatrix.set_yticks([]) > > # Plot colorbar. > axcolor = fig.add_axes([0.91,0.1,0.02,0.8]) > pylab.colorbar(im, cax=axcolor) > > # Display and save figure. > fig.show() > fig.savefig('dendrogram.png') > -------------- next part -------------- > A non-text attachment was scrubbed... > Name: dm.csv > Type: text/csv > Size: 680 bytes > Desc: not available > URL: > > > ------------------------------ > > Message: 3 > Date: Fri, 24 Feb 2017 12:15:18 +0100 > From: Gregor Thalhammer > > To: SciPy Users List > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: <7CCED80F-462D-4BED-9540-21E262206886 at gmail.com > > Content-Type: text/plain; charset="us-ascii" > > > > Am 23.02.2017 um 14:12 schrieb Propadovic Nenad >: > > > > Hello Jason, than you a lot for the answer to my post. I was aware of the squeeze_me=True option, I think I mentioned it in my initial question post. > > > > However, as I stated in the answer to Gregors kind answer, I actually need a way to inspect the parts of the access path in the data structure I import form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' disappear completely: > > > > The substruct names are somewhat hidden, try this: > > y = scipy.io.loadmat('x.mat', squeeze_me=True, struct_as_record=False) > y['CanData'].msg._fieldnames > > ['RPDO2'] > > Gregor > > PS: for introspection of objects, just take a look at the __dict__ attribute. > > > > > import scipy.io > > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > cd = y['CanData'] > > msg = cd['msg'] > > print msg > > > > Output: > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, -0.1 , 0.3 ]), u'VelAct')), > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , 0.003968, 0.007978])),) > > > > And I really need to be able to find it by some kind of inspection, so that I don't return parts of the structure that don't correspond to the intention of the person searching. > > > > Thanks once again! > > > > Nenad > > > > > > > > 2017-02-22 18:00 GMT+01:00 >>: > > Send SciPy-User mailing list submissions to > > scipy-user at python.org > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://mail.python.org/mailman/listinfo/scipy-user > > > or, via email, send a message with subject or body 'help' to > > scipy-user-request at python.org > > > > > You can reach the person managing the list at > > scipy-user-owner at python.org > > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of SciPy-User digest..." > > > > > > Today's Topics: > > > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > > the data inaccessible after loadmat (Jason Sachs) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Wed, 22 Feb 2017 09:12:26 -0700 > > From: Jason Sachs >> > > To: SciPy Users List >> > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > > bug: parts of the data inaccessible after loadmat > > Message-ID: > > >> > > Content-Type: text/plain; charset="utf-8" > > > > ah, yes, here it is: > > > > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html > > > > > ---- > > > > So, in MATLAB, the struct array must be at least 2D, and we replicate that > > when we read into Scipy. If you want all length 1 dimensions squeezed out, > > try this: > > >>> > > > > >>> mat_contents = sio.loadmat('octave_struct.mat', squeeze_me=True)>>> oct_struct = mat_contents['my_struct']>>> oct_struct.shape() > > > > > > On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs >> wrote: > > > > > This looks familiar, I ran into this a few years ago, and if I recall > > > correctly, there is an option to loadmat to reduce array dimensions > > > appropriately. There is a "squeeze_me" option (unfortunately named... > > > should probably be deprecated in favor of "squeeze") which I think does > > > this. > > > > > > https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html > > > > > > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > > > gregor.thalhammer at gmail.com >> wrote: > > > > > >> > > >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad >>: > > >> > > >> Hello, > > >> > > >> bear with me for the long post that follows: it took me more than a week > > >> to get this far, and I tried to compress all the relevant information into > > >> the post. > > >> > > >> There seems to be a bug in scipy.io.loadmat; I'll present it by a short > > >> piece of code and it's output. > > >> > > >> I create file x.mat with the following: > > >> > > >> import scipy.io > > > >> > > >> d = {'CanData': > > >> { > > >> 'msg': { > > >> 'RPDO2': { > > >> 'timest': [0.0, 0.0039679999899817631, > > >> 0.0079779999941820279], > > >> 'sig': { > > >> 'VelAct': { > > >> 'Values': [-0.050000000000000003, > > >> -0.10000000000000001, 0.29999999999999999, ], > > >> 'Name': 'VelAct' > > >> }, > > >> 'PosAct': { > > >> 'Values': [61.960000000000001, > > >> 61.960000000000001, 61.960000000000001, ], > > >> 'Name': 'PosAct' > > >> } > > >> } > > >> } > > >> } > > >> } > > >> } > > >> scipy.io.savemat("x.mat", d) > > >> > > >> Matlab is happy with the file and handles it the way I expect. > > >> > > >> When I read in the data stored in the file and print it out: > > >> > > >> import scipy.io > > > >> y = scipy.io.loadmat("x.mat") > > >> # print y > > >> cd = y['CanData'] > > >> msg = cd['msg'] > > >> print msg > > >> print msg.dtype > > >> print msg.dtype.names > > >> > > >> The output is: > > >> >C:\Anaconda2\pythonw -u "test1.py" > > >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > > >> array([u'PosAct'], > > >> dtype=' > >> array([u'VelAct'], > > >> dtype=' > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > > >> 0.003968, 0.007978]]))]],)]], > > >> dtype=[('RPDO2', 'O')])]] > > >> object > > >> None > > >> > > >> Now I've read the manual, and as I see it I have no way for me to access > > >> the deeper layers of data I just put in the file x.mat, although they are > > >> obviously right there in the data read in. Access via msg['RPDO2'] gives: > > >> IndexError: only integers, slices (`:`), ellipsis (`...`), numpy.newaxis > > >> (`None`) and integer or boolean arrays are valid indices. > > >> > > >> > > >> For historic reasons, in Matlab everything is at least a 2D array, even > > >> scalars. By sprinkling some [0,0] in your code you should get what you > > >> want, e.g. > > >> > > >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] > > >> > > >> array([[ 0. , 0.003968, 0.007978]]) > > >> > > >> > > >> Gregor > > >> > > >> > > >> > > >> If I use parameter squeeze_me=True: > > >> > > >> scipy.io.savemat("x.mat", d) > > >> y = scipy.io.loadmat("x.mat", squeeze_me=True) > > >> # print y > > >> cd = y['CanData'] > > >> msg = cd['msg'] > > >> print msg > > >> print msg.dtype > > >> print msg.dtype.names > > >> > > >> I get output: > > >> >C:\Anaconda2\pythonw -u "test1.py" > > >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > > >> -0.1 , 0.3 ]), u'VelAct')), > > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > >> 0.003968, 0.007978])),) > > >> object > > >> None > > >> >Exit code: 0 > > >> > > >> All well, but the name 'RPDO2' disapeared from the data! > > >> > > >> Now I need this information; in future I won't handle what's put into > > >> x.mat, so I need a way to access through the data all the way down (and > > >> handle the variations that will come). > > >> > > >> I have found a workaround at: > > >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- > > > >> nested-structures-i-e-dictionaries/ > > >> > > >> The problem is, the workaround uses struct_as_record=False in loadmat, > > >> and which boils down to using scipy.io.matlab.mio5_params.ma t_struct, > > >> and when you read the docstring of class mat_struct, it says: > > >> > > >> ''' > > >> ... > > >> We deprecate this method of holding struct information, and will > > >> soon remove it, in favor of the recarray method (see loadmat > > >> docstring) > > >> ''' > > >> So my questions: > > >> 1) Did I miss something? Is there a way to access the data in 'RPDO2' by > > >> using this name, without using parameter struct_as_record=False in loadmat? > > >> 2) If not, where do I file a bug? The workaround is five years old, so > > >> the issue seems to be in scipy for ages... > > >> > > >> (For the records, I use scipy within Anaconda2 1.4.1, under Windows, but > > >> this does not seem to matter). > > >> > > >> Thanks a lot for the answers, in advance. > > >> > > >> Nenad > > >> > > >> > > >> _______________________________________________ > > >> SciPy-User mailing list > > >> SciPy-User at python.org > > > >> https://mail.python.org/mailman/listinfo/scipy-user > > > >> > > >> > > >> > > >> _______________________________________________ > > >> SciPy-User mailing list > > >> SciPy-User at python.org > > > >> https://mail.python.org/mailman/listinfo/scipy-user > > > >> > > >> > > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: >> > > > > ------------------------------ > > > > Subject: Digest Footer > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > > https://mail.python.org/mailman/listinfo/scipy-user > > > > > > > ------------------------------ > > > > End of SciPy-User Digest, Vol 162, Issue 6 > > ****************************************** > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: > > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 10 > ******************************************* > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user -------------- next part -------------- An HTML attachment was scrubbed... URL: From evgeny.burovskiy at gmail.com Fri Feb 24 11:37:56 2017 From: evgeny.burovskiy at gmail.com (Evgeni Burovski) Date: Fri, 24 Feb 2017 19:37:56 +0300 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: <589B0B68-10F3-42D6-AE35-DB7D1824B558@gmail.com> References: <589B0B68-10F3-42D6-AE35-DB7D1824B558@gmail.com> Message-ID: >> Why I hesitated initially, after finding it was that setting struct_as_record=False boils down to using the class scipy.io.matlab.mio5_params.mat_struct. >> >> When you look at the docstring of that class it says: "We deprecate this method of holding struct information, and will soon remove it, in favor of the recarray method?. > > > This comment has been added 7 years ago, but the mat_struct is still there (but it is not used anymore with default settings). So I think it cannot be taken seriously. Perhaps you might file an issue for removing or altering this remark. Yes please send a PR with the documentation fix! -------------- next part -------------- An HTML attachment was scrubbed... URL: From npropadovic at gmail.com Fri Feb 24 12:37:16 2017 From: npropadovic at gmail.com (Propadovic Nenad) Date: Fri, 24 Feb 2017 18:37:16 +0100 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat Message-ID: Hello Gregor, Anyhow, the field names are accessible by > > y = scipy.io.loadmat("x.mat") > y['CanData'][0, 0]['msg'].dtype.names > > ('RPDO2',) > Oh. I missed both the above and the below. :( or > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > y['CanData']['msg'].item().dtype.names > > These are the access methods I was looking for all the time. I guess before I ask why .item() has to be used I guess I'll go rtfm some more :( > I would not consider this as a bug, more a documentation issue, which is > resolved by this discussion. You might add a comment on stack overflow with > your findings. > > Gregor > Yes, I will. Thank you! Nenad -------------- next part -------------- An HTML attachment was scrubbed... URL: From matthew.brett at gmail.com Fri Feb 24 12:49:46 2017 From: matthew.brett at gmail.com (Matthew Brett) Date: Fri, 24 Feb 2017 09:49:46 -0800 Subject: [SciPy-User] How to handle a scipy.io.loadmat - related bug: parts of the data inaccessible after loadmat In-Reply-To: References: Message-ID: Hi, On Fri, Feb 24, 2017 at 4:36 AM, Propadovic Nenad wrote: > Hello Matthew, hello Gregor, > > thank you for your answers. > > Yes, values struct_as_record=False and squeeze_me=True are what the > workaround I mentioned in the initial post at: > > http://stackoverflow.com/questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/ > > is based on. > > I will use that workaround, it gives nice access to everything I need. Sure - the other option you have is to inspect the field names of the dtype to find the structure variable names. > Why I hesitated initially, after finding it was that setting > struct_as_record=False boils down to using the class > scipy.io.matlab.mio5_params.mat_struct. > > When you look at the docstring of that class it says: "We deprecate this > method of holding struct information, and will soon remove it, in favor of > the recarray method". Yes, sorry, I should have replied to that. I will remove that warning, it's become clear over time that the mini class representation that you are using does have a place, so I don't think we should plan to remove it: https://github.com/scipy/scipy/pull/7090 Best, Matthew From s.atasever at gmail.com Mon Feb 27 04:55:18 2017 From: s.atasever at gmail.com (Sema Atasever) Date: Mon, 27 Feb 2017 12:55:18 +0300 Subject: [SciPy-User] SciPy-User Digest, Vol 162, Issue 13 In-Reply-To: References: Message-ID: Kevin, Thank you for your quick reply to my email, the information was valuable to me... On Fri, Feb 24, 2017 at 7:13 PM, wrote: > Send SciPy-User mailing list submissions to > scipy-user at python.org > > To subscribe or unsubscribe via the World Wide Web, visit > https://mail.python.org/mailman/listinfo/scipy-user > or, via email, send a message with subject or body 'help' to > scipy-user-request at python.org > > You can reach the person managing the list at > scipy-user-owner at python.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of SciPy-User digest..." > > > Today's Topics: > > 1. Re: Use Distance Matrix in scipy.cluster.hierarchy.linkage()? > (Kevin Gullikson) > 2. Re: How to handle a scipy.io.loadmat - related bug: parts of > the data inaccessible after loadmat (Gregor Thalhammer) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 24 Feb 2017 14:50:23 +0000 > From: Kevin Gullikson > To: SciPy Users List > Subject: Re: [SciPy-User] Use Distance Matrix in > scipy.cluster.hierarchy.linkage()? > Message-ID: > gmail.com> > Content-Type: text/plain; charset="utf-8" > > Sema, > > It is a two step process (at least). Here is an example: > > from scipy.cluster import hierarchy > > Z = hierarchy.linkage(dm, metric='precomputed') > labels = hierarchy.fcluster(Z, n_clusters, criterion='maxclust') > > The labels array will have length n, and will look like [0, 0, 1, 2, 0, > ...]. Typically, you have to choose the number of clusters (or some other > criterion, see fcluster docs). You can do that by looking at the dendrogram > plot with this command, which will make a matplotlib plot: > > hierarchy.dendrogram(Z) # This has a bunch of useful keyword arguments > too. > > Happy clustering! > Kevin > > On Fri, Feb 24, 2017 at 12:59 AM Sema Atasever > wrote: > > > Dear SciPy list member, > > > > I want to ask you about clustering usign scipy.cluster.hierarchy. > > > > I have a *distance matrix* n*n M where M_ij is the distance between > > object_i and object_j. You can see file format in the attachment --> > > (dm.csv) > > > > I want to cluster these n objects with hierarchical clustering. > > > > For this purpose i am usign this python code that you can see in the > > attachment (scipy_code.py) > > > > I want to ask that how can get clusters values in pdf format or text > > format and how many clusters did i get and what clusters includes what > > members? > > > > Thanks in Advance, Best regards. > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > > -- > Kevin Gullikson > Data Scientist > Spark Cognition > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: attachments/20170224/657874c4/attachment-0001.html> > > ------------------------------ > > Message: 2 > Date: Fri, 24 Feb 2017 17:13:22 +0100 > From: Gregor Thalhammer > To: SciPy Users List > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > bug: parts of the data inaccessible after loadmat > Message-ID: <589B0B68-10F3-42D6-AE35-DB7D1824B558 at gmail.com> > Content-Type: text/plain; charset="utf-8" > > > > Am 24.02.2017 um 13:36 schrieb Propadovic Nenad : > > > > Hello Matthew, hello Gregor, > > > > thank you for your answers. > > > > Yes, values struct_as_record=False and squeeze_me=True are what the > workaround I mentioned in the initial post at: > > http://stackoverflow.com/questions/7008608/scipy-io- > loadmat-nested-structures-i-e-dictionaries/ questions/7008608/scipy-io-loadmat-nested-structures-i-e-dictionaries/>is > based on. > > > > I will use that workaround, it gives nice access to everything I need. > > > > Why I hesitated initially, after finding it was that setting > struct_as_record=False boils down to using the class > scipy.io.matlab.mio5_params.mat_struct. > > > > When you look at the docstring of that class it says: "We deprecate this > method of holding struct information, and will soon remove it, in favor of > the recarray method?. > > This comment has been added 7 years ago, but the mat_struct is still there > (but it is not used anymore with default settings). So I think it cannot be > taken seriously. Perhaps you might file an issue for removing or altering > this remark. > > Anyhow, the field names are accessible by > > y = scipy.io.loadmat("x.mat") > y['CanData'][0, 0]['msg'].dtype.names > > ('RPDO2',) > > or > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > y['CanData']['msg'].item().dtype.names > > > > > > So the way I understand this, I'm building my code upon something that > won't be around for long. An uncomfortable situation, I don't know if you > agree? I'm a consultant, the code will be around the clients place longer > than myself. So this was the reason I started this thread in the first > place. I really tried to explain that in the initial post, with all the > necessary detail, but it seems that the post was too confusing. Sorry for > bothering everybody for so long. I'm, however, still not sure if I should > file this as a bug, or some kind of feature request, as there seem to be > very few people who bother (I counted maybe 2-3 more related posts, plus > the post containing the workaround, at stack overflow). > > I would not consider this as a bug, more a documentation issue, which is > resolved by this discussion. You might add a comment on stack overflow with > your findings. > > Gregor > > > > > > > Regards, > > > > Nenad > > > > > > 2017-02-24 12:15 GMT+01:00 scipy-user-request at python.org>>: > > Send SciPy-User mailing list submissions to > > scipy-user at python.org > > > > To subscribe or unsubscribe via the World Wide Web, visit > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > or, via email, send a message with subject or body 'help' to > > scipy-user-request at python.org python.org> > > > > You can reach the person managing the list at > > scipy-user-owner at python.org > > > > When replying, please edit your Subject line so it is more specific > > than "Re: Contents of SciPy-User digest..." > > > > > > Today's Topics: > > > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > > the data inaccessible after loadmat (Matthew Brett) > > 2. Use Distance Matrix in scipy.cluster.hierarchy.linkage()? > > (Sema Atasever) > > 3. Re: How to handle a scipy.io.loadmat - related bug: parts of > > the data inaccessible after loadmat (Gregor Thalhammer) > > > > > > ---------------------------------------------------------------------- > > > > Message: 1 > > Date: Thu, 23 Feb 2017 17:41:34 -0800 > > From: Matthew Brett matthew.brett at gmail.com>> > > To: SciPy Users List scipy-user at python.org>> > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > > bug: parts of the data inaccessible after loadmat > > Message-ID: > > gmail.com mail.gmail.com>> > > Content-Type: text/plain; charset=UTF-8 > > > > Hi, > > > > On Thu, Feb 23, 2017 at 5:12 AM, Propadovic Nenad > wrote: > > > Hello Jason, than you a lot for the answer to my post. I was aware of > the > > > squeeze_me=True option, I think I mentioned it in my initial question > post. > > > > > > However, as I stated in the answer to Gregors kind answer, I actually > need a > > > way to inspect the parts of the access path in the data structure I > import > > > form the x.mat-file, and if I use squeeze_me=True, parts like 'RPDO2' > > > disappear completely: > > > > > > > > > import scipy.io > > > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > > cd = y['CanData'] > > > msg = cd['msg'] > > > print msg > > > > > > Output: > > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 , > > > 0.3 ]), u'VelAct')), > > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > > 0.003968, 0.007978])),) > > > > > > And I really need to be able to find it by some kind of inspection, so > that > > > I don't return parts of the structure that don't correspond to the > intention > > > of the person searching. > > > > Sorry if I'm not following, but, does this help? > > > > In [36]: y = scipy.io.loadmat("x.mat") > > > > In [37]: y['CanData'][0, 0]['msg'][0, 0]['RPDO2'] > > Out[37]: > > array([[ (array([[ 0. , 0.003968, 0.007978]]), array([[ > > (array([[(array([[ 61.96, 61.96, 61.96]]), array(['PosAct'], > > dtype=' > dtype=[('Values', 'O'), ('Name', 'O')]), array([[(array([[-0.05, > > -0.1 , 0.3 ]]), array(['VelAct'], > > dtype=' > dtype=[('Values', 'O'), ('Name', 'O')]))]], > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]))]], > > dtype=[('timest', 'O'), ('sig', 'O')]) > > > > In [54]: y2 = scipy.io.loadmat('x.mat', squeeze_me=True, > struct_as_record=False) > > > > In [55]: y2['CanData'].msg.RPDO2 > > Out[55]: > > > > Best, > > > > Matthew > > > > > > ------------------------------ > > > > Message: 2 > > Date: Fri, 24 Feb 2017 09:59:04 +0300 > > From: Sema Atasever > > > To: scipy-user at python.org > > Subject: [SciPy-User] Use Distance Matrix in > > scipy.cluster.hierarchy.linkage()? > > Message-ID: > > gmail.com GuxYitM5yZ72fLPOo_5Q at mail.gmail.com>> > > Content-Type: text/plain; charset="utf-8" > > > > Dear SciPy list member, > > > > I want to ask you about clustering usign scipy.cluster.hierarchy. > > > > I have a *distance matrix* n*n M where M_ij is the distance between > > object_i and object_j. You can see file format in the attachment --> > > (dm.csv) > > > > I want to cluster these n objects with hierarchical clustering. > > > > For this purpose i am usign this python code that you can see in the > > attachment (scipy_code.py) > > > > I want to ask that how can get clusters values in pdf format or text > format > > and how many clusters did i get and what clusters includes what members? > > > > Thanks in Advance, Best regards. > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: attachments/20170224/dbac91b0/attachment-0001.html < > http://mail.python.org/pipermail/scipy-user/attachments/20170224/dbac91b0/ > attachment-0001.html>> > > -------------- next part -------------- > > import scipy > > import pylab > > import scipy.cluster.hierarchy as sch > > import pandas as pd > > import numpy as np > > > > D=np.loadtxt(open("C:\dm.csv", "rb"), delimiter="\t", > usecols=range(1,11)) > > > > print (D) > > print (D.shape) > > > > # Compute and plot dendrogram. > > fig = pylab.figure() > > axdendro = fig.add_axes([0.09,0.1,0.2,0.8]) > > Y = sch.linkage(D, method='single') > > Z = sch.dendrogram(Y, orientation='right') > > axdendro.set_xticks([]) > > axdendro.set_yticks([]) > > > > # Plot distance matrix. > > axmatrix = fig.add_axes([0.3,0.1,0.6,0.8]) > > index = Z['leaves'] > > D = D[index,:] > > D = D[:,index] > > im = axmatrix.matshow(D, aspect='auto', origin='lower') > > axmatrix.set_xticks([]) > > axmatrix.set_yticks([]) > > > > # Plot colorbar. > > axcolor = fig.add_axes([0.91,0.1,0.02,0.8]) > > pylab.colorbar(im, cax=axcolor) > > > > # Display and save figure. > > fig.show() > > fig.savefig('dendrogram.png') > > -------------- next part -------------- > > A non-text attachment was scrubbed... > > Name: dm.csv > > Type: text/csv > > Size: 680 bytes > > Desc: not available > > URL: attachments/20170224/dbac91b0/attachment-0001.csv pipermail/scipy-user/attachments/20170224/dbac91b0/attachment-0001.csv>> > > > > ------------------------------ > > > > Message: 3 > > Date: Fri, 24 Feb 2017 12:15:18 +0100 > > From: Gregor Thalhammer gregor.thalhammer at gmail.com>> > > To: SciPy Users List scipy-user at python.org>> > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > > bug: parts of the data inaccessible after loadmat > > Message-ID: <7CCED80F-462D-4BED-9540-21E262206886 at gmail.com 7CCED80F-462D-4BED-9540-21E262206886 at gmail.com>> > > Content-Type: text/plain; charset="us-ascii" > > > > > > > Am 23.02.2017 um 14:12 schrieb Propadovic Nenad >: > > > > > > Hello Jason, than you a lot for the answer to my post. I was aware of > the squeeze_me=True option, I think I mentioned it in my initial question > post. > > > > > > However, as I stated in the answer to Gregors kind answer, I actually > need a way to inspect the parts of the access path in the data structure I > import form the x.mat-file, and if I use squeeze_me=True, parts like > 'RPDO2' disappear completely: > > > > > > > The substruct names are somewhat hidden, try this: > > > > y = scipy.io.loadmat('x.mat', squeeze_me=True, struct_as_record=False) > > y['CanData'].msg._fieldnames > > > > ['RPDO2'] > > > > Gregor > > > > PS: for introspection of objects, just take a look at the __dict__ > attribute. > > > > > > > > import scipy.io >> > > > > > > y = scipy.io.loadmat("x.mat", squeeze_me=True) > > > cd = y['CanData'] > > > msg = cd['msg'] > > > print msg > > > > > > Output: > > > ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), (array([-0.05, > -0.1 , 0.3 ]), u'VelAct')), > > > dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > 0.003968, 0.007978])),) > > > > > > And I really need to be able to find it by some kind of inspection, so > that I don't return parts of the structure that don't correspond to the > intention of the person searching. > > > > > > Thanks once again! > > > > > > Nenad > > > > > > > > > > > > 2017-02-22 18:00 GMT+01:00 scipy-user-request at python.org> >>: > > > Send SciPy-User mailing list submissions to > > > scipy-user at python.org scipy-user at python.org > > > > > > > To subscribe or unsubscribe via the World Wide Web, visit > > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> < > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user>> > > > or, via email, send a message with subject or body 'help' to > > > scipy-user-request at python.org python.org> scipy-user-request at python.org>> > > > > > > You can reach the person managing the list at > > > scipy-user-owner at python.org python.org> python.org>> > > > > > > When replying, please edit your Subject line so it is more specific > > > than "Re: Contents of SciPy-User digest..." > > > > > > > > > Today's Topics: > > > > > > 1. Re: How to handle a scipy.io.loadmat - related bug: parts of > > > the data inaccessible after loadmat (Jason Sachs) > > > > > > > > > ---------------------------------------------------------------------- > > > > > > Message: 1 > > > Date: Wed, 22 Feb 2017 09:12:26 -0700 > > > From: Jason Sachs > >> > > > To: SciPy Users List scipy-user at python.org> scipy-user at python.org>>> > > > Subject: Re: [SciPy-User] How to handle a scipy.io.loadmat - related > > > bug: parts of the data inaccessible after loadmat > > > Message-ID: > > > gmail.com 2B86dCY7fUxzB0XXcd0gZh3w at mail.gmail.com> 8NsKcn%2B86dCY7fUxzB0XXcd0gZh3w at mail.gmail.com CAOo6sOPTMBfRATCHmoGTN8NsKcn%252B86dCY7fUxzB0XXcd0gZh3w at mail.gmail.com>>> > > > Content-Type: text/plain; charset="utf-8" > > > > > > ah, yes, here it is: > > > > > > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html < > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html> < > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html < > https://docs.scipy.org/doc/scipy-0.18.1/reference/tutorial/io.html>> > > > > > > ---- > > > > > > So, in MATLAB, the struct array must be at least 2D, and we replicate > that > > > when we read into Scipy. If you want all length 1 dimensions squeezed > out, > > > try this: > > > >>> > > > > > > >>> mat_contents = sio.loadmat('octave_struct.mat', > squeeze_me=True)>>> oct_struct = mat_contents['my_struct']>>> > oct_struct.shape() > > > > > > > > > On Wed, Feb 22, 2017 at 9:11 AM, Jason Sachs jmsachs at gmail.com>>> wrote: > > > > > > > This looks familiar, I ran into this a few years ago, and if I recall > > > > correctly, there is an option to loadmat to reduce array dimensions > > > > appropriately. There is a "squeeze_me" option (unfortunately named... > > > > should probably be deprecated in favor of "squeeze") which I think > does > > > > this. > > > > > > > > https://docs.scipy.org/doc/scipy/reference/generated/ > scipy.io.loadmat.html scipy/reference/generated/scipy.io.loadmat.html> < > https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.loadmat.html > scipy.io.loadmat.html>> > > > > > > > > On Wed, Feb 22, 2017 at 9:02 AM, Gregor Thalhammer < > > > > gregor.thalhammer at gmail.com > >> > wrote: > > > > > > > >> > > > >> Am 22.02.2017 um 12:02 schrieb Propadovic Nenad < > npropadovic at gmail.com npropadovic at gmail.com >>: > > > >> > > > >> Hello, > > > >> > > > >> bear with me for the long post that follows: it took me more than a > week > > > >> to get this far, and I tried to compress all the relevant > information into > > > >> the post. > > > >> > > > >> There seems to be a bug in scipy.io.loadmat; I'll present it by a > short > > > >> piece of code and it's output. > > > >> > > > >> I create file x.mat with the following: > > > >> > > > >> import scipy.io http://scipy.io/>> > > > >> > > > >> d = {'CanData': > > > >> { > > > >> 'msg': { > > > >> 'RPDO2': { > > > >> 'timest': [0.0, 0.0039679999899817631, > > > >> 0.0079779999941820279], > > > >> 'sig': { > > > >> 'VelAct': { > > > >> 'Values': [-0.050000000000000003, > > > >> -0.10000000000000001, 0.29999999999999999, ], > > > >> 'Name': 'VelAct' > > > >> }, > > > >> 'PosAct': { > > > >> 'Values': [61.960000000000001, > > > >> 61.960000000000001, 61.960000000000001, ], > > > >> 'Name': 'PosAct' > > > >> } > > > >> } > > > >> } > > > >> } > > > >> } > > > >> } > > > >> scipy.io.savemat("x.mat", d) > > > >> > > > >> Matlab is happy with the file and handles it the way I expect. > > > >> > > > >> When I read in the data stored in the file and print it out: > > > >> > > > >> import scipy.io http://scipy.io/>> > > > >> y = scipy.io.loadmat("x.mat") > > > >> # print y > > > >> cd = y['CanData'] > > > >> msg = cd['msg'] > > > >> print msg > > > >> print msg.dtype > > > >> print msg.dtype.names > > > >> > > > >> The output is: > > > >> >C:\Anaconda2\pythonw -u "test1.py" > > > >> [[ array([[ ([[(array([[ ([[(array([[ 61.96, 61.96, 61.96]]), > > > >> array([u'PosAct'], > > > >> dtype=' > > >> array([u'VelAct'], > > > >> dtype=' > > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([[ 0. , > > > >> 0.003968, 0.007978]]))]],)]], > > > >> dtype=[('RPDO2', 'O')])]] > > > >> object > > > >> None > > > >> > > > >> Now I've read the manual, and as I see it I have no way for me to > access > > > >> the deeper layers of data I just put in the file x.mat, although > they are > > > >> obviously right there in the data read in. Access via msg['RPDO2'] > gives: > > > >> IndexError: only integers, slices (`:`), ellipsis (`...`), > numpy.newaxis > > > >> (`None`) and integer or boolean arrays are valid indices. > > > >> > > > >> > > > >> For historic reasons, in Matlab everything is at least a 2D array, > even > > > >> scalars. By sprinkling some [0,0] in your code you should get what > you > > > >> want, e.g. > > > >> > > > >> msg[0,0]['RPDO2'][0,0]['timest'][0,0] > > > >> > > > >> array([[ 0. , 0.003968, 0.007978]]) > > > >> > > > >> > > > >> Gregor > > > >> > > > >> > > > >> > > > >> If I use parameter squeeze_me=True: > > > >> > > > >> scipy.io.savemat("x.mat", d) > > > >> y = scipy.io.loadmat("x.mat", squeeze_me=True) > > > >> # print y > > > >> cd = y['CanData'] > > > >> msg = cd['msg'] > > > >> print msg > > > >> print msg.dtype > > > >> print msg.dtype.names > > > >> > > > >> I get output: > > > >> >C:\Anaconda2\pythonw -u "test1.py" > > > >> ((array(((array([ 61.96, 61.96, 61.96]), u'PosAct'), > (array([-0.05, > > > >> -0.1 , 0.3 ]), u'VelAct')), > > > >> dtype=[('PosAct', 'O'), ('VelAct', 'O')]), array([ 0. , > > > >> 0.003968, 0.007978])),) > > > >> object > > > >> None > > > >> >Exit code: 0 > > > >> > > > >> All well, but the name 'RPDO2' disapeared from the data! > > > >> > > > >> Now I need this information; in future I won't handle what's put > into > > > >> x.mat, so I need a way to access through the data all the way down > (and > > > >> handle the variations that will come). > > > >> > > > >> I have found a workaround at: > > > >> http://stackoverflow.com/questions/7008608/scipy-io-loadmat- < > http://stackoverflow.com/questions/7008608/scipy-io-loadmat-> < > http://stackoverflow.com/questions/7008608/scipy-io-loadmat- < > http://stackoverflow.com/questions/7008608/scipy-io-loadmat->> > > > >> nested-structures-i-e-dictionaries/ > > > >> > > > >> The problem is, the workaround uses struct_as_record=False in > loadmat, > > > >> and which boils down to using scipy.io.matlab.mio5_params.ma < > http://scipy.io.matlab.mio5_params.ma/>t_struct, > > > >> and when you read the docstring of class mat_struct, it says: > > > >> > > > >> ''' > > > >> ... > > > >> We deprecate this method of holding struct information, and will > > > >> soon remove it, in favor of the recarray method (see loadmat > > > >> docstring) > > > >> ''' > > > >> So my questions: > > > >> 1) Did I miss something? Is there a way to access the data in > 'RPDO2' by > > > >> using this name, without using parameter struct_as_record=False in > loadmat? > > > >> 2) If not, where do I file a bug? The workaround is five years old, > so > > > >> the issue seems to be in scipy for ages... > > > >> > > > >> (For the records, I use scipy within Anaconda2 1.4.1, under > Windows, but > > > >> this does not seem to matter). > > > >> > > > >> Thanks a lot for the answers, in advance. > > > >> > > > >> Nenad > > > >> > > > >> > > > >> _______________________________________________ > > > >> SciPy-User mailing list > > > >> SciPy-User at python.org SciPy-User at python.org > > > > >> https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> < > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user>> > > > >> > > > >> > > > >> > > > >> _______________________________________________ > > > >> SciPy-User mailing list > > > >> SciPy-User at python.org SciPy-User at python.org > > > > >> https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> < > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user>> > > > >> > > > >> > > > > > > > -------------- next part -------------- > > > An HTML attachment was scrubbed... > > > URL: attachments/20170222/e992b074/attachment-0001.html < > http://mail.python.org/pipermail/scipy-user/attachments/20170222/e992b074/ > attachment-0001.html> attachments/20170222/e992b074/attachment-0001.html < > http://mail.python.org/pipermail/scipy-user/attachments/20170222/e992b074/ > attachment-0001.html>>> > > > > > > ------------------------------ > > > > > > Subject: Digest Footer > > > > > > _______________________________________________ > > > SciPy-User mailing list > > > SciPy-User at python.org SciPy-User at python.org > > > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> < > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user>> > > > > > > > > > ------------------------------ > > > > > > End of SciPy-User Digest, Vol 162, Issue 6 > > > ****************************************** > > > > > > _______________________________________________ > > > SciPy-User mailing list > > > SciPy-User at python.org > > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > > > -------------- next part -------------- > > An HTML attachment was scrubbed... > > URL: attachments/20170224/46a54c05/attachment.html pipermail/scipy-user/attachments/20170224/46a54c05/attachment.html>> > > > > ------------------------------ > > > > Subject: Digest Footer > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user < > https://mail.python.org/mailman/listinfo/scipy-user> > > > > > > ------------------------------ > > > > End of SciPy-User Digest, Vol 162, Issue 10 > > ******************************************* > > > > _______________________________________________ > > SciPy-User mailing list > > SciPy-User at python.org > > https://mail.python.org/mailman/listinfo/scipy-user > > -------------- next part -------------- > An HTML attachment was scrubbed... > URL: attachments/20170224/0650188b/attachment.html> > > ------------------------------ > > Subject: Digest Footer > > _______________________________________________ > SciPy-User mailing list > SciPy-User at python.org > https://mail.python.org/mailman/listinfo/scipy-user > > > ------------------------------ > > End of SciPy-User Digest, Vol 162, Issue 13 > ******************************************* > -------------- next part -------------- An HTML attachment was scrubbed... URL: