From remco at gerlich.nl Mon Apr 4 12:47:19 2016 From: remco at gerlich.nl (remco at gerlich.nl) Date: Mon, 4 Apr 2016 19:47:19 +0300 Subject: [SciPy-User] *** SPAM *** Fw: new important message Message-ID: <0000206355b6$a8c35765$659b2ec2$@gerlich.nl> Hello! New message, please read remco at gerlich.nl -------------- next part -------------- An HTML attachment was scrubbed... URL: From opossumnano at gmail.com Thu Apr 7 05:07:14 2016 From: opossumnano at gmail.com (Tiziano Zito) Date: Thu, 07 Apr 2016 02:07:14 -0700 (PDT) Subject: [SciPy-User] =?utf-8?q?=5BANN=5D_Summer_School_=22Advanced_Scient?= =?utf-8?q?ific_Programming_in_Python=22_in_Reading=2C_UK=2C_September_5?= =?utf-8?b?4oCUMTEsIDIwMTY=?= Message-ID: <57062342.4a231c0a.a405e.ffffa5c2@mx.google.com> Advanced Scientific Programming in Python ========================================= a Summer School by the G-Node, and the Centre for Integrative Neuroscience and Neurodynamics, School of Psychology and Clinical Language Sciences, University of Reading, UK Scientists spend more and more time writing, maintaining, and debugging software. While techniques for doing this efficiently have evolved, only few scientists have been trained to use them. As a result, instead of doing their research, they spend far too much time writing deficient code and reinventing the wheel. In this course we will present a selection of advanced programming techniques and best practices which are standard in the industry, but especially tailored to the needs of a programming scientist. Lectures are devised to be interactive and to give the students enough time to acquire direct hands-on experience with the materials. Students will work in pairs throughout the school and will team up to practice the newly learned skills in a real programming project ? an entertaining computer game. We use the Python programming language for the entire course. Python works as a simple programming language for beginners, but more importantly, it also works great in scientific simulations and data analysis. We show how clean language design, ease of extensibility, and the great wealth of open source libraries for scientific computing and data visualization are driving Python to become a standard tool for the programming scientist. This school is targeted at Master or PhD students and Post-docs from all areas of science. Competence in Python or in another language such as Java, C/C++, MATLAB, or Mathematica is absolutely required. Basic knowledge of Python and of a version control system such as git, subversion, mercurial, or bazaar is assumed. Participants without any prior experience with Python and/or git should work through the proposed introductory material before the course. We are striving hard to get a pool of students which is international and gender-balanced. You can apply online: https://python.g-node.org Application deadline: 23:59 UTC, May 15, 2016. Be sure to read the FAQ before applying. Participation is for free, i.e. no fee is charged! Participants however should take care of travel, living, and accommodation expenses by themselves. Travel grants may be available. Date & Location =============== September 5?11, 2016. Reading, UK Program ======= - Best Programming Practices ? Best practices for scientific programming ? Version control with git and how to contribute to open source projects with GitHub ? Best practices in data visualization - Software Carpentry ? Test-driven development ? Debugging with a debuggger ? Profiling code - Scientific Tools for Python ? Advanced NumPy - Advanced Python ? Decorators ? Context managers ? Generators - The Quest for Speed ? Writing parallel applications ? Interfacing to C with Cython ? Memory-bound problems and memory profiling ? Data containers: storage and fast access to large data - Practical Software Development ? Group project Preliminary Faculty =================== ? Francesc Alted, freelance consultant, author of PyTables, Spain ? Pietro Berkes, Enthought Inc., Cambridge, UK ? Zbigniew J?drzejewski-Szmek, Krasnow Institute, George Mason University, Fairfax, VA, USA ? Eilif Muller, Blue Brain Project, ?cole Polytechnique F?d?rale de Lausanne, Switzerland ? Juan Nunez-Iglesias, Victorian Life Sciences Computation Initiative, University of Melbourne, Australia ? Rike-Benjamin Schuppner, Institute for Theoretical Biology, Humboldt-Universit?t zu Berlin, Germany ? Bartosz Tele?czuk, European Institute for Theoretical Neuroscience, CNRS, Paris, France ? St?fan van der Walt, Berkeley Institute for Data Science, UC Berkeley, CA, USA ? Nelle Varoquaux, Centre for Computational Biology Mines ParisTech, Institut Curie, U900 INSERM, Paris, France ? Tiziano Zito, freelance consultant, Germany Organizers ========== For the German Neuroinformatics Node of the INCF (G-Node) Germany: ? Tiziano Zito, freelance consultant, Germany ? Zbigniew J?drzejewski-Szmek, Krasnow Institute, George Mason University, Fairfax, USA ? Jakob Jordan, Institute of Neuroscience and Medicine (INM-6), Forschungszentrum J?lich GmbH, Germany For the Centre for Integrative Neuroscience and Neurodynamics, School of Psychology and Clinical Language Sciences, University of Reading UK: ? Etienne Roesch, Centre for Integrative Neuroscience and Neurodynamics, University of Reading, UK Website: https://python.g-node.org Contact: python-info at g-node.org From joshua.auerbach at epfl.ch Tue Apr 19 11:28:36 2016 From: joshua.auerbach at epfl.ch (Joshua Auerbach) Date: Tue, 19 Apr 2016 17:28:36 +0200 Subject: [SciPy-User] Why is hamming distance so slow? Message-ID: <57164EA4.8090307@epfl.ch> Hi, I am trying to calculate the pairwise hamming distances among a large set of boolean arrays. I would think that hamming distance would be one of the fastest distance metrics, but it seems to be much slower than alternatives. For instance I generate a 1000x1000 test boolean matrix| test = rng.rand(1000,1000) > 0.5 |Then I try to calculate the pairwise distances, e.g.| pdist(test, 'hamming') | I can run this in an average of 3.08s on my laptop, Whereas if I change /hamming/ to /sqeuclidean/ this goes down to 0.56s. These will effectively give me the same result (with the normalization factor), but I am quite surprised to see /hamming/ be so slow. In fact, computing /sqeuclidean/ distance on a real valued matrix of the same shape also runs in less than 0.6s Why should /hamming/, which is just doing bitwise /xor/ take so much longer than computing /sqeuclidean/ distances on real valued arrays? And, is there a recommended way to optimize this? Cheers, Josh -------------- next part -------------- An HTML attachment was scrubbed... URL: From ostrakismus at yahoo.de Thu Apr 28 03:58:50 2016 From: ostrakismus at yahoo.de (Udo Hoefel) Date: Thu, 28 Apr 2016 09:58:50 +0200 Subject: [SciPy-User] optimize.leastsq hessian approx. inversion? Message-ID: <5721C2BA.8090602@yahoo.de> Hi, I'm currently using scipy.optimize.leastsq to do a fit, to get to the covariance matrix, I use self.s_sq = (infodict['fvec']**2).sum()/(self.Max_Dist_Between_Dips-(2*(max(self.Channel_List)-1)+1)) self.pcov = cov_x*self.s_sq wherein self.Max_Dist_Between_Dips = Number of Datapoints and (2*(max(self.Channel_List)-1)+1) the number of free parameters. This is according to the documentation, I believe (and one question here, https://mail.scipy.org/pipermail/scipy-user/2013-March/034316.html ) My (probably stupid) question is the following: As I understand, cov_x is the unscaled Hessian, right? Shouldn't I have to invert it at some point, to get to the Covariance? My guess would have been something like self.pcov = self.s_sq*np.linalg.inv(cov_x) Big thanks in advance for any hint, Udo From domenico.delleside at le.infn.it Thu Apr 28 08:36:10 2016 From: domenico.delleside at le.infn.it (Domenico Delle Side) Date: Thu, 28 Apr 2016 12:36:10 +0000 (UTC) Subject: [SciPy-User] Shift theorem in Discrete Fourier Transform Message-ID: I'm trying to solve a problem with python+numpy in which I've some functions of type $f(x-x_i,y-y_i,z)$ that I need to convolve with another function $g(x,y,z,t)$. In order to optimize code, I performed the fft of f and g, I multiplied them and then I performed the inverse transformation to obtain the result. As a further optimization I realized that, thanks to the shift theorem, I could simply compute once the fft of f(x,y,z) and then multiply it by a phase factor that depends on $(x_i, y_i)$ to obtain the fft of $f(x-x_i,y-y_i,z)$. In particular, $\mathcal{F}(f(x-x_i,y-y_i,z)) = exp^{-2\pi j (x_i w_1 + y_i w_2)/N} \mathcal{F}(f(x,y,z))$, where N is the length of both x and y. I tried to implement this simple formula with python+numpy, but it fails for some reason that is obscure for me at the moment, so I'm asking the help of the group in order to figure out what I'm missing. I'm providing also a simple example. In [1]: import numpy as np In [2]: x = np.arange(-10, 11) In [3]: base = np.fft.fft(np.cos(x)) In [4]: shifted = np.fft.fft(np.cos(x-1)) In [5]: w = np.fft.fftfreq(x.size) In [6]: phase = np.exp(-2*np.pi*1.0j*w/x.size) In [7]: test = phase * base In [8]: (test == shifted).all() Out[8]: False In [9]: shifted/base Out[9]: array([ 0.54030231 -0.j , 0.54030231 -0.23216322j, 0.54030231 -0.47512034j, 0.54030231 -0.7417705j , 0.54030231 -1.05016033j, 0.54030231 -1.42919168j, 0.54030231 -1.931478j , 0.54030231 -2.66788185j, 0.54030231 -3.92462627j, 0.54030231 -6.74850534j, 0.54030231-20.55390586j, 0.54030231+20.55390586j, 0.54030231 +6.74850534j, 0.54030231 +3.92462627j, 0.54030231 +2.66788185j, 0.54030231 +1.931478j , 0.54030231 +1.42919168j, 0.54030231 +1.05016033j, 0.54030231 +0.7417705j , 0.54030231 +0.47512034j, 0.54030231 +0.23216322j]) In [10]: np.abs(shifted/base) Out[10]: array([ 0.54030231, 0.58807001, 0.71949004, 0.91768734, 1.18100097, 1.52791212, 2.00562555, 2.72204338, 3.96164334, 6.77009977, 20.56100612, 20.56100612, 6.77009977, 3.96164334, 2.72204338, 2.00562555, 1.52791212, 1.18100097, 0.91768734, 0.71949004, 0.58807001]) I expect that by means of shifted/base I could obtain the corresponding values of the phase factor, but as could be seen, it cannot be a phase factor, since its np.abs is >= 1! Any help is greatly appreciated. Thanks in advance, Domenico Delle Side From lev at columbia.edu Thu Apr 28 11:59:20 2016 From: lev at columbia.edu (Lev Givon) Date: Thu, 28 Apr 2016 11:59:20 -0400 Subject: [SciPy-User] Shift theorem in Discrete Fourier Transform In-Reply-To: References: Message-ID: <20160428155920.GC16017@avicenna.ee.columbia.edu> Received from Domenico Delle Side on Thu, Apr 28, 2016 at 08:36:10AM EDT: > I'm trying to solve a problem with python+numpy in which I've some functions > of type $f(x-x_i,y-y_i,z)$ that I need to convolve with another function > $g(x,y,z,t)$. In order to optimize code, I performed the fft of f and g, I > multiplied them and then I performed the inverse transformation to obtain > the result. FYI, there is an fftconvolve routine in scipy.signal that you might want to try. > As a further optimization I realized that, thanks to the shift theorem, I > could simply compute once the fft of f(x,y,z) and then multiply it by a > phase factor that depends on $(x_i, y_i)$ to obtain the fft of > $f(x-x_i,y-y_i,z)$. In particular, $\mathcal{F}(f(x-x_i,y-y_i,z)) = > exp^{-2\pi j (x_i w_1 + y_i w_2)/N} \mathcal{F}(f(x,y,z))$, where N is the > length of both x and y. > > I tried to implement this simple formula with python+numpy, but it fails for > some reason that is obscure for me at the moment, so I'm asking the help of > the group in order to figure out what I'm missing. > > I'm providing also a simple example. > In [1]: import numpy as np > In [2]: x = np.arange(-10, 11) > In [3]: base = np.fft.fft(np.cos(x)) > In [4]: shifted = np.fft.fft(np.cos(x-1)) > In [5]: w = np.fft.fftfreq(x.size) > In [6]: phase = np.exp(-2*np.pi*1.0j*w/x.size) You shouldn't be dividing by x.size in the above line because of the way fftfreq is defined; see the output of the attached modified script. Although it isn't much of an issue in the above example, in general you should zero-pad the signal being phase shifted so that the result of the circular convolution effectively performed by fft is equivalent to ordinary convolution. -- Lev Givon Bionet Group | Neurokernel Project http://lebedov.github.io/ http://neurokernel.github.io/ -------------- next part -------------- A non-text attachment was scrubbed... Name: fft_shift.py Type: text/x-python Size: 568 bytes Desc: not available URL: