Optimizing if statement check over a numpy value

[Heli Nix]

On Thursday, July 23, 2015 at 1:43:00 PM UTC+2, Jeremy Sanders wrote:
> Heli Nix wrote:
> 
> > Is there any way that I can optimize this if statement.
> 
> Array processing is much faster in numpy. Maybe this is close to what you 
> want
> 
> import numpy as N
> # input data
> vals = N.array([42, 1, 5, 3.14, 53, 1, 12, 11, 1])
> # list of items to exclude
> exclude = [1]
> # convert to a boolean array
> exclbool = N.zeros(vals.shape, dtype=bool)
> exclbool[exclude] = True
> # do replacement
> ones = vals==1.0
> # Note: ~ is numpy.logical_not
> vals[ones & (~exclbool)] = 1e-20
> 
> I think you'll have to convert your HDF array into a numpy array first, 
> using numpy.array().
> 
> Jeremy

Dear all, 

I tried the sorted python list, but this did not really help the runtime. 

I haven´t had time to check the sorted collections.  I solved my runtime problem by using the script from Jeremy up here. 

It was a life saviour and it is amazing how powerful numpy is. Thanks a lot Jeremy for this. By the way, I did not have to do any array conversion. The array read from hdf5 file using h5py is already a numpy array. 

The runtime over an array of around 16M reduced from around 12 hours (previous script) to 3 seconds using numpy on the same machine. 


Thanks alot for your help,