Remove masked elements from an array

[James J. Besemer]

holger krekel wrote:

> there are too solutions. choose yourself :-)

THREE if you include LAMBDA.

newnums = filter(lambda x: x is not None, nums )

This is like the first solution but does not require creating a function or
giving it a name.

> nums = [ 1,2,3,None,2,3]
>
> #version1
> def f(x): return x is not None
> newnums = filter(f, nums)
>
> #version2
> newnums = [ x for x in nums if x is not None]

This got me thinking, so I wrote a little test program to compare the speed of
the various alternatives.

With a large number of elements in the list (500000), Lambda/filter was the
fastest, function/filter was a close second and Comprehensions was a DISTANT
THIRD.

Here's my data on a Windows NT, Dual-P600 CPU machine:

    [C:/usr/jb]% python test.py
    lambda1: 1.29121285628
    lambda2: 1.31910440572
    lambda3: 1.31411255221
    func1: 1.43611083136
    func2: 1.44018521068
    compre: 4.75089779296
    loop: 4.33176244194

Geeze Louise, over 3.5X slower!!  Sheesh, even the loop was faster.

Results were similar, if less dramatic, on my dual P180MMX Linux server:

    [cascade jb]$ python test.py
    lambda1: 5.81
    lambda2: 5.78
    lambda3: 5.77
    func1: 5.91
    func2: 5.88
    compre: 6.47
    loop: 6.25

Also, the difference is less for smaller lists, though list comprehensions always
came out last.

The three lambda and two function forms included slight variations.

     Lambda1 the lambda expression was assigned to a global variable outside
     the loop.

     Lambda2 includes the lambda expression inline as an arg to filter.

     Lambda3 assigns the lambda expression to a local variable inside the
     loop.

     Func1 defines a local function to pass to filter.

     Func2 defines a global function to pass to filter.

     The comprehension and for loop were coded the only way they could be.

Try it yourself!  Code may be downloaded from:

    http://cascade-sys.com/~jb/Pythonetics/test.py

I think I heard somebody say that Lambda was faster but I was AMAZED at how much
more efficient it can be.  List comprehensions are SLOW by comparison.  On this
datum alone I would be bitterly opposed to getting rid of LAMBDA().

Finally, in the original application, if numeric values of 0 are impossible then
the tests in all cases can simply test the item in question, instead of "not is
None", which I expect would speed things up slightly.

Regards

--jb

--
James J. Besemer  503-280-0838 voice
http://cascade-sys.com  503-280-0375 fax
mailto:jb at cascade-sys.com