[Numpy-discussion] Optimization suggestion sought

[Justin Peel]

On Sun, Dec 26, 2010 at 7:34 AM,  <josef.pktd at gmail.com> wrote:
> On Sun, Dec 26, 2010 at 3:51 AM, Enzo Michelangeli <enzomich at gmail.com> wrote:
>> For a pivoted algorithm, I have to perform an operation that in fully
>> vectorized form can be expressed as:
>>
>>    pivot = tableau[locat,:]/tableau[locat,cand]
>>    tableau -= tableau[:,cand:cand+1]*pivot
>>    tableau[locat,:] = pivot
>>
>> tableau is a rather large bidimensional array, and I'd like to avoid the
>> allocation of a temporary array of the same size holding the result of the
>> right-hand side expression in the second line of code (the outer product of
>> tableau[:,cand] and pivot). On the other hand, if I replace that line with:
>>
>>    for i in xrange(tableau.shape[0]):
>>        tableau[i] -= tableau[i,cand]*pivot
>>
>> ...I incur some CPU overhead for the "for" loop -- and this part of code is
>> the botteneck of the whole algorithm. Is there any smarter (i.e., more
>> time-efficient) way of achieving my goal?
>
> just a generic answer:
>
> Working in batches can be a good compromise in some cases. I instead
> of working in a loop with one row at a time, loop and handle, for
> example, 1000 rows at a time.
>
> Josef
>
>>
>> TIA --
>>
>> Enzo
>>
>> _______________________________________________
>> NumPy-Discussion mailing list
>> NumPy-Discussion at scipy.org
>> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>>
> _______________________________________________
> NumPy-Discussion mailing list
> NumPy-Discussion at scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
>

If this is really such a big bottleneck, then I would look into using
Cython for this part. With just a few cdef's, I bet that that you
could speed up the for loop tremendously. Depending on the details of
your algorithm, you might want to make a Cython function that takes
tableau, cand and pivot as inputs and just does the for loop part.