[Python.NET] Efficient copy of .NET Array to ctypes or numpy array

[Nikhil Garg]

Hi,

I have looked at the following thread to copy a c# array to numpy list
using python dot net.

https://mail.python.org/pipermail/pythondotnet/2014-May/001525.html

When I try the methods mentioned in the complete thread I manage to get the
array from simple iteration method or
np.fromiter, but I have not been able to make GCHandle or Marshal.Copy work
for my case.

To explain about my issue, I use a computer code written in c# which
produces output in a custom build format. I am able to access the data from
the file system using python dot net, but it seems that it take almost half
an hour to get the data from the file.

For example my file has an array of size 5018x73x400, if i use *fromiter, *i
provide the last bit of array to copy to python due to the way file stores
output.

for j in xrange(num_nodes-1):
for i in xrange(nsteps):
temp = file_in.ReadItemTimeStep(j+1,i).Data
temp_1 = np.fromiter(temp, float)
dest[j, i] = temp_1

Other way of doing same task would be to use a for loop but tht would be
really slow.

for k in xrange(num_nodes-1):
        for k in xrange(nsteps):
                for i in xrange(nvals):
                       temp[k,j,i] = file_in.ReadItemTimeStep(k+1,
j).Data[i]

I am trying to find a faster way to extract the data from the file. I was
wondering if there is a possibility to use something like GChandle or
Marshal.Copy. I have also attached same output from Marshal.Copy

Marshal.Copy(temp, 0,
IntPtr.__overloads__[int](temp2.__array_interface__['data'][0]), len(temp))


         0.00000000e+000,   0.00000000e+000,   0.00000000e+000,
         0.00000000e+000,   3.13551399e-242,   4.20993645e-161,
         8.71245319e-119,   8.38370432e-089,   2.60762781e-075,
         1.92616374e-072,   2.89006184e-072,   3.06007265e-082,
         3.81879776e-315,   0.00000000e+000,   0.00000000e+000,
         0.00000000e+000,   1.03959661e-297,   7.22038448e-196,
         3.05237954e-130,   2.58469346e-093,   2.29495911e-070,
         3.74354820e-062,   1.74288060e-061,   2.13172553e-067,
         2.74305386e-077,   0.00000000e+000,   0.00000000e+000,
         0.00000000e+000,   0.00000000e+000,   1.02484756e-240,
         1.70193770e-154,   1.30625320e-108,   1.16793762e-075,
         2.96889340e-061,   7.84687103e-058,   1.16879604e-057,
         5.56256478e-068,   4.06437857e-315,   0.00000000e+000,
         0.00000000e+000,   0.00000000e+000,   1.11705239e-296,
         1.46978156e-194,   1.98034372e-128,   3.79615170e-091,
         5.88936509e-068,   1.85305929e-059,   2.16186001e-058,

Whereas the actual values stored in the array are
         0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
         0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
         0.00000000e+00,   0.00000000e+00,   5.66641200e-37,
         1.24961600e-30,   5.34846800e-25,   1.72519600e-20,
         1.49402200e-17,   3.27685300e-15,   4.10687500e-13,
         1.87978200e-11,   2.01397900e-10,   8.83561100e-10,
         2.14397600e-09,   2.02600300e-09,   2.47599800e-09,
         2.15987100e-09,   4.82694100e-10,   1.18759300e-10,
         3.32024100e-11,   0.00000000e+00,   0.00000000e+00,
         0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
         0.00000000e+00,   0.00000000e+00,   0.00000000e+00,
         1.45288600e-37,   7.37245900e-31,   7.59282500e-25,
         5.85463400e-20,   1.20634700e-16,   3.06414300e-14,
         5.16041300e-12,   2.87562800e-10,   3.62841500e-09,
         1.63540600e-08,   4.06988300e-08,   3.97764200e-08,
         4.88768500e-08,   4.07385000e-08,   8.78868600e-09,

I would appreciate any help i can get regarding this matter.

Nikhil
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/pythondotnet/attachments/20141026/1b36f7a0/attachment.html>