[SciPy-dev] [SciPy-user] efficiently importing ascii data

[Travis Oliphant]

Darren Dale wrote:

>Hi Travis, All,
>
>  
>
>>>The other thing that needs to be done is that string (and unicode)
>>>arrays need to convert to the numeric types, easily.  This would let you
>>>read in a string array and convert to numeric types quite cleanly.
>>>Right now, this doesn't work simply because the wrong generic functions
>>>are getting called in the conversion routines.  This can and should be
>>>changed, however.  The required code to change is in arraytypes.inc.src.
>>>
>>>I could see using PyInt_FromString, PyLong_FromString,
>>>PyFloat_FromString, and calling the complex python function for
>>>constructing complex numbers from a string.   Code would need to be
>>>written to fully support Long double and complex long double conversion,
>>>but initially they could just punt and use the double conversions.
>>>
>>>Alternatively, sscanf could be called when available for the type, and
>>>the other approaches used when it isn't.
>>>
>>>Anybody want a nice simple project to get themselves up to speed with
>>>the new code base :-)
>>>      
>>>
>>I'll look into it (after hours, just started a new job). I haven't worked
>>much with C or wrapping C, and I need to learn sometime.
>>    
>>
>
>Now that I have studied the source, I'm not sure I am up to this. Between 
>trying to learn C and wade through the (what appears to me) extremely 
>abstracted source code, I might as well deciphering Linear A.
>  
>
If you don't already know C, then the extra code-generation comments 
could be more confusing.

The code generation itself is very simple.  It's just a Python program 
that takes a .src file and constructs actual c-files by repeating code 
between tags substituting for names defined in the /**begin repeat**/  
header.


For example:


/**begin repeat
#name=mine,yours,ours#
#num=1,2,3#
**/

this gets repeated as @name@ and @num@

/**end repeat**/

would produce a file with the contents:


this gets repeated as mine and 1

this gets repeated as yours and 2

this gets repeated as ours and 3



This is a simple way to produce lots of similar code very quickly and 
maintain it more easily.


-Travis