[Tutor] Help for Python Beginner with extracting and manipulating data from thousands of ASCII files

[Cecilia Chavana-Bryant]

On Mon, Oct 1, 2012 at 12:33 AM, Alan Gauld <alan.gauld at btinternet.com>wrote:

> On 30/09/12 23:07, Cecilia Chavana-Bryant wrote:
>
>> Hola again Python Tutor!
>>
>> With a friend's help I have the following code to extract reflectance
>> data from an ASCII data file, do a bit of data manipulation to calibrate
>> the data and then write the calibrated file into an out file.
>>
>
>
> <snip>
>
>
>  I have successfully calibrated one ASCII file at a time with this code.
>> However, I have 1,000s of files that I need to calibrate so I would like
>> some help to modify this code so it can:
>>
>> 1. Use one calibration file (Cal_FileP17.txt) on data files created from
>> July to the 18th Sep and a different calibration file (Cal_FileP19.txt)
>> for data files created from the 19th of Sep onwards.
>>
>> 2. Find all the .txt files in a folder called ASCII_files, which is
>> subdivided into 12 different folders and calibrate all these files
>>
>
>
> Number 2 is easier to solve and the os.walk() and glob.glob()
> functions should provide all the tools you need.
>
> Number 1 is more tricky since there is no obvious way to determine the
> arbitrary start/stop dates you specify. So I'd suggest you need to
> generalise the requirement to take a start/stop date as well as the
> calibration file name and the input data file pattern. Use those as input
> parameters to a function that generates the list of files to process and
> then calls your existing code (wrapped in a new function) and possibly
> provide default values for all/some of the parameters.
>
> Another option is to add the start/end dates to the calibration file if
> you have control of that, but personally I'd stick with input parameters...
>
> Many thanks Alan for your reply. I have added start and end dates as part
of the header information for the calibration files in the date format:
01/07/2011. So, I now need to write some code to take this into
consideration, any suggestions?

>
> --
> Alan G
> Author of the Learn to Program web site
> http://www.alan-g.me.uk/
>
> ______________________________**_________________
> Tutor maillist  -  Tutor at python.org
> To unsubscribe or change subscription options:
> http://mail.python.org/**mailman/listinfo/tutor<http://mail.python.org/mailman/listinfo/tutor>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/tutor/attachments/20121001/3a3d304f/attachment.html>