Addendum to Strategy/ Advice for How to Best Attack this Problem?

[Peter Otten]

Saran Ahluwalia wrote:

> On Sunday, March 29, 2015 at 7:33:04 AM UTC-4, Saran Ahluwalia wrote:
>> Below are the function's requirements. I am torn between using the OS
>> module or some other quick and dirty module. In addition, my ideal
>> assumption that this could be cross-platform. "Records" refers to
>> contents in a file. What are some suggestions from the Pythonistas?
>> 
>> * Monitors a folder for files that are dropped throughout the day
>> 
>> * When a file is dropped in the folder the program should scan the file
>> 
>> o IF all the records in the file have the same length
>> 
>> o THEN the file should be moved to a "success" folder and a text file
>> written indicating the total number of records processed
>> 
>> o IF the file is empty OR the records are not all of the same length
>> 
>> o THEN the file should be moved to a "failure" folder and a text file
>> written indicating the cause for failure (for example: Empty file or line
>> 100 was not the same length as the rest).
> 
> Below are some functions that I have been playing around with. I am not
> sure how to create a functional program from each of these constituent
> parts. I could use decorators or simply pass a function within another
> function.

Throwing arbitrary code at a task in the hope that something sticks is not a 
good approach. You already have given a clear description of the problem, so 
start with that and try to "pythonize" it. Example:

def main():
    while True:
        files_to_check = get_files_in_monitored_folder()
        for file in files_to_check:
            if is_good(file):
                move_to_success_folder(file)
            else:
                move_to_failure_folder(file)
        wait_a_minute()

if __name__ == "__main__":
    main()

Then write bogus implementations for the building blocks:

def get_files_in_monitored_folder():
    return ["/foo/bar/ham", "/foo/bar/spam"]

def is_good(file):
    return file.endswith("/ham")

def move_to_failure_folder(file):
    print("failure", file)

def move_to_success_folder(file):
    print("success", file)

def wait_a_minute():
    raise SystemExit("bye") # we don't want to enter the loop while 
developing

Now successively replace the dummy function with functions that do the right 
thing. Test them individually (preferrably using unit tests) so that when 
you have completed them all and your program does not work like it should 
you can be sure that there is a flaw in the main function.

> [code]
> import time
> import fnmatch
> import os
> import shutil
> 
> 
> #If you want to write to a file, and if it doesn't exist, do this:

Hm, are these your personal notes or is it an actual script?
 
> if not os.path.exists(filepath):
>     f = open(filepath, 'w')
> #If you want to read a file, and if it exists, do the following:
> 
> try:
>     f = open(filepath)
> except IOError:
>     print 'I will be moving this to the '
> 
> 
> #Changing a directory to "/home/newdir"
> os.chdir("/home/newdir")

Never using os.chdir() is a good habit to get into.

> def move(src, dest):
>     shutil.move(src, dest)
> 
> def fileinfo(file):
>     filename = os.path.basename(file)
>     rootdir = os.path.dirname(file)
>     lastmod = time.ctime(os.path.getmtime(file))
>     creation = time.ctime(os.path.getctime(file))
>     filesize = os.path.getsize(file)
> 
>     print "%s**\t%s\t%s\t%s\t%s" % (rootdir, filename, lastmod, creation,
>     filesize)
> 
> searchdir = r'D:\Your\Directory\Root'
> matches = []
> 
> def search

Everytime you post code that doesn't even compile you lose some goodwill.
In a few lines of code meant to demonstrate a problem a typo may be 
acceptable, but for something that you probably composed in an editor you 
should take the time to run it and fix at least the syntax errors.

> for root, dirnames, filenames in os.walk(searchdir):
>     ##  for filename in fnmatch.filter(filenames, '*.c'):
>     for filename in filenames:
>         ##      matches.append(os.path.join(root, filename))
>         ##print matches
>         fileinfo(os.path.join(root, filename))
> 
> 
> def get_files(src_dir):
> # traverse root directory, and list directories as dirs and files as files
>     for root, dirs, files in os.walk(src_dir):
>         path = root.split('/')
>         for file in files:
>             process(os.path.join(root, file))
>                     os.remove(os.path.join(root, file))
> 
> def del_dirs(src_dir):
>     for dirpath, _, _ in os.walk(src_dir, topdown=False):  # Listing the
>     files
>         if dirpath == src_dir:
>             break
>         try:
>             os.rmdir(dirpath)
>         except OSError as ex:
>             print(ex)
> 
> 
> def main():
>     get_files(src_dir)
>     del_dirs(src_dir)
> 
> 
> if __name__ == "__main__":
>     main()
> 
> 
> [/code]