Addendum to Strategy/ Advice for How to Best Attack this Problem?

[Dave Angel]

On 03/29/2015 07:37 AM, Saran Ahluwalia wrote:
> On Sunday, March 29, 2015 at 7:33:04 AM UTC-4, Saran Ahluwalia wrote:
>> Below are the function's requirements. I am torn between using the OS module or some other quick and dirty module. In addition, my ideal assumption that this could be cross-platform. "Records" refers to contents in a file. What are some suggestions from the Pythonistas?
>>
>> * Monitors a folder for files that are dropped throughout the day
>>
>> * When a file is dropped in the folder the program should scan the file
>>
>> o IF all the records in the file have the same length
>>
>> o THEN the file should be moved to a "success" folder and a text file written indicating the total number of records processed
>>
>> o IF the file is empty OR the records are not all of the same length
>>
>> o THEN the file should be moved to a "failure" folder and a text file written indicating the cause for failure (for example: Empty file or line 100 was not the same length as the rest).
>
> Below are some functions that I have been playing around with. I am not sure how to create a functional program from each of these constituent parts. I could use decorators or simply pass a function within another function.

Your problem isn't complicated enough to either need function objects or 
decorators.  You might want to write a generator function, but even that 
seems overkill for the problem as stated.  Just write the code, 
top-down, with dummy bodies containing stub code.  Then fill it in from 
bottom up, with unit tests for each completed function.

More complex problems can justify a different approach, but you don't 
need to use every trick in the arsenal.

>
> [code]
> import time
> import fnmatch
> import os
> import shutil
>

If you have code fragments that aren't going to be used, don't write 
them as top-level code.  Either move them to another file, or at least 
enclose them in a function with a name like   dummy_do_not_use()

My own convention for that is to suffix the function name with a bunch 
of uppercase ZZZ's  That way the name jumps out at me so I'll recognize 
it, and I can be sure I'll never actually call it.

>
> #If you want to write to a file, and if it doesn't exist, do this:
>
> if not os.path.exists(filepath):
>      f = open(filepath, 'w')
>
> #If you want to read a file, and if it exists, do the following:
>
> try:
>      f = open(filepath)
> except IOError:
>      print 'I will be moving this to the '
>
>
> #Changing a directory to "/home/newdir"
> os.chdir("/home/newdir")

As Peter said, chdir can be very troublesome.  Avoid at almost all 
costs.  As you've done elsewhere, use os.path.join() to combine 
directory paths with relative filenames.

>
> def move(src, dest):
>      shutil.move(src, dest)
>
> def fileinfo(file):
>      filename = os.path.basename(file)
>      rootdir = os.path.dirname(file)
>      lastmod = time.ctime(os.path.getmtime(file))
>      creation = time.ctime(os.path.getctime(file))
>      filesize = os.path.getsize(file)
>
>      print "%s**\t%s\t%s\t%s\t%s" % (rootdir, filename, lastmod, creation, filesize)
>
> searchdir = r'D:\Your\Directory\Root'
> matches = []
>
> def search
> for root, dirnames, filenames in os.walk(searchdir):

Why are you using a directory tree when your "spec" said the files would 
be in a specific directory?

>      ##  for filename in fnmatch.filter(filenames, '*.c'):
>      for filename in filenames:
>          ##      matches.append(os.path.join(root, filename))
>          ##print matches
>          fileinfo(os.path.join(root, filename))
>
>
> def get_files(src_dir):
> # traverse root directory, and list directories as dirs and files as files
>      for root, dirs, files in os.walk(src_dir):
>          path = root.split('/')
>          for file in files:
>              process(os.path.join(root, file))
>                      os.remove(os.path.join(root, file))

Probably you shouldn't have os.remove in the code till the stuff around 
it has been carefully tested.  Besides, nothing in the spec says you're 
going to remove any files.

>
> def del_dirs(src_dir):
>      for dirpath, _, _ in os.walk(src_dir, topdown=False):  # Listing the files
>          if dirpath == src_dir:
>              break
>          try:
>              os.rmdir(dirpath)
>          except OSError as ex:
>              print(ex)
>
>
> def main():
>      get_files(src_dir)
>      del_dirs(src_dir)
>

Your description says "monitor".  That implies to me an ongoing process, 
or a loop.   You probably want something like:

def main():
     while True:
         process_files(directory_name)
         sleep(10000)

>
> if __name__ == "__main__":
>      main()
>
>
> [/code]
>


-- 
DaveA