New to Programming: TypeError: coercing to Unicode: need string or buffer, list found

[Saran A]

On Thursday, April 2, 2015 at 9:06:49 AM UTC-4, Chris Angelico wrote:
> On Thu, Apr 2, 2015 at 11:46 PM, Saran A <ahlusar.ahluwalia at gmail.com> wrote:
> > @ChrisA - this is a smaller function that will take the most updated file. My intention is the following:
> >
> > * Monitor a folder for files that are dropped throughout the day
> >
> > * When a file is dropped in the folder the program should scan the file
> >
> > o IF all the contents in the file have the same length (let's assume line length)
> >
> > o THEN the file should be moved to a "success" folder and a text file written indicating the total number of records/lines/words processed
> >
> > o IF the file is empty OR the contents are not all of the same length
> >
> > o THEN the file should be moved to a "failure" folder and a text file written indicating the cause for failure (for example: Empty file or line 100 was not the same length as the rest).
> >
> 
> Sounds like a perfect job for inotify, then. Your function will be
> called whenever there's a new file.
> 
> > Here is the code I have written:
> >
> > def initialize_logger(output_dir):
> >     logger = logging.getLogger()
> >     ...
> >     def file_len(filename
> >         with open(filename) as f:
> >             for i, l in enumerate(f):
> >                 pass
> >             return i + 1
> 
> These functions are all getting defined inside your
> initialize_logger() function. I suspect you want them to be flush left
> instead.
> 
> >     def copyFile(src, dest):
> >         try:
> >             shutil.copy(src, dest)
> >         # eg. src and dest are the same file
> >         except shutil.Error as e:
> >             print('Error: %s' % e)
> >         # eg. source or destination doesn't exist
> >         except IOError as e:
> >             print('Error: %s' % e.strerror)
> 
> Recommendation: Skip the try/except, and just let exceptions bubble
> up. Don't just print out messages and keep going.
> 
> > def move_to_failure_folder_and_return_error_file():
> >     os.mkdir('Failure')
> >     copyFile(filename, 'Failure')
> >     initialize_logger('rootdir/Failure')
> >     logging.error("Either this file is empty or the lines")
> 
> This doesn't move the file, it copies it. Is that your intention?
> 
> Moving a file is pretty easy. Just use os.rename().
> 
> > if __name__ == '__main__':
> >    import sys
> >    validate_files(sys.argv[1:])
> 
> I've no idea what validate_files() does, as you haven't included that.
> 
> I think you could code this fairly efficiently as a simple callback
> off pyinotify, or if you're not on Linux, with one of the equivalent
> services. What you're doing here (watching for files, looking inside
> them, and moving them when done) is pretty common around the world.
> 
> ChrisA

@ChrisA

validate_files will:

#double check for record time and record length - logic to be written to either pass to Failure or Success folder respectively. I welcome your thoughts on this.  

def validate_files(): 
    creation = time.ctime(os.path.getctime(added)) 
    lastmod = time.ctime(os.path.getmtime(added)) 


Does this address the issue. I particularly like writing my own exceptions as they provide me with more information on what could be the root cause. I know that in other circumstances, try and except are not the best practice. I appreciate the reminder though.  

Does this modification to copyFile do the job of moving the file? I haven't written a test yet. 

Thanks for catching the indentation for the helper functions. 
   
def copyFile(src, dest):
> >         try:
> >             shutil.rename(src, dest)
> >         # eg. src and dest are the same file
> >         except shutil.Error as e:
> >             print('Error: %s' % e)
> >         # eg. source or destination doesn't exist
> >         except IOError as e:
> >             print('Error: %s' % e.strerror)