[Tutor] Executing a command from a specific directory

[Kurt Bendl]

Hello,

On Sep 18, 2009, at 5:16 AM, Ansuman Dash wrote:

> I have written it like that. It is like press 1 and it ll download  
> file1 and press 2 it ll download file2 etc....
>
> But my question was I am using "time.sleep()" to make my script to  
> wait for the file download and then validate it in log file, so is  
> there any other way I can synchronize my code with the download.
>
> I am asking this because some files are very huge (120MB) and  
> download also depends on N/W bandwidth so sometimes it ll be done in  
> 30mins and some times it takes 60 mins. So I can't rely on  
> "time.sleep()"


I had a similar problem.
I used pyinotify on a linux box. inotify is a kernel hook
that you can use to trigger actions on events... like when
a file write is completed.

Note: I'm a total hack at this. I'm sure there's a more
elegant way to do what I'm dong, but I needed a fix fast,
and this worked for me.  I'd appreciate any tips anyone has
to offer to make this cleaner and/or more pythonic. :-)

I'll be glad to try to answer any questions about this hackery.


Best,
   Kurt


Here's a slightly cut-down version of my code:
http://pastebin.com/f239b0413

inotify_published_file_handler.py
#####################################################
#!/usr/bin/env python
# -*- coding: utf-8 -*-
# encoding: utf-8
"""
inotify_published_file_handler.py
rev. 20090706-01
----

Created by Kurt Bendl.


Usage
---------
sudo su - www-data -c 'cd /web/log/; \
     nohup /web/scripts/inotify_published_file_handler.py > \
     /web/log/inotify-errors.txt 2>&1  &'


Purpose
---------
Monitor the $WEBROOT/aapracing/import/publish directory.
Once a file is closed, act on it:

1. Copy the PDF and XML files from source_dir to web_filebin_dir
2. If the file is an.XML, copy it to the xml_ftp_dir
3. Make the dummy file for PHP publishing process
4. Remove the source file

Requirements
------------
  * Linux kernel 2.6.13 +
  * pyinotify 2.8.6 +


Installation
--------------
To install pyinotify on ubuntu/debian:

`sudo easy_install pyinotify`


Docs
-----
Docs on pyinotify can be found here: http://trac.dbzteam.org/pyinotify/wiki

"""

import os
import re
import shutil
import pyinotify
import datetime


### production config info
source_dir = "/web/site/aapracing/publish/data/publish/"
web_filebin_dir = "/web/site/filebin/downloads/"
reference_path = '/web/site/aapracing/publish/data/published/'
xml_ftp_dir = "/home/ftp/private/xml/"
filez = '(PDF|pdf|XML|xml|txt)'
logfile_path = "/web/log/inotify.log"

event_mask = pyinotify.IN_CLOSE_WRITE
wm = pyinotify.WatchManager()

def getNow():
   """Just return the current time for timestamping logs"""
   return datetime.datetime.now().strftime("%Y-%m-%d %H:%M:%S")

def makeReferenceFile(tfile):
   open(tfile, 'w').close()


class SourceMonitor(pyinotify.ProcessEvent):
   """ Watches the source dir for IN_CLOSE_WRITE event"""

   def process_IN_CLOSE_WRITE(self, event):
     """when an IN_CLOSE_WRITE happens, do something"""
     if re.search('(.*\.'+filez+'$)', event.pathname):
       # We have a match, put a copy into the filebin dir
       shutil.copy2(event.pathname, web_filebin_dir)
       logfile = open(logfile_path, "a")
       logfile.write("%s: %s moved to %s. \n" %
                     (getNow(), event.pathname, web_filebin_dir))
       if re.search('(.*\.(XML|xml)$)', event.pathname):
         # If it's and XML, put a copy in the FTP dir
         shutil.copy2(event.pathname, xml_ftp_dir)
         logfile.write("%s: %s moved to %s. \n" %
                       (getNow(), event.pathname, xml_ftp_dir))
       # Make the dummy file marker to enable PHP
       # to know the file is really published
       fhandle = os.path.basename(event.pathname)
       open(reference_path + fhandle, 'w').close()
       # Now, whack the source file since we're done with it.
       os.remove(event.pathname)

p = SourceMonitor()
notifier = pyinotify.Notifier(wm, p)
wdd = wm.add_watch(source_dir, event_mask)
print "This should have been started with:\n\n"
print "  sudo su - www-data -c 'cd /web/log/; nohup /web/scripts/ 
inotify_published_file_handler.py >  /web/log/inotify-errors.txt 2>&1   
&' \n\n"

notifier.loop()

--
Kurt Bendl, Consultant
Internet Tool & Die
http://tool.net/
502-759-7104