compress_web.py

[Mike Hostetler]

I have a web page that doesn't have many large pieces, but it has many
large ones.  When I added a page, I would have to update the index, etc.
It was hard to remember which pages I updated, so I just tar'ed them all
up.  This got old, because I had to upload a sort-of large tarball
(~100k gziped) when I only created one html file less then 3k.  I
thought, "You know, it would be really cool if I wrote a program that
would check the date of the files, then just compress the new files." 
So I did.  The original was written in Perl, and it sucked.  Then I
discovered Python.  The Python version was not only cleaner, but it's
even faster.

So here it is, submitted for your approval.  I'm including it here in
case it's useful for someone, either to use the script yourself or to
learn a bit more about Python (I know I learned a lot of Python writing
this).  The license is GPL'ed.  I also signed it with gpg, for those
interested/worried about such things.  My gpg signature is available at
http://www.binary.net/thehaas/mikeh.gpg 

Email me if you have questions, comments, or sympathies.

- mikeh

-- 
Mike Hostetler          
thehaas at binary.spam_must_die.net 
http://www.binary.net/thehaas 
GnuPG key: http://www.binary.net/thehaas/mikeh.gpg
-------------- next part --------------
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

#!/usr/local/bin/python
######################################
# compress_web.py
# (C) 2000  by Mike Hostetler
######################################
#    This program is free software; you can redistribute it and/or
#    modify
#    it under the terms of the GNU General Public License as published
#    by
#    the Free Software Foundation; either version 2 of the License, or
#    (at your option) any later version.
#
#    This program is distributed in the hope that it will be useful,
#    but WITHOUT ANY WARRANTY; without even the implied warranty of
#    MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
#    GNU General Public License for more details.
#
#    You should have received a copy of the GNU General Public License
#    along with this program; if not, write to the Free Software
#    Foundation, Inc., 675 Mass Ave, Cambridge, MA 02139, USA.
#
######################################

import os,stat,time,string

#####################################
def compress_web(tar,days,webdir,bad_files):

	os.chdir(webdir)

	newfiles = []

	os.path.walk(webdir,find_time,newfiles)
	
	if (newfiles != []):
		for i in newfiles:
			tar = tar+ " " + i
		os.system(tar)
	else: 
		print "\n### no files found that needed archived\n"

#####################################
# this function is called by the os.path.walk for each file in webdir 
def find_time(newfiles, dirname, names):

	strlen = len(webdir)+1
	time_limit = time.time() - (86400 * days)

	for file in names:	
		filename= dirname+"/"+file

# Wow, this line sucks.  But it works.
# What it does:
#        1st condition - is it in list of files we don't want?
#        2nd condition - is it a regular file (i.e. not a directory)?
		if not(file in bad_files) and (stat.S_ISREG(os.stat(filename)[stat.ST_MODE])):

# This checks to see if the file meets the time requirement
			if os.stat(filename)[stat.ST_MTIME] > time_limit:

# It is important to strip "filename" by the strlen - this makes sure we
# grab the directory name of the file as well.  If we just used "file",
# then we wouldn't grab the directory.  That would be bad.
				newfiles.append(filename[strlen:])
	return newfiles
#####################################

if __name__ == '__main__':

	webdir = "/home/mikeh/public_html"
	bad_files = ("web.tar.gz","dan_o.dat")
	days = 2
	tar_command = "tar cvzf web.tar.gz"

	compress_web(tar_command,days,webdir,bad_files)

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.0.0 (GNU/Linux)
Comment: For info see http://www.gnupg.org

iD8DBQE5AKFuaP33v4T41CURAkLwAKCmZEptU0rtPt05dT1qKhM25qDJSQCgi9ms
naDnMjhIo3Op0e4JAzKgw8w=
=qj24
-----END PGP SIGNATURE-----