[Tutor] Nested for loops, possibly?

[Bob Williams]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On 05/02/15 14:59, DaveA wrote:
> 
> 
> On February 5, 2015 8:27:29 AM EST, Bob Williams
> <linux at barrowhillfarm.org.uk> wrote:
>> Hi,
>> 
>> My script is running under Python 3.4.1 on a 64bit openSUSE
>> linux system. It is a backup script making calls to rsync and
>> btrfs-tools, and backing up several different paths. Here is the
>> script, my question follows below:
>> 
>> **Code** import datetime import glob import os, os.path import
>> subprocess import sys
>> 
>> if not os.getuid() == 0: print("\n*** This script must be run as
>> root. ***\n") sys.exit()
>> 
>> mnt_path = "/home/bob/A3"
>> 
>> subprocess.call(["mount", "LABEL=backup", mnt_path]) if not
>> os.path.ismount(mnt_path): print("\nBackup drive is not
>> mounted\nCheck if it is attached.\n") sys.exit() else: 
>> print("\nBackup drive mounted at", mnt_path, "\n")
>> 
>> src_path = "/home/bob"
>> 
>> today = datetime.datetime.now() fname =
>> today.strftime("%y-%m-%d_%H-%M")
>> 
>> doc_retain = datetime.timedelta(days=90) pic_retain =
>> datetime.timedelta(days=90) misc_retain =
>> datetime.timedelta(days=90) etc_retain =
>> datetime.timedelta(days=30) mus_retain =
>> datetime.timedelta(days=30) web_retain =
>> datetime.timedelta(days=30) rep_retain =
>> datetime.timedelta(days=30)
>> 
>> doc_srcpath = os.path.join(src_path, "Documents") pic_srcpath =
>> os.path.join(src_path, "Pictures") misc_srcpath = src_path 
>> etc_srcpath = "/etc" music_srcpath = os.path.join(src_path,
>> "music") www_srcpath = "/srv/www" repo_srcpath =
>> os.path.join(src_path, "download")
>> 
> Three things bother me about this portion of code, Too much code at
> top-level, too many separate variables, too many global.The last
> point doesn't matter much, since they're constant.
> 
> I'd put these Values into a class, and make a list of instances. If
> you're not yet comfortable with that, it would also be possible to
> make a list per "paragraph", and use zip to combine them, as
> already suggested.
> 
I also felt that there was something wrong with that long list of
variables. However, I'm new to classes, so I'll need to do some
background reading. Thank you for the pointer.

> The class would hold retain, srcpath, syncpath, snappath, etc. And
> your list would have 7 instances of that class currently,
> corresponding to your doc, pic, misc, ...
> 
> That list would be THE global, replacing these 35 or so. It would
> be populated something like:
> 
> def initialize (worklist=[]): worklist . append  (Job (90,
> src_path, "Documents", "documents", "docsnaps") worklist .append
> (Job (90, src_path,  "Pictures", ... .... return worklist
> 
> Now all the joins and globs are done in the Job initializer, just
> once.
> 
> .....th.join(mnt_path, "documents")
>> pic_syncpath = os.path.join(mnt_path, "pictures") misc_syncpath =
>> os.path.join(mnt_path, "miscellaneous") etc_syncpath =
>> os.path.join(mnt_path, "etc") music_syncpath =
>> os.path.join(mnt_path, "music") www_syncpath =
>> os.path.join(mnt_path, "www") repo_syncpath =
>> os.path.join(mnt_path, "repo")
>> 
>> doc_snappath = os.path.join(mnt_path, "docsnaps", fname) 
>> pic_snappath = os.path.join(mnt_path, "picsnaps", fname) 
>> misc_snappath = os.path.join(mnt_path, "miscsnaps", fname) 
>> etc_snappath = os.path.join(mnt_path, "etcsnaps", fname) 
>> music_snappath = os.path.join(mnt_path, "musicsnaps", fname) 
>> www_snappath = os.path.join(mnt_path, "wwwsnaps", fname) 
>> repo_snappath = os.path.join(mnt_path, "reposnaps", fname)
>> 
>> doc_snaplist = glob.glob(mnt_path + "/docsnaps/*") pic_snaplist =
>> glob.glob(mnt_path + "/picsnaps/*") misc_snaplist =
>> glob.glob(mnt_path + "/miscsnaps/*") etc_snaplist =
>> glob.glob(mnt_path + "/etcsnaps/*") music_snaplist =
>> glob.glob(mnt_path + "/musicsnaps/*") www_snaplist =
>> glob.glob(mnt_path + "/wwwsnaps/*") repo_snaplist =
>> glob.glob(mnt_path + "/reposnaps/*")
>> 
>> def do_sync(source, dest): subprocess.call(['rsync', '-av',
>> '--safe-links', '--delete-excluded', '-F', source, dest]) 
>> print("\n")
>> 
>> def create_snaps(newsnap, snapdest): subprocess.call(['btrfs',
>> 'subvolume', 'snapshot', newsnap, snapdest]) print("\n")
>> 
>> def expire_snaps(snaplist, today, expiry_interval): x = 0 
>> print("\nDeleting snapshots older than",
>> str(expiry_interval)[:7], "...") for i in range(0,
>> len(snaplist)): snap_date =
>> datetime.datetime.strptime(snaplist[i][-14:], "%y-%m-%d_%H-%M") 
>> if today - snap_date >= expiry_interval: 
>> subprocess.call(['btrfs', 'subvolume', 'delete', snaplist[i]]) x
>> += 1 if x == 0: print("... No snapshots older than",
>> str(expiry_interval)[:7], "found - nothing to do.") else: 
>> print("...", x, "snapshot(s) deleted.") print("\n")
>> 
>> def main(): print("Backing up ", src_path, "/Documents\n",
>> sep='') do_sync(doc_srcpath, doc_syncpath) 
>> create_snaps(doc_syncpath, doc_snappath) print("Documents backup
>> completed.") expire_snaps(doc_snaplist, today, doc_retain)
> 
> At this point main becomes something like
> 
> jobs = initialize () for job in jobs: do_sync (job.srcpath,
> job.syncpath) create_snaps (job.syncpath,  job.snappath) 
> expire_snaps (job.snaplist, ...
>> 
>> print("Backing up ", src_path, "/Pictures\n", sep='') 
>> do_sync(pic_srcpath, pic_syncpath) create_snaps(pic_syncpath,
>> pic_snappath) print("Pictures backup completed.") 
>> expire_snaps(pic_snaplist, today, pic_retain)
>> 
>> print("Backing up Miscellaneous files\n") do_sync(misc_srcpath,
>> misc_syncpath) create_snaps(misc_syncpath, misc_snappath) 
>> print("Miscellaneous backup completed.") 
>> expire_snaps(misc_snaplist, today, misc_retain)
>> 
>> print("Backing up /etc\n") do_sync(etc_srcpath, etc_syncpath) 
>> create_snaps(etc_syncpath, etc_snappath) print("Backup of /etc
>> completed.") expire_snaps(etc_snaplist, today, etc_retain)
>> 
>> print("Backing up ", src_path, "/music\n", sep='') 
>> do_sync(music_srcpath, music_syncpath) 
>> create_snaps(music_syncpath, music_snappath) print("Music backup
>> completed.") expire_snaps(music_snaplist, today, mus_retain)
>> 
>> print("Backing up Web server\n") do_sync(www_srcpath,
>> www_syncpath) create_snaps(www_syncpath, www_snappath) print("Web
>> server backup completed.") expire_snaps(www_snaplist, today,
>> web_retain)
>> 
>> print("Backing up Download repository\n") do_sync(repo_srcpath,
>> repo_syncpath) create_snaps(repo_syncpath, repo_snappath) 
>> print("Download repository backup completed.") 
>> expire_snaps(repo_snaplist, today, rep_retain)
>> 
>> print("\nAll backups completed.") print("\nUnmounting backup
>> drive.\n") subprocess.call(['umount', mnt_path]) print("\n>>>
>> Please power down the Quad external drive enclosure <<<\n")
>> 
>> if __name__ == "__main__": main() **/Code**
>> 
>> I would like to reduce all those repeated calls to do_sync() in
>> main(), for example, to one by putting the *_srcpath and
>> *_*syncpath variables into lists (eg. source_list and sync_list)
>> and using a for loop to get the first item out of each list, then
>> the second item, etc. Something like:
>> 
>> for i in range(0, len(source_list)): for j in range(0,
>> len(sync_list)): do_sync(source_list[i], sync_list[j])
>> 
>> but this will get all the values of sync_list[j] for each value
>> of source_list[i], which is not what I want.
> 
> If you're sure all the lists are the same length,  you can use zip.
> But I'd recommend a list of objects.
> 
>> 
>> I hope this is clear enough to see my problem? I realise that the
>> print statements will need some work, I'm just trying to get the 
>> functionality working.
>> 
>> TIA
>> 
>> Bob
> 


- -- 
Bob Williams
System:  Linux 3.16.7-7-desktop
Distro:  openSUSE 13.2 (x86_64) with KDE Development Platform: 4.14.3
Uptime:  06:00am up 7:55, 3 users, load average: 0.16, 0.05, 0.06
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2

iEYEARECAAYFAlTTqYIACgkQ0Sr7eZJrmU7s0gCeNfl1WCW2kvahVlfmBfoZdEZ4
FU8AnitA7R8ewK8Uc5/24HB6aGlq8GHQ
=hQva
-----END PGP SIGNATURE-----