[Tutor] More efficient than glob?

[Doug.Shawhan@gecits.ge.com]

Wow! Thanks. This is a big help.

-----Original Message-----
From: Yigal Duppen [mailto:yduppen@xs4all.nl]
Sent: Thursday, November 07, 2002 10:20 AM
To: Shawhan, Doug (CAP, ITS, US); tutor@python.org
Subject: Re: [Tutor] More efficient than glob?


-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

>> for example:
>>
>> from __future__ import generators
>> def infiniteList():
>> 	start = 1
>> 	while 1:
>> 		yield start
>> 		start += 1
> Woah. Pretty neat.
> What practical use for such a list is there?

None that I know of ;-)
But if we look back at your example:

def log_glob():
        for i in range(11):
                month = "%02d" % (i+1, )
                files = glob.glob('\\tmp\\%s-*' % (month, ))
                for f in files:
                        yield f

for f in log_glob():
	do_stuff(f)

the use becomes more obvious.

The for f in log_glob() call starts the generator. This means the generator 
function will run until the first 'yield'. Then do_stuff will be called.
Then 
the next generator pass will be started, etc...

Since 'files = glob.glob ' takes a lot of memory, this is very important.
Instead of loading all files at once in memory, the generator allows us to 
keep only parts of the list (in this case, each part identified by a month) 
in memory! Whenever one of the globs has been exhausted, it can be thrown 
away and the next batch can be loaded. And the rest of the code will never 
notice!

Without generators this would have been impossible (or at least Very Hard).

YDD
- -- 
http://www.xs4all.nl/~yduppen
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.0 (GNU/Linux)

iD8DBQE9ypKzLsKMuCf5EdwRAvewAKDXdakROR2gcZC2AoOx8Ro59QpPHACePWtQ
T3uCJcbU5ZsCUevBszGQpAM=
=vpEd
-----END PGP SIGNATURE-----