parse/slice/...
Albert Hopkins
marduk at letterboxes.org
Tue Jan 6 15:05:46 EST 2009
On Tue, 2009-01-06 at 11:23 -0800, rcmn wrote:
> I'm not sure how to call it sorry for the subject description.
> Here what i'm trying to accomplish.
> the script i'm working on, take a submitted list (for line in file)
> and generate thread for it. unfortunately winxp has a limit of 500
> thread . So I have to parse/slice the file by chunk of 500 and loop
> until the list is done.
> I though i would of done it in no time but i can't get started for
> some reason.
> And i can't find a good way to do it efficiently . Does anyone have
> something similar to this.
If your thread is smart enough to handle the file you can do something
like:
class Worker(threading.Thread):
def __init__(self, file_object):
self.file_object = file_object
threading.Thread.__init__(self)
def run(self):
while True:
line = self.file_object.readline()
if line == '':
return
# process line
NUM_WORKERS = 500
bigfile = open('somefile', 'r')
workers = [Worker(bigfile) for i in range(NUM_WORKERS)]
for worker in workers:
worker.start()
More information about the Python-list
mailing list