Threading

[MRAB]

On 2020-01-24 21:33, Matt wrote:
> So I would create 10 threads.  And each would pop items off list like so?
> 
> 
> def start_test():
>      while big_list:
>          list_item = big_list.pop()
>          print list_item, port
>          sleep(1)
> 
> port = 80
> for i = 1 to 10
>      t = Thread(target=start_test)
>      t.start()
> 
> t.join()
> 
Not quite.

1. Create a list of threads.

2. Put the items into a _queue_, not a list.

3. Start the threads.

4. Iterate over the list of threads, using .join() on each.

If you're going to start the threads before you've put all of the items 
into the queue, you can also put a sentinel such as None into the queue 
after you've finished putting the items into it. When a thread sees the 
sentinel, it knows there are no more items to come. You could have one 
sentinel for each thread, or have only one and have each thread put it 
back when it sees it, for the other threads to see.

> 
> 
> Would that be thread safe?
> 
> On Fri, Jan 24, 2020 at 2:44 PM Chris Angelico <rosuav at gmail.com> wrote:
>>
>> On Sat, Jan 25, 2020 at 7:35 AM Matt <matt.mailinglists at gmail.com> wrote:
>> >
>> > I am using this example for threading in Python:
>> >
>> > from threading import Thread
>> >
>> > def start_test( address, port ):
>> >     print address, port
>> >     sleep(1)
>> >
>> > for line in big_list:
>> >     t = Thread(target=start_test, args=(line, 80))
>> >     t.start()
>> >
>> > But say big_list has thousands of items and I only want to have a
>> > maximum of 10 threads open.  How do work my way through the big_list
>> > with only 10 threads for example?
>>
>> First off, it is high time you move to Python 3, as the older versions
>> of Python have reached end-of-life.
>>
>> The best way is to create your ten threads, and have each one request
>> "jobs" (for whatever definition of job you have) from a queue. Once
>> the queue is exhausted, the threads terminate cleanly, and then you
>> can join() each thread to wait for the entire queue to be completed.
>>
>> ChrisA
>> --
>> https://mail.python.org/mailman/listinfo/python-list
>