Queues - Is Infinity all right?

[Peter Hansen]

ykingma at accessforall.nl wrote:
> 
> Peter Hansen wrote:
> 
> ...
> >
> > That said, I'd consider using a fixed queue only when I was greatly
> > concerned about whether my consumer thread was going to be able to
> > keep up with my producer thread(s), and was therefore concerned about
> > overflowing memory with unconsumed input.
> >
> > In other words, I'd worry more about the robustness of the app
> > than about optimizing it before it worked...
> >
> 
> I've been bitten a few times by unbounded queues when the consumer
> thread could not keep up: the queue grows to memory limits and
> no robustness is left.
> 
> So I changed my default choice for queue implementations to bounded
> queues. They are much more predictable under unexpected loads
> as they evt. force the producer to wait for the consumer.
> 
> The next question is off course what maximum queue size to use.
> You can get good answers from queuing theory, but 10 to 20
> works well in many cases.

Reading your reply, and my original response, I am second-thinking
my above-quoted philosophy.  Certainly, as even I said above, robustness
should be the concern, and not performance.  Nevertheless, clearly
this is an issue of robustness primarily, so it would seem that a 
predefined limit, well above what is expected to be required, would be a
good idea to help ensure robustness.

-Peter