String concatenation - which is the fastest way ?

[Stefan Behnel]

przemolicc at poczta.fm, 11.08.2011 16:39:
> On Thu, Aug 11, 2011 at 02:48:43PM +0100, Chris Angelico wrote:
>> On Thu, Aug 11, 2011 at 2:46 PM,<przemolicc at poczta.fm>  wrote:
>>> This is the way I am going to use.
>>> But what is the best data type to hold so many rows and then operate on them ?
>>>
>>
>> List of strings. Take it straight from your Oracle interface and work
>> with it directly.
>
> Can I use this list in the following way ?
> subprocess_1 - run on list between 1 and 10000
> subprocess_2 - run on list between 10001 and 20000
> subprocess_3 - run on list between 20001 and 30000
> etc
> ...

Sure. Just read the data as it comes in from the database and fill up a 
chunk, then hand that on to a process. You can also distribute it in 
smaller packets, just check what size gives the best throughput.

Still, I'd give each work parcel a number and then reorder the results 
while collecting them, that allows you to vary the chunk size and the 
process size independently, without having to wait for a process that 
happens to take longer.

Stefan