creating size-limited tar files

[Dave Angel]

On 11/14/2012 11:16 AM, andrea crotti wrote:
> 2012/11/14 Dave Angel <d at davea.name>:
>> On 11/14/2012 10:56 AM, andrea crotti wrote:
>>> Ok this is all very nice, but:
>>>
>>> [andrea at andreacrotti tar_baller]$ time python2 test_pipe.py > /dev/null
>>>
>>> real  0m21.215s
>>> user  0m0.750s
>>> sys   0m1.703s
>>>
>>> [andrea at andreacrotti tar_baller]$ time ls -lR /home/andrea | cat > /dev/null
>>>
>>> real  0m0.986s
>>> user  0m0.413s
>>> sys   0m0.600s
>>>
>>> <snip>
>>>
>>>
>>> So apparently it's way slower than using this system, is this normal?
>>
>> I'm not sure how this timing relates to the thread, but what it mainly
>> shows is that starting up the Python interpreter takes quite a while,
>> compared to not starting it up.
>>
>>
>> --
>>
>> DaveA
>>
> 
> 
> Well it's related because my program has to be as fast as possible, so
> in theory I thought that using Python pipes would be better because I
> can get easily the PID of the first process.
> 
> But if it's so slow than it's not worth, and I don't think is the
> Python interpreter because it's more or less constantly many times
> slower even changing the size of the input..
> 
> 

Well, as I said, I don't see how the particular timing has anything to
do with the rest of the thread.  If you want to do an ls within a Python
program, go ahead.  But if all you need can be done with ls itself, then
it'll be slower to launch python just to run it.

Your first timing runs python, which runs two new shells, ls, and cat.
Your second timing runs ls and cat.

So the difference is starting up python, plus starting the shell two
extra times.

I'd also be curious if you flushed the system buffers before each
timing, as the second test could be running entirely in system memory.
And no, I don't know offhand how to flush them in Linux, just that
without it, your timings are not at all repeatable.  Note the two
identical runs here.

davea at think:~/temppython$ time ls -lR ~ | cat > /dev/null

real	0m0.164s
user	0m0.020s
sys	0m0.000s
davea at think:~/temppython$ time ls -lR ~ | cat > /dev/null

real	0m0.018s
user	0m0.000s
sys	0m0.010s

real time goes down by 90%, while user time drops to zero.
And on a 3rd and subsequent run, sys time goes to zero as well.

-- 

DaveA