creating size-limited tar files

[Dave Angel]

On 11/14/2012 03:43 PM, Andrea Crotti wrote:
> <SNIP>
> Anyway the only thing I wanted to understand is if using the pipes in
> subprocess is exactly the same as doing
> the Linux pipe, or not.

It's not the same thing, but you can usually assume it's close.  Other
effects will probably dominate any differences.
> 
> And any idea on how to run it in ram?
> Maybe if I create a pipe in tmpfs it might already work, what do you think?
> 
> 

In a good virtual OS, such as Linux, there's very little predictable
difference between running in RAM (which is to say reading and writing
to the swap file) or reading and writing to a file you specify.  In
fact, writing to a file can frequently be quicker, if it's sequential.

Why?  Linux is using any given piece of physical RAM to map a file, or
an allocated buffer, or shared memory, or nearly anything.  About the
only special cases are the kind of RAM that has to be locked into RAM
for hardware reasons.

Linux decides which pieces to keep in memory, whether it calls it
caching, swapping, memory mapping, or whatever.  And frequently,
attempts to "beat the system"  result in counterintuitive results.

If in doubt, measure.  But choose your measures carefully, because lots
more things will change the measurement than you might expect.


-- 

DaveA