Most space-efficient way to store log entries

[Cameron Simpson]

On 30Oct2015 00:35, Marc Aymerich <glicerinu at gmail.com> wrote:
>Usually I use my home router (which has an attached HDD) for
>downloading movies and stuff (big files) from the WAN... it has a
>800Mhz mips cpu... anyway my experience with it is that:
>
>rsync tops at ~400Kbps

Rsync is not a maximally efficient file transfer tool. What it excels at is 
incremental transfer, when you have some of a set of files, or a portion of a 
file you need to extend, such as updating a copy of a growing log file.

>apache+wget tops at ~1.1Mbps
>netcat tops at ~1.4Mbps

You might also try scp or sftp. On a LAN or if one endpoint is CPU constrained 
(probably the data source) you may find that compression is actually a lose. If 
your source data is already low redundancy (video, many audio formats, most 
image data, already compressed files like .gz or .zip) then compression is 
definitely a lose.

Finally, on a LAN you may be able to mount one end or the other as a remote 
filesystem. Then you can benchmark plain old "cp" over SMB or NFS etc.

Cheers,
Cameron Simpson <cs at zip.com.au>