[Borgbackup] status of cache resync improvements

Dan Christensen jdc at uwo.ca
Sat Dec 5 21:50:55 EST 2015


Dan Christensen <jdc at uwo.ca> writes:

> As an example with attic, one repository I have has
>
>                        Original size      Compressed size    Deduplicated size
> All archives:                1.61 TB            976.84 GB             30.92 GB
>
> This repo contains 188 archives.
>
> Rebuilding the cache, over gigabit ethernet, takes 40 minutes, and is
> CPU bound on the local machine.

I'm embarrassed to say that on the machine where I was having trouble,
msgpack-python was falling back to its pure python implementation.  
I installed a pre-compiled Ubuntu package, and now the 40 minutes is
reduced to 2 minutes.

For another slightly smaller repo, the time is down from 2 hours to 5.5
minutes.

This is still not ideal, but is also not a deal breaker.

Sorry for missing this basic point.  I wish msgpack gave a loud warning
during "pip install msgpack-python" if it doesn't find what it needs in
order to compile itself...

Dan


More information about the Borgbackup mailing list