[Borgbackup] Possibility to push-backup to remote through an intermediary proxy-cache?

Wolfgang 'datenwolf' Draxinger list+borgbackup at datenwolf.net
Wed Oct 4 05:48:15 EDT 2023


Hi,

I'm wondering if there's a way to setup Borg in a way, that it
performs push-backups through intermediary proxy-caches?

__Motivation__

My current setup looks like this: I have several machines in the
local network (laptop, desktop, etc.) which backup to Borg
repositories hosted on SSD in my self-built edge router, which is
then replicated to a Hetzner storage box. It's set up this way, due
to the abysmally bad physical conditions of the telephone lines
here, permitting some 15 MBit/s upload at most, so compared to the 1
GBit/s in the LAN that's a ~100× difference. Especially for mobile
devices like laptops: 1 minute of Borg push equates to ~100 minutes
in replication.

This is working fine so far. However with additional devices in the
network, and increasing storage capacity for each of them, I hit the
capacity limit of the SSD in the router. I could – of course – go
for a NAS. Trouble is: I've yet to find a NAS that meets my
expectations; specifically I want this thing to draw significantly
less than 10W while idle, so that it can passively cool while not
doing "stuff", for audible noise reasons.


__Idea__

Instead of keeping full repositories on the router SSD, use it as a
cache for the push-backup deltas. Essentially I imagine it to happen
like this: Machine A is pushing to remote repository R, where both
ends are also talking to an intermediary proxy-cache C. A is
coordinating with R to determine the actual delta, but pushing the
data to C, which will then "drip-feed" it to R. The connection
schemes I'm thinking of could look either like this

+---+ <---------------> +---+
|   |                   |   |
| A | fast         slow | R |
|   | <---> +---+       |   |
+---+ ===>> | C | ===>> +---+
            +---+

or this

+---+                   +---+
|   | <---> +---+ <---> |   |
| A | fast  | C |  slow | R |
|   | ===>> +---+ ===>> |   |
+---+                   +---+

<--->: control connection
===>>: push data transfer


__Questions__

Is this something that can be implemented using Borgbackup? As far
as I understand it (i.e. what I gather from the documentation, and
some experiments), it's not (yet).

Is this something that is worthwhile to look into implementing? If
so, where to start and how to do it?


Cheers,
dw


More information about the Borgbackup mailing list