[Borgbackup] Recovery from missing segments

[David Gasaway]

On Wed, Jun 27, 2018 at 4:05 AM, Thomas Waldmann <tw at waldmann-edv.de> wrote:

> borg uses the client-side chunks index to determine whether a chunk "is
> present in the repo". the problem in the case pictured above is that
> your index does not correspond to the (corrupted) repo any more.

This much I understand.

> to detect such corruption, there is borg check.

Yes, but I'm trying to find a check mode that will repair this
condition without reading the segment files, as that would take a very
long time.

> also, it is a good idea to have multiple repositories on different
> targets (preferably at different places) so that in case one repo
> suffers from corruption (e.g. caused by bad hardware) or other issues,
> you still have another good repo.

I do have a second repo, but that doesn't help me repair the first.

> Without --verify-data, the archives check (2nd phase) will check the
> presence of the chunk (in the chunks index). The index will be a freshly
> built one from the repo check phase (first phase). It is built from the
> segment files, so if segment files are missing, chunks will be missing
> in the freshly built index, too.

I'm trying to avoid a repo check as it downloads all the segment
files.  I believe you are saying that '--archives-only' only checks
the index and would do nothing to detect or repair the condition I
described.  Can such a mode be added?  Hypothetical mode would simply
check the filesystem for the expected segment files, and if not found,
replace the associated chunks in the index with zero chunks so the
chunks/segments get recreated at the next backup (assuming source data
is still present).  Again, I don't care that a particular archive is
corrupt or has references to missing chucks, so long as I can get the
backups back on track.

> If you lose segment files you very likely have lost data. Due to the
> deduplication, every chunk of data in a repo is present only once and if
> you lose that, you have lost that piece of data(*). Same thing applies
> to metadata.

Right, but again, I'm talking about the loss of new segment files and
I don't care that a particular archive is corrupt.

> (*) you might be lucky if you still have the input data that produces
> this chunk. Files having lost that chunk might get healed by running
> borg check --repair, borg create, borg check --repair sequence.

My source data isn't going anywhere.  Or if it has, it's no great loss
so long as I have the most recent data.

Thanks.

-- 
-:-:- David K. Gasaway
-:-:- Email: dave at gasaway.org