[Borgbackup] Recovery from missing segments

[Thomas Waldmann]

> I'd like to ask if borg can recover gracefully from missing segments.
> That is, say a new segment is created and recorded, but the next time
> that borg runs, the segment file is no longer there.  Will borg
> recognize this on the next backup

borg uses the client-side chunks index to determine whether a chunk "is
present in the repo". the problem in the case pictured above is that
your index does not correspond to the (corrupted) repo any more.

to detect such corruption, there is borg check.

also, it is a good idea to have multiple repositories on different
targets (preferably at different places) so that in case one repo
suffers from corruption (e.g. caused by bad hardware) or other issues,
you still have another good repo.

> The docs for 'borg check' say about an archive check: "For all files
> (items) in the archive, for all chunks referenced by these files,
> check if chunk is present."  Does that mean checking for  existence of
> the segment where the chuck is expected to be stored, for the chunk
> object in the archive metadata, or both?

Without --verify-data, the archives check (2nd phase) will check the
presence of the chunk (in the chunks index). The index will be a freshly
built one from the repo check phase (first phase). It is built from the
segment files, so if segment files are missing, chunks will be missing
in the freshly built index, too.

With --verify-data, it will also read and cryptographically verify each
chunk (slow).

In general, be aware that borg check is complex code, might be imperfect
and can not do wonders anyway.

If you lose segment files you very likely have lost data. Due to the
deduplication, every chunk of data in a repo is present only once and if
you lose that, you have lost that piece of data(*). Same thing applies
to metadata.

(*) you might be lucky if you still have the input data that produces
this chunk. Files having lost that chunk might get healed by running
borg check --repair, borg create, borg check --repair sequence.

-- 

GPG ID: 9F88FB52FAF7B393
GPG FP: 6D5B EF9A DD20 7580 5747 B70F 9F88 FB52 FAF7 B393