Proving backups actually restore

By Lucas Flores, IT Systeme Flores UG. Last updated 2026-10-29

A backup that has never been read back through the restore path is a theory, not evidence. Restow treats "backed up" and "proven restorable" as two different, separately-shown states, and a backup without a verified read-back is shown as not done, not as a quieter shade of success.

How the weekly restore check works

Back up
Mail, calendar, contacts and OneDrive, incremental and encrypted, on your storage.
Pick a sample
Every week, per mailbox and OneDrive: about 20 mails, 20 files, calendar and contacts.
Read it back
From the latest backup, through the same path a real restore uses.
Compare
Byte for byte against the recorded checksums. Only a match counts.

Shown per mailbox as recovery readiness

ReadyLatest check passed
AttentionWarnings in the check
Not restorableCheck failed
UnverifiedNot read back yet

What a restore check reads, how often it runs, and what green, red and "Unverified" mean, in plain language for customers and IT service providers: how Restow proves your backups can be restored.

What's checked today

A weekly sampled recovery-readiness check runs per mailbox, OneDrive and SharePoint site: Restow reads a sample back through the restore path and compares it with the recorded hashes, then shows the result plainly as a sample-based check, not as complete coverage it hasn't earned. The check does not restore into a Microsoft 365 mailbox, OneDrive or IMAP account; the optional test restore into a separate target is not wired up yet (see below). Where a check fails, that failure shows up in the mailbox or OneDrive's own status: it does not get averaged away into an overall green dashboard.

An object removed from its source is rated differently. Since 0.4.7, a mailbox deleted in Microsoft 365, for example, no longer turns red because its last backup grows old: nothing backs it up any more, and retention always keeps its newest restore point. It is rated by whether that backup restores, without age reasons such as "The latest backup is N days old", and it is never overdue; a kept backup that does not restore stays red. The table says "Removed from source, backups kept", and the report page opens with "Removed from the source" and lists the age as "Not counted: ...". The same rating applies in the statistics, the objects list, the tenant's overall readiness, the integration API and the verify.completed notification. An object whose newest backup was never checked gets one more check from the weekly planner.

Microsoft 365 objects that Restow no longer backs up (the account is disabled, it has no licence, Microsoft locked it, or it was removed from the source) first read amber "Not backed up any more, please confirm", with the reason. That is no failure, no red and not overdue, but the tenant's readiness stays at least yellow, never green, and the integration API counts them in readiness.toConfirm, until an administrator confirms "Yes, intended"; the object then reads grey "Intentionally no longer backed up". Disabled or unlicensed mailboxes and locked OneDrives and sites read "Not backed up: account disabled", "Not backed up: no licence" or "Not backed up: locked" and are neutral: they add no failed count.

Servers and clients are covered too

For a server or client backed up with the agent (see server and client backup), the check runs after every new good backup. The agent records the SHA-256 of up to 20 random files, the Restow server reads them back from the repository and compares the hashes, and the result is green only when they match. On top of that the repository is checked once a week. Both results feed Recovery Readiness, so a machine counts as recoverable only after files were actually read back with matching hashes. Proxmox VE guests (a preview) count in readiness as well. A failed check says what happened, why and what to do: see the failure explanations in the documentation.

File shares are covered too

For a file share (see file shares), every new restore point is checked: Restow reads up to 20 sampled files back and compares them with what was backed up, and the repository gets a weekly restic check of 5 percent of its data. Recovery readiness rates a share by its newest restore point. A restore point without samples, because every file changed in the hour before the run, is rated yellow and says why. Copy jobs use only a restore point that passed its check.

Proxmox VE: a check after every backup, and boot tests

Since 0.4.2, every new restore point of a Proxmox VE VM or container (a preview) is read back within minutes of its backup: the server compares a sample of its blocks with the recorded SHA-256 (for a container, a restic check of a subset), besides the weekly check of every guest. It is on by default (Installation › Proxmox VE) and can be switched off for the installation. "Check now" and "Check all unverified" in Recovery readiness start checks by hand, and a red check shows the error it found. It reads data back from storage; it does not start the guest.

A check that could not run is not a failed check. Since 0.4.3, when another run holds the lock on a container's restic repository, the check is postponed and tried again ten minutes later, with the previous result standing meanwhile. After two hours the restore point reads "Could not check: repository locked": yellow (attention) in Recovery readiness, never red and never green, with restic's report of who holds the lock. Stale locks are removed before the check (details). A locked repository therefore no longer produces a red "Restore check failed" that the data never earned, and it never produces a green one either: a restore point that no check could read stays unproven. Checks that failed red because of a lock before 0.4.3 stay red until the next check ("Check now").

Proxmox VE (a preview) has more than the sample check. A job can boot the newest restore point of each guest, weekly or after every backup, and "Boot test now" on a restore point asks for one by hand. It is off by default and has a memory limit and a wait for the guest agent. The guest's node restores the restore point as a new guest in the pool restow-restore, isolates it before the start (every network interface disconnected or on an isolated bridge, passthrough devices, serial ports and bind mounts removed), starts it, waits for the QEMU guest agent (a container: its init) and destroys it in every case. One test runs per node at a time, never beside a backup. Green means the guest agent answered, yellow that the VM only started and nothing answered, red that the restore, the start or the boot failed. Recovery readiness rates the newest restore point by its boot test where there is one. It does not check ports or applications inside the guest. It needs restow-pve 0.4.0 on the node and was tested against a simulated node, not against a real Proxmox VE cluster.

The storage check: damaged is not the same as unreadable

Next to the restore checks, a storage check reads packs back and compares them with their recorded size and checksum (a sample every week, every pack every month). Since 0.4.2 it tells a pack it could not read for the moment (a download cut off mid-transfer, a reset connection, a timeout, S3 asking to slow down, denied access) from a damaged one. It reads such a pack again, up to three times; one that stays unreadable is listed in Recovery readiness › Storage integrity as "temporarily unreadable", with the error, how many consecutive checks could not read it and when. Objects using it are rated "Attention", not "Not restorable": that needs a pack proven damaged (missing, wrong size or checksum), or one that could not be read in 3 consecutive checks over at least 24 hours. When the packs read intact again, the finding ends and the objects' own restore checks rate them again.

Ransomware detection judges, it does not prove

Every new restore point is also judged for signs of ransomware, from data the backup already has. That is a different question from "can this be restored": a restore point without findings proves nothing, and a finding is a reason to look. See ransomware detection.

The archive has its own check

The archive's SHA-256 hash chain is verified separately from backups. The chain check verifies every link, the daily anchors (written every night) and, if you want, a sample of message contents. It is part of the archive, in every edition, see the archive.

What is planned

A full test restore into a separate target for every kind of backup is planned, and so is verifying every backed-up item rather than a sample; the Proxmox VE boot test covers guests only. Until then, the sampled check is the honestly-labelled interim state. See the roadmap for exact status, and how Restow is built for the automated release smoke run that gates every release, including an IMAP backup and restore with a byte-for-byte comparison and a restore with the server stopped.

What the release checks cover

A release smoke run gates every release: a fresh install, restores with a hash comparison, endpoint backup, file share backup against Samba, archive capture from an IMAP account and mail import, among 13 checks. In every release since 0.4.4, 12 of 13 checks passed in full for the full images and 1 was skipped; for the Community images, 9 of 13 passed in full and 4 were skipped, among them, by design, the checks that need a Business module or a second tenant: journal receipt, the storage targets and, since 0.4.4, archive capture. From 0.4.0 to 0.4.3 the run had 12 checks (the file share check came with 0.4.0): 11 of 12 passed in full for the full images and 1 was skipped, and for the Community images 9 of 12 passed in full and 3 were skipped. From 0.2.1 to 0.3.6 it had 11 checks: 10 of 11 passed in full for the full images and 1 was skipped, and for the Community images 8 of 11 passed in full and 3 were skipped. The report, with the reason for every skipped check, is attached to each release.

The skipped check is number 4, the automated Microsoft 365 backup and restore check against a real tenant, run against a dev tenant. It is not active yet and was skipped in every release so far. Microsoft 365 is covered by tests against a simulated Graph API. Restow is a beta: run it alongside your existing backups until you have verified restores against your own data. In the release checks, the agent's install script runs only inside a Linux container; the macOS agent ships, but its installer is not part of the release checks (the agent's own tests ran natively on macOS).

Frequently asked

How does Restow verify that a backup can actually be restored?

By reading it back through the restore path and checking it against the recorded hashes, not by assuming a successful backup job means the data is recoverable. Today that runs as a weekly sampled recovery-readiness check per mailbox, OneDrive and SharePoint site, shown as a sample, not as complete proof. The check does not restore into a mailbox, OneDrive or IMAP account. Servers, clients and file shares are checked after every new backup, Proxmox VE guests within minutes of every new backup (a preview, since 0.4.2) and additionally by boot tests if you turn them on. A full test restore into a separate target for every kind of backup, and verification of every backed-up item, is planned. See the roadmap.

What happens if a verification check fails?

It's shown, not hidden. The affected mailbox or OneDrive's recovery-readiness status reflects the failure immediately, the same honesty rule that applies to Graph throttling waits and every other known limit in the product. A failed check also says what happened, why, and what to do, with the technical details kept for a support case. Red needs evidence (data that is missing, differs or does not decode); a check that could not complete, for example because the storage did not answer, rates nothing, sends no alert and is retried.

Is an unverified backup shown as successful?

No. A backup that hasn't been read back and verified is treated, and displayed, as not proven, never quietly rounded up to "done."

Are file shares verified too?

Yes. After every new restore point of a file share, Restow reads up to 20 sampled files back and compares them with what was backed up, and the repository gets a weekly restic check of 5 percent of its data. Recovery readiness rates the share by its newest restore point. A restore point without samples, because every file changed in the hour before the run, is rated yellow and says why.

Can Restow check that a Proxmox VE backup boots?

Optionally, as a preview. A Proxmox VE job can boot the newest restore point of each guest weekly or after every backup (off by default). The guest's node restores it as a new guest in the pool restow-restore, cuts its network before the start, waits for the QEMU guest agent and destroys the test guest again in every case. Green means the guest agent answered, yellow that the VM only started, red that the restore, the start or the boot failed. It does not check ports or applications inside the guest.

Is a Proxmox VE backup checked right after it is made?

Since 0.4.2, yes, by default (Installation › Proxmox VE, owner): every new restore point of a VM or container is read back within minutes of its backup, a sample of its blocks compared with the recorded SHA-256, for a container a restic check of a subset, besides the weekly check of every guest. It is a preview, tested against a simulated Proxmox VE node. "Check now" and "Check all unverified" in Recovery readiness start checks by hand, and a red check shows the error it found.

What does "Could not check: repository locked" mean for a Proxmox VE backup?

That the restore check could not run, not that the backup is damaged. Since 0.4.3, when another run holds the lock on a container's restic repository, the check is postponed and tried again ten minutes later, with the previous result standing. If the lock stays for two hours, the restore point reads "Could not check: repository locked" in yellow (attention) in Recovery readiness, with restic's report of who holds the lock. It is never red, because nothing was found, and never green, because nothing was read back. Stale restic locks are removed before the check, and "Check now" tries again at once. Checks that a lock failed red before the update stay red until the next check.

What does Restow do when the storage check cannot read a pack?

Since 0.4.2 it does not call that damage. A pack the storage check could not read for the moment (a download cut off, a reset connection, a timeout, S3 asking to slow down, denied access) is read again, up to three times; if it stays unreadable it is listed as temporarily unreadable, with the error and how many checks in a row could not read it, and objects using it are rated Attention. They become Not restorable only when a pack is proven damaged (missing, wrong size or checksum) or could not be read in 3 consecutive checks over at least 24 hours. Every later check reads it first, and "Re-check these packs" reads only the flagged ones again.

Are servers and laptops verified too?

Yes. After every new good backup of a server or client, the agent records the SHA-256 of up to 20 random files. The Restow server then reads those files back from the repository and compares the hashes, and the check is green only when they all match. The repository itself is also checked once a week. Both results feed Recovery Readiness.

Why restore tests matter more than backups →