Not About Backups, It’s About Restores

Years ago, a new backup product came to market. The product name featured the words “Backup and Restore.” I remember asking one of their salespeople, “Isn’t that kind of wordy?” He turned and asked me, “Yes, but have you used backup products that ONLY have backup in the name…and did the restores always work?”

He had a point. Before modern backup APIs, file-based backups didn’t always capture system state. Image-based backups of entire systems were often complicated, expensive, and relatively rare. Restores could involve long incremental chains, feeding tapes one by one. Backups would happily fail for weeks or months if nobody read the reports every morning. Often, we were recovering one failed server, with VSS, secondary copies, and application-level recovery providing a safety net for a missing backup. Modern, coordinated encryption and destruction of data by advanced ransomware operators have changed that threat landscape. Complete backups and working restores matter more than ever. Part of assessing your ransomware response posture is making sure you have fast, reliable recovery from backups that are protected and separated from production. What does this require?

Start with copies an attacker cannot take with them

The familiar 3-2-1 rule is a useful starting point: three copies of your data, on two types of media, with one copy offsite. It forces you to think beyond the production datastore. Three folders on the same storage system do not accomplish the same thing.

Ransomware adds another question: can someone who compromises production also delete your recovery copies? Offsite helps with a building failure. It does less for you if the same compromised administrator account controls both sites. Protect recovery infrastructure with separate administrative access, appropriate network boundaries, and offline or immutable copies. CISA’s ransomware guidance explicitly calls for offline backups and regular recovery testing.

A local snapshot remains useful, but losing the datastore can take it with the VM. As I covered in Snapshots Are Your Best Friend, remote protection belongs alongside local recovery options.

Make restore testing routine

A backup job reporting success tells you that the backup software finished its job. You still need to find out whether the application works.

Automate recovery into an isolated “bubble” network on hardware separated from production. VMware Cloud Foundation (VCF) networking can provide the overlay networks and policies to build that environment. Configure and test the isolation explicitly, including outbound access; creating another network segment alone does not make it a clean room. Restored copies should not register themselves in production DNS, send customer emails, or start processing the same payment queue as the live application.

Then test beyond power-on. Can users authenticate? Does the database recover? Can the application complete a representative transaction? Record the restore point, results, and elapsed time, and rerun the tests as the application changes. A booting VM is a useful milestone. A working service is the recovery objective. A server that has a corrupt bootloader will back up fine, but it will not restore fine, and the only way to know recovery is possible is regular testing. 

Find a restore point you can trust

The newest backup may contain encrypted data. An older one may contain the attacker’s foothold. Immutability preserves a restore point; it does not establish that the contents are clean.

This is where the cyber-recovery capabilities in VCF Protection and Recovery, previously VMware Live Recovery, become useful. Guided restore-point selection and behavioral analysis help teams evaluate candidate recovery points in an isolated environment before returning workloads to service. VMware’s overview explains that workflow. VCF 9.1 also supports a customer-managed, on-premises clean room with the VMware Advanced Cyber Compliance add-on, including isolation and EDR integration. The vSAN Protection and Recovery overview covers the on-premises option.

Give the security team time and space to examine candidates, remediate vulnerabilities, and address compromised credentials. A malware scan is evidence for that decision, not a guarantee that every trace of an intrusion has disappeared.

Keep enough history without making the vault unaffordable

If you discover an intrusion today, yesterday’s copy may already be too recent. Recovery history needs to reach back far enough to give you useful choices.

A vSAN cyber vault can provide the protected storage behind that process. vSAN ReadyNodes certified for cyber recovery support high-density QLC flash, giving recovery environments a more economical all-flash capacity option. Global deduplication in vSAN ESA, generally available in VCF 9.1, can reduce duplicate blocks across the cluster. Actual savings depend on the data; data already compressed or encrypted inside the guest or application will offer fewer opportunities.

Size for retained history, incoming changes, and the working space needed to validate recovery candidates. The vault needs room for the investigation as well as the copies. Use certified hardware and account for QLC endurance and the supported recovery use case when selecting a profile.

Put the critical applications on a fast recovery path

Restoring one accidentally deleted file and restoring the business are very different storage workloads. A repository that comfortably absorbs overnight backups may struggle when hundreds of VMs need to read from it at once.

Tape and capacity-focused magnetic storage can still have a place in the retention strategy. But if your critical application has a four-hour recovery objective, discovering that its restore takes 17 hours is a design failure you want to find during a test. Watching a large restore crawl along is like watching paint dry, except someone from the business keeps asking whether it is dry yet.

Keep your most critical services on a recovery path that can meet their objectives. Flash-resident replicas and recovery points can avoid some of the delays of retrieving and reconstructing data from slower tiers. Test the whole path, though: network throughput, concurrent restores, security validation, application startup, and dependencies all consume time. Fast storage helps; it does not turn an untested runbook into a four-hour recovery plan.

Recover foundational services such as identity, DNS, and key management in the order the applications actually need them. Measure success when the service is usable. You do not want storage throughput to become the reason the business starts entertaining an attacker’s sales pitch.

Have somewhere to recover that is not the crime scene

Your recovery plan needs a destination that the incident has not compromised. That might be a separate customer-owned environment or a VMware-powered cloud service provider. Confirm the capacity, access, connectivity, and recovery procedures before you need them.

The recovery compute does not necessarily have to sit idle all year. A suitably isolated, general-purpose cluster could run lower-priority test/dev workloads during normal operation. The runbook can specify which workloads shut down to free capacity, who can authorize that decision, and how critical recovered services receive priority.

Use vSphere resource pools, reservations, and DRS deliberately, and test the plan under contention. Shares control relative access to resources; they do not create capacity, and their effect depends on the resource-pool hierarchy. Keep the vault and clean-room security boundaries intact even when you share compute. A QLC recovery-storage profile and the compute needed to run production applications are separate sizing and support decisions.

Pick a critical application and walk the entire recovery path: select a protected copy, validate it in isolation, recover its dependencies, and have the application owner prove it works. Record how long it took and fix what slowed it down. That is how recovery becomes another working layer of your defense in depth.

The backup report can tell you what was copied last night. The restore test tells you whether you can bring the business back.