Modern distributed applications consist of many components: containers, virtual machines, databases, load balancers, Kubernetes namespaces and other objects. Like all applications, you must protect against data loss, service outages and disasters. Data protection applications must contend with new types of protection objects and complex topologies. The Astrolabe data protection framework enables discovery and backup/replication of data and the restoration of complex applications. It provides a data protection-centric model of applications with APIs for snapshotting, data extraction, copying and restoration.
Data protection is usually the last subject tackled when a new system is designed. The lack of standard APIs for data protection forces data protection vendors to write new interfaces to each new application they plan to protect. As a result, many users opt to forgo data protection, use a stop-gap solution or wait until their data protection vendor supports the application before deploying.
Astrolabe introduces a core concept, the Protected Entity (PE), as the focus for data protection. The PE can be nearly anything, including:
- virtual disks
- filesystems
- Kubernetes namespaces
- databases
- service meshes
- virtual machines
The core requirement for a (PE) is that it can be created from a serialized copy. A PE may also have component PEs. For example, a Kubernetes namespace may have a set of Persistent Volumes (virtual disks) that are components of it. PEs form a graph of objects that need to be snapshotted and backed up together.
Providing basic DP oriented APIs such as snapshot operations, PEs are used to create/delete operations, and provide data paths for retrieving/restoring data. A standard data path (S3 compatible) is specified for all PEs. Higher performance data paths can be specified, as well. A key goal of Astrolabe is that all PEs can be backed up by the basic Astrolabe client.
The Astrolabe project consists of:
- OpenAPI specification
- Reference server implementation
- Reference Protected Entity implementations
The reference implementation currently includes support for vSphere virtual disks, a generic “file system” and a S3 generic repository that can be used to store any type of PE. We currently have a Kubernetes namespace PE under development. The reference server is implemented in Go. Client bindings and server skeletons for other languages are easily generated from the OpenAPI specification.
Astrolabe is just starting. We are committed to developing this in an open fashion and with community collaboration. We welcome your comments and contributions via GitHub to help shape the direction of the project. We look forward to collaborating with users, partners and developers to determine the features and capabilities of Astrolabe and to make it an open standard.
