Home > Blogs > VMware vSphere Blog


What’s New with vSphere Data Protection (VDP) 5.5

VMware vSphere Data Protection (VDP) is a backup and recovery solution that was introduced with vSphere 5.1. VDP leverages proven technology from EMC Avamar to provide reliable, space-efficient, disk-based data protection for VMware virtual machines (VMs). VDP is fully integrated with VMware vCenter Server and the vSphere Web Client. One of the real beauties of VDP is its ease of deployment and management which explains why thousands of VMware customers have already downloaded VDP since it was introduced last year.

Simply put, VDP is backup and recovery for VMware, from VMware.

Last week at VMworld, VDP 5.5 was introduced which further extends the capabilities of VDP. Here are details on these new capabilities in VDP 5.5…

- Replication of backup data to EMC Avamar

Replication provides a more efficient, secure means to move backup data offsite for disaster recovery purposes (versus legacy tape solutions). Service providers will be able to offer services for receiving, storing, and archiving backup data replicated from VDP at a customer site to Avamar located in service provider data centers. If an organization has an existing Avamar implementation, this can also be used as a target for backup data replicated from VDP.

The replication engine is very robust, efficient, and flexible. It is possible to replicate backup data for all VMs backed up by the VDP appliance or select individual VMs. Since the backup data is de-duplicated at both the source and destination and only changed data segments are sent across the wire, bandwidth utilization is minimized – no need for WAN acceleration technologies. Replication schedules and retention policies can be customized and are separate from backup job schedules and retention. For example, VDP can have a backup job with a daily schedule and a 60-day retention and then replicate the backup data to a service provider weekly with a retention policy of one year at the service provider location. It is even possible to restrict which backup data is replicated, if desired. For example, the three most recent backups or backups that occurred within a specific time period (by date).

SunGard and Secure-24 are planning to offer a backup data replication target service for VDP and I expect many more service providers will offer this service in the near future.

- Direct-to-Host Emergency Restore

The restore of a virtual machine directly to a host can be accomplished without the need for vCenter Server and the vSphere Web Client. This capability is especially interesting for those who plan to use VDP to backup vCenter Server. This was a popular feature request and we are now happy to deliver with VDP 5.5!

- Backup and restore of individual .vmdk files

Speaking of feature requests, this was the most requested: The option to select individual .vmdk files when creating/editing a backup job or performing a restore. Granted, this capability should have been in the initial release of VDP – all I can say is better late than never.   :)

- Granular scheduling for backup and replication jobs

Specific times can be selected to start backup and replication jobs. For example, Backup Job 1 is scheduled to start at 9:05 PM, Backup Job 2 is scheduled to start at 10:20 PM, and Replication Job 1 is scheduled to start at 1:30 AM. Of course, backup jobs should still be scheduled during the backup window.

- Flexible VDP storage management

When deploying VDP, separate data stores can be selected for the VDP guest OS and backup data partitions. It is also possible to mount an existing VDP backup data partition to a new VDP appliance during deployment. As an example, this is useful if by chance the .vmdk file containing the VDP guest OS is corrupted or lost. A new VDP appliance could be deployed and the existing backup data partition added to the new appliance thus preserving that existing backup data.

As many of you already know, there are two flavors of vSphere Data Protection (VDP) There is VDP, which is included with vSphere Essentials Plus and higher, and there is VDP Advanced, which is more robust and sold separately.

VDP Advanced 5.1 provides an agents for SQL Server and Exchange (unlike VDP) that are installed in the guest operating system of a VM running either of these applications. The agents enable more granular backup and restore capabilities. For example, you can select individual databases for backup or restore. The agent also provides application quiescing, client-side de-duplication, and more advanced backup options such as log truncation, full or incremental backups, and the ability utilize multiple backup streams.

VDP Advanced 5.1 also allows dynamic capacity growth (unlike VDP). For example, you can start with a VDP Advanced appliance that is configured with 2TB of de-duplicated backup data capacity and grow that capacity to 4TB, 6TB or 8TB as needed. VDP (not Advanced) is limited to 2TB. It is possible to migrate from VDP to VDP Advanced and preserve existing VDP backup data.

Understanding the latest versions of vSphere Data Protection (VDP) are VDP 5.5 and VDP Advanced 5.1, that begs the question “Are the new features in VDP 5.5 described above also in VDP Advanced 5.1?” Short answer: No, these new VDP 5.5 features are not in VDP Advanced 5.1. As most of you know, we cannot discuss specific road map information in forums such as this. However, we can take a brief look at the history of VDP releases and theorize a bit: VDP 5.1 (not Advanced) was initially announced with vSphere 5.1 at VMworld in 2012. Early this year, VMware introduced VDP Advanced 5.1 with all of the features of VDP 5.1 – and more! At VMworld last week, VMware introduced VDP 5.5 and its new features with vSphere 5.5. I’ll let you take it from there…   :)

By the way, I did not include any screen shots. I have something better – check out this site if you want to “walk through” some of the new features of VDP 5.5, as well as, many other features and products that are available in VMware vCloud Suite 5.5:

http://vmwarewalkthroughs.com/vCloudSuite5-5/

@jhuntervmware

37 thoughts on “What’s New with vSphere Data Protection (VDP) 5.5

  1. James Montgomery

    Hi Jeff,

    Thanks for the article.

    I’ve noticed the walk-through screenshot uses “root” to authenticate with the target Avamar system.

    Please can you tell me if it can be give Avamar Repluser credentials instead, and “root” was just an example?

    Can I get VDP 5.5 documentation anywhere?

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi James – I used the root account for simplicity’s sake. Repluser can be used. I just tested it in my lab. The documentation and release notes will be posted when VDP 5.5 becomes generally available (GA).

      Reply
  2. Pingback: A Summary of What’s New in vSphere 5.5 | VMware vSphere Blog - VMware Blogs

  3. Benjamin

    Thanks for Granular scheduling. Some blackbox VMs doesn’t support VM hot snapshot and they need to be shutdown before doing backup job for data consistency.

    With this feature, it could be possible to perform backup granular scheduling and vCenter scheduled tasks for VM guest OS shutdown and Power On in order to have automated VM consistency cold backup.

    Don’t hesitate to tell me if my thinking about this feature is wrong.

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi Ben,

      I think that is a good use case, if you can afford the downtime. Example:
      Scheduled task shuts down VM at 9:00 PM
      VDP starts backup job at 9:05 PM
      Scheduled task powers on VM at 9:30 PM

      Reply
      1. Benjamin

        That’s exactly what I mean and if VDP works like VDR for the initial backup process (perform VM snapshot then backup snapshot to appliance), the Power On task could append just after VDP scheduled backup time (one or two minutes after for security) since VMware cold snapshots are very fast.

        Do you know if it is possible to create an unscheduled backup job (just define the source and destination) on VDP and call it from vSphere Orchestrator (plugin or CLI) in order to create a workflow?

        Reply
        1. Jeff HunterJeff Hunter Post author

          Unfortunately, there is no VCO plugin or even a basic supported CLI method to manage VDP jobs. This is a popular request and VDP product management is aware. However, it will probably be a while before this functionality is available (assuming that it will be someday).

          Reply
  4. Pingback: VMware: What’s new in vSphere 5.5 | VMpros

  5. Lee

    Hi Jeff,
    In regards to replication to a service provider Avamar grid is the replication supported to a Avamar grid with its storage on Data domain or only to primary avmar grid storage?

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi Lee,

      The VDP appliance has no “visibility” as to what storage it is replicating to. When creating a replication job in VDP 5.5, four items are configured for the destination: IP address or FQDN, TCP port number, username, password. If you have Avamar configured to store replicated data on Data Domain, that will work fine.

      Reply
  6. Steven

    A very nice feature would be an option to limit the number of concurrent backups, all VWs starting a backup together is quite killing the I/O on low/mid-end systems!

    Reply
    1. Lee

      Yes we also found this issue on the previous version of the product, you can limit this by editing a file somewhere in /var directory which limits the number of proxies used for backup/restore. Even on high end systems we started peaking at 1.2GB/s throughput and latency started to increase so you would still need to tune the number of proxies used to the environment. Sorry i can’t be more specific on the location as I cant remember the location and the demo is offline now.

      Reply
    2. Jeff HunterJeff Hunter Post author

      Steven – I agree. I have mentioned this a few times to product management and the development team. I think it is being considered for a future release. As mentioned by Lee, it is possible to change the number of proxies in use by running the registerproxy.sh script located in /usr/local/avamarclient/etc/. Please understand this is something that has not been tested and it is NOT SUPPORTED. I strongly recommend against making any kind of changes like this in a production environment.

      Reply
      1. Javier

        I have hit this as well within our environment. VDP becomes unresponsive and backups stall leaving a bunch of snapshots unhandled and some of them locked and mounted to the VDP Appliance. (which needs to be powered off if you want to release that snapshots)

        Job concurrency control is really necessary, supported or not :(

        Reply
  7. Pingback: Novedades vSphere 5.5 : While(1){ } – A Infinite Loop

  8. Pingback: Installation u. Konfiguration vSphere Data Protection (VDP) | blog.bistron.eu

  9. Dennis

    Hello Jeff,

    when will the VMWare Data Protection Advanced 5.5 available for Download ?
    or is there a way to use VDPA 5.1 with vCenter 5.5 ?

    regards Dennis

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi Dennis – the latest release of VDP Advanced 5.1 is compatible with vCenter Server 5.5. I can’t say for sure when VDP Advanced 5.5 will be GA (generally available – available for download, in other words), but we are hoping to make it available sometime this quarter. VDP Advanced 5.5 was announced at VMworld in Barcelona this week. As with most announcements like this, the product typically becomes available for download in the weeks following the announcement. However, as with any “futures” information, there are no guarantees on when a product becomes available for download and exactly what features it will contain.

      Reply
  10. Bill Wright

    Jeff,

    I’m looking for some help in getting VDP to work. We have VSphere Essentials Plus on a three host cluster. The marketing promise of being able to recover our VMs was a major factor in our company making the switch to VMWare and investing in it. We’ve gone from 5.0 to 5.1, and we’ve tried to get DR and now VDP to work. We’ve installed VDP 5.5 but that didn’t do it. We’ve spent a great deal of time and energy working on our system and with VMWare tech support, but to no avail. At this point, we’ve given up on VDP and out of desperation we purchased a license for PHD virtual, which worked out of the box and continues to work smoothly.

    We’re running VMWare and VDP on VM certified hardware and we’ve been escalated through tech support to the ‘highest level’. Tech support finally told us that our storage hardware was ‘too low end’, but when we asked for a spec that would work with VDP, they couldn’t provide us with any help. So here we are with VMWare certified hardware, the latest version of all software, and a system that fails to work with VDP but works smoothly with competing software.

    Any suggestions on help you could find for us to break this logjam? I’d really like to be using VDP and not being pursued by PHD to upgrade to Yet Another Version…

    Thanks,

    Bill Wright

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi Bill. VMware Support is the best avenue for getting help with the issues you are experiencing. It is true that VDP (and many other backup and recovery products) require storage that can handle a reasonable amount of I/O. As mentioned in a previous comment, it is possible to change the number of proxies in use, which should lighten the load on the storage. This is done by running the registerproxy.sh script located in /usr/local/avamarclient/etc/. Please understand this is something that has not been tested and it is NOT SUPPORTED, but it may help in troubleshooting the root cause of the issue. For example, if you reduce the number of proxies in use down to one or two and things work fine, but things do not work fine when all eight proxies are enabled, that is a pretty good indication that storage performance (lack thereof) is to blame. VDP 5.5 also includes a storage performance test that can be ran during deployment or after VDP has been deployed. This may also provide some answers if the issue is indeed storage performance.

      Reply
  11. Mark Burgess

    Hi Jeff,

    Version 5.5 certainly looks like a big leap forward with the replication and Data Domain integration, but I am still very confused about how a customer (I work for a partner) should handle low term retention.

    Surely any half decent backup product needs the ability to backup to tape, disk or cloud with the normal granular restore capabilities, on a monthly or even less frequent basis.

    Without this capability I cannot see how VDPA can become a viable solution for anything but the smallest of organisations.

    Or am I missing something?

    Many thanks
    Mark

    Reply
    1. David Fairfield

      I agree, I think the new product sounds great and has a lot of new features, but it would appear to be lacking the cloud/tape options that other vendors provide (at a cost of course). We’re a small organization, 3 hosts on Essentials Plus, less than 20 VM’s and use vDP 5.1 currently for our local VM backups. We need the ability to take that volume or vDP VM directly out to the cloud somewhere without having to use something like Veaam to do a local copy backup first and then turn around and take that out to the cloud. We use an Equallogic PS4100 exclusively for vDP to keep our production SAN (PS6100) free from that traffic. How would vDP 5.5 handle that for us considering we’re one office with no remote sites? We’ve been researching options and given 5.5 has some new things that would help, it doesn’t appear to solve it all for us…

      Reply
      1. Jeff HunterJeff Hunter Post author

        Hi David and Mark,

        Keep in mind VDP (not Advanced) 5.5 is included at no additional charge if you have vSphere 5.1 Essentials Plus or higher. VDP can replicate to EMC Avamar. It is likely we will soon see service providers offering “backup data target as a service” for VDP 5.5 where the service provider hosts an Avamar grid for its customers and provides restore services, i.e., a cloud option.
        VDP Advanced 5.5 was announced at VMworld in Barcelona. VDP Advanced will be able to securely and reliable replicate backup data to another VDP Advanced appliance – either at a secondary customer site or a service provider. Tape is a legacy technology – tapes tend to be cumbersome, unreliable, and potentially not secure.
        Of course, there is a licensing cost for VDP Advanced, but it includes quite a bit of additional functionality as outlined here: http://blogs.vmware.com/smb/2013/10/vsphere-data-protection-advanced-5-5-gaining-momentum.html

        Reply
        1. Mark Burgess

          Hi Jeff,

          Thanks for your comments.

          I think to call tape a legacy technology is a bit of a cop-out.

          What percentage of VMware’s customers that VDPA is targeted at (i.e. SMB/SME) are still using tape?

          Very high I suspect as technologies like Avamar and Data Domain are still too expensive for long term retention.

          I agree that the cloud is the way forward, but probably not using hosted services like Avamar/DD as they will be too expensive to compete with tape.

          Also customers do not want to be locked into a proprietary standard like Avamar/DD as inevitably it will be expensive.

          The beauty of tape, raw disk (i.e. CIFS shares) and cloud storage like Amazon Glacier is that they are cheap and are flexible so the customer can change at any point, which is exactly would you need for long term retention.

          Products like Riverbed Whitewater provide an ideal bridge between the backup application and the cloud and I would like to see it supported by VDPA, especially as it is available as a VM appliance – true software defined storage!!!

          Above all customers want choice and they do not want to be forced to sign-up with an expensive service provider and probably pay more for a hosted Avamar solution for long term retention than for VDPA – it just does not make sense to me.

          For me long term retention to tape or any disk/cloud solution is the achilles heel of EMC’s backup strategy – they do not ever want you to store data on storage not provided by EMC.

          This strategy is not sustainable and I hoped that VMware would have seen this and been able to rise above the politics.

          Because of this issue as a VMware partner I would find it very difficult to recommend VDPA over Veeam as clearly the product is being held back for non-technical reasons.

          It is such a shame as VDPA 5.5 looks superb in most other areas.

          Or as before am I missing something?

          As I work for a partner (EMC and VMware) please respond directly if you prefer.

          Reply
          1. Jeff HunterJeff Hunter Post author

            Hi Mark – Thank you for your comments, as well. In my opinion, tape is a legacy solution – it has been around for decades. That is not a cop-out. The reasons many still use it run the gamut. However, ask your customers how much they like their tape solution. I strongly suspect the majority are not happy, but continue to use it because there aren’t many (any?) better, reasonably priced alternatives. Avamar supports tape and VDP is based on Avamar. I doubt It would take much development effort to enable tape support in VDP, but then we would start tying the solution to physical hardware. It makes more sense to put development efforts into building a better solution, not continuing support for an existing solution that most organizations would like to get away from. I think you are making assumptions that pricing for service provider based VDP replication targets is going to be too expensive. As a partner/reseller, you know that it boils down to customer requirements. If a customer requires long-term archiving and/or a tape solution, then I will be the first to tell the customer VDP or VDP Advanced is not the right solution for that use case. However, VDP and VDP Advanced do meet the requirements of many customers. We have seen tens of thousands of downloads for VDP and when VDP Advanced 5.5 becomes GA, I suspect we will see many organizations investing in it. I can tell you my inbox and phone are already very busy since the announcement of VDP Advanced 5.5 at VMworld in Barcelona. Hopefully, you will consider adding it to the portfolio of backup products you recommend to customers.

  12. Mark Burgess

    Hi Jeff,

    As per my previous comment I totally agree with you that customers want to get away from tape, but I think we both agree that it is still the best value solution for most customers – and that is what counts especially in these tough economic times.

    I can fully appreciate why VMware/EMC would not want to invest in putting a proper tape-out solution into Avamar/VDPA especially as my understanding is that Avamar and NetWorker will be consolidated into a single next-generation product hopefully sometime next year – it will be interesting to see if this has tape out (if I was a betting man I would say yes as I do not believe the world is quite ready to let go of tape, but I am sure it will once we enter the next decade).

    What would be nice to see in the short-term is the ability to schedule a copy of a backup job to a CIFS or NFS share (with full restore capabilities). This is what Avamar ATO has done for years (but is very clunky) – this would then allow the customer to spool this off to tape or use next-generation cloud gateway products like Riverbed Whitewater.

    Have you done a survey of VMware customers to determine what customers want from their backup solution in terms of long term retention – I would be very interested to review the results?

    Best regards
    Mark

    Reply
  13. Pingback: vSphere Data Protection Advanced 5.5 is Generally Available | VMware vSphere Blog - VMware Blogs

  14. Pingback: vSphere Data Protection Advanced 5.5 is Generally Available | VMware SMB Blog - VMware Blogs

  15. Steve

    Hello!

    I’d like to use VDP to backup VMs at my protected site and replicate those backups to a recovery site. I want to use array based replication to replicate the volume. Is it possible to attach the replicated volume to a new instance of VDP at the DR site in the event of a disaster and recover the VMs? (i.e. attach the replicated backup volume to new VDP instance at install time at DR site?).

    Thanks!

    Reply
    1. Jeff HunterJeff Hunter Post author

      Hi Steve. Technically, the approach you are describing might work, but I am not sure I would rely on it. Check out this earlier blog post I wrote:

      http://blogs.vmware.com/vsphere/2012/12/recover-replicated-vdp-appliance.html

      I realize you are using array replication (not vSphere Replication as I did with the blog article), but the result is the same: The replicated copy will be a crash-consistent copy. If there is a requirement to replicate backup data from one site to another, I encourage you to consider investing in VDP Advanced 5.5, which has replication built in.

      Reply
  16. Pingback: VMware Link Collection | Life

Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>