Aria Automation
VMware Aria Automation Assembler VMware Aria Automation Templates

Introducing Allocation Helpers in VMware Aria Automation

VMware Aria Automation is a powerful tool for managing your IT infrastructure and automating your processes. One of the most recent additions to the platform is the Allocation Helpers feature, which introduces a new way to use the allocation capabilities of VMware Aria Automation based on infrastructure profiles. In this blog post, we’ll explore what Allocation Helpers are and how they can benefit your organization.

What is Allocation?

Allocation in VMware Aria Automation refers to the process of selecting and reserving the most suitable infrastructure, such as availability zone, VPC (Virtual Private Cloud), or a storage account to a specific workload (i.e., Virtual Machine) upon creation of that workload.

The process ensures that infrastructure resources are given to the right users and user-groups based on their roles and responsibilities, leading to efficient resource utilization.

Allocation is driven by policies and rules that guarantee the distribution of resources based on predefined criteria set by the platform administrator. Users with the administrator role assigned have access to VMware Aria Automation Assembler’s Infrastructure tab where they can define Cloud Zones, Network Profiles, Image and Flavor Mappings, Tags and other resource infrastructure configurations that guide the allocation process.

For example, the administrator may create the following policies:

  • A Cloud Zone that includes all computes (availability zones, clusters, resource pools) having a specific tag.
  • Network Profiles to support multiple target deployment environments – a small test environment where an existing network has outbound access only or a large load-balanced production environment that needs a set of security policies.
  • Image Mapping with different AMIs (Amazon Machine Images) for each region and cloud account.
  • Custom Naming convention for the provisioned workloads based on the username, project or timestamp.

When deploying multi-tier applications that consist of Virtual Machines, Networks, Load Balancers and Volumes with VMware Aria Automation Templates, the infrastructure policies described above are used to guide the allocation process to select placement targets that satisfy the requirements of each workload component both individually and as a group together.

For Example, consider the following VMware Aria Automation Template of a two-tier WordPress web application:

Two-Tier WordPress Web Application
inputs: {}
resources:
  mysql:
    type: Cloud.Machine
    properties:
      name: mysql
      flavor: small
      image: ubuntu
      constraints:
        - tag: production
      networks:
        - name: ${wpnet.name}
          assignPublicIpAddress: false
          network: ${resource.wpnet.id}
      cloudConfig: |
        #cloud-config
        repo_update: true
        repo_upgrade: all
        packages:
         - mysql-server
        runcmd:
         - sed -e '/bind-address/ s/^#*/#/' -i /etc/mysql/mysql.conf.d/mysqld.cnf
         - service mysql restart
         - mysql -e "GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY 'mysqlpassword';"
         - mysql -e "FLUSH PRIVILEGES;"
  wordpress:
    type: Cloud.Machine
    properties:
      name: wordpress
      flavor: medium
      image: ubuntu
      count: 2
      constraints:
        - tag: production
      networks:
        - name: ${wpnet.name}
          assignPublicIpAddress: false
          network: ${resource.wpnet.id}
      cloudConfig: |
        #cloud-config
        repo_update: true
        repo_upgrade: all
        packages:
         - apache2
         - php
         - php-mysql
         - libapache2-mod-php
         - php-mcrypt
         - mysql-client
        runcmd:
         - mkdir -p /var/www/html/mywordpresssite && cd /var/www/html && wget -O wordpress.tar.gz https://wordpress.org/wordpress-5.0.3.tar.gz && tar -xzf wordpress.tar.gz -C /var/www/html/mywordpresssite --strip-components 1
         - i=0; while [ $i -le 10 ]; do mysql --connect-timeout=3 -h ${mysql.networks[0].address} -u root -pmysqlpassword -e "SHOW STATUS;" && break || sleep 15; i=$((i+1)); done
         - mysql -u root -pmysqlpassword -h ${mysql.networks[0].address} -e "create database wordpress_blog;"
         - mv /var/www/html/mywordpresssite/wp-config-sample.php /var/www/html/mywordpresssite/wp-config.php
         - sed -i -e s/"define('DB_NAME', 'database_name_here');"/"define('DB_NAME', 'wordpress_blog');"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define('DB_USER', 'username_here');"/"define('DB_USER', 'root');"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define('DB_PASSWORD', 'password_here');"/"define('DB_PASSWORD', 'mysqlpassword');"/ /var/www/html/mywordpresssite/wp-config.php && sed -i -e s/"define('DB_HOST', 'localhost');"/"define('DB_HOST', '${mysql.networks[0].address}');"/ /var/www/html/mywordpresssite/wp-config.php
         - service apache2 reload || service apache2 start
  myapp-lb:
    type: Cloud.LoadBalancer
    properties:
      name: myapp-lb
      network: ${resource["appnet-public"].id}
      instances: ${wordpress.id}
      routes:
        - protocol: HTTP
          port: '80'
          instanceProtocol: HTTP
          instancePort: '80'
          healthCheckConfiguration:
            protocol: HTTP
            port: '80'
            urlPath: /mywordpresssite/wp-admin/install.php
            intervalSeconds: 6
            timeoutSeconds: 5
            unhealthyThreshold: 2
            healthyThreshold: 2
      internetFacing: true
      constraints:
        - tag: production
  wpnet:
    type: Cloud.Network
    properties:
      name: wpnet
      networkType: outbound
      constraints:
        - tag: production
  appnet-public:
    type: Cloud.Network
    properties:
      name: appnet-public
      networkType: public
      constraints:
        - tag: production

To satisfy the infrastructure requirements of this application, the allocation process selects and reserves the following infrastructure components, also known as placement targets:

  • A compute (availability zone) to place all components in.
  • A VPC and CIDR block for the wpnet private network with enough IPs to host the MySQL VM and the WordPress cluster VMs.
  • An existing public network for the myapp-lb load balancer.

As all VMware Aria Automation Template components in this example are logically grouped together, the platform ensures that the three placement targets – Compute, CIDR, and Existing network – are selected in a manner that guarantees interconnectivity between them. For example, when the allocation process selects a public network, it will consider that the network needs to be connected to the about-to-be-selected availability zone. Furthermore, the allocation process makes sure that all selected placement targets are tagged with “production” tag due to the constraints specified inside the Template yaml.

What are Allocation Helpers?

Allocation Helpers are a new type of resource in VMware Aria Automation Templates that can help streamline the process of infrastructure allocation. They are essentially meta-components that can be added to a Template and are designed to work with other resources in your environment to ensure that they are allocated correctly.

Allocation Helpers

There are several types of Allocation Helpers, each dedicated to selecting a specific placement target.
For instance, the Compute Helper selects Account, Region and Placement Compute based on the assigned Cloud Zones from the Project and optionally – by applying constraints-versus-tags matching, whilst the Network Helper selects Account, Region, Subnet, Network Domain (VPC) and other network-related settings based on the available Network Profiles.

Just like “regular” resources, Allocation Helpers can be dropped on the design canvas and configured via editing the yaml on the right-side code panel of the Template. Upon deploying however, Allocation Helpers do not create any actual infrastructure on the cloud hyperscalers. Instead, they live entirely within VMware Aria Automation’s own database. In that sense, they are meta-resources whose purpose is to select and reserve infrastructure from your profiles and give that infrastructure to actual application resources.

Let’s see some examples:

EC2 Instance with Compute Helper

EC2 Instance with Compute Helper
formatVersion: 1
inputs:
  instance_name:
    type: string
resources:
  Allocations_Compute_1:
    type: Allocations.Compute
    properties: 
  Idem_AWS_EC2_INSTANCE_1:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: ${input.instance_name}
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      image_id: 'ami-0aa7d40eeae50c9a9'

In this simple Template, the actual EC2 Instance resource binds its account, region, and availability_zone properties to the selected* output properties of the Compute Helper. Because of that, the values for account, region, and availability_zone will be dynamically populated by the allocation process during deploy time according to the policies and rules that the platform administrator has set – in that case – the Cloud Zones and the Computes included within them.

Two EC2 Instances on the same Subnet

Two EC2 Instances on the same Subnet
formatVersion: 1
inputs:
  instance_name:
    type: string
  second_instance_name:
    type: string
resources:
  Allocations_Compute_1:
    type: Allocations.Compute
    properties: {}
  Idem_AWS_EC2_INSTANCE_1:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: ${input.instance_name}
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      image_id: ami-0aa7d40eeae50c9a9
      subnet_id: ${resource.Allocations_Network_1.selectedSubnet.id}
  Allocations_Network_1:
    type: Allocations.Network
    properties:
      networkType: existing
  Idem_AWS_EC2_INSTANCE_2:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: ${input.second_instance_name}
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      subnet_id: ${resource.Allocations_Network_1.selectedSubnet.id}
      image_id: ami-0aa7d40eeae50c9a9

Here the Network Helper is used to select an existing subnet for our EC2 instances, according to the Network Profiles defined in the system. The two EC2 instances bind the selectedSubnet.id property of the same Network Helper which guarantees that they will be placed by the allocation process on the same subnet.

EC2 Instance with Volume

EC2 Instance with Volume
formatVersion: 1
inputs:
  instance_name:
    type: string
  volume_name:
    type: string
resources:
  Allocations_Compute_1:
    type: Allocations.Compute
    properties: {}
  Idem_AWS_EC2_INSTANCE_1:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: ${input.instance_name}
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      image_id: ami-0aa7d40eeae50c9a9
      block_device_mappings: 
        - volume_id: ${resource.Idem_AWS_EC2_VOLUME_1.resource_id}
          device_name: /dev/sdb
  Idem_AWS_EC2_VOLUME_1:
    type: Idem.AWS.EC2.VOLUME
    properties:
      name: ${input.volume_name}
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      size: 15
      volume_type: io2
      iops: 100

Here, the same Compute Helper is used by two different resources – EC2 Instance and Volume.
In doing so, the engine will try to place the two resources into the same account, region and availability zone guaranteeing connectivity between the components.

How do Allocation Helpers work?

To get the best understanding of how Allocation Helpers work, it is important to break down the concept into smaller, more manageable parts and thoroughly examine each part.

Inputs and Outputs

The primary concept to understand is that every helper in a Template has a set of input properties that can be configured in the yaml. While some inputs are mandatory, others are optional. The platform uses those inputs to guide the allocation process during placement target selection.
In other words, the inputs can influence the allocation process, and perhaps the most common example of an input is the constraints property which is used to direct the allocation flow towards selecting a placement target that is labeled with a specific tag. Furthermore, each helper has a set of output properties that start with selected*.  During allocation, the platform engine dynamically selects placement targets for each helper based on the helper inputs and infrastructure profiles. Upon successful allocation the selected placement targets are returned as part of the selected* outputs of the allocation helper.

All inputs and outputs of each Allocation Helper, along with their type (string, integer, array, etc.), and brief descriptions, can be viewed by clicking on the properties section.

Allocation Helper Properties
Allocation Helper Properties

The computed output values can then be bound to actual resource properties with the following syntax:

${resource.<name-of-allocation-helper>.selected*}

Example:

formatVersion: 1
resources:
  Allocations_Image_1:
    type: Allocations.Image
    properties:
      image: 'ubuntu'
  Allocations_Compute_1:
    type: Allocations.Compute
    properties:
      constraints:
        - tag: env:aws-main
  Idem_AWS_EC2_INSTANCE_1:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: ec2Instance1
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      availability_zone: ${resource.Allocations_Compute_1.selectedPlacementCompute.id}
      image_id: ${resource.Allocations_Image_1.selectedImageId}

Affinity and anti-affinity

Depending on the use case it is sometimes necessary to combine multiple allocation helpers inside the same Template. In one of our earlier examples, we used both Compute Helper and Network Helper to configure the account, region, availability_zone and subnet_id properties of a single EC2 instance.
But how does the platform engine ensure that both helpers will select compatible placement targets?
Imagine the complications that would arise if the engine selected a subnet that is not connected to the selected availability zone – deploying such EC2 instance will most certainly fail.

To solve this problem, the allocation engine builds a graph out of all resources in the canvas, including helpers, through their property bindings. Allocation Helpers that belong to the same component of the graph are regarded as “grouped” and consequently, the engine applies affinity by making sure the chosen placement targets within the allocation group are interconnected.

Two EC2 Instances with two Compute Helpers
formatVersion: 1
inputs: {}
resources:
  Allocations_Compute_1:
    type: Allocations.Compute
    properties: 
      constraints: 
        - tag: pci:true:soft
  Allocations_Image_1:
    type: Allocations.Image
    properties:
      image: 'ubuntu'
  Idem_AWS_EC2_INSTANCE_1:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: 'mysql'
      region: ${resource.Allocations_Compute_1.selectedRegion.id}
      account: ${resource.Allocations_Compute_1.selectedCloudAccount.name}
      image_id: ${resource.Allocations_Image_1.selectedImageId}
  Allocations_Compute_2:
    type: Allocations.Compute
    properties: 
      constraints: 
        - tag: prod:hard
  Allocations_Flavor_1:
    type: Allocations.Flavor
    properties:
      flavor: 'small'
  Idem_AWS_EC2_INSTANCE_2:
    type: Idem.AWS.EC2.INSTANCE
    properties:
      name: 'postgres'
      region: ${resource.Allocations_Compute_2.selectedRegion.id}
      account: ${resource.Allocations_Compute_2.selectedCloudAccount.name}
      image_id: 'ami-0aa7d40eeae50c9a9'
      instance_type: ${resource.Allocations_Flavor_1.selectedInstanceTypeName}

In this example, there are two graph components and therefore two groups of allocation helpers – [Allocations_Compute_1, Allocations_Image_1] and [Allocations_Compute_2, Allocations_Flavor_1].
The engine will guarantee affinity between placement targets within each group, however the two groups may end up in a different region.

Grouping of helpers can be manually overwritten by specifying the groupId input that each Allocation Helper has. By setting the same groupId in all helpers, one can force the allocation process to apply affinity even between helpers that are part of different graph components within the canvas. The following diagram depicts how the allocation process works when evaluating placement targets for 3 Allocation Helpers – Compute, Image and Flavor.

Allocation Helpers

What are the benefits of using Allocation Helpers?

Overall, using Allocation Helpers and dynamic placement based on tags and infrastructure profiles in VMware Aria Automation can offer several benefits:

  1. Efficient resource utilization: By using tags, one can ensure that resources are allocated efficiently. For example, one can tag VM hosts based on their intended usage, such as development, testing or production.
  2. Automation: Resources can be automatically allocated and provisioned based on tags, which can help reduce manual effort and ensure consistent deployment.
  3. Simplified management: By using tags, one can simplify resource management by grouping resources based on their characteristics, such as application on environment. This can help streamline tasks like patching, backups and upgrades.
  4. Flexibility: Infrastructure profile allocation and dynamic placement can supply flexibility, allowing one to easily adjust resources based on changing demands. For example, when a newer image version is released the platform administrator can roll it out by updating the image mapping for the affected region.

Previously, the ability to use allocation was restricted to only a handful of resource types such as Virtual Machines, Load Balancers, and Subnets. The allocation settings for these types of resources were contained within their respective definitions. However, with the introduction of allocation helpers, the allocation settings have been separated from the resources themselves and placedin a distinct meta-component. This new approach allows for all types of resources to benefit from allocation, expanding beyond the earlier limitation of just Virtual Machines and a few other resources.

Summary

Allocation Helpers are a powerful new tool in VMware Aria Automation’s design canvas that enables users of the platform to configure allocation capabilities for every resource type. This offers several advantages over static Template configurations such as more efficient resource utilization, flexibility, and simplified management.

We have created the following demo video on our TechZone site for additional information and to see allocation helpers in action. Getting Started with Plug-in Based Designs in Aria Automation

Related Posts:

Related Articles