In the IoT Single Point of View whitepaper, we looked at the key foundational decisions for successfully implementing an Internet of Things (IoT) solution. In this blog, we’ll break down the high level architecture in the infrastructure plane. This is one part of the Deployment Architecture foundational decision as highlighted in the whitepaper.
An IoT solution by its very nature will span a geographic area. This could be a factory, a city, a country, the globe or even space. Whereas we’ll be negotiating different technical barriers and constraints as we expand the geographic and spatial span of an IoT solution, we must recognize that we’ll need multiple infrastructure sub-systems with differing roles. Below, we describe briefly the role these sub-systems play in an IoT solution.
Infrastructure Sub-Systems of an IoT Solution
Devices
At the frontier of an IoT solution facing the real world that the solution interacts with are the devices. These use sensors to collect data from the environment in which they are placed. The ratio of sensors to devices can vary quite a bit – from a single sensor per device to devices with multiple sensors. For example, a temperature sensor as a device measures only temperature and communicates that into the IoT solution, but a device monitoring the weather may collect various readings, including but not limited to temperature, barometric pressure, rainfall, humidity and wind speed using different sensors connected to the same device. In an IoT solution, the devices communicate the data back into the IoT solution, primarily to the Device Edges, which are usually the immediately connected IoT infrastructure sub-system. Devices and Device Edges usually use special purpose protocols over mostly wireless and sometimes wired connections.
Device Edges
Like Devices, Device Edges sit at the frontier of an IoT solution. However, unlike Devices, they are expected to be more resilient, have more capacity for storage, comparatively more compute capacity, possibly have an ability to run models for drawing inferences/predictions from data coming through and implement IP to connect to the rest of the IoT solution. Device Edges usually connect to multiple Devices and could possibly be proprietary in nature supporting special purpose protocols and wireless connections to Devices.
Depending on requirements of an IoT solution and the feature set of Device Edges, they may be the bridge between Devices at the frontier and the rest of the IoT solution, and service as low key Compute Edges, also. With the scale and spread of an IoT solution, new methods to manage these remote sub-systems will be required. VMware’s Pulse IoT offering caters to this challenge by collaborating with various vendors and manufacturers so that your IoT solution can be enabled with Devices and Device Edges that fit your purpose, yet be managed using a single product.
Protocol Gateways
Devices use a variety of protocols to communicate with Device Edges to transmit among other things, device status, sensor data, logs and events. Each protocol has a number of characteristics that should inform the decision to employ them in a deployment. Al-Sarawi, Anbar, Alieyan et al. compare seven protocols in their paper Internet of Things (IoT) Communication Protocols: Review. In addition to the options for the physical, link and network layer protocols, the application layer has myriad options also. In a survey of technologies enabling IoT, Al-Fuqaha, Guizani, Mohammadi et al provide an overview of machine to machine (M2M) connectivity from devices to applications.
For an architect designing an IoT solution, it is evident very quickly that Devices may be unable to communicate with Device Edges without additional “help” primarily due to the diversity of protocols at different layers. Whereas one can enable a Device Edge with multiple services to cater to the diversity of application layer protocols, additional components called Protocol Gateways may be required to hide the diversity at the physical, link and network layers. These devices act as concentrators, bridges, routers or a combination of those to connect an IoT Device network in one protocol to a network in another protocol catering to the Device Edges.
Network Connectivity
Beyond the Device Edges, we’re essentially in the datacenter territory most of us in the IT infrastructure space are familiar with, the only caveat being that the IT infrastructure here is geographically spread out. Network technologies created to cater for this spread begin to appear with the Device Edges or just after. Any network technology in use to connect the infrastructure sub-systems of IoT will need to be as a minimum:
- Secure, since it’ll be carrying data from devices. This could contain Personally Identifiable Information (PII), data for components of a sensitive system, such as power grids and airplanes
- Software Defined, to allow administrators of the IoT system to test and effect changes at scale remotely
- Reliable, to compensate somewhat for the general unreliability in connection that may be expected from Devices and Device Edges purely due to the nature of the environment in which they might be deployed
- Scalable, because IoT among other things is a problem in geographic scale
Whereas many vendors claim to have solutions for a network that might form the backbone of your future IoT solution, we encourage you to look at VMware’s SD-WAN by VeloCloud and Cloud Backbone Network to address the key features you’ll need to underpin your IoT Solution’s Wide Area Network (WAN) requirements and connect across different IoT infrastructure sub-systems.
Compute Edges
Depending on the use-case and capabilities of the Devices in use, a Device Edge and Compute Edge could be combined into one. However, to draw a distinction in purpose, a Compute Edge hosts significant compute capacity and is an extension of the datacenter closer to the edge.
Often when we think about IoT, the imagination is drawn towards what we can do with the data generated or collected by IoT Devices and not towards what goes into making an IoT solution operational; that is activities such as, provisioning, securing, managing, monitoring, scaling and decommissioning of Devices and Device Edges. Whereas the possibilities that the data offers elicits most of our imagination and effort, the operational aspects of an IoT solution also deserve similar interest and effort. Compute Edges, in addition to supporting solution components that contribute to consuming data from an IoT solution and making that productive, also support a large number of components that provide supporting services to the rest of the IoT solution. For example:
- Networking components that connect Devices and Device Edges to the rest of the IoT Solution,
- Data filtering solutions that curate the data flowing from the Devices via Protocol Gateways and Device Edges,
- Intrusion detection services,
- Solutions to roll out patches and updates to Devices, Device Edges and Protocol Gateways,
- Prediction and inference components at the edge, and
- Components to carry out distributed training at the Compute Edge
These capabilities may be delivered by augmenting Compute Edges with VMware offerings such as SD-WAN and Cloud Backbone Network, and implementing products such as NSX-T, VMware AppDefense, and Pulse IoT.
Depending on the use-case and deployment architecture of IoT solutions, Compute Edges may be physically placed at sites that belong to IoT solution owners, or they may leverage the geographic spread of public cloud provider datacenters and utilize hosted solutions. Any which way you may choose to proceed, VMware has offerings in the space, namely:
- VMware Cloud on Dell EMC, VMware’s Project Dimension, or VMware Cloud on AWS Outposts for on-site deployments, and
- VMC on AWS for cloud-provider based vSphere stack deployments
Core Datacenter
The core datacenter represents the main datacenter where all data ingested by the IoT solution may be deposited for analyses. It may be tempting to think that the IoT Solution’s center of gravity is at the Core Datacenter, but this may not be true. The Core Datacenter will however be the orchestration center of the IoT Solution. There is also no reason for the Core Datacenter to exist on-premises and not in the cloud! As might be evident from what you’ve been reading so far, this choice is dependent on the IoT use-case and the deployment architecture.
Since datacenter virtualization has been VMware’s core business for quite some time, we have mature options in this space across a spectrum of needs. You may want to look at the following options for a start:
- Leveraging VMware’s Software Defined Datacenter Products to create your own datacenter,
- Leveraging the VMware vCloud Network Function Virtualization if you’re a telecommunications company or consuming from a telecommunications provider that implements it if you are not, and
- Consuming from a provider that’s part of the VMware Cloud Provider Program
In addition to the aforementioned options, all options applicable to the Compute Edge are also applicable to Core Datacenter.
Cloud Services
Public Cloud services such as Amazon Web Services (AWS), Google Cloud and Microsoft Azure Cloud have made it easy to develop new ideas quickly. This quote in an article by CIO.com attributed to Amanda Hendley, managing director and president of Computer Measurement Group Inc. frames it perfectly:
“A hybrid cloud infrastructure and new guidelines for the development teams accelerated the path to innovation,” she continued. “For instance, teams were able to take ownership of their work, try out new technology features, and deploy faster with heightened security.”
The consumption of technologies from Public Cloud allows application developers to experiment with features that might otherwise require for instance, specialized hardware, lots of effort and time to bring up an underpinning technology to perform adequately, multiple global locations or large quantities of data. These are made available now to everyone to experiment with and innovate in what is being termed the Democratization of AI for example, in the case of making Artificial Intelligence (AI) more accessible. Many Public Cloud providers also offer generous free tiers with usage beyond that being charged by units of consumption.
A great parallel to equate the phenomena of Public Cloud is the Public Transport system. It would be cost prohibitive for many residential developments, businesses, and small cities to create a always accessible public transport system from scratch for their own use. It would also be an immense waste of resources and funds, and duplication for many small institutions to do this in isolation from other institutions. However, when cities, businesses and residential developments come together to fund, create and utilize one public transport system, it creates for a feature rich and cost effective transport system with each user being charged on the unit of that system that they consume, for example, time, trip numbers, or trip length.
The methods required to derive information from massive quantities of data may require specialized algorithms and specialized hardware in addition to expertise and time to implement. IoT has great potential – as it is proving in many cases – to be the supplier of rich data in massive quantities. Public Cloud is helping marry this data with the expertise and specialization required to extract actionable insights and information.
What’s Next?
Whereas it might seem that this is the last stage in an IoT solution, we posit that it is a cycle. We call it the Ingest, Analyze and Engage cycle. Businesses will ingest data, analyze that data, engage with their infrastructure and customers using actionable insights and information leading to improvement in their Key Performance Indicators (KPIs). This cycle will then repeat till it reaches a peak when the process helps them stay on the ball. It’s constant improvement by constantly monitoring the pulse of your business interactions. More on that later.
About the author:
As Staff Architect in Solutions Development, Professional Services Engineering at VMware, Neeraj Arora leads the development of service offerings for IoT, Edge Computing and NFV. Previously, Neeraj was part of VMware Professional Services field organization delivering integrations to Fortune 500 companies using VMware and non-VMware products. Industry experience includes gaming, utilities, healthcare, communications, finance, manufacturing, education and government sectors. Neeraj has published research papers in the areas of Search Engines, Standards Compliance, and use of Computer Science in Medicine.
