How do you earn your chair on the Carnegie Hall stage? Practice, practice, practice.
How do you get a network that you can trust? Test, test, test.
There is a fundamental shift in the way mobile networks are deployed, and communication service providers (CSPs) must adapt their design and deployment models to accommodate this change.
Traditionally, CSPs have used end-to-end solutions provided by a single vendor. Single-vendor solutions often cause CSPs to miss out on advances in technology, prevent CSPs from selecting best-of-breed components and typically cost CSPs more money than multi-vendor solutions. However, single-vendor solutions do have one key advantage: well-defined interoperability and performance. Network equipment manufacturers spend lots of time ensuring all their components interoperate. By using hardware-based components, vendors can confidently predict the network capacity of each component. As CSPs move toward a software-based network solution using elements from multiple vendors and running on off-the-shelf hardware, CSPs cannot deploy new network elements as freely as they used to without doing proper testing first. To learn more about this design shift be sure to check out our previous blog.
Optimize for each network function role
Network functions (NFs) are the key building blocks of a 5G network. Some of the NFs of today’s network include the Authentication Server Function (AUSF), the Access and Mobility Management Function (AMF), the Data Network (DN) and the Policy Control Function (PCF). Each NF has specific interoperability requirements, as well as different types of protocols, traffic patterns and workloads each must handle. Before deploying NFs into a production network, CSPs must conduct proper testing to be sure that the NF is running on optimized hardware, to ensure that each NF is configured for optimum performance and to verify that the NF can handle real-world traffic patterns and loads.
To confirm these key elements, there are four types of testing CSPs must run on each NF individually and as a system prior to a wide-scale rollout. The key test types are:
- Platform testing
- Functional testing
- Performance and benchmarking
- Use case testing
When performing these different tests, an engineer must take several steps to validate the NFs’ capabilities to interact with each other and to handle production workloads. Proper testing allows for network design optimization, tweaking and the ability to see how different hardware handles various workloads and performs for each NF type. Not all hardware/software combinations will support each NF equally well, so it’s important to test different configurations to achieve optimum performance.
Before deploying an NF in a production environment, it’s important to first understand how each VNC/CNF behaves with the infrastructure. One must also understand what other network elements the NF will communicate with to fully understand the types and amounts of traffic the NF must process. Finally, one must also decide if the NF will be deployed on virtual machines to make it a virtual network function (VNF) or in a container to make it a cloud-native network function (CNF). Once all of these elements are understood, testing can validate the design decisions and allow CSPs to make optimizations before deploying in the real world.
Platform Testing
Platform testing determines the behavior of the hardware platform (e.g. cloud/hypervisor) used to deploy an NF and how to best configure the platform for a specific NF type. The goal is to verify the capability of the platform that provides resources to the NF, such as CPU, memory, network and storage. Performance testing measures the impact of the NF on the platform resources.
While performing platform testing, it’s important to identify all the platforms the NF will support and to test each type. Ideally, one can test the platform before NF deployment to establish a baseline. For example, get the performance metrics of vSwitch, PCI passthrough and SR-IOV throughput benchmarking for the platform. This will help create a better design for NF validation and avoid surprises.
A few other key elements to check during platform testing include:
- NF lifecycle management covers VNF/CNF instantiation, configuration, update/upgrades, scale-out/in, scale-up/down, update/upgrades, NF terminate and NF heal.
- Record compute resource utilization like CPU, memory and hard disk utilization before/during/after sending traffic to VNF/CNF.
- Test NF functionality when compute resources are highly utilized. This will be a valuable input to the cloud monitoring systems and help ensure acceptable resource utilization. For example, develop scripts that will bring compute utilization to 90% and check the NF functionality.
- Look at server hardware. Monitor the vCPU performance of the NF on different cloud platforms (or hypervisors) and different server hardware.
- If possible, test the NF while the L3 cache is under a stress test.
Functional testing
The goal of functional testing is to verify that a VNF/CNF functions properly under all possible load conditions and responds appropriately to different traffic types. Functional testing should cover the following points:
- Decide on the type of network traffic to test with. Ideally, simulate the type of network that will traverse/interact with the VNF/CNF. If you’re uncertain of what protocols/traffic to emulate, use an iMix that mimics a traditional mix of network traffic.
- Example: If a virtual router is deployed on the cloud platform, use a load generator that can emulate L3 traffic with fixed packet size, random packet size and iMix traffic.
- Get the requirements of the hardware for VNF/CNF from the VNF/CNF vendor/SPOC. This step is crucial because based on this, the engineer needs to decide the hardware.
- Example: Find out if this VNF/CNF is going to run with SR-IOV, passthrough devices or CPU pinning, etc.
- Once you decide on the traffic patterns, select a load generator that can create traffic specific to that VNF/CNF. There are common tools made by Spirent and Keysight (formerly known as Ixia) to help. If appropriate, use stateful traffic to create a more realistic traffic scenario.
- Test high availability at the application level and compute level.
Performance and benchmarking
Benchmarking plays a major role in certifying that the VNF/CNF will work under real-world and worst-case scenarios. Accomplish benchmark testing in the following manner:
- Decide the KPIs/metrics you’ll capture for a particular VNF/CNF. Sample KPIs include determining the acceptable packet loss, latency, jitter and throughput.
- Example: If vIMS or vEPC is the VNF, capture #subscribers, #sessions, #transaction and throughput.
- Test VNF/CNFs with performance enablers like DPDK, CPU pinning, NUMA and SR-IOV. These are the enablers most widely used in the telecom cloud.
- Example: Test virtual routers on compute node with DPDK and without DPDK. Results will help you to design the implementation of vRouter.
- Most of the packet core VNF/CNFs use CPU pinning. Make sure you consider this when you plan VNF/CNF validation. Also, test VNF/CNF on NO NUMA domains and shared CPU pinning.
- When possible, use an isolated setup and not a shared setup to avoid inconsistent results.
Use case testing
Use case testing plays a major role in the final step of user acceptance testing (UAT). With use case testing, the focus is on real-world scenarios to get the UAT. The engineer has to collaborate with the respective SPOCs to identify the real-time scenarios. Use case testing should cover the following:
- Create as realistic use cases and scenarios as possible.
- Example: If a user is validating a virtual load balancer, create a service chain scenario where the traffic will start from the end device, go to the firewall, then to the load balancer, and then to the end device with port mirror configurations.
- Derive use cases from current existing setups and potential future setups. This is especially important for interoperability testing. Make sure that you cover the integration of the VNF manager with the NFV manager along with multi-vendor VNF integrations as needed.
- Focus on north-south traffic and east-west traffic scenarios.
- Corner cases are not required but are nice to check if time permits. Start with the most widely used scenarios and work toward corner cases if your schedule allows.
If you’re ready to move to a 5G network and are not sure where to start, VMware Professional Services is eager to help you propel your transformation forward.
Trust and verify with confidence
With proper planning, deployment and testing, CSPs can leverage new design options with 5G. Moving to VNFs and CNFs allows CSPs to use commodity open-source hardware and not be locked into a single vendor. This open architecture allows CSPs to save money and expedite network rollout. However, multi-vendor networks do require CSPs to perform diligent testing to be sure everything works correctly. Fortunately, with a little planning and a proper test plan, CSPs can be confident they’re rolling out the best network possible to meet the demands of today’s customers.
