The integration of software-defined architectures in 4G and 5G networks has fundamentally altered the telecommunications attack surface, shifting the defensive bottleneck from vulnerability discovery to remediation.

Transition to 5G infrastructure represents a migration to cloud-native network functions, from hardware-centric networks to software-defined networking (SDN) and network functions virtualization (NFV). While this allows telecom providers to scale rapidly and deploy updates flexibly, it significantly broadens the attack surface. Modern 5G networks rely heavily on colossal open-source codebases and legacy protocols. Historically, securing these environments depended on human-led penetration testing and limited static analysis tools, creating a severe asymmetry where malicious actors only needed to find a single flaw.

The Frontier AI Inflection: Why Communication Service Providers Should Care

Frontier AI models have permanently disrupted this dynamic, proving that AI can now autonomously hunt for and validate zero-day vulnerabilities at a scale previously thought impossible. These models raise the risk from traditional cyberattacks to intelligent, autonomous threats that can identify vulnerabilities, exploit APIs, and target cloud-native telecom networks at scale.

The window between discovery and weaponization — historically months — has collapsed to hours or eliminated entirely for whoever holds access. For 5G Mobile Core running 24/7 with near-zero acceptable downtime, this changes every assumption in the patch-management playbook.

Critically, AI models are dual-use. The same scaffolding that finds a flaw in a User Plane Function (UPF) can write the exploit for it.

Sophisticated adversaries actively exploit the complex supply chains and converged architectures of modern telecommunications. A recent U.S. congressional investigation into the “Salt Typhoon” hacking campaigns revealed that state-backed Chinese telecommunications firms continue to maintain deep footholds in the U.S. internet ecosystem. Despite previous regulatory bans, these entities can exploit retained hardware, data center access, and interconnection agreements to establish “trusted backdoors,” enabling the misrouting and interception of data on a massive scale.

Simultaneously, the danger of converged networks over cellular infrastructure was starkly demonstrated in a recently uncovered cyberattack in Poland. Attackers successfully breached a local heat and power plant’s industrial control systems by pivoting through a private cellular data network, originally accessed via compromised wind farm substations. By exploiting widespread cellular network misconfigurations that allowed unrestricted device-to-device communication, hackers seamlessly bridged the gap between telecommunications routing and critical physical infrastructure.

In response to this shifting threat landscape, security agencies are now explicitly urging operators to stop treating private cellular networks as inherently trusted infrastructure, mandating they apply the same rigorous security controls and zero-trust principles they would use for any public, internet-facing connection.

Together, these incidents illustrate that the shift to highly interconnected, software-defined networks not only accelerates the pace of vulnerability exploitation but also deeply entwines telecom security with the survival of critical national infrastructure.

In that sense 3GPP TS 33.501 defines five security domains: network access, network domain, user domain, application domain, and SBA security (3GPP TS 33.501). Key controls:

5G AKA and EAP-AKA mutual authentication with SUCI (encrypted Subscription Permanent Identifier) subscription concealment, defeating false-base-station identification and IMSI catchers (GSMA FS.40).
Paging is no longer derived from SUPI but from 5G-GUTI, preventing subscriber tracing (GSMA FS.40).
Within the RAN F1 interface, ENISA notes mandatory confidentiality, integrity, and replay protection for F1-C but only optional integrity for F1-U — an asymmetry that illustrates, though does not fully define, the broader CP/UP security gap (ENISA).
3GPP and ENISA guidelines dictate that vendors must construct full confidentiality, integrity, and replay protection capabilities for both control plane data (Xn-C) and user plane data (Xn-U) interfaces.
IPsec ESP + IKEv2 (certificate-based) required on N2/N3; DTLS support on N2 (ENISA).
SEPP on N32 for inter-PLMN CP; IPUPS (a UPF function) for GTP-U protection on N9 (3GPP TS 33.501).

5G Threats Layers

The control plane security protects decisions and signaling. The user plane security protects the traffic that flows once those decisions are made. AI must be applied to each with different data (data sensitivity), latency budgets, and failure modes.

Control-Plane Threat Surface

Because the control plane (CP) is now exposed via RESTful SBI APIs over HTTP/2 + TLS, it inherits all the attack classes of any API-driven microservice fabric as well as telecom-specific signaling attacks.

Signaling / DDoS storms: Registration and PDU-session establishment procedures can involve dozens of messages across NAS/NGAP/SBI and, when sessions are established or modified, Packet Forwarding Control Protocol (PFCP). An attacker inducing many such procedures with modified UEs exhausts AMF/SMF resources with modest effort, amplifying impact far beyond the injected traffic. Attach-request flooding and PFCP session attacks against the N4 interface are now well-characterized testbed attack classes.
N4 / PFCP manipulation: Session establishment and modification messages crossing the SMF–UPF N4 interface can be intercepted or forged, enabling session hijack or denial of service.
SBI / API abuse and the mTLS + OAuth2 trust model: NRF, PCF, AMF, SMF and other NFs expose internal SBI APIs. NEF is the primary northbound exposure point. Broken object-level authorization, token replay, or excessive-trust NEF integration can enable policy manipulation, slice reassignment, or unauthorized capability exposure.
Rogue / compromised gNB: A false base station attempts to force AKA outcomes or capture identifiers; Subscription Concealed Identifier (SUCI) mitigates identification but a compromised legitimate gNB can still inject Next Generation Application Protocol (NGAP) and manipulate the CP.
Slice isolation breaches: Cross-tenant resource or signaling leakage between slices undermines the slice-as-a-product guarantee.
Interconnect (N32): Roaming and inter-PLMN signaling remains a historically rich attack surface; the SEPP is the trust boundary.
Baseband protocol attacks: Differential testing of 5G basebands (e.g., 5GBaseChecker) shows real deviance in control-plane protocol state machines across vendors.

VMware Telco Cloud Platform and 5G Control-Plane Security Controls

When 5G core NFs (AMF, SMF, NRF, PCF, NEF, AUSF, UDM, NSSF, SEPP) run as VMs or CNFs on Broadcom’s Telco Cloud Platform, built on VMware Cloud Foundation (VCF). The platform itself becomes the first enforcement layer for control-plane zero-trust. The capabilities below are not add-on overlays; they are intrinsic to the hypervisor and orchestration fabric.

Hypervisor-native microsegmentation of every NF

VMware vDefend (the evolution of NSX Distributed Firewall, now plug-and-play add-on over VCF) enforces stateful L2-L7 firewalling directly in the hypervisor kernel, transparently at each VM’s vNIC — independent of guest OS, network topology, or whether workloads are on the same subnet or host. For the 5G control plane, this means each NF can be individually isolated in its own microsegment: the AMF only accepts NGAP from gNB address ranges; the NRF only accepts SBI calls from NF types registered in its allow-list; the SEPP sits in a DMZ security pool isolated from internal NFs.

VMware Telco Cloud Platform, powered by VCF, adds VPC-aware lateral security for per-tenant isolation and self-service microsegmentation where infrastructure teams set guardrails and application owners (or CNF management pipelines) create fine-grained DFW rules within them via APIs in DevOps CI/CD. This is the platform’s enforcement point for the zero-trust principle that 3GPP mTLS/OAuth2 specify at the application layer.

Hypervisor-level intrusion detection for SBI and signaling traffic

vDefend Advanced Threat Prevention (ATP) includes IDS/IPS, Network Traffic Analysis (NTA), Network Detection and Response (NDR), and Malware Prevention — all operating at the hypervisor level with no traffic hair-pinning. The NDR engine is a multi-context correlation system that aggregates detection signals into unified intrusion campaigns, and includes GenAI-assisted investigation (Intelligent Assist) for simplified triage. For 5G control plane defense, vDefend IDS/IPS inspects east-west traffic between NF VMs/CNFs at the hypervisor, providing flow-level and threat-detection telemetry for SBI paths. VMware Telco Cloud Platform, powered by VCF, extends this to unified lateral threat prevention for VMs and VKS VMware vSphere Kubernetes Service (VKS) workloads, meaning CNF pods receive the same IDS/IPS inspection as VM-based VNFs — one console, one policy model across VMs and containers.

Infrastructure-enforced slice isolation.

VMware Telco Cloud Platform, powered by VCF, through its components such as NSX enables security groups with dynamic membership criteria defined by tags and governed by policy, so VNFs and CNFs can be isolated into trust domains by risk and sensitivity level. For 5G slicing, this means each NFs can be placed in their own NSX security group with tag-based DFW rules that prevent cross-slice signaling leakage at the infrastructure layer, complementing the OAuth2 slice-scoped tokens at the SBA layer.

CNF lifecycle security, RBAC, and supply-chain protection

VMware Telco Cloud Automation orchestrates VNF/CNF deployment and lifecycle with RBAC for NFVO, VNFM, VNF Designer, and API roles in a multi-tenant environment, integrating with any VIM and Kubernetes. For CNFs, Broadcom’s Telco Cloud Platform, built on VMware Cloud Foundation (VCF), can be configured to operationalize supply-chain controls including: RBAC for trusted container image registries, vulnerability scanning, inspection against security benchmarks, automated security patching, and isolation/monitoring of CNF microservice communications. The OS provides a security-hardened Linux container host optimized for running CNFs on vSphere. This supply-chain protection is the infrastructure analog of mTLS certificate validation; only known, signed, and scanned artifacts run.

Management plane isolation and compliance

Broadcom’s Telco Cloud Platform, built on VCF management components, can be isolated from the virtualization fabric via microsegmentation. Element managers can be separated and blocked from communicating with one another to prevent man-in-the-middle attacks; access is secured with RBAC, least privilege, and separation of duties, with MFA for VMware ESX, vCenter, and VCF Automation. VMware Telco Cloud Platform supports a three-tiered risk management approach aligned to NIST mapping controls to compliance frameworks.

Encryption and secure boot

VMware Telco Cloud Platform, components such as vSphere and vSAN provide data-at-rest encryption driven by software policy with external KMS; ESX enforces secure boot with cryptographic attestation, hardware roots of trust, and only digitally signed code execution. This protects NF VM images and CNF container images at rest and ensures platform integrity at boot.

How Frontier AI Augments VMware Telco Cloud Platform Controls in the 5G Control Plane

VMware Telco Cloud Platform provides the enforcement and telemetry foundation; Frontier-class AI provides the reasoning that turns that telemetry into action. The integration is synergistic, not overlapping.

From detection to assessment

vDefend’s NDR engine can correlate multi-context detection signals into unified intrusion campaigns and includes GenAI-assisted investigation. Frontier extends this: where vDefend NDR flags an anomalous SBI call pattern (e.g., an SMF making unexpected NRF discovery queries outside its registered scope), Frontier AI models can ingest the NDR alert alongside the vDefend DFW flow logs, the NRF token-issuance logs, and the NF’s registered profile, then reasons across all of them to determine whether this represents a cross-service token attack scope-confusion exploit, a compromised SMF CNF, or a benign config change.

Signaling-storm triage with vDefend IDS/IPS telemetry

vDefend IDS/IPS at the hypervisor level can detect anomalous traffic patterns between NF VMs/CNFs. For a signaling-storm attack, Frontier AI models can ingest the vDefend IDS alerts alongside VMware Telco Cloud Platform analytics (which can correlate physical and virtual infrastructure). vDefend DFW can then be programmatically updated via its API to drop, isolate, or steer the offending flows; Frontier AI models can draft the DFW policy change, a human approves, and the enforcement happens at hypervisor speed without touching the network topology.

Slice isolation verification

NSX security groups and tags enforce slice isolation at the infrastructure layer; Frontier AI models can audit the DFW rule base for contradictions (e.g., a DFW rule that accidentally allows AMF-to-UPF traffic across slices) and verify that the NSX tag-based group membership matches the 3GPP S-NSSAI assignments in the NRF. This cross-layer verification — does the infrastructure policy match the SBA authorization policy? — is exactly the kind of reasoning a frontier AI model excels at and no single tool provides.

In-line CP IDS

Research has demonstrated an AI/ML intrusion-detection system embedded as a core NF in the control plane via the AMF’s SBI, inspecting IP and non-IP CIoT flows across both planes using Transformer-encoder architectures, aligned with 3GPP Release 17 — non-intrusive and compliant. vDefend IDS/IPS at the hypervisor layer and this NF-embedded IDS operate at different layers (infrastructure vs. SBA) and Frontier AI models sit above both as the reasoning/correlation layer.

User-Plane Threat Surface

The user plane (UP) is high-volume, latency-sensitive, and the natural target for volumetric attacks.

GTP-U DDoS and reflection: The UPF is a volumetric choke point; overwhelming it disrupts every session it serves. The GTP-based user plane is an explicitly identified threat vector, with N6-directed flooding shown to overwhelm UPF resources .
GTP-U-in-GTP-U / packet-crafting: CVE-2021-45462 demonstrated a single anomalous GTP-U packet crafted from a UE crashing the UPF, due to improper error checking in the packet core. GTP-U on N3 (port 2152) is the attack ingress.
Optional UP integrity: Because user-plane integrity (F1-U) is optional in the spec, modified-in-transit user traffic is a real risk where operators have not activated it.
Data exfiltration and C2: Once a PDU session exists to the DN, the UP is a general-purpose data path; DGA, beaconing, tunneling, and exfiltration traverse N6 just as on any internet link.
Edge threats: User-plane breakouts at the edge place UPFs and application workloads closer to (and sometimes co-located with) less-trusted environments.
Subscriber / application behavior anomalies: Compromised IoT fleets, SIM-farm signaling, and automated abuse all manifest as behavioral deviation in UP flows.

VMware Telco Cloud Platform and 5G User-Plane Security Controls

When the UPF runs as a VM or CNF on VMware Telco Cloud Platform, powered by VCF, the same hypervisor-native security fabric that protects the control plane extends to the user plane — but with different enforcement points and different performance requirements.

Microsegmentation of the UPF workloads

The UPF, whether deployed as a CNF on Telco Cloud Platform can be protected by vDefend Distributed Firewall (DFW) at the hypervisor level. DFW rules can isolate the UPF from non-UPF workloads, restrict N6 egress to authorized Data Network (DN) destinations, and isolate multi-access edge computing User Plane Function (UPF) instances from central network functions (NFs) — all without redesigning the physical network topology. vDefend can apply where UPF traffic traverses vSphere/NSX-managed vNICs or supported VKS or VM paths. To secure high-performance UPFs utilizing Enhanced Data Path (EDP), DPDK, or direct-path passthrough architectures, traditional kernel-based firewalls must be bypassed. Because these dataplanes copy packets directly from the network interface to user-space memory, enforcement and telemetry must move to the hardware, fabric, and application layers. VMware Telco Cloud Platform provides unified lateral threat prevention extends IDS/IPS inspection to VKS (Kubernetes) workloads, so UPF CNF pods on Telco Cloud Platform receive the same hypervisor-level threat prevention as VM-based UPFs.

IDS/IPS and NDR: User-plane threat detection at the hypervisor

vDefend ATP (IDS/IPS, NTA, NDR, Malware Prevention) can inspect traffic at the UPF VM/CNF’s vNIC without traffic hair-pinning. Network Detection and Response (NDR’s) multi-context correlation engine aggregates detection signals into unified intrusion campaigns. GenAI-assisted Intelligent Assist provides built-in triage, and integration with third-party SIEMs enables rapid threat detection across the broader security stack. For data exfiltration and detection, vDefend’s Network Traffic Analysis profiles east-west flows and identifies anomalous communication patterns — the same capability used for ransomware lateral-movement detection in enterprise VCF, applied to telecom user-plane traffic.

IDPS Turbo Mode high-performance threat prevention.

vDefend introduces IDPS Turbo Mode for high-performance threat prevention, addressing the user plane’s throughput requirements where traditional IDS/IPS would impose unacceptable latency. This is critical: the user plane cannot tolerate the inspection overhead that the control plane can, and Turbo Mode provides a hypervisor-native detection layer that scales to high-throughput UPF workloads.

Correlated physical/virtual observability

VMware Telco Cloud Platform components like VCF Operations provides analytics, network intelligence, correlating physical devices with virtual environments for rapid insight and automated actions. Combined with a telemetry and network observability tool, it can profile and monitor traffic segments, types, and destinations to recommend security rules, identify policy violations, and detect aberrant traffic between trust domains. This is the telemetry foundation for AI-driven user-plane analytics.

How Frontier AI Augments VMware Platform Controls in the 5G User Plane

The user plane demands a layered model: the hypervisor-native controls of VMware Telco Cloud Platform (vDefend DFW, IDS/IPS Turbo Mode, NDR) provide Tier 1 detection and enforcement; Frontier AI reasoning sits above as the Tier 2 assessment and correlation layer. No frontier LLM runs inline at 400 Gbps.

User-plane threat assessment: vDefend’s NDR engine can correlate multi-context detection signals into unified intrusion campaigns with GenAI-assisted investigation (Intelligent Assist). Frontier AI can extend this by ingesting vDefend NDR alerts alongside telemetry and cross-referencing with the control-plane event stream. For a GTP-U-in-GTP-U attempt, the model can confirm the packet structure matches a specific (like CVE CVE-2021-45462 family) and correlate with vDefend IDS/IPS signatures that fired on the anomalous GTP-U packet (where traffic traverses the managed path), identify the source from DFW flow metadata, and recommend a response — as a result a DFW policy update to drop/isolate the offending workload, a UPF/SMF policy action, or trigger NWDAF/GTP-aware sensor escalation.
Wire-speed ML screening — research-tier integration with platform telemetry: On VCF, vDefend IDPS Turbo Mode can provide the platform-native analog — hypervisor-level IDS/IPS at performance levels suitable for high-throughput UPF workloads.
Multi-interface unified detection with vDefend cross-plane correlation: On VMware Telco Cloud Platform, vDefend DFW can provide visibility into all east-west traffic between NF VMs/CNFs across across N4, N2, and N6 interfaces at the hypervisor level, so the unified IDS research architecture maps naturally onto vDefend’s existing flow visibility. Frontier AI sits above both, correlating a UP anomaly on N6 (detected by vDefend IDS) with a CP event on N4/N2 (from the CP IDS or vDefend NDR) — the cross-plane correlation a reasoning model needs.
CNF hardening: VMware Telco Cloud Platform enables CNF supply-chain security through features like trusted image registry, vulnerability scanning, and signed images provided by Harbor, and elevates the platform-level gate.
Edge threat response: For deployments where UPFs and application workloads share edge infrastructure, vDefend’s VPC-aware lateral security can provide per-tenant isolation at the edge, and Frontier AI can reason across the vDefend DFW flow logs to detect cross-tenant lateral movement attempts.

Securing 5G on VMware Telco Cloud Platform

The architectural strengths of VMware Telco Cloud Platform uniquely enable this autonomous, zero-day remediation for the 5G Core.

Multi-Tiered Isolation: Combining VMs and Kubernetes

Modern 5G Core networks rely on Cloud-Native Network Functions (CNFs) deployed on Kubernetes. While containers offer incredible agility, relying solely on bare-metal Kubernetes creates a flat security perimeter where a single container breakout could compromise the entire cluster.

VMware Telco Cloud Platform solves this by leveraging multiple layers of virtualization. It runs carrier-grade Kubernetes (vSphere Kubernetes Service) on top of a hardened VM hypervisor layer.

When the AI detects a vulnerability in a CNF image or the K8s orchestration layer, it can test and validate its patch within a completely sandboxed, hypervisor-isolated environment. Even if an AI-generated test exploit escapes the container, it remains trapped by the underlying VM, ensuring the testing phase never threatens the broader production network.

Cryptographic Multi-Tenancy for 5G

5G network slicing requires true multi-tenancy, where critical services may share the same physical infrastructure as consumer mobile traffic.

The multi-tenant resource pooling of VMware Telco Cloud Platform ensures strict, hardware-level isolation between these tenancies. As the AI continuously hunts for lateral movement vulnerabilities, it leverages the micro-segmentation and distributed firewall capabilities of VCF and vDefend. The platform allows the AI to instantly quarantine a compromised tenant slice at the hypervisor level while it compiles and deploys the necessary CNF patch, ensuring zero disruption to neighboring tenants.

Confidential Computing at the Hypervisor: Data-in-Use Protection for Both Planes

All the controls discussed thus far — microsegmentation, IDS/IPS, OAuth2, mTLS — protect data in transit and at rest. They do not protect data in use: subscriber credentials, authentication context, session keys, and subscriber payload resident in NF VM/CNF. Starting with 9.0, VCF supports VM-level confidential computing with AMD SEV-SNP and Intel TDX, both available on the same vSphere platform. Confidential computing mitigates this gap by encrypting memory and CPU state at the hardware level, reducing the ability of the hypervisor or co-resident workloads to read the workload’s data. VCF 9.1 adds a rewritten Virtual Machine Monitor running in deprivileged user mode and Supervisor Mode Access Prevention, further reducing the hypervisor’s attack surface. The control plane’s most sensitive assets are cryptographic materials and subscriber identity context — exactly the data that data-in-use encryption is designed to protect. The user plane’s data-in-use challenge is different. It is high-throughput, latency-sensitive, and handles subscriber payload (not just signaling metadata). Running the UPF as a CVM (SEV-SNP or TDX) helps ensure that subscriber payload is encrypted in DRAM, protecting against hypervisor-level snooping, memory scraping, and side-channel attacks from co-resident workloads.

Hypervisor EDR: Protection for Both Planes

Confidential computing protects data in use; vDefend protects data in transit at the vNIC. EDR protects the ESXi host itself — the hypervisor process, its file system, its running processes, and its network connections. VCF 9.1 introduces ESX EDR partner integration, enabling third-party EDR agents to run directly on the ESXi hypervisor. ESX EDR can detect the indicators of host compromise that precede NF VM attacks: unauthorized SSH sessions, rogue processes (e.g., cryptocurrency miners, reverse shells), unexpected network connections from the host to external IPs, modifications to ESXi system files, and suspicious host logins or host-management activity. For a host running control-plane AUSF/UDM (the authentication credential store), early detection of host compromise is critical. It is the difference between a contained incident and a full subscriber credential breach.

Automated Policy Control and Governance

Deploying AI-generated code directly into a live telecom environment introduces extreme risk if not tightly governed. A patch might fix a security flaw but accidentally violate a national data sovereignty law or break a 3GPP compliance standard.

VMware Telco Cloud Platform integrates advanced, declarative Policy Control via its Kubernetes Policy Manager (leveraging Open Policy Agent and Gatekeeper). Before any AI-generated patch or configuration change is deployed to a live cluster, it must pass through these immutable policy guardrails. The system ensures that the AI’s remediation complies with your organization’s required security posture, resource allocation limits, and zero-trust framework.

For VMware Telco Cloud Platform, this architecture changes the software lifecycle from reactive patching (waiting for a bug bounty report or an active breach) to the continuous, autonomous hardening of the Telco Cloud.

Conclusion

5G’s cloud-native decomposition exposed both the control plane and the user plane as independently attackable software surfaces, each producing huge telemetry data. Frontier AI models close an urgent gap with autonomous vulnerability discovery, cross-node alert correlation, and machine-speed assessment.

But the value is conditional. It depends on separating the two planes (CP for decisions and signaling; UP for subscriber traffic), layering fast narrow ML below a slower reasoning tier, isolating the model from raw subscriber data in secure enclaves, and gating high-blast-radius actions behind human or policy approval. It also depends on treating the model itself as a controlled, dual-use asset subject to export controls and containment risk.

AI does not replace 5G security controls. Applied correctly — plane-aware, layered, enclave-isolated, human-gated — it is the force multiplier that makes zero-trust, cloud-native 5G operations defensible against an adversary who increasingly has the same tools.

By combining the proactive intelligence of Frontier AI with the absolute isolation, multi-tenancy, and policy enforcement of VMware Telco Cloud Platform, telecom operators can securely patch and harden CNFs before they are ever pushed to live 5G cell sites.