ALL YOUR PACKET ARE BELONG TO US

If InfoSec ruled the world … well, OK, maybe not the world … if InfoSec ruled the data center network design, and if money was no object, we would probably have something like this. Every server in the data center directly connected to its own port on one massive firewall. Every packet sent from every server would be inspected against a stateful security policy before going anywhere. And every packet received by every server would pass one final policy check before hitting the server’s NIC receive buffer. The firewall wouldn’t care about the IP address of the servers, for the simple reason that it’s directly connected to every server. E.g. “The server on this port can talk to the server on that port, on TCP port X”. And if that wasn’t good enough, the firewall knows everything about the servers connected to it, and can create rules around a rich set of semantics. All of this with no performance penalty. That would be awesome, right?

Let’s pretend money was not the issue. How would you design this massive omnipresent data center firewall? I can think of three ways off hand.

1. 1. You design a monstrous power sucking stateful firewall chassis with thousands of line-rate ports. At this point it’s time to route a ghastly mess of cables from every server to this centralized mega firewall core chassis – but that’s somebody else’s problem. Oh, and don’t forget you’ll need two of those bad boys for “redundancy”. Your monster firewall is pretty freaking awesome at security, but only so-so at basic L2 and L3 networking. But so what — the network team can learn to like it or find a new job. And if you run out of ports … no worries; just wait another few years for a bigger chassis and do the rip/replace routine.
   2. You design a line rate stateful firewall ToR switch. Rip out the network team’s favorite ToR and put this one in its place. Tell them to stop throwing a fit and just deal with it. You’ll have hundreds of these ToR firewalls to manage and configure consistently. No problem … just let the network team re-apply for their jobs as firewall engineers.

Go ahead and pinch yourself now. This is nothing but a fantasy nightmare.

The interests of security often poorly translate into networking. Comprehensive security ~= Compromisednetworking.

What about design #3? More on that in a minute. (Hint: title of the post)

In the real world, rest assured we do have firewalls to provide some security. But this security is not ubiquitous, nor is it assured. Instead, we have firewalls (physical or virtual) hanging off the network somewhere catching steered packets – and we can only hope the network was configured correctly to steer the right traffic to the right policy.

In this post we’ll briefly review the physical and virtual firewall, followed by a discussion on the Distributed Firewall.

Physical Firewall

This is the firewall we’ve grown up with; the trusty old physical firewall appliance. It has some very specialized hardware that does network security really well – within certain performance specifications of packets/sessions per second and raw throughput. 10Gbps would be considered high end (and costly). There aren’t enough ports on the firewall to connect every server directly to it, so instead we hang it off the Core switch and tell the network team to figure out how to steer the traffic through it.

For east-west traffic between servers it’s a choke point. It can only go so fast – but don’t tell the network team that or they might start blaming the firewall for every little performance problem. Meanwhile, we’ll just throw more firewalls at the problem, hang them off more ports on the Core network switch, and tell the network team to figure out the traffic steering part all over again.

The physical firewall is hanging off the network somewhere catching packets – it’s not directly connected to the servers. Consequently, security policy is only as good as the information available in the packets; such as IP addresses and TCP/UDP port numbers. So we build our firewall security rules around that basic context. E.g. “This IP address can talk to that IP address on TCP port X”, and so on.

Deploying a new App? No problem! Just submit a change ticket to the InfoSec team describing the App you want to bring online, along with its list of server IP addresses and TCP/UDP port numbers. A few weeks later (after it’s been decided which physical firewall your App will be physically anchored to) the security rules for your App will be added to the existing 5000 lines of rules collecting dust. A few weeks after that, the network team will (hopefully) engineer the traffic steering to the right interfaces on the right firewall. And if the App is decommissioned later on? Don’t ask. The rules are never cleaned up because that’s too much work, and that would require another change ticket anyway.

Deploying a virtualized multi-tenant agile cloud with automated provisioning? OK, that might be a problem. The physical firewall doesn’t do a very good job at automation and multi-tenancy.

Maybe virtualized Apps need a virtual firewall?

Virtual Firewall

The virtual firewall is just like the trusty old physical firewall but only now it’s delivered as a virtual machine instead of a physical appliance. This might appeal to you if it was determined that automation and multi-tenancy are important. The rationale being that each App instance (or tenant) can deploy their own virtual firewalls in their own enclaves, and if the firewall is a virtual machine it can be deployed automatically like any other virtual machine. And performance of a virtual appliance is just as good as a physical.

The virtual firewall is, again, this choke point hanging off the network somewhere catching packets (this time on a vswitch somewhere). We still need to steer traffic through it, and it’s still an object that needs to be managed and cared for instance by instance. And the rule structure still revolves around the basic context of IP addresses – no real improvement there.

Deploying an App in a virtualized multi-tenant cloud? No problem! Just tell the App owner that security is now their problem, and point them to the URL where they can download the firewall virtual machine and user manuals. When one virtual firewall becomes too much of a traffic bottleneck, the App owner can just throw more virtual firewalls at the problem and figure out how to design, manage, and automate around that. If the App is decommissioned later on the virtual firewalls and their rules just go away.

Did you say the auditor is here to validate compliance? OK, that might be a problem. There are hundreds of Apps in this cloud, which means potentially hundreds of virtual firewalls floating around. Let’s hope the App owners deployed them correctly – if at all. And it might take some time to find that one virtual firewall we need to look at and grab its configuration – cross your fingers!

Maybe virtualized apps in a distributed infrastructure need a distributed firewall?

Distributed Firewall

As we’ve just discussed, the physical and virtual firewalls are really the same old firewall model, just in different form factors. The distributed firewall, on the other hand, is when the firewall is no longer a form factor at all, rather the “Firewall” is now embedded as-a-service in the programmable hypervisor kernel networking stack. All participating hypervisors collectively become one “Firewall”. Every virtual server is connected to a hypervisor. By consequence, in this model, every virtual server is directly connected to one omnipresent “Firewall”. A firewall that knows everything about those virtual servers. Sound familiar? ALL YOUR PACKET…

Every packet sent from a virtual machine can be inspected by the stateful firewall service present in the hypervisor kernel, before the packet even touches the network. Only after packets have been permitted through the stateful “Firewall”, they hit a network built with any standard networking equipment and architecture of choice. No need to steer traffic. No need to compromise the network design. And every packet delivered to a virtual machine can be inspected one last time by the same firewall service, at the destination hypervisor, before the packet finally hits the virtual NIC.

The distributed firewall security policy is centrally defined (via API) through a notion of logical “containers”. Workloads are then placed in containers programmatically, and the enforcement is programmed into the hypervisor kernels. The security rules are based on container membership – not based on IP addresses (although you can still do that if you want). For example, things in container “Web” can talk to container “App” on TCP port X, and so on. Of course this means the IP address of your virtual machines can change, without changing the security policy. And container membership can be static or dynamic, programmed to trigger on just about any arbitrary metadata about the workload; such as user identity, the presence of a virus, various operating system characteristics, etc. Thus, security policy is decoupled from IP addressing.

The distributed firewall is multi-tenant and capable by virtue of its presence in the hypervisor kernel. It’s attached directly to the workloads and follows them wherever they go. Just place the workload in its security container and you’re done. Every packet sent or received flows through a stateful “Firewall”, and the security policy follows the virtual machine when it migrates to another hypervisor. With a distributed firewall, traffic steering is completely removed from the process of implementing security policy – for the simple reason that it’s directly connected to every virtual machine. In other words, security is omnipresent, at the very first and last hop.

With security decoupled from both traffic steering and IP addressing, it becomes possible to consider more simplified, flatter application architectures. For example, Web and App “tiers” could be on the same network segment, separated by their container membership.

Deploying an App in a virtualized multi-tenant cloud? No problem! The App owner just describes the desired security from one group of virtual machines to another. No need to fumble around with virtual firewalls, and the infrastructure owner can enforce certain enterprise policies right at the first hop. And if the App is removed later on its associated security rules are removed along with it.

Bottlenecks? Choke points? Not anymore. The distributed “Firewall” performance scales out with the hypervisors. Meaning, the theoretical throughput of the distributed firewall is some calculation of (number of Hypervisor machines) * (Specs of each machine).

The auditor just showed up? No problem! Go to the central policy console and view everything in real time, in one place.

Despite all of its awesomeness, the distributed firewall is not the firewall to end all other firewalls. The slam dunk deployment will be for securing east-west traffic within and between tiers of a virtualized application, Web > App, etc. and virtual server (or desktop) access control. At the north most edge of an App architecture you might still want a traditional physical or virtual firewall providing an additional set of perimeter edge services, such as NAT, VPN, etc.

Distributed Firewall: Key Takeaways

• Directly connected to virtual machines
• No traffic steering
• Omnipresent security at the first and last hop
• Security implementation decoupled from network architecture
• Not a choke point
• Scale-out distributed kernel resident data plane
• No device to manage – Firewall as-a-service
• Centralized policy automated with APIs
• Security rules decoupled from IP addressing
• Workloads added to “containers”, with static or dynamic membership
• Security enforced based on container membership
• Ideal for east-west application traffic, and access control
• Stateful security that migrates with the workload
• May be combined with traditional physical or virtual firewalls in an overall solution

Cheers,
Brad

**Editors Note: This post originally appeared at bradhedlund.com.