Troubleshooting Distributed Firewall in NSX-V - How to check firewall rules for a VM

Troubleshooting Distributed Firewall in NSX-V

Blog reference:

https://docs.vmware.com/en/VMware-NSX-Data-Center-for-vSphere/6.4/com.vmware.nsx.troubleshooting.doc/GUID-20234847-3E7A-4FE8-AEE1-31FFB3652481.html

 In my earlier post on Microsegmentation, we referenced the below topology and we put the workloads for different tiers - Web, App and DB on the same NSX Logical Switch.
With NSX micro segmentation, firewall is applied at vnic level of each virtual machine.

Topology

Below firewall rules are configured on DFW

Notice from the output above that firewall rules are only being applied wherever required using Applied To field


One of the troubleshooting techniques when you are checking DFW issues is to look for the firewall rules being applied at vnic level of a specific VM.


There are two ways of checking the rules
1. From the CLI of host where VM resides.
2. From NSX Manager CLI

1. From the host CLI

 

Login to the host where the VM resides. In our case, the VM is on host esx-03a

We will first get the dvfilter name using below command.
Notice that we are looking for output relevant to the VM name - hr-web

With this, we have got the dvfilter name.
Much easier than checking from NSX Manager CLI as you will notice.
The dvfilter name is nic-70100-eth0-vmware-sfw.2

Now we can check the firewall rules corresponding to this dvfilter using command below.

Look carefully and you will observe that only firewall rules relevant to hr-web-01a have been applied to the vnic of this VM.

From the below pic, only rule number 1, rule number 2 and the default rule is applied to this VM.

Rule number 3 is not applied because that is related to app and db virtual machines.
Hence rule number 3 will be applied to the vnic of App and DB virtual machines.
Also notice that logging is enabled for these three rules.

Filter and check the default deny rule

Addr Sets used in DFW rules

We are keen on checking the address sets and this can be done using the command above.

2.  Let's also check the firewall rules applied to vnic of VM from NSX Manager CLI

For this, we will begin by checking the host where our VM resides. 

 

Notice from the output above that the VM resides on host esx-03a 

Let's also check the cluster corresponding to this VM

 Next up, we need the cluster ID to know the host ID 

We now need to find dvfilter for the VM using the host ID, host ID in our case is host-32

Now that we know the dvfilter, we can go ahead and check the rules applied to the vnic of VM

Notice from the output above that you are able to view the TCP rules applied to the vnic of VM hr-web-01a
Corresponding address sets are also listed.

==================================================================

Troubleshooting DFW using Vrealize Log Insight

For the integration of Vrealize Log Insight server along with NSX:

1. We need to forward logs from the hosts to Vrealize Log Insight VRLI server
2. Forward logs of NSX edges and DLR to Vrealize Log Insight Server
3. Forward logs of NSX Manager to VRLI server.
4. Forward logs of NSX controllers to VRLI server.

We will be sending ICMP traffic towards the web server. DFW is blocking this traffic.
We should see this event on the web console of VRLI server.

ICMP traffic is blocked by DFW as per configuration

HTTPS traffic towards the web server is allowed

 

VMware NSX Microsegmentation - Securing Collapsed Architectures

VMware NSX Microsegmentation - Securing Collapsed Architectures

As depicted in above topology, NSX-V Distributed Firewall feature is enabled.
And as shown in figure above, firewall is effectively applied at each vNic of virtual machine.

In this topology:

1. BGP is used as routing protocol
2. iBGP is used within NSX
3. eBGP is used between NSX edges and the physical ToR switches
4. Physical switches are Juniper QFX Virtual Chassis.
5. ToRs are advertising a default route towards NSX edges only. NSX domain uses this default route for egress traffic from the NSX domain.

A different logical switch is used for each of the three tiers viz. Web, App and Database.

In this post, we are going to showcase a key ability of NSX whereby it can impose a zero trust model and secure VMs from different tiers on the same logical switch. We are not creating a logical switch per tier in such a topology.
This new topology is as shown below.

Notice, there is a single logical switch to which Web, App and DB VMs are connected.

We will leverage Vrealize Network Insight to plan security here.

VMware vcenter server and VMware NSX manager both are added as data sources in Vrealize Network Insight.


Using the information presented to us by Vrealize Network Insight, we will populate rules on our NSX Distributed Firewall.

The IP addresses of the VMs connected to the logical switch are:
Web - 172.16.60.10
App - 172.16.60.11
DB - 172.16.60.12

Using flow queries on Vrealize Network Insight, we are able to fetch information related to destination ports.

Next, we are going to create security groups using dynamic membership criteria.
This is to showcase the ability of NSX to create dynamic security groups based on VM names.

In future, if they add more virtual machines with VM name as hr-web-02, then such a virtual machine will automatically be a part of the security group based on dynamic membership criteria as defined above.

We then define the rules on NSX Distributed Firewall.

Here, we are making sure that only required communication is allowed on the distributed firewall and everything else is blocked.
This can be seen by the default rule in pic above which is set to Block

We then verify that our rules are working as intended:

Ping from Web to App fails

We are able to connect to the web server and even though the three virtual machines belong to the same logical switch / subnet, the web server is unable to ping other machines on the same logical switch.
Effectively this is a zero trust model.


Further verification commands have been executed on Web and App VMs to illustrate that firewall rules are effective allowing only what is required and denying everything else.