Configuring virtual NAM Probes

When configuring a virtual machine for a Virtual NAM Probe, the virtual machine should be configured for one or two vCPUs and each vCPU should be configured with a reservation setting equal to the clock speed of the physical core.

The idea is to use full cores and prevent other virtual machines from accessing the cores used by the Virtual NAM Probe. The virtual machine's memory should be configured for 8 GB in total. A reservation of 6 GB and a limit of 8 GB should be sufficient. If the Virtual NAM Probe is being deployed on a dedicated host, memory values can be adjusted according to need.

Configuring the virtual machine for one or two vCPUs

The number of vCPUs should not exceed two. The reason for this is that when a dual vCPU virtual machine places a request for cycles, that request goes to the host and is placed in a queue where it is held until at least two cores (or hyperthreads) become available. If the host is exhibiting CPU contention, the more vCPUs you have allocated to your virtual machine, the longer the virtual machine may have to be held before the total number of cores become available and the request for cycles is granted.

Adding virtual adapters

A number of virtual adapters will have to be added to the virtual machine. One will be used for standard communications and the rest will be used for packet monitoring (one per virtual switch). The recommended virtual adapter is the VMXnet3 adapter. All packet monitoring adapters need to be assigned to a dedicated port group that will require promiscuous mode set to accept and VLAN ID set to 4095 . This will allow the NAM Probe to see the entire traffic on the switch for all VLANs that are present. Placing the virtual switch in promiscuous mode is not advised since this will make all packets visible to all virtual machines on the switch. This is not only a security risk, but it also places a large load on the server because it needs to make multiple copies of all packets available to the virtual machines.

Caution:

A complete NAM Probe installation consists of a supported version of Red Hat Enterprise Linux installed on either a physical server configured as recommended or virtual machine with the NAM Probe application installed on top of that platform. For more information, see Operating Systems and Databases supported by DCRUM.

Be aware that the VMXnet3 adapter requires the use of libpcap for packet capture, which results in some performance degradation. Also be aware that the VMXnet3 adapter is geared for efficiency and, to that point, uses very large packets when communicating with other VMXnet3 adapters. These packet sizes can reach up to 64 KB in size. Unfortunately, the NAM Probe is only capable of handling packets as large as 16 KB. To handle anything greater would require even more memory to be added to the receive buffers, resulting in an increased memory footprint that would be undesirable in a virtual environment. The technique that creates this situation is called Large Receive Offload (LRO). VMware has the ability to accommodate this situation by providing a parameter setting that disables LRO. Large aggregated packets are then segmented down to standard packet sizes, allowing the NAM Probe to process them.

Setting network performance parameters

Finally, because the packets are traveling at the speed of the server backplane, more memory must be allocated to the receive buffers than would normally be required when receiving data over a traditional LAN. In this case, network performance parameters net.core.rmem_default and net.core.rmem_max need to be adjusted to accommodate the faster throughput. Otherwise, these buffers will be easily overrun and packets will be dropped. Testing shows that these parameters should be set to 16000000 . The following scripts disable LRO and set the network performance parameters. These scripts can run after reboot or added to the contents of /etc/rc.local .

 # disable LRO
rmmod vmxnet3
modprobe vmxnet3 disable_lro=1

# set kernel network buffer
sysctl -w net.core.rmem_max=16000000
sysctl -w net.core.rmem_default=16000000

To check whether the script was executed, run:

 >	sysctl -q net.core.rmem_max

If the command returns 16000000, the parameters are set properly.

For more information, see Preparing Virtual NAM Probe machine.