Analyzing traffic internal to Cisco data center fabric (VN-tag support)

The traffic internal to the Cisco data center fabric and containing VN-Tag can be analyzed by the NAM Probe without requiring additional configuration. It is important, however, to ensure that the monitoring NAM Probe taps into the most appropriate position within the data center fabric.

VN-tag in Cisco data center fabric

The term VN-Tag (Virtual NIC tag) relates to the Cisco technology used for tagging the traffic internal to the Cisco data center fabric: an additional Ethernet tag is inserted into the Ethernet frame and is used by the Nexus 5000/2000 combination and the Cisco UCS Fabric Interconnect/IO Module.

In effect, VN-Tag can be thought of, and is often referred to as a virtual cable. More precisely, it is an Interface Virtualization (IV) scheme that moves interfaces from a virtual machine to an external, or secondary, Ethernet switch and makes them virtual.

VN-Tag can be used to enable the connection for a virtual network interface card (vNIC) or a virtual host bus adapter (vHBA), hosted on a network interface virtualization adapter (NIV), to an upstream virtual Ethernet or virtual fiber channel port (FC) on an NIV-capable switch. If the server is not equipped with a NIV-capable adapter, VN-Tag can also be used to connect a physical port on a fabric extender (FEX) to an upstream virtual Ethernet port on the switch.

The implementation of the VN-Tag can be done either in hardware by a VN-Tag capable adapter, or in software by the hypervisor, and will be applied as traffic traverses from one of its virtual adapters.

Note

Any FEX in the path passes the traffic containing VN-Tag upstream to the switch terminating the other end of the IV. However, if a plain non-NIV adapter is connected to a fabric extender, the FEX will apply the VN-Tag. In this latter scenario, half of the connection is made using a physical cable (server-to-FEX), while the other half is virtual (FEX-to-switch).

The following diagrams show example scenarios in which VN-Tag is used:

Cisco UCS with VIC connecting directly to NIV-capable

Cisco UCS with VIC connecting directly to a NIV capable switch

Example of a generalized scenario showing the use of VN-Tag

A generalized scenario showing the use of VN-tag

Connecting a NAM Probe to Cisco data center fabric

Unless there are special limitations that need to be placed on the traffic being analyzed by the NAM Probe, in scenarios involving a FEX, connect the NAM Probe tap to the vPC link between the FEX and its peer upstream switch. This way you will be able to monitor the traffic for all of the hosts sharing the FEX.

While it may be possible to use a mirror port on a switch to extract the traffic that is free of VN-Tag, use this approach with caution as it can lead to performance problems by placing too large a load on the switch. Oversubscription, source port density, and load need to be considered plus any perceived mirror output load that is likely to exceed approximately 5 GB (destined for a 10 GB destination port) is not recommended. In this situation, to analyze the data center traffic, tap the NAM Probe onto the un-stripped traffic containing VN-Tag.

In the case of a single physical host (potentially containing many virtual servers), you can connect the NAM Probe to analyze the traffic between the host and an upstream NIV-capable switch or a fabric extender. To analyze traffic to/from all of the hosts connected to a given fabric extender, connect the NAM Probe between the FEX and its switch. Note that a FEX is, in effect, a line card for virtual connections (virtual cables) to all of the hosts that are connected to it. Connecting the tap on the vPC link between the FEX and the switch is the recommended option.

The following diagrams show example scenarios of connecting a NAM Probe into the UCS fabric, to monitor VN-Tagged traffic:

Monitoring traffic between a single Cisco UCS host with a VIC connected directly to an NIV-capable switch

Monitoring traffic between Cisco UCS host with VIC connected to NIV-capable switch

Monitoring a vPC link between FEX and Cisco UCS switch

Note that for the unlikely scenario of a single host being connected to a FEX, the same result can also be achieved by connecting the tap on the link between the host and the FEX. For a scenario with no FEX, you can connect the tap between the host and the switch.

When connecting the tap to the data center fabric, consider the following:

  • Monitoring Ethernet channel
    For the connection between the FEX and its switch, it is possible to use straight links or to group the links using Ethernet Channel. Both of the scenarios are supported by the NAM Probe.
  • Avoiding scenarios where nested VN-Tags may appear by monitoring only FEX vPC uplinks
    Monitoring of stacked VN-Tags is not supported by NAM Probe. Instead, connect the NAM Probe in such a way to avoid the possibility of nested tags. Nested VN-Tag occur when multiple UCS interconnects are port channels attached individually to host switches, for example, a Nexus 7x facing a Nexus 5x, as a part of a vPC domain. In this situation, the data coming in from the FEX can take different paths across the fabric and may have to pass between different, non-optimal interfaces, potentially resulting in stacked tags.
    As a best practice, we recommend that you install the tap point in a location between the FEX and bridge/switch to prevent potential stacked tags from appearing in the monitored data. The tags, if any, will exist only on the path that interconnects the bridges/switches in the vPC domain but not on the FEX uplinks.