NAM Probe sampling

What is sampling mode?

Sampling mode is a method by which the NAM Probe handles traffic volume that exceeds its capabilities. When faced with an overwhelming traffic volume, instead of degrading the quality of measurements of the monitoring software, the NAM Probe artificially limits the volume by not analyzing all client-server communications (all TCP/IP sessions between given pair).

How does sampling mode work?

Sampling mode is activated as soon as the monitoring software detects that the NAM Probe is dropping network packets. Once activated, sampling mode decreases the traffic volume to be analyzed by 5% until the NAM Probe stops dropping packets. The reduction of traffic volume is achieved by ignoring the observed traffic (ignoring whole client-server sessions). This method generally does not affect metrics based on averages, but statistics for volume are necessarily skewed as the amount of traffic analyzed is reduced.

Sampling mode remains active as long as the packet drops occur on the NAM Probe. Persistent packet drops downgrade (5% at a time) the analysis volume until it reaches a minimum of 5% of the total observed volume. This means that if your NAM Probe is continuously dropping packets, it will eventually analyze only 5% of observed traffic and 95% will be analyzed based on sampled data.

Figure 1. Single traffic spike with default NAM Probe sampling setting
While a single traffic spike will not cause persistent NAM Probe sampling, in the extreme case where your analyzed volume drops to 5%, the return to the full volume analysis may take a long time using the default 5% per hour recovery setting.

NAM Probe sampling single spike
NAM Probe sampling single spike

The NAM Probe remembers the volume of traffic it had to activate for each level of sampling. Once the traffic volume reduces, the NAM Probe will automatically lower the sampling level and eventually analyze all observed traffic again. If no changes in traffic volume and no packet drops occur for a certain time (1 hour by default), the NAM Probe will lower sampling by one step (5%). You can set the time for the volume analysis recovery by modifying the interval at which the sampling mode increases the analysis volume. This setting is located on the NAM Probe in the rtm.config file.

Change the default value of 3600 seconds (which is 1 hour) to the number of seconds you want between sampling mode increases.

 sampling.config.noDropTimePeriod=3600
Note

We recommend that you do not set this value lower than 900 seconds (which is 15 minutes).

Figure 2. Single traffic spike with custom NAM Probe sampling setting
An example of a single traffic spike and NAM Probe sampling set to 15 minutes. Once the traffic load returns to normal, and the NAM Probe stops dropping packets, NAM Probe sampling decreases while the analyzed traffic increases, both by 5% every 15 minutes until analyzed traffic reaches 100% and NAM Probe sampling becomes inactive.

NAM Probe sampling 15 minutes
NAM Probe sampling 15 minutes

When should I disable sampling?

The sampling mode is enabled by default however, it is not active. There is no need to disable NAM Probe sampling in most deployments monitoring and analyzing an acceptable traffic volume. It is beneficial to have sampling mode enabled in the event an unnoticed increase in traffic volume downgrades NAM Probe performance.

You might consider disabling sampling mode if your deployment experiences expected spikes in traffic volume. Such spikes (for example, a regularly scheduled backup tasks) can cause the NAM Probe to briefly drop packets, which will activate sampling mode, which in turn will bring down the analyzed traffic volume down by 5% for each second the NAM Probe is dropping packets. The recovery of analyzed traffic to 100% is, by default, scheduled to increase by 5% every 1 hour. So, for example, a scheduled backup occurring every hour and generating a traffic spike that causes sampling mode to possibly reduce the analyzed volume to 70% can have an ongoing effect because the NAM Probe never returns to full volume analysis before the next scheduled backup.

Note

Keep in mind that NAM Probe sampling, when active, ignores whole client-server sessions, which has a minor effect on volume-related statistics and an insignificant effect on averages. However, if you disable NAM Probe sampling, any network packets that are dropped are done so in an uncontrolled manner (without regard to sessions). This may have a significant effect on metrics related to errors, volume, and averages.

Figure 3. Scheduled traffic spikes with default NAM Probe sampling setting
If your scheduled traffic spikes occur on a regular basis and sampling mode does not recover in time, your NAM Probe will not operate at its full capability. If your monitoring profile does not require you to analyze the spike traffic, it might be beneficial to allow the NAM Probe to randomly drop packets during such spikes but maintain the proper analysis volume the rest of the time.

Here is an example of scheduled traffic spikes that cause persistent NAM Probe sampling. Such cases warrant either a custom NAM Probe sampling setting, or disabling NAM Probe sampling.

Scheduled_Spikes
Scheduled_Spikes

How do I read the NAM Probe performance data on NAM Server reports?

The NAM Probe performance data is presented on the NAM Probe global packet stats report on NAM Server. To examine your NAM Probe sampling to received packets ratio, hide all the metrics but leave the RX packets and Dropped (sampling) visible. The RX packets is the volume of packets received by the driver and Dropped (sampling) is the volume of packets dropped that were sampled. The higher Dropped (sampling) volume, the more your RX packets is sampled. You can review your NAM Probe sampling for a specific date by adjusting the Resolution and Time range.