Configuring URL monitoring

You can create named URL definitions to monitor specific URLs and you can specify URLs to be excluded from monitoring. You can also specify a virtual HTTP server to handle scenarios in which many websites reside under a single IP address.

Skill level: advanced user

This screen offers functionality suitable for expert users.

  • If you are new to software services, start with Software services for beginners before you come here.
  • If you want to monitor a well-known software service, start with Autodiscovered Software Services to see if your work has already been done for you.
  • If you find you still need to define your own software service, try to use the wizard or a template to walk you through the process. You can always use the manual screens to tweak a software service after you create it with the wizard. See Software Services for details.

Access

  1. In NAM Console ► Deployment ► Manage devices, select NAM Probe Configuration ► Open configuration.
  2. On the NAM Probe Configuration screen, select Software Services ► User-Defined Software Services.
  3. On the User-Defined Software Services screen, select the software service for which you want to edit dimensions, metrics, or attributes.
  4. Right-click in the Rules table to add or edit a rule.
  5. On the Edit Rule screen, open the URL Monitoring tab.

Configuration

It is assumed for this task that you have already created a user-defined software service for this protocol and have specified one or more rules containing the essential components such as the IP address and port of the software service to be monitored. For more information, see Configuring rules for user-defined software services.

Note

The order in which you arrange URLs is important. When adding several URLs of the same type, make sure that you arrange the definitions from the most specific to the most general, as the URLs are processed from top to bottom. Especially, if you add a specific excluded URL, make sure you place it before a more general monitored URL, otherwise the exclusion will be ignored.

To specify definitions for the monitoring of URLs for a user-defined HTTP software service, create or edit one or more URL definitions:

On the URL Monitoring tab, add or open a definition for a URL to be monitored or to be excluded from monitoring.

In the URL Definitions table, right-click and choose Add Monitored URL to create a new definition for monitoring URLs, Add Excluded URL to create a new definition for URLs excluded from monitoring, or Open to open an existing definition. The Configure Monitored URL or Configure Excluded URL window will open.

Select a URL type.

The option you select here determines the type of URL information that you will need to enter further down in the URL Definition section:

  • Virtual HTTP server
  • Static URL part
  • URL as regular expression

Enter a URL definition string.
The information you enter here depends on the URL type selection you made in the previous step.

Virtual HTTP server
This option refers to monitoring a host where many websites reside under a single IP address. Using a virtual HTTP server causes all of the reported pages that have no separate definitions to be aggregated into one record and reported together. This does not apply to those pages from the IP address that are defined separately in a monitoring configuration. Such individual definitions do not require that you select this option.

For example, a valid virtual HTTP server address is http://server.domain.com, without a trailing slash.

Static URL part

A fully qualified URL, containing the protocol to be used, the server to be contacted, and the file to be requested, such as http://server.domain.com/page .

This URL is added to the list of monitored URLs regardless of the limit of monitored URLs.

Ignore host port (HTTP Options tab)

This setting is per software service and is selected by default.

When Ignore host port is selected, NAM monitors all URLs without regard to port numbers. All number of operations observed for a specific URL (regardless of the port) are summed up. This is a recommended setting if you are using auto-learning.

When Ignore host port is cleared, NAM differentiates operations based on the URL and the port number. All reported URLs contain the port number, and operations are matched per URL:port combination.

Avoid inflating other operations count

Typically, in manually defined static URLs you are not required to include port numbers. If you clear the Ignore host port setting, NAM will attempt to match the observed operations to a URL:port definition. Since typical URL definitions do not include port numbers, all observed operations will be unmatched in regards to the URL definitions and counted as other operations .

To avoid counting all operations as other operations, when you clear the Ignore host port setting, edit all your Static URL part definitions for this software service to include port numbers (including common port numbers such as port 80 or 443).

URL as regular expression

An extended POSIX regular expression describing a set of URLs. For more information, see Regular expression fundamentals.

The syntax allows you to use parentheses “()” to select one or more sub-expressions (specific portions of the results). If this mechanism is used, only the specified portions are reported; if more than one portion is specified, the portions are concatenated.

Note

When using a regular expression to specify a set of URLs to monitor, you must:

  • Explicitly include the string “http:// ” or “https:// ” in the expression. Thus, for example, you should not start the expression with “.* ” and expect that the “http:// ” or “https:// ” strings will be assumed or resolved as a part of the regular expression.

  • The parentheses you use to select the part of the URL to be extracted must include the above strings “http:// ” or “https:// ” and they must also include the name of the host. However, the name of the host does not have to be provided explicitly, but can be resolved by the regular expression. Thus, for example, “(http://www.someserver.com/)report/(myreport) ” is correct, and so is “(http://.*/)report/(myreport) ”.

  • The regular expression must be constructed such that, after extracting the portions delimited by parentheses, the resulting string does not end with a slash character (“/”). This rule applies to all URLs except home pages (URLs consisting only of a protocol specification and a host name). Such URL specifications should end with a slash.

    For example (http://www.someserver.com/)report/(myreport)/abc is legal, but (http://www.someserver.com/)report/(myreport/)abc is not legal. Note also that a specification ending with (myreport/*) is not valid because it can be matched by a string ending with a slash, as the asterisk can match an empty string.

You can test the patterns that will be used by the NAM Probe using the Regular Expressions Test tool, which is activated after you click Test located next to the regular expression pattern field.

Example: Using a regular expression to specify monitored URLs

The use of parentheses in a regular expression is demonstrated in the following example:

(http://www.puternews.net/report)/[0-9]+,[0-9]+,[0-9]+

The above expression matches URLs such as:

  • http://www.puternews.net/report/12345,11111,11111
  • http://www.puternews.net/report/12345,22222,22222
  • http://www.puternews.net/report/12345,33333,33333 but only the bracketed portion (“http://www.puternews.net/report ”) will be reported.

Example: More complex example of using a regular expression to specify monitored URLs

The following is a more complex example that demonstrates concatenation of bracketed portions:

A site contains URLs of the form:

 http://www.mylife.fr/assurance/assurances/!ut/p/kcxml/
04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLN4o39w0BSYGYRiGB-
pFoYsamaEIG8Y4IEW99X4_83FT9AP2C3NDQiHJHRQDwwo2X/
delta/base64xml/L3dJdyEvUUd3QndNQSEvNElVRS82XzJfNVVN?WCM_GLOBAL_CONTEXT=/
assurance/wcm/connect/My Life.fr/Aide/Accueil Aide&WT.tz=1&WT.bh=12&WT.ul=en-us&WT.cd=32&WT.sr=
1400x1050&WT.jo=Yes&WT.ti=AssuranceRetraitePERP&WT.js=Yes&WT.jv=1.3&WT.fi=
Yes&WT.fv=3.0&WT.sp=@@SPLITVALUE@@

where only the part after “...wcm/connect/ ”, in this case My Life.fr/Aide/Accueil Aide, is relevant for differentiating this page from other pages of this site, the rest being session ID and various parameters.

If you use the following regular expression to define monitored URLs:

 (http://www.mylife.fr/)assurance/assurances/.*WCM_GLOBAL_CONTEXT=/
assurance/wcm/connect/([^&]*)

the reported URL for this page is:

 http://www.mylife.fr/My life.fr/Aide/Accueil Aide
Note

A large number of URLs defined by using regular expressions can have an adverse effect on the performance of the NAM Probe, because resolving regular expressions is processor-intensive.

If you are configuring excluded URLs, this step completes this particular definition. If you are defining monitored URLs, proceed to the next step.

Optional: Specify a page name. Not supported by HTTP Express.

Page names are not supported by HTTP Express analyzer.

Optional: Specify an operation time threshold. Not supported by HTTP Express.

Operation time threshold relates to the time a page takes to load. Operation loads above the threshold limit are considered to be slow. You can specify this limit separately for this URL definition, or you can use the value as defined for this entire monitoring rule in the Options tab.

To specify an individual value for this URL definition, clear the Inherit setting from the rule check box and select the value in the Operation Time Threshold threshold selection boxes.

To use the value defined for the entire monitoring rule, ensure that the Inherit setting from the rule check box is selected. For more information, see Configuring rules for user-defined software services.

Optional: Specify a server time threshold.

Server time threshold relates to the server time portion of an overall page load time. Server times above the threshold limit are considered to be slow due to the poor data center performance. You can specify this limit separately for this URL definition, or you can use the value as defined for this entire monitoring rule in the Options tab.

To specify an individual value for this URL definition, clear the Inherit setting from the rule check box and select the value in the Server Time Threshold threshold selection boxes.

To use the value defined for the entire monitoring rule, ensure that the Inherit setting from the rule check box is selected. For more information, see Configuring rules for user-defined software services.

Optional: Specify the data generation options. Not supported by HTTP Express.

To specify individual values for this URL definition, clear the Inherit setting from the rule check boxes. To use the value defined for the entire monitoring rule, ensure that the Inherit setting from the rule check box is selected. For more information, see Configuring rules for user-defined software services.

This controls the scope of data generated by the NAM Probe that is used in NAM Server reporting.

Data generation

Advanced Diagnostics on Demand Data

When controlling ADoD data generation, you can either disable it completely or decide on the depth of available data.

  • Disabled
    No ADoD data is generated.
  • Operation loads
    The NAM Probe will generate data enabling you to access essential operation-level information.
  • Operation loads and hits
    The NAM Probe will generate data enabling you to access a deep drilldown report that represents an HTTP page hit broken down into specific HTTP elements.
  • Operation loads, hits, and headers
    The NAM Probe will generate data enabling you to access even deeper drilldown information retrieved from related request and response headers for the hit.

Optional: Select additional options.

In the Options section, select or clear the desired options as required:

Report URL after redirect

Not supported by HTTP Express. This option causes addresses after the last redirection to be reported for redirected pages. By default, redirections are reported as addresses of the originating page, before redirection takes place. The final target page will be reported regardless of how many redirects are detected in between. To use the value defined for the entire monitoring rule, ensure that the Inherit setting from the rule check box is selected. For more information, see Configuring rules for user-defined software services.

Report Redirects as Page

Not supported by HTTP Express. You can configure page redirects as single regular pages and report them separately. This makes it possible to combine the redirects with the originating or target page (depending on the setting in Report URL after redirect). In this way, redirects can become operations and you can create transactions consisting of more than one step. The per-URL or per-URL parameter settings always take precedence.

Report analyzed HTTP Method

All methods of passing HTTP parameters, can be distinguished if this option is selected. To use the value defined for the entire monitoring rule, ensure that the Inherit setting from the rule check box is selected.

The All methods option allows for processing all detected HTTP methods including the WebDAV HTTP extension. The extended WebDAV methods automatically identified include:

  • PROPFIND
    Retrieves properties and a directory hierarchy of a remote system.
  • PROPPATCH
    Changes and deletes multiple properties is a single operation.
  • MKCOL
    Creates directories or collections.
  • COPY
    Copies a resource from one URI to another.
  • MOVE
    Moves a resource from one URI to another.
  • LOCK
    Puts a lock on a resource.
  • UNLOCK
    Removes a lock from a resource.
Note

Monitoring WebDAV software services requires a specific configuration options. In order to properly report a hit as a separate operation, you must define a URL with regex matching all URLs (http://.*) and content types.

For more information, see Configuring rules for user-defined software services.

Report long pages, incoming over many monitoring intervals

This option allows for reporting so-called long pages (pages that load continually). This type of page is used, for example, for providing constantly updated information such as stock market reports. There are a number of different techniques for providing this functionality, such as by using streaming objects or server PUSH. All pages to be treated as long pages must be specified explicitly.

Long pages are reported on reports, but no transaction-related information is included in reports. The only information collected for such pages are network metrics.

Report Only URL Part When Parameters Do Not Match

Select this option to cause this URL to be reported even if none of the parameter sets specified for the URL has been matched with the actual parameters seen in the monitored traffic. Parameters are defined in a separate configuration window.

Optional: Specify frameset recognition options. Not supported by HTTP Express.

NAM Probes are able to recognize HTML pages containing frames. The association between frames and their contents can be performed automatically, by analyzing the HTML tags: FRAMESET and IFRAME . In addition, the NAM Probe allows explicit definitions of framesets in cases where the actual contents of frames are loaded dynamically, for example, using JavaScript. This latter method of frame recognition is referred to as static .

Configure the following:

Enable or disable automatic multi-frame page recognition

The check box Enable Automatic Multi-Frame Page Recognition is available at the global level as well as for an individual service or URL. Service-specific settings take precedence over global settings and URL settings take precedence over service rule settings.

Subframe URI strict matching patterns

For the static frame recognition method, this option allows you to define subframe URIs. For a subframe to be recognized, you need to enter the entire and complete URI. If you have set URL cut method to nocut in global HTTP settings, you will also need to specify the parameters to match, otherwise enter the URI up to the cut point. For information on setting the cut method, see HTTP - Recognition and Parsing of URLs.

Subframe URI regex patterns

For the static frame recognition method, this option allows you to define subframe URIs as regular expressions. This method of defining URIs is very powerful and flexible, though you should bear in mind that regular expression pattern matching does use up more processing power than simple string matching, as in the case of subframe URI strict matching above. Also note that the cut method specified in global HTTP settings does not apply here and therefore the regular expression you enter should take into account URI parameters, if any are expected in the URI you wish to match.For more information, see Regular expression fundamentals.

Optional: Specify the monitored content types. Not supported by HTTP Express.

Optional: Define the custom metrics for the URL. Not supported by HTTP Express.

Optional: Define the operation attributes for the URL. Not supported by HTTP Express.

Optional: Configure the automatic page name recognition for the URL. Not supported by HTTP Express.

Optional: Define the end-of-page components for the URL Not supported by HTTP Express.

Optional: Configure the Correlation ID extraction for the asynchronous HTTP transactions. Not supported by HTTP Express.

Save or publish your changes.
For more information, see Configuring rules for user-defined software services.

What to do next

If you require URL recognition that includes parameter matching, you need to define parameter information for this URL definition.