Monitoring XML operations

To monitor operations belonging to a software service that uses XML, you first need to arrange for the software service to be monitored, and then you need to define the actual transaction patterns and indicate the tags from which information will be extracted.

Skill level: advanced user

This functionality is suitable for expert users.

  • If you are new to software services, start with Software services for beginners before you come here.
  • If you want to monitor a well-known software service, start with Autodiscovered Software Services to see if your work has already been done for you.
  • If you find you still need to define your own software service, try to use the wizard or a template to walk you through the process. You can always use the manual screens to tweak a software service after you create it with the wizard. See Software Services for details.
  • See SOAP and XML monitoring for details specifically on SOAP and XML configuration.

The transaction patterns can be defined on the XML Operations tab while defining a rule for the software service.

The Request and Response tags are top-level XML tags for the transaction. Within those tags, you can specify tags from which various information can be extracted, such as operation ID or user ID. If you do not specify a regular expression for a particular item, the entire contents of the tag is extracted, but not the tags themselves. If you specify a regular expression, the regular   expression is applied to the entire markup, that is the contents of the tags as well as the tags themselves, plus any potential tag parameters.

Note

In addition to extracting the user ID and operation name from lower-level tags, you can also extract these from the top-level Request tag. To do so, repeat the top-level tag name in the appropriate entry field - Operation User ID (n) or Operation Name.

If it happens that the name of one of the lower-level tags is the same as the name of the top-level Request tag, the top-level tag will take precedence. That is, if you specify the top-level tag name in an Operation User ID (n) or Operation Name field, the operation ID or operation name will be extracted from the top-level tag and not from any lower-level tag of the same name.

The Occurrence counter:

Those sections of the configuration screen that refer to particular XML tags, are equipped with an Occurrence counter. This counter specifies which occurrence of the tag, within the parsed XML document, will be processed to obtain the relevant information.

Request and Response tags:

  • Tag
    The XML tag to be recognized. For an XML request, the tag parameter is required. For the XML response, this parameter is optional. If no response tag is supplied, it is assumed that the response tag is the same as the request tag.
  • Include tag
    A tag to be examined as a nested component of a parent tag, where the parent tag is defined as a Tag parameter. This include tag parameter is available only for the XML over MQ analyzer.
  • Regular expression
    A regular expression describing the pattern of the XML over MQ request or XML over MQ response. This parameter is available only for the XML over MQ analyzer.

Operation ID (request):

  • Tag
    Optional: For a request and response to match, the contents of the Operation ID tags have to agree. If ID tag information is not provided, requests and responses are matched based on the order in which they appear.
  • Regular expression
    Optional: A regular expression for forming the Operation ID by extracting portions of the Operation ID tag or its contents. See the description of the Operation ID tag option above.
    • If you use this option, the corresponding tag must also be included in the syntax.
    • The regular expression is applied to the entire tag: to the contents of the tag as well as to the tag name and any parameters present. For more information, see Regular expression fundamentals

For Operation response ID (response):

  • Tag
    Response operation ID tag. If Tag information is not provided, the content of the tag defined with idattribute will be used.
  • Regular expression
    Response operation ID tag regular expression. If Regular expression is empty, the regular expression defined by Tag for Operation ID is used.

Operation user ID (n) (request):

Note

For XML traffic over HTTP or HTTPS, if a user ID is not specified, the user ID is derived from the underlying protocol: it is derived from the HTTP Authorization tag or, if this property is not present, the ID is based on the HTTP user ID property.

  • Tag
    Optional: User ID tag name (this can also be the top-level Request tag name). The user ID as it appears on transaction monitoring reports can be constructed using the information derived from a named tag and its contents. You can use the Regular expression option to specify which portions of the tag or its contents to use. By default, if Regular expression is not specified, only the contents of the tag is used,not the tag itself or its attributes.

    Operation User ID tags are searched for only in operation requests.
    You can configure up to four scope tags (Operation user ID, Operation user ID (2), Operation user ID (3) and Operation user ID (4)) and corresponding regular expressions to extract part of the element content when creating user name recognition rules. The new configuration parts would be checked one by one until a match is found.

  • Regular expression
    Optional: Specifies a regular expression for forming the user ID by extracting the desired portions of the Operation user ID tag or its contents. See the description of the Operation user ID tag option above.

    • If you use this option, the corresponding tag must also be included in the syntax.
    • The regular expression is applied to the entire tag: to the contents of the tag as well as to the tag name and any parameters present. For more information, see Regular expression fundamentals

The tag communicationFunctionSnd specified in the Tag for Operation user ID is searched for first. If communicationFunctionSnd is found, the corresponding Regular expression for Operation user ID is applied and the resulting user identification is used to identify the transaction. If communicationFunctionSnd is not found, the MessageHeader configured in Tag for Operation user ID (2) is searched for. If it is found, Regular expression for Operation user ID (2) is applied and the resulting string is used to identify the transaction. If Tag is not found for Operation user ID and Operation user ID (2), user name identification fails and the corresponding transaction is reported without user identification.

Correlation ID (request):

  • Tag
    Name of the XML tag containing the ConversationId identifier. This tag is optional for standard XML operations but required for processing asynchronous XML operations. TheConversationIdparameter is the only parameter that links messages exchanged between different net nodes.

    To match asynchronous messages exchanged between different addresses (for example, client X, Server Y and client A, Server B), it is necessary to set up pairs of configuration parameters.

  • Regular expression
    Optional: Specifies a regular expression for forming the user ID by extracting the desired portions of the Correlation ID tag or its contents.

    • If you use this option, the corresponding tag must also be included in the syntax.
    • The regular expression is applied to the entire tag: to the contents of the tag as well as to the tag name and any parameters present. For more information, see Regular expression fundamentals

Operation name prefix:

  • Optional: The operation name prefix. It is not derived from the actual contents of the monitored operation, but is provided so you can introduce your own operation name prefix that will appear in the operation name on traffic monitoring reports. The operation name will be composed of the operation name prefix and the contents of operation name tags, if any such tags are defined and found in a given operation. Single-space characters are used as separators when composing the operation name out of the above components.

    The XML over HTTP analyzer can report on the URL as part of the operation name, but not the parameters. This analyzer can also report the SOAPAction value as part of the transaction name. Operation name prefix can be used with two variables:

    • Use $url to add the URL or a regular expression based on the URL, to the prefix of the operation name. To strip or filter parts of the URL, use the URL regular expression attribute to apply a regular expression to the URL.
    • Use $soapaction to insert the SOAPAction value (header content) to the prefix of the operation name. To strip or filter the SOAPAction value, use the SOAP header regular expression option to apply a regular expression. You can also create additional parts of the operation name based on entries created in the Operation name table by specifying tags and regular expressions which help you filter the contents of the tag.

URL regular expression:

  • The regular expression entered in this field will be applied to the URL string, before the latter is used as the URL value, when constructing the operation name prefix. This applies only to cases where $url is specified in the Operation name prefix field. For SOAP header regular expression:
  • Use this option to extract or filter text passed in the SOAPAction field in the HTTP header to replace the $soapaction variable in the operation name prefix. This mechanism can take effect only when the variable $soapaction is used as the value of the Operation name prefix parameter.

Operation name (request):

  • Tag
    The operation name as it appears on operation monitoring reports is constructed using the operation name prefix (if defined) and information derived from up to three operation name tags and their contents. (Here you can also specify the top-level Request tag name.)
    Operation name tags are searched for only in operation requests.
  • Regular expression
    Specifies a regular expression defining which portions of the tags or their contents to use. By default, if a regular expression is not defined, only the contents of the tags are used,notthe tags themselves or their attributes.

Operation errors (response):

Optional: Operation error tags and values. Patterns specified here are used to recognize operation errors. In each error definition, you should specify the Tag name. You can specify up to five different operation error types corresponding to five different patterns. Use the asterisk (*) wildcard character to classify a number of error responses into one type corresponding to one error number. For example, error definition ERROR_CODE=001 will match all codes that begin with 001 (including error responses such as <ERROR_CODE id="...">0015: @32kjngk34@@</ERROR_CODE>).

Operation error tags are searched for only in operation responses.

Note:

  • The error numbers you define here correspond to the XML operational error numbers reported by NAM Server reports. However, reports display errors per software service and not per operation. Errors from different operations but possessing the same error tag numbers are aggregated. Therefore, it is important to define error types so that aggregation produces meaningful results.
  • By default, different error types are shown on reports as XML Error 1, ..., 5 . The report server reports enable you to customize these default names to suit your particular software service.

Notes on using regular expressions in XML transaction name configuration:

  • A regular expression (regex) is always associated with an XML tag.
  • If a regex is not defined for a corresponding tag, the value returned is the tag content.
  • If a regex is defined, first the XML element is searched for and then regex rules are applied to it as a whole (the XML element's tag, element's attributes, and its value). A regular expression can help you find attribute values or can work on any combination of characters between (and including) the start tag and end tag.
  • By default, it is assumed that the POSIX Basic Regular Expressions standard is used. To use POSIX Extended Regular Expression syntax, you must enter the ${posix_extended} marker before the regular expression and for each attribute defined.

POSIX Extended Regular Expression used to search and extract user name

${posix_extended}##<[^>/]*From>[[:space:]]*<[^>/]*PartyId[^>]*>([[:alnum:][:punct:]]+)</[^>]*PartyId>[[:space:]]*</[^>]*From>##\1

Example of XML operation patterns

Consider the following example of XML traffic:

POST http://www.somecompany.com/index.html HTTP/1.0
User-Agent: Mozilla/Gecko [en] (WinNT; I)
Host: www.somecompany.com
Accept: image/gif, image/x-xbitmap, image/jpeg,, */*
Cookie: user=somecompany;

<?xml version="1.0" encoding="UTF-8" ?&gt;
<SaleRequest>
   <NewTransaction>
      <ClientInfo>
         <ClientName>Client 1</ClientName>
      </ClientInfo>
      <VendorInfo>
         <VendorName>Vendor 1</VendorName>
      </VendorInfo>
      <ProductDesc>H12QW1</ProductDesc>
      <Amount>23</Amount>
      <Price value="dollar">23</Price>
      <TransID>2005-07-26_0001</TransID>
      <CorrID>QWER12345-111</CorrID>
      <UserName>John Smith</UserName>
   </NewTransaction>
   <NewTransaction>
    <ClientInfo>
         <ClientName>Client 2</ClientName>
      </ClientInfo>
      <VendorInfo>
         <VendorName>Vendor 2</VendorName>
      </VendorInfo>
      <ProductDesc>H12QW2</ProductDesc>
      <Amount>81</Amount>
      <TransID>2005-07-26_0002</TransID>
      <CorrID>QWER12345-112</CorrID>
      <UserName>John Smith</UserName>
   </NewTransaction>
</SaleRequest>
HTTP/1.1 200 Ok
Date: Wed, 01 Dec 1999 16:00:37 GMT
Server: Apache/1.2.6
Last-Modified: Wed, 01 Dec 1999 16:00:37 GMT
ETag: \"350ca-6398-363ace72\"
Content-Type: text/html
<?xml version="1.0" encoding="UTF-8"?>
<SaleResponse>
   <NewTransaction>
      <ClientInfo>
         <ClientName>Client 1</ClientName>
      </ClientInfo>
      <VendorInfo>
         <VendorName>Vendor 1</VendorName>
      </VendorInfo>
      <UserName>John Smith</UserName>
      <TransID>2005-07-26_0001</TransID>
      <CorrID>QWER12345-111</CorrID>
      <TransStatus>OK</TransStatus>
      <TransRespCode>0xx232000</TransRespCode>
   </NewTransaction>
   <NewTransaction>
      <ClientInfo>
         <ClientName>Client 2</ClientName>
      </ClientInfo>
      <VendorInfo>
         <VendorName>Vendor 2</VendorName>
      </VendorInfo>
      <UserName>John Smith</UserName>
      <TransID>2005-07-26_0002</TransID>
      <CorrID>QWER12345-112</CorrID>
      <TransStatus>Error - not exist</TransStatus>
      <TransRespCode>0xx232250</TransRespCode>
   </NewTransaction>
</SaleResponse>

What will be reported:

  • For the Operation ID section, the tag TransID and regular expression 2005* will match both <TransID>2005-07-26_0001</TransID> and <TransID>2005-07-26_0002</TransID>. As a result, 2005-07-26_0001 and 2005-07-26_0002 will be reported.
  • For the Operation name prefix section, the operation name prefix XML will be added at the beginning of the operation name.
  • For the Operation user ID section, the tag UserName and an empty regular expression will match everything that is included in tag <UserName>, such as <UserName>John Smith</UserName>. As a result, John Smith will be reported.
  • For the Correlation ID section, the tag CorrID and regular expression QWER12345-* will match both <CorrID>QWER12345-111</CorrID> and <CorrID>QWER12345-112</CorrID>. As a result, QWER12345-111 and QWER12345-112 will be reported.
  • For the Operation name section, the tag ClientName and regular expression Client* will match both <ClientName>Client 1</ClientName> and <ClientName>Client 2</ClientName>. As a result, Client 1 and Client 2 will be reported.
  • For the Operation errors section:
    • For level1, the tag TransRespCode and wildcard 0xx232000 will match <TransRespCode>0xx232000</TransRespCode>.
    • For level2, the tag TransRespCode and wildcard 0xx2322* will match <TransRespCode>0xx232250</TransRespCode>.
      As a result, XML Errors (1) and XML Errors (2) will be reported.