IBM MQ

Learn how to use Dynatrace to trace IBM MQ messages and monitor IBM MQ.

  • The IBM MQ extension isn't required for tracing IBM MQ messages. Dynatrace provides out-of-the-box automatic detection and monitoring of IBM MQ queues and messages as described in the IBM MQ message tracing section below.
  • The IBM MQ extension is required for monitoring IBM MQ. See the IBM MQ monitoring section for details.

IBM MQ message tracing

Out of the box, Dynatrace automatically detects and monitors IBM MQ queues and messages in your applications.

  • Dynatrace OneAgent displays IBM MQs in the smartscape topology view.

  • All services that use an IBM MQ are listed in the Sends to queue and Receives from queue sections of the related service overview page.

  • As long as each message is sent and received within the same queue, you get an end-to-end view of each message trace.

  • Service-level backtraces can follow messages backward to their origins.

  • When you look at a single PurePath, you can understand the end-to-end flow of the message.

Tracing aliases, remote queues, and clusters

If your requests use aliases, remote queues, or cluster queues, your end-to-end traces will be incomplete without the new IBM MQ tracing functionality enabled. This results in two service requests that aren’t connected, one on the sending and one on the receiving side.

Dynatrace automatically discovers aliases, remote queues, and cluster queues for the queue manager that it monitors when you enable topology retrieval in your IBM MQ plugin and provide the API endpoint and token.

  1. Select Retrieve queue topology for improved transaction tracing.
  2. Enter the API endpoint and API token key.

In addition to the fully automatic discovery of these types of queues, you can also provide this information to Dynatrace via the IBM MQ tracing API. For tracing of these queues to work in all cases, you need to supply Dynatrace with the respective data.

Tracing messages to IBM IMS on the mainframe

Traces towards IBM IMS on the mainframe also require special attention. For technical reasons, Dynatrace needs to know which queues are direct entries into IMS.

Using the IBM MQ tracing API to provide this information is quite simple; tell Dynatrace the queueName and queueManagerName that represent entry points into IBM IMS.

This API allows you to fully automate your IBM MQ and IBM IMS monitoring within Dynatrace and leverage an end-to-end view, from your most modern microservices to your proven business logic in more traditional systems like the mainframe.

IBM MQ monitoring

You need the IBM MQ ActiveGate extension to monitor IBM MQ.

Getting started

To get started, contact a Dynatrace ONE Product Specialist. (Just click the chat button in the upper-right corner of the Dynatrace menu bar.) Dynatrace ONE will make sure you have what you need to monitor IBM MQ.

Prerequisites

The IBM MQ ActiveGate extension has the following requirements:

  • IBM MQ 8.0+ for Windows/Linux/AIX/MQ Appliance (z/OS and iSeries are not supported).
  • An Environment ActiveGate version 1.155+ that has the ActiveGate plugin module installed and isn't used for synthetic or mainframe monitoring.
    • One Environment ActiveGate can typically support 30-50 IBM MQ queue managers.
  • IBM MQ client installed on Environment ActiveGate.
  • A server-connection channel on the queue manager for communication with the plugin.

SSL-specific requirements

If you are using SSL, you also need:

  • GSKit

  • Two key repositories:

    • A key repository on the ActiveGate where the MQ Client resides.
    • A key repository on the MQ Server.

    where:

    • Each repository has the other’s public key certificate.
    • The certificate label of the MQ Server is ibmwebspheremq<queue_manager>, where <queue_manager> is your queue manager name, and the entire label is all lowercase.
    • The certificate label of the MQ Client on the ActiveGate is ibmwebspheremq<username>, where <username> is the user in which the Remote Plugin process runs, and the entire label is all lowercase.

    See the SSL configuration section for details.

IBM MQ client configuration

See the current version here. Alternatively, you can search the IBM website for the MQ client version download that matches your environment.

  • For Windows, install the MQ client as specified.
  • For Linux-based ActiveGates, just install MQSeriesRuntime.rpm, MQSeriesClient.rpm, and MQSeriesGSKit.rpm.
    1. Add the MQ client lib64 libraries to LD PATH by creating a new file in /etc/ld.so.conf.d/. The content of this file is only the path to the lib64 libraries, which is /opt/mqm/lib64 by default.
    2. Save and restart the LD configuration with sudo ldconfig.

IBM MQ connectivity to ActiveGate

The IBM MQ plugin extension must connect to the MQ server passing a user and password. The user:password pair from the Dynatrace UI is passed in an MQCSP structure block.

However, depending how the MQ server is configured, the MQ server may ignore the MQCSP block and instead authenticate the "UserID" passed, which is the user running the Remote Plugin Module process. If this is the case, that user must exist on the MQ server and must have permissions to access channels and queues.

You are also able to connect without a user or password, but only if your Queue Manager permits it and has properly configured channels (IDPWOS, IDPWLDAP, etc.). The user must have at least the following permissions: connect, display, browse, put, inquire, and change (only required if requesting Enqueue/Dequeue counts).

Extension installation

  1. Get the IBM MQ plugin file (custom.remote.python.ibmmq.zip) from your Dynatrace ONE Product Specialist via live chat. Don't rename the file.

  2. Unzip custom.remote.python.ibmmq.zip to the plugin_deployment directory of your ActiveGate host.

  3. If the resulting directory structure isn't

    .\plugin_deployment\custom.remote.python.ibmmq\  
    

    make the necessary changes.

  4. Restart the Dynatrace Remote Plugin Module service.

    • On Linux, restart the service using the following commands with admin rights:
      systemctl restart remotepluginmodule.service  
      
    • On Windows, run these two commands in a Command Prompt launched as Admin:
      sc stop "Dynatrace Remote Plugin Module"  
      sc start "Dynatrace Remote Plugin Module"  
      
  5. In Dynatrace, select Settings > Monitoring > Monitored technologies.

  6. Select Add new technology monitoring and then Add ActiveGate plugin.

  7. Select Upload plugin and upload custom.remote.python.ibmmq.zip.

  8. Enter the following information to connect to your IBM MQ Queue Manager.

    Setting Details
    Endpoint name Enter a meaningful endpoint name.
    User The user to authenticate against MQ Server. If you leave this blank, make sure Queue Manager and the Server-connection channel are configured to allow this.
    Password The user password.
    Comma-separated Queue Manager hosts:ports This is a connection name list so you can enter more than one host and IP address (with ports) for one queue manager only. Example: 192.168.55.180(1414), 192.168.55.181(1414), 192.168.55.182:1415, 192.168.55.183:1414
    Server-connection channel Channel for the plugin to communicate with the queue manager.
    Single queue manager Name of queue manager to collect data from. Only one queue manager per endpoint. You can add other endpoints to other queue managers.
    Channels List of channels to collect data for. You can use wildcards (*) and exclusions (-). Examples:
    • abc* will get all channels that start with "abc"
    • abc*, -mq.chan** will get all channels that start with "abc" and exclude all channels that start with "mq.chan"
    Comma separated queues A comma-separated list of queues to collect data for. You can use wildcards (*) and exclusions (-). Examples:
    • abc* will get all queues that start with "abc"
    • abc*, -amq* will get all queues that start with "abc" and exclude all queues that start with "amq"
    Listener channel Leave empty for none. Enter * for all.
    Path to key repository The path to the SSL key repository on the ActiveGate.
    • If you are not using SSL, leave it empty.
    • If you are using SSL:
      • Provide the path on the ActiveGate where the SSL key repository is located. The path must end in the key name minus the extension (for example, D:/Some/path/SSL/key).
      • Two key repositories must exist: one on the ActiveGate where the MQ Client resides, and the other on the IBM MQ server.
      • Each repository must have the other’s public key certificate.
      • The certificate label of the MQ Server must be ibmwebspheremq<queue_manager>(all lowercase).
      • The certificate label of the MQ Client on the ActiveGate must be ibmwebspheremq<username> (all lowercase).
      • The username is the account through which the Remote Plugin process runs the Remote Plugin process.
    Cipher spec Select the Cipher specification for encryption used in channel communication. Must match the Cipher specification of the Server-Connection channel configuration.
    Exclude SYSTEM Check this box if you want to automatically exclude all queues and channels that start with “SYSTEM.”
    Run Reset Statistics This will issue the RESET_STATS command to queues in order to collect Enqueue/Dequeue count metrics. This requires CHG permissions on queues.
    Retrieve queue topology for improved transaction tracing This checkbox enables a flag to use Dynatrace Configuration API to feed the MQ topology for local, alias, remote, and cluster queues. This way, application transaction tracing will have improved visibility to MQ calls.
    Cluster environment or tenant Your current cluster environment or tenant URL.
    API Token A Dynatrace API token with permissions for "Write configuration" and "Access problem and event feed, metrics, and topology".
    Name of group If the device is part of a cluster, enter the name here to group the devices in the GUI. This will group your devices in your Technologies view.
    ActiveGate Choose the ActiveGate where the plugin resides which will poll MQ Server every minute.

Installation troubleshooting

Also see Troubleshoot ActiveGate plugins.

Self-signed certificates

  1. Your keystore on your MQ server must have a certificate with the proper label. Your keystore can have any name, but the certificate label must be in this format:
    ibmwebspheremq<queue_manager_lower_case>
    where <queue_manager_lower_case> is the name of your queue manager in lowercase.

    Example: if your queue manager is QM_ORANGE, the certificate label must be ibmwebspheremqqm_orange

    • You can create a new keystore on your MQ server with the following command:
      /opt/mqm/bin/runmqakm -keydb -create -db /var/mqm/qmgrs/QM_ORANGE/ssl/qm_orange.kdb -pw changeit -type cms -stash

    • Then you can create the certificate in the keystore with the following command:
      /opt/mqm/bin/runmqakm -cert -create -db /var/mqm/qmgrs/QM_ORANGE/ssl/qm_orange.kdb -pw changeit -label ibmwebspheremqqm_orange -size 2048 -sigalg SHA512withRSA -san_dnsname qm_orange.dynatrace.com -dn "CN=qm_orange.dynatrace.com,OU=Extensions,O='Dynatrace',L='Detroit',ST=Michigan,C=US" -expire 1825

    • The filename can be anything, and the location of the keystore is up to you. The example below uses the default location, which reflects the same location in the Queue Manager properties page.

  2. Your keystore on you client side (where ActiveGate is, where your MQ client is installed) must have a certificate with the proper label. Your client keystore can have any name, but the certificate label must be in this format:
    ibmwebspheremq<user_running_plugin_process>
    where <user_running_plugin_process> is the operating system user that is running the Dynatrace Remote Plugin process/service (not the ActiveGate). For Windows, it's likely that "Local Service" is the account that runs it and you should change it to a real user (service account, for example). So if the user is svcdyn, the certificate label on the client must be ibmwebspheremqsvcdyn

    Create the client certificate in the client keystore:

    The username running the process must match the username in the client certificate:

  3. Both keystores (on the MQ server and on the MQ client where ActiveGate is) must be in CMS format (not JKS or PK12) and must contain, at minimum, .kdb, .rdb, and .sth files. The stashed password is STH, so be sure to stash the password when you create the keystore.

  4. The MQ client keystore must have the public key of the MQ server certificate imported, so export the certificate ibmwebspheremq<queue_manager> from the MQ server keystore and import it into the client keystore.

    • You can export the certificate from the MQ server keystore with the following command:

      /opt/mqm/bin/runmqakm -cert -extract -db /var/mqm/qmgrs/QM_ORANGE/ssl/qm_orange.kdb -pw changeit -label ibmwebspheremqqm_orange -target /var/mqm/qmgrs/QM_ORANGE/ssl/ibmwebspheremqqm_orange.arm -format ascii

    • Copy the extracted certificate file to the ActiveGate, then import the extracted certificate into the ibmmqkeystore.kdb keystore on the ActiveGate:

  5. The MQ server keystore must have the public key of the client certificate imported, so export the ibmwebspheremq<username> certificate from the client keystore and import it into the MQ server keystore.

    • Export the certificate from the ActiveGate keystore:

    • Copy the extracted client certificate to the MQ server and then import it with the following command:

      /opt/mqm/bin/runmqakm -cert -add -db /var/mqm/qmgrs/QM_ORANGE/ssl/qm_orange.kdb -pw changeit -label ibmwebspheremq**diego** -file /var/mqm/qmgrs/QM_ORANGE/ssl/client_cert.arm -format ascii

  6. Now your MQ server should have a keystore with its own certificate and the client certificate, and your ActiveGate should have its own keystore with its own certificate and the MQ server certificate.

  7. Make sure that your server-connection channel has the proper cipher spec selected and you select the same spec in the plugin configuration UI.

  8. If you're using a Peer Name (PNRP) on the server-connection channel (Distinguished Name), make sure the Distinguished Name exists in the CLIENT certificate (the one labeled ibmwebspheremq<user>).

  9. The path to the repository field on the plugin UI is the path to the client keystore, including the keystore name but not including the extension.

    Example: if the keystore is
    D:\ssl\clientkeystore.kdb
    the path to repository field must be
    D:\ssl\clientkeystore

CA-signed certificates

  1. Your MQ server and MQ client (ActiveGate) need to have separate CMS keystores. They must be CMS, not JKS or PK12 or other. The keystore names don't matter.

  2. Your MQ server keystore must have your CA root certificate imported as a "Signer Certificate".

  3. You must create a Certificate Signer Request (CSR) from your MQ server keystore because it writes a record in the .rdb file of your keystore. You can't create a CSR from elsewhere.

  4. Create a signed certificate using your Certificate Authority and your Signer Request. For OpenSSL, you could execute the following command:
    openssl x509 -req -in certcsr.arm -CA myCARoot.crt -CAkey myCARoot.key -CAcreateserial -out mysignedcert.crt -days 500 -sha256

  5. Import/Receive your signed certificate into your keystore in Personal Certificates. Make sure your certificate has the label (alias) ibmwebspheremq<queue_manager>, where <queue_manager> is your queue manager name in lowercase.

  6. Repeat steps 2-4 for your MQ client (ActiveGate) keystore.

    • CA root certificate import
    • CSR creation
    • Signed certificate creation
  7. Import/Receive your signed certificate into your keystore in Personal Certificates. Make sure your certificate has the label (alias) ibmwebspheremq<username>, where <username> is the user that runs your Remote Plugin Module process (not your ActiveGate process).

  8. Place all files for MQ server in the proper location and configure the Queue Manager to look for the keystore in that location.

  9. Configure your SSL server-connection channel to use the right cipher spec. They must match on both sides of the communication pipeline.

  10. For the MQ client:

    • Place the files in your location of choice.
    • In the configuration UI of the plugin, enter the path to the keystore WITHOUT the .kdb extension (enter /path/to/SSL/keystore/filename in the SSL Repository field without the .kdb extension).
    • Verify that the proper cipher spec is selected.
  11. Refresh your SSL configuration on your Queue Manager to pick up the new SSL changes to your Queue Manager.

Metrics

Queue Manager

  • Availability %
  • Connections
  • Active channels

Channel (split by channel)

  • Availability: Whether the channel is running or not.
  • Messages: Number of messages sent and received, including MQI calls for all channel instances.
  • Bytes Sent/Received: Number of bytes sent and received for all channel instances.
  • Buffers Sent/Received: Number of buffers sent and received for all channel instances.
  • Last Message Date/Time: Time last message was sent.
  • Current Sharing Conversations: Number of conversations being shared for this channel.

Queues (split by queue)

  • Queue Depth: Current queue depth value
  • Percent queue depth: Calculated using the Current Queue Depth over Max Queue Depth value.
  • Inhibit Get/Put: Property value denoting whether PUT and GET are allowed
  • Open Input/Output Count: Open input and output handles. IPPROCS/OPPROCS
  • Oldest Message Age: Time in seconds of oldest message.
  • Uncommitted Messages: Number of messages that have not been committed.
  • Enqueue/Dequeue Rate: Number of messages in queue that were PUT or RETRIEVED and not committed per second.
  • Time Indicator: Time messages stay in queue
  • Last get/Last put: Time in milliseconds when MQGET and MQPUT commands were executed on this queue.

Listener (split by listener)

  • Availability %