• Home
  • Deploy Dynatrace
  • Dynatrace OneAgent
  • Installation and operation
  • Dynatrace for z/OS
  • z/OS installation overview
  • Install the CICS code module

Install the CICS code module

With the CICS code module, you can get observability for your CICS transactions and programs including IBM MQ and database calls.

Observability forIncluding

CICS transactions

Transactions initiated using

  • IBM MQ Bridge and Trigger Monitor
  • CICS Transaction Gateway, HTTP/S, SOAP over HTTP/S, JSON using non-Java JSON pipeline
  • 3270 terminal

CICS programs

  • Programs invoked using CICS LINK
  • Transaction details for DPL LINK and START TRANSACTION requests in a distributed trace

Database calls

Database calls with their SQL statements from CICS to DB2 and IMS DB via the DL/I access method

File access

File access from CICS via the VSAM and BDAM access methods.

Trace your CICS transactions end-to-end with Dynatrace

Analyze the performance of your transactions end-to-end using the Service flow.

CICS service flow

Use the PurePath distributed traces to drill down to the code level and to optimize your programs.

CICS code-level

Understand exceptions in the context of your transactions down to the DB2 database layer.

CICS distributed trace with exception

Detect CICS anomalies and isolate fault domains with Dynatrace

Davis® AI automatically pinpoints the root cause of problems and assesses their user impact so that you can prioritize mitigation strategies and reduce the mean time to repair.

CICS automatic fault domain isolation

Analyze failures with exception details in the context of transactions.

CICS failure analysis

Installation

The CICS code module includes a PLT program that initiates at CICS startup. This program uses hooks to instrument CICS terminal and application owning regions, creating events of interest. It forwards monitoring data to the zDC subsystem via shared buffers.

You need to install the CICS code module in every CICS region you want to monitor. To update your CICS code module without restarting the CICS region, see Update the CICS code module without region restart.

You need to add the z/OS Java code module to each CICS Transaction Gateway you want to monitor.

CICS library definition

You can dynamically add the load library as a CICS library definition in the CSD. The CEDA definition is in SZDTSAMP member CICRDO. You can find it in the example in the next section.

If you don't want to use the CICS Library definition, you need to add the following PDS (or its contents) to the DFHRPL concatenation:

plaintext
// DD DISP=SHR,DSN=<hlq>.SZDTLOAD

Where <hlq> is the high-level qualifier you set during Download z/OS product datasets.

Dynatrace CICS programs and transaction

The sample CICS definitions below install programs and a transaction that are required by the CICS code module. A copy of these definitions can be found in SZDTSAMP member CICRDO, which is provided for use with the batch RDO utility DFHCSDUP.

We recommend to use these transaction and group names, but you can change them in accordance with your installation policies. Coordinate the group name and group list name with your CICS administrator. Replace XYZLIST with the name of your group list (GRPLIST).

Show sample definition
plaintext
CEDA DEFINE LIBRARY(DYNATRA) DESCRIPTION(DYNATRACE LOAD Library) GROUP(DTA1) DSNAME01(<hlq>.SZDTLOAD) CEDA DEFINE TRANSACTION(DTAX) DESCRIPTION(DYNATRACE CODE MODULE UTILITY TRAN) GROUP(DTA1) PROGRAM(ZDTPLT) TASKDATAKEY(CICS) TASKDATALOC(ANY) CEDA DEFINE PROGRAM(ZDTPLT) DESCRIPTION(DYNATRACE CODE MODULE UTILITY PROG) GROUP(DTA1) DATALOCATION(ANY) EXECKEY(CICS) CEDA DEFINE PROGRAM(ZDTPLTSD) DESCRIPTION(DYNATRACE CODE MODULE SHUTDOWN PROG) GROUP(DTA1) DATALOCATION(ANY) EXECKEY(CICS) CEDA DEFINE PROGRAM(ZDTAGT68) DESCRIPTION(DYNATRACE CTS 5.1 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT69) DESCRIPTION(DYNATRACE CTS 5.2 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT70) DESCRIPTION(DYNATRACE CTS 5.3 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT71) DESCRIPTION(DYNATRACE CTS 5.4 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT72) DESCRIPTION(DYNATRACE CTS 5.5 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT73) DESCRIPTION(DYNATRACE CTS 5.6 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTAGT74) DESCRIPTION(DYNATRACE CTS 6.1 AGENT) GROUP(DTA1) CEDF(NO) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTSOAPH) DESCRIPTION(DYNATRACE SOAP HEADER PROG) GROUP(DTA1) EXECKEY(USER) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(ASSEMBLER) CEDA DEFINE PROGRAM(ZDTDC2) DESCRIPTION(COBOL2 PRIMER PROGRAM) GROUP(DTA1) EXECKEY(CICS) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(COBOL) CEDA DEFINE PROGRAM(ZDTDC2A) DESCRIPTION(COBOL2 PRIMER PROGRAM) GROUP(DTA1) EXECKEY(CICS) CONCURRENCY(THREADSAFE) DATALOCATION(ANY) LANGUAGE(COBOL) CEDA DEFINE TDQueue(ZDTQ) DESCRIPTION(DYNATRACE FOR DTAX Messages) GROUP(DTA1) TYPE(Extra) DAtabuffers(001) DDname(ZDTQ) Opentime(Deferred) Erroroption(Ignore) TYPEFile(Output) RecordSize(00136) BlockSize(00140) RECORDFormat(Variable) BLOCKFormat(Unblocked) DIsposition(Shr) CEDA INSTALL GROUP(DTA1) CEDA ADD GROUP(DTA1) LIST(XYZLIST)

CICS startup program list table

Add the CICS startup program (ZDTPLT) after the DFHDELIM entry in your PLTPI source code and assemble the table.

This step is optional for test installations because the DTAX transaction can be used instead to enable the code module after CICS initialization. We recommend that you place the ZDTPLT entry immediately before the TYPE=FINAL specification.

The JCL procedure DFHAUPLE in CICSHLQ.SDFHINST(DFHAUPLE) can be used to build the PLTPI table.

An example of the PLTPI table that contains the CICS startup program
plaintext
* * PLT USED TO SUPPORT DYNATRACE CODE MODULE INITIALIZATION * DFHPLT TYPE=INITIAL,SUFFIX=SI DFHPLT TYPE=ENTRY,PROGRAM=DFHDELIM * Other PLT startup programs here... DFHPLT TYPE=ENTRY,PROGRAM=ZDTPLT DFHPLT TYPE=FINAL END
Compuware Xpediter Global storage protection users

The PLT startup program (ZDTPLT) initializes the CICS code module's exit work area, which CICS obtains on its behalf. Products such as Compuware Xpediter/CICS may be configured to enforce strict storage access controls and may abend ZDTPLT and prevent the CICS code module from starting unless it is excluded from these controls. If you use the Xpediter/CICS global storage protection feature, add a monitor exceptions entry to the XDDBPINP DD in the CICS region JCL to exclude ZDTPLT*. For example:

plaintext
DBPA 17.02 TRAN=*,PROGRAM=ZDTPLT*,CSECT=*

CICS shutdown program list table

Add the CICS shutdown program (ZDTPLTSD) before the DFHDELIM entry in your PLTSD source code and assemble the table.

We recommend to place the ZDTPLTSD entry immediately after the TYPE=INITIAL specification.

The JCL procedure DFHAUPLE in CICSHLQ.SDFHINST(DFHAUPLE) can be used to build the PLTSD table.

An example of the PLTSD table that contains the CICS shutdown program
plaintext
* * PLT USED TO SUPPORT DYNATRACE CODE MODULE SHUTDOWN * DFHPLT TYPE=INITIAL,SUFFIX=SD DFHPLT TYPE=ENTRY,PROGRAM=ZDTPLTSD * Other PLT shutdown programs here... DFHPLT TYPE=ENTRY,PROGRAM=DFHDELIM DFHPLT TYPE=FINAL END

Connect CICS code module to a zDC subsystem

The PLT startup program (ZDTPLT) automatically connects to the default zDC subsystem at CICS region initialization.

If multiple zDCs subsystems are running, it connects to the zDC that specifies DEFAULT(YES), unless an INITPARM override parameter in the CICS SYSIN parameters specifies that it must connect to a zDC with a particular name:

plaintext
INITPARM=(ZDTPLT='MEPC,<option>'),

<option> sets the log level for the CICS code module; see Logging.

To verify the connectivity between the CICS code module and the zDC subsystem, send a ping message.

Customization

CICSPlex name grouping

You can group CICS regions belonging to the same CICSPlex into a single process group. To do so

  1. In the Dynatrace menu, go to Settings > Mainframe > Transaction monitoring.
  2. Turn on Group CICS regions that belong to the same CICSPlex.
  3. Add MASPLTWAIT(YES) to your LMAS parameter. It instructs the CICS region to wait for the CICSPlex to become available before proceeding. If the CICSPlex isn't available, the code module can't consider it.
  4. optional The MASINITTIME(10) timeout interal defaults to 10 minutes. You can customize it in the range of 5 minutes to 59 minutes.

If you enabled CICSPlex name grouping after the CICS region is up, you need to run the DTAX transaction DTAX DISABLE and DTAX ENABLE.

CICS web services support

The CICS code module can trace the CICS web services invoked through a SOAP request or a JSON request (non-Java JSON pipeline).

To trace the CICS web service provider programs that are invoked by DFHPITP from CICS SOAP pipelines or invoked by DFHPIJT from non-Java JSON pipelines, update the service provider pipeline definitions for the services that should be traced.

Add a <handler> stanza that invokes the supplied ZDTSOAPH header program.

xml
<?xml version="1.0" encoding="EBCDIC-CP-US"?> <provider_pipeline xmlns="http://www.ibm.com/software/htp/cics/pipeline"> <transport> <default_http_transport_handler_list> <handler> <program>ZDTSOAPH</program><handler_parameter_list/> </handler> </default_http_transport_handler_list> </transport> <service> <terminal_handler> <handler> <program>DFHPIJT</program><handler_parameter_list/> </handler> </terminal_handler> </service> </provider_pipeline>

You need to add a <headerprogram> stanza that invokes the supplied ZDTSOAPH header program.

For example, add the sample stanza below to the cics_soap_1.1_handler section after any existing header program sections.

xml
<headerprogram> <program_name>ZDTSOAPH</program_name> <namespace>*</namespace> <localname>*</localname> <mandatory>true</mandatory> </headerprogram>

To trace outbound SOAP requests that originate within CICS transactions that are traced by a CICS code module, add the previous <headerprogram> stanza to the service requester pipeline definitions of those SOAP services that should be traced. Outbound SOAP requests that occur within CICS transactions that aren't traced are ignored. However, tracing isn't limited to requests from SOAP programs that act as CICS SOAP service providers.

Route DTAX messages using TDQueue

To optionally route DTAX messages to Dynatrace TDQueue (Transient Data Queue), use the ZDTQ resource definition provided above in your CICRDO member.

By default, the queue remains closed through the OPENTIME(DEFERRED) attribute. You must manually open it by using the CEMT INQUIRE|SET TDQUEUE command. To default the queue to open at install time, use the OPENTIME(INITIAL) attribute.

Logging

You can control the CICS code module log level either by using the DTAX transaction or by specifying an optional INITPARM at CICS region startup.

plaintext
INITPARM=(ZDTPLT='MEPC,<Option>'),

<Option> sets the logging level for CICS code module. Accepted values are:

  1. FINE | F for fine logging. We recommend to enable it only when the CICS code module has difficulties during startup.
  2. INFO | I for info logging. This is the default.
  3. WARNING | W for warning messages logging.
  4. SEVERE | S for severe messages logging.
plaintext
Example: INITPARM=(ZDTPLT='MEPC,SEVERE'),

There are two different sets of CICS logs:

  • One set of messages occurs when the DTAX transaction issues the DISABLE and ENABLE commands. These messages are written to the CICS CSMT Transient Data Queue(usually written to MSGUSR). View these messages in the CICS job spool. DTAX also writes a set of messages to the CEEOUT SYSOUT statement when errors occur in the connection between the zDC and the DTAX transaction. View these messages in the CICS Job spool. As long as the DTAX transaction can connect to the zDC, it logs its messages to the zRemote.
  • The CICS code module monitoring transaction activity routes its log messages to the zDC, and subsequently to the zRemote. The log shows if any corrupted distributed traces, timeouts, or other errors occurred. You may also see some statistical information in these logs.

You can access the CICS logs via the zRemote logs.

Update without region restart

To update your CICS code module to a newer version without restarting the region

  1. Download z/OS product datasets and follow the instructions to get the newest version.
  2. Disable the DTAX transaction in the CICS region with the DISABLE command.
  3. Copy the CICS code modules in the SZDTLOAD dataset into the Dynatrace DFHRPL dataset defined to your CICS region.
  4. Use the CICS command CEMT I PROG(ZDT*) to display the CICS code modules. Use the SET PROG(ZDT*) NEWCOPY command to tell CICS a new version of each program will be used.
  5. Enter CICS command CEDA INSTALL GROUP(DTA1) to install the new code module to the CICS region.
  6. Enable the DTAX transaction with the ENABLE command. Verify that the new CICS code module version is displayed on the DTAX panel.

Troubleshooting

Verify the CICS code module group and resources are installed correctly
Note

The group name might be different, as well as the two-character suffix representing the CICS release of the code module (for example, CTS52 uses 69).

From the CICS region, look for these messages to validate the CICS code module resources have been defined:

plaintext
CICSAPPL Install for group XXXX has completed successfully. CICSAPPL OWNER CSSY Resource definition for ZDTAGT72 has been added. CICSAPPL OWNER CSSY Resource definition for ZDTDC2 has been added. CICSAPPL OWNER CSSY Resource definition for ZDTDC2A has been added. CICSAPPL OWNER CSSY Resource definition for ZDTPLT has been added. CICSAPPL OWNER CSSY Resource definition for ZDTPLTSD has been added. CICSAPPL OWNER CSSY Resource definition for ZDTSOAPH has been added. CICSAPPL OWNER CSSY TRANSACTION definition entry for DTAX has been added.
Verify the CICS code module PLT program is invoked

The CICS code module logs initialization messages to the zRemote log.

The zRemote log can be accessed from within Dynatrace, in the same manner as all the other code module log files. Look for log entries similar to the following:

plaintext
2019-05-09 20:19:11.789 UTC [d37f9842] info [native] Registering a pgi for the job: HVBAC021, host=10.30.220.41, groupId= f39f4801966aa7c7, pgir.groupInstanceID= fad6dee63cfd1522, hostID= 95c0bb0371704b8c, nodeID= fad6dee63cfd1522, groupName=HVBAC021, hostGroup=, processGroupType= 28 2019-05-09 20:19:11.789 UTC [d37f9842] info [native] Registered SubAgent[C021,51,32aa8d038887d1c9] with zDC[Z021,52], rc=true 2019-05-09 20:19:11.789 UTC [d37f9842] info [native] ASID[51], smfID[S0W1], sysid[C021], jobName[HVBAC021], subagentid[32aa8d038887d1c9] snaId[NETD .HVBAC021], CICS release 54 was successfully registered with zdc[52] using protocol version=7.2.0, allocator=pooled. 2019-05-09 20:19:13.789 UTC [d37f9842] info [native] ASID[52], smfID[S0W1], sysid[Z021], jobName[AFVBZ021] - ZDC955I - Dynatrace connection being processed ZDC-Job/ID:AFVBZ021/Z021. 2019-05-09 20:19:13.790 UTC [d37f9842] info [native] ASID[51], smfID[S0W1], sysid[C021], jobName[HVBAC021] - ZDTP008I - ZDTP008I ZDTAGT71. 2019-05-09 20:19:13.790 UTC [d37f9842] info [native] ASID[51], smfID[S0W1], sysid[C021], jobName[HVBAC021] - ZDTP020I - ZDTP020I Active Sensors: MQ DB2 DB2R SOAP CTG DB2Fetch DLI DLIR HTTP .
System under diagnosis (CICS region) could not connect to the zDC

Check the job log of the affected CICS regions for the following message, where yyyy is the subsystem ID of the zDC that the CICS region is trying to connect to. It might be blank if the CICS region is trying to connect to the default subsystem that is configured with the DEFAULT(YES) parameter. We recommend to simply search for the error message code.

plaintext
ZDTP004W zDC yyyy unavailable

Verify that the zDC with that subsystem ID is started. If so, try to issue the DTAX transaction command ENABLE to re-enable the connections.

Version incompatibility

Ensure that the CICS code module version is less than or equal to zRemote module version. Don't connect newer CICS code modules to older zRemote modules. Following is a sample message in the zRemote log when an CICS code module version is incompatible with the zRemote version.

plaintext
severe [native] CICS14CR1[asid = 108] is trying to initialize with an invalid protocol version number : x.xxx.xx