Customize the zDC subsystem
Learn how to customize the zDC subsystem depending on your needs.
z/OS MODIFY commands
The format of the z/OS MODIFY command is defined as follows:
MODIFY <jobName or stcProcName>,DT1 <command>
The zDC subsystem supports the following z/OS MODIFY commands.
Command | Description |
---|---|
DISPLAY | Displays Dynatrace subtask processor stats. |
DTM%V | Forces display (ZDC952) of the utilization of the zLocal message queue. |
LOG=? | Queries the log level of the zDC. |
LOG=3 | Sets the log level of the zDC. Log level values range from |
START | Starts the zLocal. |
STDO | Appends previously unreported lines from zLocal |
STOP | Stops the zLocal. |
ZLALOGLEVEL=INFO | Sets the log level of the zLocal. Log level values include |
The DISPLAY
command skips most rows with zero counters if the log level is at or higher than the default level of 3. To list the complete set of counters, you need to increase the log level to 1.
MODIFY MEPx,DT1 LOG=1
MODIFY MEPx,DT1 DISPLAY
A zDC sample output from DISPLAY
with LOG=1
.
14:13:33.854126 ZDC027I MODIFY COMMAND RECEIVED
14:13:33.857174 ZDC028I DT1 DISPLAY
14:13:33-977432 ZDC994I ShoS Counters
35:Msgs sent from MAgent to zDC using unnamed pipe
1:QUEUE_MAX_EVENTS(EQH)High watermark elements used
128:EQH queue elements defined
128:EQH elements currently free
0:Internal Performance:XmPosts avoided
0:Internal Performance:XmPost Skip DupSrbs
1,758:zDC Elapsed time (approx sec)
0: Bytes read from DTM Msgs
49:SMO Msgs written
34:Internal:Redundant Post bypassed
15:Internal:Attempt Rd=Wr cursor for vStorage Perf.
15:z/OS Unix has no ready Msgs, waiting for work
48:z/OS Unix Msgs read
3,808: Bytes read from SMO Msgs
2,267:TBC Total Transactional Buffers
2,267:TBC number in Free queue
25:TBC Msgs written
25:TBC Msgs read
10:TBC TBCs written
10:TBC TBCs read
10:TBC Post bypass
41:TBC No Ready Q
1,905:TBC Bytes written
1,905:TBC Bytes read
2:TBC Age<=PingDelay*.5
2:TBC Age<=PingDelay*.75
2:TBC Age<=PingDelay*1
2:TBC Age<=PingDelay*1.25
2:TBC Age>=PingDelay*1.25
10,485,760:DTMSG_SMOSIZE Bytes allocated
9,248:DTMSG_SMOSIZE Bytes used
14:13:36-712702 ZDC995W DispTcb: 009D1438 0.900sec
14:13:36-712792 ZDC995W DispTcb: 009BBAD0 0.577sec
14:13:36-712832 ZDC995W DispTcb: 009BBCF0 0.013sec
14:13:36-712882 ZDC995W DispTcb: 009BBE88 0.012sec
14:13:36-714022 ZDC995W DispTcb: 009BE170 0.924sec
14:13:36-714062 ZDC995W DispTcb: 009BE390 0.590sec
14:13:36-714112 ZDC995W DispTcb: 009BE528 0.251sec
Rows that end with an exclamation mark (!
) should have zero counts.
Multiple TCP/IP stacks on a LPAR
An LPAR with multiple TCP/IP stacks requires an additional configuration step in the ZDCMEPC
started task PROC from SZDTSAMP
.
The BPXTCAFF
program associates a specific TCP/IP stack with the current address space.
The PARM
value is the job name that starts the desired TCP/IP stack.
//MEPC PROC
...
//STEP0 EXEC PGM=BPXTCAFF,PARM='TCPIP_stack_name'
//*
//ZDCMAIN EXEC PGM=ZDCMAIN,REGION=0M,PARM='LANGUAGE=EN'
//STEPLIB DD DISP=SHR,DSN=hlq.SZDTAUTH
//* Notes:
//* SYSPRINT is required in all cases.
//* SYSPRINT must be allocated to spool, not disk, on JES2 systems.
//* SYSPRIN3 is required on JES3 systems only.
//SYSPRINT DD SYSOUT=*
//SYSPRIN3 DD SYSOUT=*
//SYSIN DD DISP=SHR,DSN=hlq.SZDTSAMP(ZDCSYSIN)
Multiple zDC/zRemote pairs on an LPAR
A zDC/zRemote pair connects the modules on a LPAR to a single Dynatrace environment. For example, it may be necessary to configure multiple zDC/zRemote pairs on an LPAR
- When you want to maintain different application groups in different Dynatrace environments.
- When you want to optimize your monitoring performance with high transaction volumes.
Install an additional zDC/zRemote pair
-
Install an additional zRemote module.
-
On your LPAR, copy the current JCL SYSIN member
ZDCSYSIN
to another name (for example,ZDCSYSI2
). -
Edit the new JCL SYSIN member by changing the following SYSIN parameters:
//SYSIN DD DISP=SHR,DSN=<hlq>.SZDTSAMP(ZDCSYSIN2) DTAGTCMD(zremoteagent=<ipaddress>[:port] name=zlocal_<lpar>) SUBSYSTEM_ID(<subsystem>) DEFAULT(NO)
- Point
zremoteagent=<ipaddress>[:port]
to the IP address and port of the new zRemote module. - Change
name=<zlocal_<lpar>>
to a new zLocal name. - Change
<subsystem>
to a new name. - Change
DEFAULT(YES)
toDEFAULT(NO)
. The first zDC is designated as the default. Only one zDC per LPAR can specifyDEFAULT(YES)
. Additional zDCs fail to initialize unless they specifyDEFAULT(NO)
.
- Point
-
Make a copy of the current started task PROC and change the JCL SYSIN member to point to the new SYSIN member.
-
Start the new zDC and check if it connects to the new zRemote.
Connect the modules to the new zDC
Create an INITPARM
parameter override in the CICS system initialization to target the new zDC subsystem.
Change the ZDC=
parameter in the injection utility parameters to target the new zDC subsystem.
See injection utility of IMS control region.
Change the ZdcName
parameter in the dtconfig.json
file to target the new zDC subsystem.
See the Download procedure of the z/OS Java module.
Failover processing for zDC/zRemote
Disclaimer: This is not load balancing.
The zDC subsystem can automatically switch to the next active zRemote module if a problem occurs with the primary zRemote.
zLocal version 1.1.0+ zRemote module version 1.261+ Once failover happens, and the secondary zRemote is connected, the zDC checks every 5 minutes to see if the primary zRemote is available. If not, the secondary zRemote connection remains intact, and the cycle starts anew. If the primary zRemote is available, the secondary zRemote is disconnected, and an immediate reconnect is attempted to the primary zRemote.
The list of zRemote addresses used for failover processing is maintained in the JCL SYSIN member ZDCSYSIN
. Edit it by changing the following SYSIN parameters:
//SYSIN DD DISP=SHR,DSN=<hlq>.SZDTSAMP(ZDCSYSIN2)
DTAGTCMD(zremoteagent=<ipaddress>[:port];<ipaddress>[:port];<ipaddress>[:port]
nobootstrap=true)
-
Set
zremoteagent=<ipaddress>[:port]
to include a list of zRemote addresses. -
Set
nobootstrap=true
to disable automatic updates for the zLocal; they're not supported during failover processing.
Update zLocal during failover processing
To update the zLocal during failover processing
-
Edit the JCL SYSIN member
ZDCSYSIN
by changing the following SYSIN parameters://SYSIN DD DISP=SHR,DSN=<hlq>.SZDTSAMP(ZDCSYSIN2) DTAGTCMD(zremoteagent=<ipaddress>[:port] nobootstrap=false)
- Set
zremoteagent=<ipaddress>[:port]
to the primary zRemote module. - Set
nobootstrap=false
to enable automatic updates for the zLocal.
- Set
-
Start the zDC. The bootstrapper
dtzagent
downloads the latest zLocal binarylibdtzagent.so
to the/u/dt/agent/downloads/native/a.b.c.d/zos-s390-64
directory. -
Copy the zLocal binary
libdtzagent.so
to the/u/dt/agent/lib64
directory. Now the zDC is ready for failover processing with the latest zLocal binary. -
Edit the JCL SYSIN member
ZDCSYSIN
by changing the following SYSIN parameters://SYSIN DD DISP=SHR,DSN=<hlq>.SZDTSAMP(ZDCSYSIN2) DTAGTCMD(zremoteagent=<ipaddress>[:port];<ipaddress>[:port];<ipaddress>[:port] nobootstrap=true)
- Set
zremoteagent=<ipaddress>[:port]
to include a list of zRemote addresses. - Set
nobootstrap=true
to disable automatic updates for the zLocal.
- Set
-
Restart the zDC.
Terminate the zDC
The zDC typically releases system resources (especially common storage) when it terminates due to a shutdown command or abnormal due to a cancel command or a program abend. Recovery (ESTAEX) routines are always in effect to trap abnormal terminations and to free all system resources.
However, there may be a situation where zDC can't be canceled due to a z/OS anomaly or failure, which precludes the functioning of the cancel command. If you want to terminate the zDC, try the commands below in the specified order until the zDC terminates. Go on to step 2 only if step 1 is unsuccessful, go on to step 3 only if step 2 is unsuccessful, and so on.
To avoid manually cleaning up system resources using the ZDCDELET
process below, wait a short while between commands to permit dumps to be taken and system resources to be freed.
- Issue the
STOP jobname
command. - Issue the
MODIFY jobname,SHUTDOWN IMMED
command. - Issue the
CANCEL jobname
command. - Issue the
FORCE jobname
command.
Attempt to restart the zDC. If the restart is successful, you can continue to run this job. If the restart is not successful and zDC indicates that it can't continue, you can execute the emergency zDC cleanup program named ZDCDELET
.
ZDCDELET
is a stand-alone job that attempts to terminate a specified zDC job and release all system resources held by a zDC process. The JCL to execute ZDCDELET
is shown below:
//*
//* PLACE YOUR JOB STATEMENT HERE
//*
//DELETE EXEC PGM=ZDCDELET,PARM=xxxx
//STEPLIB DD DISP=SHR,DSN=<hlq>.R1nnnxx.SZDTAUTH
The PARM=
on the EXEC
statement must specify the 4-character subsystem name of the zDC process you want to terminate.
- User abends
100
and101
may occur and messages describing the abends are written to the job log. - User abend
100
indicates that thePARM=
is not exactly 4 bytes long. - User abend
101
indicates that the subsystem name specified on thePARM=
operand can't be found in the system. - Messages
ZDC400
throughZDC403
may be written to the job log.
Refer to the zDC user abends and z/OS module messages for a list of all zDC messages.