The zDC (z/OS Data Collection subsystem) handles the life cycle of the zLocal Agent, formerly called the zUNIX Agent or USS (Unix System Services) Agent. It controls communication between the zLocal Agentand CICS and IMS Agents. You can enter Operator start, stop, and modify commands from a system console to control the zDC and the lifecycle of the zLocal Agent.
Installing the zDC
The AppMon zDC runs as an authorized z/OS process (typically as a system task). This means the programs must reside in an authorized library. It is intended to start automatically as a started task (STC) at a system IPL. This gives the zDC continuously availability to collect PurePath data from CICS and IMS Agents and forward it to the AppMon Server.
The zDC can also be run as a batch job. The service class of the zDC must be high enough so that it is always available for keep-alive messages from the zRemoteAgent. The zDC should have a priority equal to or higher than the CICS application regions and the IMS dependent regions.
Download the z/OS components to the mainframe. Make a note of the high-level qualifier used in the download procedure. The sample high-level qualifier of
DT.DYNA is used in this procedure. Substitute your value as needed.
Create the RACF user ID for the zDC process.
This ID must have a z/OS Unix segment. The recommended home directory name is
/u/dt. If a directory other than
/u/dt is used, update the DTAGTCMD argument in the zDC parameter file (the SYSIN DD of the proc) to specify the correct path to the
dtzagent binary, which defaults to
/u/dt/agent/lib64/dtzagent. Only the
/u/dt/ part can be changed, because the path to the Agent binary must remain
agent/lib64/dtzagent, regardless of the high-level qualifier used for this directory path. The
/u/dt directory or the high-level qualifier that replaces it must be writeable by the zDC process. If the zDC is run as a started task, the ID must be capable of representing the zDC started task. Give the ID the same name as the zDC started task and add it to the access list of the RACF class STARTED with at least READ authority.
Change the dataset high-level qualifier represented by hlq in member
SZDTSAMP to the value you used in the Download and SMP/E install of the z/OS Agent Components procedure.
Run the COPYAGNT job to create the required z/OS Unix subdirectories under the home directory of the zDC started task, copy the z/OS zLocal Agent binary file from
SZDTSAMP, and set it to be executable. If using the default home directory of
/u/dt/, the resulting file is
/u/dt/agent/lib64/dtzagent. If the home directory has been changed, the paths in this job must be changed to correspond. File and directory names for z/OS zLocal are case-sensitive. The path names below the home directory cannot be changed.
COPYAGNT job should run under the user ID that was created for the zDC process, so that user ID owns the
dtzagent file. If this is inconvenient, you can also use the
chgrp commands to reset the owning user and group for the
dtzagent file after running
LZDT700.SZDTAUTH.Substitute hlq with the high-level qualifier value you used in the Download and SMP/E install of the z/OS Agent Components procedure. If you are also planning on using the IMS Agent, you must also authorize the
For example, create a member named
Then issue the console command:
Copy the ZDCMEPC sample zDC started task procedure from SZDTSAMP to a system proclib used for started tasks.
The default name for the procedure is MEPC. Customize it as necessary for local standards. If a different subsystem name is chosen, change the SUBSYSTEM_ID() in the zDC SYSIN parameters to match. The proc and the two sysin members ZDCSYSIN and ZDCMEPCA that it uses contain dataset names that must be edited to replace HLQ. with the appropriate high-level qualifiers.
Update the zDC startup parameters that are supplied through the SYSIN DD of the zDC proc, which defaults to the
ZDCSYSIN member of
DTAGTCMD(/u/dt/agent/lib64/dtzagent– Update with the correct path to this program if the default was not used.
- nobootstrap=false - Leave the default value of false if you want to use the bootstrap Agent. If you do not want to use a bootstrap Agent set the value to true. If this parameter is not specified the default is to use the bootstrap Agent.
name=<agentname>– This name does not require an agent mapping in a system profile. It is simply used for the name of the log files. This name should be reflective of the fact that this is a zlocalagent, and it could include the smfid of the lpar that it services. For example, “name=zlocalagent_<lpar>”. This name should be different than the corresponding zremoteagent, so that the log files can be easily distinguished. This agent does not connect to the AppMon Server directly.
- zremoteagent=<host>[:port] - Update with the IP address of the zRemote Agent. The zRemote Agent is a required parameter. The port parameter is optional and defaults to 8898.
See the NON-BOOTSTRAP Agent section for additional information on using the NON-BOOTSTRAP Agent.
See zRemote Agent for additional information on how to set up the zRemote Agent.
Optional but recommended: Add a command to your system startup to automatically start the zDC subsystem at IPL.
zDC reference information
The following sections describe the basic operation and configuration parameters for the zDC.
The zDC manages the lifecycle of the z/OS Local Agent named
dtzagent that runs in the z/OS Unix environment:
- It starts the z/OS Local Agent when the zDC initializes at system startup or is manually started.
- It terminates the Agent when the zDC is stopped.
COPYAGNT job creates the z/OS Unix file containing this program.
libdtzagent.so - the NON-BOOTSTRAP Agent
If you want to run the
dtzagent NON-BOOTSTRAP, it must be downloaded from the AppMon Server. Complete the instructions to start the zDC and connect it to the AppMon Server using the standard
dtzagent binary. The first time you connect to the AppMon Server, do not use the
nobootstrap=true parameter. If you do use this parameter the first time, the zDC will not start successfully. You can comment it out for this first execution.
Assuming that you have installed the product to
/u/dt (if you have not, then replace this folder with your installation folder name), when the zDC successfully connects to the AppMon Server, the agent
libdtzagent.so is copied to a folder
/u/dt/agent/downloads/rr.rr.rr.bbbb /native/zos-s390-64 (where
rr.rr.rr is the release number and
bbbb is the build number). In the
SZDTSAMP dataset, the member
OCOPYAGT is a job that copies the
libdtzagent.so to the
/u/dt/agent/lib64 folder. When you uncomment or add the
nobootstrap=true parameter In the
ZDCSYSIN, the non-bootstrap agent that was copied into the
lib64 folder is used.
The following are examples of log messages in the zDC job output. The first example is with a bootstrap Agent. The second example is from a zDC using a non-bootstrap Agent.
When using the bootstrapped
libdtzagent.so creates a log directory and log files in its working directory, which defaults to the home directory specified in the user ID’s RACF z/OS Unix segment, so write permissions are required for this location. Permissions assigned to files created by
libdtzagent.so can be controlled by setting DT_UMASK. The default is umask(022).
No attempt is made by the zDC to limit which user IDs can enter operator commands.
The zDC uses TCP/IP sockets communication, so it may be necessary to update your security system’s access rules if they are configured to deny TCP/IP access by default.
General SYSIN dataset format
Positions 1 through 71 of each statement can contain parameter data. Positions 72 through 80 are ignored and can contain a sequence number, but are not required to do so.
Each parameter takes the form KEYWORD(value). Each keyword and the requirements for the values associated with it are documented in the ZDCSYSIN member of the sample dataset SZDTSAMP.
zDC SYSIN parameters
All values that tailor the execution of the zDC reside in the dataset pointed to by DDNAME SYSIN.
The sample started task procedure (ZDCMEPC) refers to the ZDCSYSIN member of SZDTSAMP as shown below:
//SYSIN DD DISP=SHR,DSN=<hlq>.LZDT700.SZDTSAMP(ZDCSYSIN)
The dataset must contain 80 character records, blocked to a multiple of 80 bytes.
When a change is made to the SYSIN dataset, the ZDC must be stopped and restarted for the change to take effect.
Commonly used parameters
nobootstrap=true- The NON-BOOTSTRAP agent
libdtzagent.so. If the nobootstrap=true parameter is not used, the bootstrap version of the z/OS Local Agent is loaded and attached by the zDC. Automatic updates of this binary are performed. This “bootstrap” Agent queries the AppMon Server to see if it includes a more current version of the z/OS Local Agent. If it does, that Agent is downloaded from the Server and used. This check is performed each time the zDC is started.
<Agent name>- This name does not require an agent mapping in a system profile. It is simply used for the name of the log files. This name should be reflective of the fact that this is a zlocalagent, and it could include the SMF id of the LPAR that it services. For example, “name=zlocalagent_<lpar>”. This name should be different than the corresponding zremoteagent, so that the log files can be easily distinguished. This agent does not connect to the Server.
[zremoteagent=ipaddress:port])– The port parameter is optional and defaults to 8898. See zRemote Agent for information on how to register the Windows service or start the Linux/Unix daemon with the right zdclistenerport, name, and server parameters.
- loglevel=info - the loglevel used by the z/OS Local Agent
SUBSYSTEM_ID(MEPC)– The name of the zDC subsystem. Specify the desired subsystem name here if MEPC is not appropriate.
DTLOGLEVEL(3)– The logging level produced by the zDC. Set this parameter to
4to suppress informational messages. It should not be set to less than
3unless requested by Customer Support. Values less than
3are used for diagnostic debugging.
DTMSG_SMOSIZE(10)– A number that defines the maximum amount of storage (in Mbytes) for messages that can be queued in the zDC shared memory object pending their writing to the AppMon Agent. The default is 10 MB. This default value may need to be increased if very high volumes of transactions are traced.
DTCHDIR(/u/dt)– Used to change the z/OS Unix current directory where
dtzagentcreates temporary files for
stderr. It defaults to the home directory of the associated user ID.
ZIIP_ENABLE(YES)requests the zLocal agent to offload Agent message data processing and transmission to a zIIP CPU if available. ZIIP_ENABLE(NO) is the default. ZIIP_ENABLE(YES) will make virtually all of the CPU time that is consumed by the zDC to process Agent data eligible to run on a zIIP assist processor. The relatively small amount of background activity that occurs when the zDC is idle is not affected by this option and continues to run on a general CPU.
DTMSG_TRANBUFSIZE(n,m)overrides the default number and size of Transaction buffers. Transaction buffers offer better performance for CICS and IMS agents by placing event messages related to each PurePath into dedicated buffers. Rather than send individual event messages for each PurePath as they occur, they are blocked into one or more buffers and sent together. Additionally, the buffer of related messages is processed more efficiently by the zremoteagent . The n parameter is the number of buffers in thousands. For example, 2 = 2000 buffers. A zero value disables Transaction buffers. The
mparameter must be either
4to specify 2K or 4K buffer size. The 4K buffer size is recommended unless storage consumption is a significant concern. The minimum non-disabled value is
1,2. The maximum is
248,2corresponding to total size of 512 Megabytes. An upper bound for the number of buffers that may be required is one for each IMS message region Agent and MAXTASK times the number of CICS Agents, but actual requirements are likely to be considerably smaller.
If a parameter value must span multiple lines, specify the value through position 71 and continue the value on the next statement starting in position 1
A comment is specified by placing an asterisk in position 1 of a statement. The entire statement is considered part of the comment.
A comment can be enclosed between a slash-asterisk and an asterisk-slash:
Multiple lines can be contained within comments of this type.
Running multiple zDC systems on a single z/OS image
A zDC connects one or more CICS or IMS Agents running on the same z/OS system to a single AppMon Server, so it may be necessary to configure multiple zDC-started tasks on a single z/OS system when different application groups are being traced by different AppMon Servers. At times, it may be useful to configure an alternate zDC for testing purposes as well.
Only one zDC on a particular z/OS system can specify DEFAULT(YES) in its SYSIN parameters. Additional zDCs will fail to initialize unless they specify DEFAULT(NO).
See Adding SIT override INITPARM for Non-Default ZDC sub-systems to configure a CICS Agent to connect to a non-default zDC.
Check maintenance level
Browse Install Library:
<*hlq*>.LZDT700.SZDTAUTH where <hlq > is the High Level Qualifier of the SMPE target zone.
Browse the beginning of the ZDCDTAGT load module. You should see the header:
ZDCDTAGT 00BASE YYYYMMDD HH.MM VER 06.03.00 COPYRIGHT (C)… In the example above, 00BASE indicates this is the release
7.0.0 GA version of the module without any maintenance. When service is applied, the field contains a problem number, and release numbers and dates will vary over time.
The manager for the dynaTrace z/OS zLocal Agent. ZDCDTAGT can produce diagnostic messages with this format:
ZDC99<n><i> messages where n=loglevel and i=severity
zDC commands supported by the z/OS modify command
Modify <JobName or StcProcName>,DT1 <cmd>
|DISPLAY||Display z/OS Unix Agent subtask processor stats.|
|DTM%V||Force display (ZDC952) of utilization of the zLocal Agent message queue.|
|LOG=n||Set loglevel to n (0-8) where lower numbers increase the level of detail. Default is 3.|
|START||Start zLocal Agent.|
|STDO||Append previously unreported lines from zLocal agent stdo.log to SYSPRINT.|
|STOP||Stop zLocal Agent.|
|ZLALOGLEVEL=Xxx||Set zLocal agent loglevel to Xxx (FINEST,FINER,FINE,CONFIG,INFO,WARNING,SEVERE,DEBUG,NONE). No default. Available in 6.5 and above.|
zDC sample output from MODIFY MEPx,DT1 DISPLAY
The display skips most rows with zero counters if the log level is at or above the default level of 3. The loglevel can be changed for diagnostic purposes to greatly increase logging, including displaying the full set of counters
using MODIFY MEPx,DT1 LOG=1):
Rows that end with an exclamation mark should have zero counts.
Emergency zDC termination
zDC makes every effort possible to release system resources (most especially common storage) when it terminates. This is true if the termination is normal due to a shutdown command or abnormal due to a cancel command or a program abend.
Recovery (ESTAEX) routines are always in effect to trap abnormal terminations and to free all system resources.
However, there may be a situation where zDC cannot be cancelled due to a z/OS anomaly or failure, which precludes the functioning of the cancel command.
If you are unable to communicate with zDC and want to terminate it, try the commands below in the specified order until the zDC terminates. Go on to step 2 only if step 1 is unsuccessful, go on to step 3 only if step 2 is unsuccessful, and so on. To avoid the need to manually clean up system resources using the ZDCDELET process below, wait a short while between commands to permit dumps to be taken and system resources to be freed.
- Issue the
- Issue the
MODIFY jobname,SHUTDOWN IMMEDcommand.
- Issue the
- Issue the
Attempt to restart the zDC. If the restart is successful you can continue to run this job. If however the restart is not successful and zDC indicates that it cannot continue, you can execute the emergency zDC cleanup program named ZDCDELET.
Messages ZDC400 through ZDC403 may be written to the joblog as well. These can be found in the ZDC Messages section below.
Refer to the zOS Agent Messages page for a list of all zDC messages.
Multiple TCP/IP stacks
A z/OS system with multiple TCP/IP stacks presents an additional complication. In this situation, an additional step is needed in the server job:
The BPXTCAFF program is used to associate a specific TCP/IP stack with the current address space. The PARM value is the job name used to start the desired TCP/IP stack.