Created: | Updated:
Tracing z/OS Connect REST API processing creates SMF type 123 records. The records contain monitoring information like the start and end time of each request, information about the requesting application call and information about the system-of-record that processed the request.
SMF 123 records have 2 versions, which stipulates the granularity of the information provided.
Version 1: captures basic data for individual requests.
Version 2: captures enhanced data for each API provider request.
OpenTelemetry uses W3C Trace-Context in the REST API headers call. These trace headers can be used to selectively extract traces that come from applications using the OpenTelemetry specification for tracing observability.
Pay careful attention to the sequence if there are any previously configured auditInterceptors. For more information about server.xml and zosconnect_auditInterceptor configuration elements, refer to the IBM z/OS Connect product documentation in the IBM Knowledge Center.
Step 1: Back up the server.xml for the z/OS Connect server that you want to integrate with z/IRIS
Step 2: Configure an audit interceptor
Step 2a: Open the server.xml for editing
Step 2b: Configure auditing for requests with tracestate and traceparent in the request headers
<zosconnect_auditInterceptor id="openTelemetryInterceptor" sequence="3" apiProviderSmfVersion="2" apiProviderRequestHeaders="tracestate, traceparent" />
id should be unique in this server.xml file
Step 3: Enable the audit interceptor for this z/OS Connect server
Add the new zosconnect_auditInterceptor id to the interceptorRef attribute of the zosconnect_zosConnectInterceptors or create a new zosconnect_zosConnectInterceptors if one was not previously configured.
<zosconnect_zosConnectInterceptors id="myServerInterceptorList" interceptorRef="zirisMetrics, openTelemetryInterceptor"/>
Step 4: Refresh the z/OS Connect server
Step 6a: Dynamic refresh
If the updateTrigger="mbean" config element was configured in the server.xml file, a refresh can be dynamically performed to activate the new configuration using the MVS z/OS Connect MODIFY command. If updateTrugger is not configured, a manual restart will be required (Step 6b)
Example z/OS Connect refresh command
Step 6b: Stop and start the z/OS Connect
/S BAQSTRT,PARMS='<serverName> [--clean]
Step 5: Verify tracing is activated
Check the logs for confirmation that the audit interceptor configuration has been accepted and activated.
Example z/OS Connect messages.log audit interceptor confirmation
[2/11/21 9:31:29:906 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor zirisMetrics is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester. [2/11/21 9:31:30:603 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor openTelemetryInterceptor is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester. [2/11/21 9:31:30:668 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0803I: Request headers to be recorded by the audit interceptor openTelemetryInterceptor are [tracestate, traceparent]. [2/11/21 9:31:32:883 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor hrAuditInterceptor is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester. [2/11/21 9:31:33:584 GMT] 000004de com.ibm.zosconnect.internal.ServiceManagerImpl A BAQR7096I: z/OS Connect EE interceptor list myServerInterceptorList is active at the global level with interceptors: zirisMetrics,openTelemetryInterceptor.