z/OS Connect - How to trace requests with OpenTelemetry (SMF Type 123 version 2)
Created: | Updated:
Tracing z/OS Connect REST API processing creates SMF type 123 records. The records contain monitoring information like the start and end time of each request, information about the requesting application call and information about the system-of-record that processed the request.
SMF 123 records have 2 versions, which stipulates the granularity of the information provided.
Version 1: captures basic data for individual requests.
Version 2: captures enhanced data for each API provider request.
OpenTelemetry uses W3C Trace-Context in the REST API headers call. These trace headers can be used to selectively extract traces that come from applications using the OpenTelemetry specification for tracing observability.
Pay careful attention to the sequence if there are any previously configured auditInterceptors. For more information about server.xml and zosconnect_auditInterceptor configuration elements, refer to the IBM z/OS Connect product documentation in the IBM Knowledge Center.
Step 1: Back up the server.xml for the z/OS Connect server that you want to integrate with z/IRIS
Step 2: Configure an audit interceptor
Step 3: Enable the audit interceptor for this z/OS Connect server
Step 4: Refresh the z/OS Connect server
Step 5: Verify tracing is activated
Check the logs for confirmation that the audit interceptor configuration has been accepted and activated.
Example z/OS Connect messages.log audit interceptor confirmation
[2/11/21 9:31:29:906 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor zirisMetrics is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester.
[2/11/21 9:31:30:603 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor openTelemetryInterceptor is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester.
[2/11/21 9:31:30:668 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0803I: Request headers to be recorded by the audit interceptor openTelemetryInterceptor are [tracestate, traceparent].
[2/11/21 9:31:32:883 GMT] 000004de om.ibm.zosconnect.audit.smf.internal.AuditSMFInterceptorImpl I BAQR0801I: When active, the audit interceptor hrAuditInterceptor is set to record SMF 123 subtype 1 version 2 records for API provider and version 1 records for API requester.
[2/11/21 9:31:33:584 GMT] 000004de com.ibm.zosconnect.internal.ServiceManagerImpl A BAQR7096I: z/OS Connect EE interceptor list myServerInterceptorList is active at the global level with interceptors: zirisMetrics,openTelemetryInterceptor.