z/IRIS v 1.5.0

Features

CPU activity metrics (RMF SMF type 70 records)

Resource Measurement Facility (RMF) is IBM's strategic performance management product for z/OS and provides the ability to gather resource usage data with additional reporting and analysis functionality.
z/IRIS supports RMF SMF records, starting with record type 70, which provides CPU activity data. The RMF SMF records are streamed and parsed in real-time and integrated into open-source metric monitoring software.

z/IRIS Metrics Streaming provides detailed documentation on the various metrics and data schemas created by z/IRIS.

InfluxDB integration support for metrics

z/IRIS supports streaming metrics to an InfluxDB 2.0 time series database for retention. InfluxDB instances can be used as data sources for analytics and visualization applications like Grafana. 

Streaming metrics to an alternate DBMS through z/IRIS can be requested.

Example of z/IRIS InfluxDB 2.0 RMF 70 schema

click to enlarge image

Grafana dashboard JSON

For quick start-up assistance, z/IRIS provides users with an importable .json file that creates the following panels in a Grafana dashboard:

Current in-ready work unit queue distribution per system

click to enlarge image

CPU contention per system

click to enlarge image

Mean CPU usage

click to enlarge image

z/IRIS v 1.4.2

Improvements

Time zone customization

• Configure z/OS time zones globally, per Sysplex or per LPAR. This improves the consistency of timestamps. The default time zone is UTC.

  • Old timestamp in local time:

    2020-10-06T12:34:49.930

  • New timestamp format with explicit UTC time zone:  

    2020-10-06T10:34:49.930Z

Instana APM integration

• Db2 Performance

  Db2 Performance spans provide detailed performance information sourced from Db2 for z/OS IFCIDs 58-66:

  IFCID	Instana call.name
  IFCID 58	Db2 Performance : End Sql QW0058SN
  IFCID 59	Db2 Performance: Fetch start QW0059SN
  IFCID 60	Db2 Performance: QW0060ST start QW0060SN
  IFCID 61	Db2 Performance: QW00612ST start QW0061SN
  IFCID 62	Db2 Performance: QW0062ST start QW0062SN
  IFCID 63	Db2 Performance: SQL statement
  IFCID 64	Db2 Performance: QW0064ST start QW0064SN
  IFCID 65	Db2 Performance: Open Cursor start QW0065SN
  IFCID 66	Db2 Performance: Close Cursor start QW0066SN

  For information on the IFCID fields used in Instana, please see "Custom tags in Instana APM".

• Details regarding the Instana custom tags created from the performance class Db2 for z/OS IFCIDs can be found in "Custom tags in Instana APM".
• Db2 Performance, Db2 Accounting and Db2 Deadlock spans are now Exit spans to prevent merging with the parent JDBC call spans while keeping all custom tags searchable via the Instana Analyze Dashboard.
• Db2 Performance, Db2 Accounting and Db2 Deadlock spans are assign the Instana service.name Db2 and are of type DATABASE

z/IRIS v 1.4.1

Improvements

Distributed Db2 for z/OS traces for Instana APM

• Instana traces created by z/IRIS for distributed Db2 for z/OS JDBC calls have been adapted to support Instana changes to custom trace processing.
  • "SQL related IFCIDs" call name has been changed and split into two new call names for improved search and filter capabilities.
    • Db2 Accounting (for IFCID 3 related performance information).
    • Db2 Deadlock (for IFCID 172 related Db2 resource and JDBC application data).
  • Call endpoints have been updated to point to the related Db2 for z/OS subsystem.
• Enhance distributed Db2 for z/OS tags created by z/IRIS:
  • Db2 for z/OS related tags are prefixed with "zos.db2.*".
  • All tags have been updated for uniformity.

Deadlock Events for Instana APM

• Each deadlock record (IFCID 172) processed by z/IRIS triggers a deadlock event in the Instana Events dashboard now.
• Each deadlock event describes the impacted Db2 system, the number of Db2 resources affected and the Db2 for z/OS timestamp of the deadlock event. The trace.id, that contains all details regarding the deadlock, is provided in the event details.

Db2 Deadlocks

Db2 Deadlock calls provide the deadlock event, impacted resources as well as the holder and waiter sessions for each resource. Each child spans of the Db2 Deadlock trace provides the following information:

• The Db2 for z/OS Resource Name (RNA) affected by the deadlock and the sessions holding-up or waiting for the RNA.
• If the holder and waiter sessions (JDBC connections) are instrumented by Instana agents, z/IRIS will provide the relevant Instana trace.id under the following tags: zos.db2.deadlock.holder.trace.id and zos.db2.deadlock.waiter.trace.id. Use the Instana Analyze dashboard to locate the application calls identified by z/IRIS as participants of a deadlock.
• Each Instana trace.id, identified by z/IRIS as a participant in the deadlock, is tagged as erroneous and enriched with the deadlock trace information required to assist DevOps with identifying all application calls affected by deadlocks in Db2 for z/OS.
• z/OS and Db2 identifiers related to the holder and waiter JDBC calls, i.e: zos.db2.luw.id, zos.lu.name, zos.network.id, zos.system.name and zos.subsystem.name.
• If an IPv4 address is available for holder and waiter sessions, z/IRIS will provide the network address under the following tags: zos.db2.holder.network.address and zos.db2.waiter.network.address.

Db2 Accounting

Db2 Accounting traces provide resource consumption and performance information within the z/OS system and Db2 subsystem, for JDBC calls, including:

• In-Db2 CPU, zIIP processor and elapsed time (zos.db2.cpu.time, zos.db2.ziip.time and zos.db2.elapsed.time).
• Total CPU, zIIP processor and response time (zos.cpu.time, zos.ziip.time and zos.db2.response.time).
• Service Units Factor required to calculate mainframe MSU costs for JDBC calls (zos.db2.su.factor).
• Workload manager service class applied to the Db2 for z/OS JDBC call (zos.db2.workload.service.class.name).
• z/OS and Db2 identifiers for the JDBC connection, i.e: zos.db2.luw.id, zos.lu.name, zos.network.id, zos.system.name and zos.subsystem.name.

Known issues

Instana APM integration

• By default, Instana spans created by z/IRIS are delayed by 60 seconds to avoid unpredictable merging with parent spans. When merging occurs, certain span metrics are overwritten in the call tree, by z/IRIS span metrics  (i.e. endpoint and call name). Users can customize the delay as described in the 'z/IRIS - Configuring integration into Instana' article.
• Start time for DB2 Accounting and Db2 Deadlock is derived from the SMF Header Timestamp and not from the DB2 Timestamps available in IFCID 3 or IFCID 172.
• Instana "fixes" the timestamp provided by z/IRIS, which adjusts the start time of spans created z/IRIS. The issue has been reported to Instana and will be fixed in an upcoming release.
• Instana custom tags are either string or boolean. Numeric-logical comparisons for tag values, are not yet available for custom traces in Instana.

z/IRIS v 1.4.0

Features

z/OS common address space work record streaming (SMF 30)

• Parsing, normalizing and formatting of SMF Record Type 30, subtypes 1-6 in near real-time.
• Intermittent storage in Apache Kafka cluster.

Instana APM z/OS common address space work traces

• Traces are created when:
• A batch job starts and ends.
• A job step completes.
• Interval accounting is active (frequency depends on the SMFPRMxx interval and synchronizing settings).
• System address space accounting is active (frequency depends on the SMFPRMxx interval and synchronizing settings).
• Error signal and event generation when steps and jobs end abnormally to alert DevOps teams timeously of any issues related to z/OS work.
• All traces are enriched with custom tags to facilitate searching and filtering to easily locate mainframe traces for batch analysis.

Fixes

• Corrected bugs related to path specifications in the ziris.comm.config file.

z/IRIS v 1.3.0

Features

Load Balancing & High Availability with Apache Kafka Integration

• In order to guarantee High Availability and Load Balancing, streaming SMF records through an Apache Kafka Cluster is now possible.
• z/IRIS IronTap subscribes to an Apache Kafka Cluster to consume SMF records.
• The z/IRIS z/OS Client can be configured as an Apache Kafka Producer.

Improvements

z/IRIS IronTap Linux Server

• z/IRIS IronTap can consume SMF records from Apache Kafka.

• The following SMF data is now included in the span and tag section in Instana APM: 

  SMF Field Name	Instana Tag
  SMFxSID ( System identification from the LPAR SID parameter)	SystemId
  SMFxSSI (Subsystem identification value set by the SUBSYS=option specified in the SMF macros)	SubsystemId

• z/IRIS IronTap metrics are visible in Instana:

z/OS Client

• All z/OS Client logging is now controlled over the java.util.logging api. The default properties are provided in the $APP_HOME/lib/logging.properties file.

• Throughput logging can be activated using the monitor.thruput.interval ziris.comm.config parameter. SMF Record throughput statistics will only be written to STDOUT when the logging level set to FINE as follows in logging.properties:

  ziris.comm.config extract example

  # monitor.thruput.interval defines an interval in seconds to display throughput
  #  statistics. Only takes effect if com.msc.zos.ziris=FINE in logging.properties
  # default : 60 seconds                                                  
  # syntax : monitor.thruput.interval={5-3600}                            
  monitor.thruput.interval=5                                              

  logging.properties logging level for throughput monitoring

   com.msc.zos.ziris=FINE

Fixes

• Stabilized z/OS Client reconnect feature for mode=tcp.

Reported issues

• None

z/IRIS v 1.2.1

Improvements

IronTap Linux Server

• The names of the following fields have been changed to improve readability. The values were converted into CPU Seconds: 

  old field name	new field name
  	Total CPU Seconds
  CPU Start Time	(not displayed, used to calculate "Total CPU Seconds)
  CPU End Time	(not displayed, used to calculate "Total CPU Seconds")
  CPU Time Executing DB2	Total MVS
  WLM SP Address Spaces	SP
  UDF processed in SP	UDF
  Main Appl. Triggers	Appl. Triggers
  SP Sql Stmts in WLM Address Space	SP SQL
  	UDF SQL
  On ZIIP Total	Total
  In-DB2 on ZIIP	In DB2
  Trigger Exeution on ZIIP	Appl. Trigger
  ZIIP Eligible on Std CPU	On CP
  SP on ZIIP Class 1 and 2	Appl. SP
  UDF on ZIIP CPU	Appl. UDF
  SP on Std CPU Class 1 and 2	Appl. SP
  UDF in-DB2 Class 1 and 2	Appl. UDF
  SP Requests on ZIIP in-DB2	SP in WLM
  SP SQL Stmts on ZIIP in-DB2	UDF in WLM
  UDF in SP on ZIIP	UDF in WLM 2
  UDF in-DB2 on ZIIP	In Db2 UDF in WLM
  Nested Task Triggers on ZIIP	Nested Triggers
  DB2 Processing of Accel. Eligible SQL	Accelerator SQL on CP
  DB2 Processing on ZIIP of Accel. Eligible SQL	Accelerator SQL on ZIIP

z/OS Client

• Improved decision branching for streaming modes (in preparation for ETL streaming option between client and server).

• Message flooding prevention when reconnecting after a TCP connection/IO Failure. The z/OS Client will log 5 reconnect attempts and thereafter logging for the reconnect will be suppressed until a connection has been established. The following message will be logged in the STDOUT DD:
  
  

  Example

  Jul 15, 2019, 10:31:41 AM ! ZIRISUAT 1 error reconnecting via TCP, EDC8128I Connection refused. (Connection refused)
  Jul 15, 2019, 10:31:41 AM ! ZIRISUAT 1 Reconnect failed, automatic retry every 10 seconds.
                           Reconnect logging will be SUPPRESSED until a successful connection has been established

Fixes

• None

Reported issues

• None

z/IRIS v 1.2.0

Major Features include:

• Db2 LUWID correlation and ordering.

• Detecting Stored Procedure metrics and tagging for Instana APM users.

• Detecting Deadlock metrics and tagging for Instana APM users.

Improvements:

• Instana APM - Response Time alignment to Instana standard (<1ms).
• Simplified SQL Communication Area Output.

Fixes:

• IFCID 3 parsing corrections between v11 and v12.
• Statement Type in IFCID 63 - offset corrected to parse the self defining section.
• Timestamp corrections.

Reported Issues

• No reported issues

Downloading z/IRIS Software

All z/IRIS Software can be downloaded from the mainstorconcept Download Portal. 
New users should register here first, before attempting to download any z/IRIS Software.