Kafka CLI - How to identify Kafka consumer performance issues
Created , Updated
Identify Kafka consumer performance issues by analyzing the Kafka consumer lag using Kafka’s CLI.
Step-by-Step Guide
1. Open a terminal which allows to use Kafka CLI commands
2. If desired, start a screen
screen -S kafkacli
3. Watch the GetOffsetShell command
Issue the following command, inserting the relevant broker-list and topic name for your environment
watch -n 5 kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list localhost:10000 --topic ziris-smf
Output example:
Output format:
topic-name:partition-number:partition-offset
If partition-offset is increasing steadily, then this indicates that there is a steady workload on the specified topic.
If partition-offset is increasing rapidly, then this indicates a heavy workload on the specified topic.
4. Watch the kafka-cosumer-groups.sh command
Issue the following command, insert the relevant bootstrap-server and consumer group to observe lag results for the specified consumer group
watch -n 5 kafka-consumer-groups.sh --bootstrap-server localhost:10000 --group consumer-group1 --describe
Output example:
Monitor the LAG column for steadily or rapidly increasing consumer lag.
A steady increase may indicate that there is currently a higher workload than the consumer group can process.
A rapid increase may indicate that your consumer group is experiencing more severe issues, for example some or all consumers are currently inactive or unable to process any messages.
Possible workarounds
Restart all (inactive) consumers.
Increase the capacity to process high workloads and to ensure that messages are processed closer to real-time as possible by running additional consumers for this consumer group.