5

I am reading Kafka, the definitive guide and I came across the below point for consumers.

heartbeat.interval.ms must be lower than session.timeout.ms, and is usually set to one-third of the timeout value.

May I know why is it, I thought both should be the same, let's say if I set the session.timeout.ms to 10s and heartbeat to 3s, then if the consumer takes longer time to process the record, the broker already sends the poll and figures out the consumer already dead and rebalances, in that case, what is the purpose of session timeout property. Am I interpreting it wrong?

Ramya B
  • 59
  • 2

2 Answers2

0

These are Kafka consumer configurations as you mentioned in question. So obviously it is unknown for Kafka brokers. Consumer is polling and sending the heartbeat to brokers to confirm brokers are active, and is there any pending message to read from topic paritions; not other way as you mentioned in your question.

Hopefully this will make sense for you!

Steephen
  • 101
  • 2
0

It is 1/3rd of the time in order to have chance to get heartbeat even if one time fails, the consumer is sending the heartbeat to the consumer group in order to show that it is alive, in current versions of kafka clients the heartbeat process is done in background thread and not interfering your 'main' thread. If heartbeat would fail / not get in the session timeout interval , your consumer group controller would decide that your consumer is not active and evict it from the group with rebalance event

Ran Lupovich
  • 111
  • 2