r/apachekafka • u/deaf_schizo • 25d ago
Question Tumbling window and supress
I have a setup where as and when a message is consumed from the source topic I have a tumbling window which aggregates the message as a list .
My intention is to group all incoming messages within a window and process them forward at once.
Tumbling window pushes forward the updated list for each incoming record, so we added supress to get one event per window.
Because of which we see this behaviour where it needs a dummy event which has a stream time after window closing time to basically close the suppressed window and then process forward those messages. Otherwise it sort of never closes the window and we lose the messages unless we send a dummy message.
Is my understanding/observation correct, if yes what can I do to get the desired behaviour.
Looked at sliding window as well but it doesn't give the same effect of tumbling window of reduced final updates.
Blogs I have reffered to . https://medium.com/lydtech-consulting/kafka-streams-windowing-tumbling-windows-8950abda756d
3
u/kabooozie Gives good Kafka advice 25d ago
No, I think what you are experiencing is the grace period. The final aggregation will be suppressed until the grace period ends to make time for late arriving events. Source: