ExtendedKafkaConsumer in DefaultKafkaConsumerFactory restricts the customizations and extensions of CustomKafkaConsumers

[atul-raghuwanshi]

• As part of commit Get rid of proxy in DefaultKafkaConsumerFactory we got rid of createProxy which created a proxy on the consumer returned by createRawConsumer

• Instead we have added ExtendedKafkaConsumer which extends kafka consumer and provides the listener functionality by wrapping the code.

• This change however restricts me from extending DefaultKafkaConsumerFactory and returning a CustomKafkaConsumer which extend from KafkaConsumer and also using listner code of ExtendedKafkaConsumer. This is due to restriction in java not allowing multiple inheritance.

• Essentially i would want my ExtendedKafkaConsumer to act on my CustomKafkaConsumer instead of KafkaConsumer.

• This would be easily possible with the proxy method as i can return any custom Kafka Consumer from createRawConsumer and proxy will be created then on that object with the listener code.

• To support this in newer way i have to maintain the listener code of ExtendedKafkaConsumer inside my CustomKafkaConsumer.

• Note that i already have a CustomKafkaConsumer extending from KafkaConsumer which is used in non spring environment as well and so not possible to extend from ExtendedKafkaConsumer which is mentioned in the commit message.

I would like to discuss here that is it correct to keep the new way of ExtendedKafkaConsumer instead of old proxy way. If so then how can i create a KafkaConsumerFactory which uses the same code, however just uses my CustomKafkaConsumer.

cc: @artembilan @garyrussell author of the commits

[artembilan]

Thank you for feedback!

Well, we got rid of proxy for performance and reflection reasons.
This is called default for a reason as well, so giving it some customization would not sound like a "default".
That's why we have that KafkaConsumerFactory strategy to let you have any possible custom KafkaConsumer.

There might be some other hooks to further customize KafkaConsumer behavior without extending it, e.g. ConsumerPostProcessor, RecordInterceptor, or KafkaListenerObservation.

You may also extend an ExtendedKafkaConsumer the way that it delegates to your CustomKafkaConsumer keeping an extension for that DefaultKafkaConsumerFactory.

Summarizing my post, we are not going back to the proxy.