Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add allow_auto_create_topics to make automatic topic creation configurable #1556

Merged
merged 3 commits into from
Jun 25, 2024

Conversation

kumaranvpl
Copy link
Collaborator

@kumaranvpl kumaranvpl commented Jun 24, 2024

Description

We used to always create topics automatically and provided no way to make it configurable. This PR introduces a new variable called allow_auto_create_topics to make automatic topic creation configurable.
I have included a warning when allow_auto_create_topics is set to False because confluent-kafka-python backfires/crumbles terribly if it tries to access an non existent topic as mentioned here - confluentinc/confluent-kafka-python#1547.
There were few blocking calls in the __init__ method of Async Confluent Producer and Consumer. I have used run_in_executor to make them non-blocking.

Fixes #1486, #1363

Type of change

Please delete options that are not relevant.

  • Documentation (typos, code examples, or any documentation updates)
  • Bug fix (a non-breaking change that resolves an issue)
  • New feature (a non-breaking change that adds functionality)
  • Breaking change (a fix or feature that would disrupt existing functionality)
  • This change requires a documentation update

Checklist

  • My code adheres to the style guidelines of this project (scripts/lint.sh shows no errors)
  • I have conducted a self-review of my own code
  • I have made the necessary changes to the documentation
  • My changes do not generate any new warnings
  • I have added tests to validate the effectiveness of my fix or the functionality of my new feature
  • Both new and existing unit tests pass successfully on my local environment by running scripts/test-cov.sh
  • I have ensured that static analysis tests are passing by running scripts/static-analysis.sh
  • I have included code examples to illustrate the modifications

@kumaranvpl kumaranvpl requested a review from Lancetnik June 24, 2024 12:18
@kumaranvpl kumaranvpl marked this pull request as draft June 24, 2024 12:18
@kumaranvpl kumaranvpl marked this pull request as ready for review June 24, 2024 13:04
Lancetnik
Lancetnik previously approved these changes Jun 24, 2024
@kumaranvpl kumaranvpl enabled auto-merge June 25, 2024 15:56
@kumaranvpl kumaranvpl added this pull request to the merge queue Jun 25, 2024
Merged via the queue into main with commit 4a2341d Jun 25, 2024
29 checks passed
@kumaranvpl kumaranvpl deleted the handle-auto-create-topics branch June 25, 2024 16:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants