refactor: improve kafka-check-schemas commit hook #34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Creating this PR to open the discussion about how to improve the pre-commit hook for checking kafka schemas.
The current hooks works by calling a specific target in scala projects,
generateKafkaSchemas
, and checking if this leads to a schemas file different or not (which would mean the schemas is not properly updated)This works but has some disadvantages:
sbt
which is slow to start so this slows down the pre-commit execution. This is mitigated by the fact that it only runs when the src/../models folder is updated but this is not entirely satisfyingBasically, we want the schemas generation to be fast and not get in the way of the developers, but we also would like to detect and warn the devs if for some reason these schemas are not up to date anymore when they commit (either they forgot to commit the new schemas files or the schemas update mechanism doesn't work anymore).
One option that has been mentioned is to include that step in the compile stage so that no time in lost in multiple sbt executions. This means the schemas files would be refreshed at each compilation and we would need to find a way to check if the committed ones have been refreshed.