Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Validation module compatible with batch module #1496

Open
jeromevdl opened this issue Oct 30, 2023 · 3 comments
Open

Validation module compatible with batch module #1496

jeromevdl opened this issue Oct 30, 2023 · 3 comments
Labels
enhancement New feature or request priority:2 High - core feature or affects 60% of the users RFC status/staged-major-release This change will go with the next major release v2 Version 2
Milestone

Comments

@jeromevdl
Copy link
Contributor

jeromevdl commented Oct 30, 2023

Is your feature request related to a problem? Please describe.

The @Validation annotation should be compatible with the batch module (SQS, Kinesis), i.e. if one message is wrong do not fail the whole batch but add the message to the partial failure list.

There should not be any (maven) dependency between the 2 modules.

Describe the solution you'd like

  • browse the list of messages and validate each one (here),
  • log a warn for each invalid message
  • remove the ones that are not good (no need to pass them to the rest of the batch processing),
  • add them to the list of failure by creating an SQSBatchResponse / StreamsEventResponse and put the messages in error in the partial error. (example)
  • then the handler is called with all the remaining messages (or directly return if no message is valid), and here the batch module will work (it wil create an SQSBatchResponse / StreamsEventResponse and potentially add partial failures).
  • finally the end of the validation will complete the response returned by the handler (batch module) with the validation failure from initial steps.

Describe alternatives you've considered

eventually put the @Validation annotation on the processRawMessage but it would be much complex to develop (much more changes) and I do not see the benefit.

@jeromevdl jeromevdl added enhancement New feature or request v2 Version 2 RFC labels Oct 30, 2023
@jeromevdl jeromevdl added the good first issue Good for newcomers label Nov 23, 2023
@jeromevdl
Copy link
Contributor Author

let's have #1489 merged before doing that one, then it's an "easy" one

@scottgerring
Copy link
Contributor

This should be done in conjunction with #1469

@scottgerring scottgerring added priority:2 High - core feature or affects 60% of the users and removed good first issue Good for newcomers labels Dec 20, 2023
@scottgerring scottgerring added this to the v2 milestone Dec 21, 2023
@jeromevdl
Copy link
Contributor Author

Done with #1621

@jeromevdl jeromevdl added the status/staged-major-release This change will go with the next major release label Jul 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request priority:2 High - core feature or affects 60% of the users RFC status/staged-major-release This change will go with the next major release v2 Version 2
Projects
None yet
Development

No branches or pull requests

2 participants