Issue 191: Add at-least-once stream processing example#192

Open

claudiofahey wants to merge 82 commits intopravega:devfrom claudiofahey:issue-191-streamprocessing

+3,093

Commits on Aug 21, 2020

Issue 191: Add non-Flink stream processing example.
Claudio Fahey
committed
Issue 191: Moved inner classes to top level. Removed obsolete classes.
Claudio Fahey
committed
Issue 191: Various updates. This PR is still incomplete.
Claudio Fahey
committed
Issue 191: Processor is now stateless to avoid complications with managing state.
Claudio Fahey
committed
Issue 191: Cleanup
Claudio Fahey
committed
Issue 191: Allow variable number of workers; simplify recovery
Claudio Fahey
committed
Update to use EventStreamClientFactory. Removed ExactlyOnceMultithreadedProcessor.
Claudio Fahey
committed
Add test structure
Claudio Fahey
committed
Add AtLeastOnceProcessor
Claudio Fahey
committed
Add https://github.com/pravega/pravega/blob/c7ac009970787df2633163644691aa3d72fc1a4e/client/src/test/java/io/pravega/client/state/examples/MembershipSynchronizer.java
Claudio Fahey
committed
Add ReaderGroupPruner (incomplete)
Claudio Fahey
committed
Add ReaderGroupPruner (incomplete)
Claudio Fahey
committed
Bug fixes in MembershipSynchronizer
Claudio Fahey
committed
Document new classes. Add various parameters.
Claudio Fahey
committed
Use new AppConfiguration class. Update license. Mark obsolete sections in README.md.
Claudio Fahey
committed
Add writer to AtLeastOnceApp
Claudio Fahey
committed
Add graceful shutdown
Claudio Fahey
committed
Add logging
Claudio Fahey
committed
Fix typo in comment
Claudio Fahey
committed
Create runAtLeastOnceApp.sh
Claudio Fahey
committed
Now using SampleEvent class with JSON serializer. InstanceId can now be set as an application parameter.
Claudio Fahey
committed
Update README (incomplete)
Claudio Fahey
committed
Update README (incomplete)
Claudio Fahey
committed
Remove unused test code. Revert changes to logback.xml. Add new scripts to build.gradle.
Claudio Fahey
committed
Remove unused test class.
Claudio Fahey
committed
EventGenerator logs when events are acked.
Claudio Fahey
committed
Minor logging and formatting changes.
Claudio Fahey
committed
Move runAtLeastOnceApp.sh
Claudio Fahey
committed
Documentation, logging, and formatting updates
Claudio Fahey
committed
Fix typo.
Claudio Fahey
committed
Fix /README.md.
Claudio Fahey
committed
Add integration test (incomplete)
Claudio Fahey
committed
Add integration test (incomplete)
Claudio Fahey
committed
Add integration test (incomplete)
Claudio Fahey
committed
Basic graceful shutdown test working
Claudio Fahey
committed
AtLeastOnceProcessor now uses suppliers. Added basic structure for fault injection.
Claudio Fahey
committed
Add pause and unpause to ReaderGroupPruner
Claudio Fahey
committed
Basic pause of worker working
Claudio Fahey
committed
Use TestContext
Claudio Fahey
committed
Now validating that stopped instances do not process events
Claudio Fahey
committed
Add license headers
Claudio Fahey
committed
Add unit tests for TestEventValidator
Claudio Fahey
committed
Add testing documentation
Claudio Fahey
committed
Add test forcing duplicates.
Claudio Fahey
committed
Add pause/unpause to AtLeastOnceProcessorInstrumented. Add failing tests handleExceptionDuringFlushTest and killAndRestart1of1ForcingDuplicatesTest.
Claudio Fahey
committed
Add different write modes. Split writeEventsAndValidate.
Claudio Fahey
committed
Now using EndToEndTestConfig with builder
Claudio Fahey
committed
Add timeout to failing tests
Claudio Fahey
committed
Created failing test handleExceptionDuringProcessTest
Claudio Fahey
committed
Bug fix: Fix handling of position when an exception occurs during processing or flushing
Claudio Fahey
committed
Fix killAndRestart1of1ForcingDuplicatesTest
Claudio Fahey
committed
Updated documentation. Now using in-process Pravega cluster.
Claudio Fahey
committed
Update README.
Claudio Fahey
committed
Remove unneeded dependency JUnitParams.
Claudio Fahey
committed
Add description to getStream1Name and getStream2Name
Claudio Fahey
committed
Randomize execution period of ReaderGroupPruner
Claudio Fahey
committed
Revert log level to ERROR
Claudio Fahey
committed

Commits on Sep 12, 2020

Move inline class definition to top-level class SampleEventProcessor
Claudio Fahey
committed
Clean up EventDebugSink: Make methods private. Add final.
Claudio Fahey
committed
Remove commented code
Claudio Fahey
committed
Replace some try-with-resource blocks with Lombok @cleanup
Claudio Fahey
committed
Add final
Claudio Fahey
committed

Commits on Mar 8, 2021

Merge branch 'dev' into issue-191-streamprocessing
Claudio Fahey
authored

Commits on Mar 18, 2021

Merge branch 'dev' into issue-191-streamprocessing
RaulGracia
authored

Commits on Mar 19, 2021

Move annotations to separate lines
Claudio Fahey
committed
Fix samples version number and improve feedback in runAtLastOnceApp.sh
Claudio Fahey
committed
Fix typo in streamprocessing/README
Claudio Fahey
committed
Fix formatting
Claudio Fahey
committed
Removed all commented and unused code from SetupUtils.java
Claudio Fahey
committed

Commits on Mar 25, 2021

Decrease heartbeat interval for kill5of6Test to speed up dead reader detection.
Claudio Fahey
committed
Adding logging
Claudio Fahey
committed
Add retry to killAndRestart1of1ForcingDuplicatesTest
Claudio Fahey
committed
Fix retry in killAndRestart1of1ForcingDuplicatesTest
Claudio Fahey
committed

Commits on Mar 26, 2021

Commits on Mar 27, 2021

Make methods private
Claudio Fahey
committed
Create private method createStream
Claudio Fahey
committed
Make SampleEvent immutable. Replace @cleanup with try-with-resources.
Claudio Fahey
committed
Update README
Claudio Fahey
committed
Add createStreams method to AtLeastOnceApp
Claudio Fahey
committed
Add createStreams private method to EventGenerator
Claudio Fahey
committed

Commits on Jul 27, 2021

Merge branch 'dev' into issue-191-streamprocessing
RaulGracia
authored