Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

feat: stream events to timeline service in batches #3683

Merged
merged 3 commits into from
Dec 9, 2024
Merged

feat: stream events to timeline service in batches #3683

merged 3 commits into from
Dec 9, 2024

Conversation

matt2e
Copy link
Collaborator

@matt2e matt2e commented Dec 9, 2024 •
edited
Loading

  • Each timeline client now batches events to publish to timeline service
  • Timeline service no longer guarantees that events are in time order.
  • Steaming fixes:
    • Streaming now uses event id for paging
    • Streaming now works when order is set to descending (each batch is identical to ascending now, but in reverse order)
    • Added integration test to ensure streaming works with the paging logic

closes Rate limit and batch new events when publishing to timeline #3671

@matt2e matt2e requested review from a team and alecthomas as code owners December 9, 2024 03:28
@matt2e matt2e requested review from wesbillman and removed request for a team December 9, 2024 03:28
@matt2e matt2e marked this pull request as draft December 9, 2024 03:28
@matt2e
feat: timeline batching
4118dab 
# Conflicts:
#	cmd/ftl-controller/main.go
#	frontend/cli/main.go
@matt2e matt2e force-pushed the matt2e/timeline-batch branch from f7a2904 to 4118dab Compare December 9, 2024 03:32
@matt2e matt2e marked this pull request as ready for review December 9, 2024 03:48
@matt2e matt2e enabled auto-merge (squash) December 9, 2024 03:57
@matt2e matt2e merged commit 4c88c60 into main Dec 9, 2024
96 checks passed
@matt2e matt2e deleted the matt2e/timeline-batch branch December 9, 2024 04:00
@ftl-robot ftl-robot mentioned this pull request Dec 9, 2024
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@alecthomas alecthomas Awaiting requested review from alecthomas alecthomas is a code owner

@wesbillman wesbillman Awaiting requested review from wesbillman wesbillman was automatically assigned from TBD54566975/ftl-frontend-team

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Rate limit and batch new events when publishing to timeline
1 participant
@matt2e