Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Bloom-compactor Sharding #11154

Merged
merged 17 commits into from
Nov 10, 2023
Merged

Bloom-compactor Sharding #11154

Show file tree
Hide file tree
Changes from 3 commits
Commits
Show all changes
17 commits
Select commit Hold shift + click to select a range
49d8a39
Bloom-compactor sharding
salvacorts Oct 31, 2023
f0abf1d
Merge branch 'main' into salvacorts/bloomcompactor-sharding
salvacorts Nov 8, 2023
85b6fae
merge main followups
salvacorts Nov 8, 2023
18fdaa2
Remove duplicated logic inside runCompact
salvacorts Nov 8, 2023
5211754
WIP apply CR feedback
salvacorts Nov 8, 2023
12b31f4
Fix test
salvacorts Nov 8, 2023
e0fa945
Chnage metrics to counter
salvacorts Nov 8, 2023
27240d7
Change default shard size
salvacorts Nov 8, 2023
368f8f0
Extract common logic for sharding from bloom-gw and bloom-compactor i…
salvacorts Nov 8, 2023
8172c22
WIP CR feedback
salvacorts Nov 8, 2023
7f5bd55
Remove Series struct in favor of Job
salvacorts Nov 8, 2023
930d21e
Fix tests and lint issues
salvacorts Nov 8, 2023
8b1b3cc
Update docs
salvacorts Nov 8, 2023
8556a72
CR feedback
salvacorts Nov 10, 2023
8705806
Merge branch 'main' into salvacorts/bloomcompactor-sharding
salvacorts Nov 10, 2023
6b7d7a3
Fixes after merge
salvacorts Nov 10, 2023
34bc1ff
More fixes
salvacorts Nov 10, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
15 changes: 15 additions & 0 deletions docs/sources/configure/_index.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2532,6 +2532,16 @@ ring:
[enabled: <boolean> | default = false]

[working_directory: <string> | default = ""]

[compaction_interval: <duration>]

[compaction_retries: <int>]

[tables_to_compact: <int>]

[skip_latest_n_tables: <int>]

[max_compaction_parallelism: <int>]
```

### limits_config
Expand Down Expand Up @@ -2927,6 +2937,11 @@ shard_streams:
# CLI flag: -bloom-gateway.shard-size
[bloom_gateway_shard_size: <int> | default = 1]

# The shard size defines how many bloom compactors should be used by a tenant
# when computing blooms.
# CLI flag: -bloom-compactor.shard-size
[bloom_compactor_shard_size: <int> | default = 0]

# Allow user to send structured metadata in push payload.
# CLI flag: -validation.allow-structured-metadata
[allow_structured_metadata: <boolean> | default = false]
Expand Down
Loading
Loading