chore(k212): release 3.1.0

[loki-gh-app]

🤖 I have created a release beep boop

3.1.0

3.1.0 (2024-07-23)

⚠ BREAKING CHANGES

• update helm chart to support distributed mode and 3.0 (#12067)
• api: Fail log queries when executed on instant query endpoint (#13421)

Features

• Add a version of the mixin dashboards for meta monitoring (#12700) (ec1a057)
• Add backoff to flush op (#13140) (9767807)
• Add CLI to inspect RF=1 WAL segments (#13552) (150e653)
• add detected-fields command to logcli (#12739) (210ea93)
• Add ingester_chunks_flush_failures_total (#12925) (cc3694e)
• add lokitool (#12166) (7b7d3d4)
• Add metrics for Ingester RF-1 (#13510) (d4179aa)
• Add metrics for number of patterns detected & evicted (#12918) (bc53b33)
• Add metrics to WAL Manager (#13490) (bfe97d7)
• Add new Drain tokenizer that splits on most punctuation (#13143) (6a0fdd0)
• Add pattern ingester support in SSD mode (#12685) (19bfef4)
• add profile tagging to ingester (#13068) (00d3c7a)
• add recalculateOwnedStreams to check stream ownership if the ring is changed (#13103) (e7689b2)
• Add settings for cpu/mutex/block profiling options (#13278) (f06eabb)
• Add step param to Patterns Query API (#12703) (7b8533e)
• Add tokenizer interface for Drain Training (#13069) (797bb64)
• add toleration for bloom components (#12653) (fcb2b0a)
• Add utf8 support to Pattern Lexer to support utf8 chars (#13085) (f6f8bab)
• add warnings to metadata context directly (#12579) (c4ac8cc)
• Added getting started video (#12975) (8442dca)
• Added getting started video for ingesting Otel logs (#13226) (5e560f9)
• Added Interactive Sandbox to Quickstart tutorial (#12701) (97212ea)
• Added video and updated Grafana Agent -> Alloy (#13032) (1432a3e)
• API: Expose optional label matcher for label names API (#11982) (8084259)
• api: Fail log queries when executed on instant query endpoint (#13421) (ce71f1c)
• area/promtail: Added support to install wget on promtail docker image to support docker healthcheck (#11711) (ffe684c)
• blooms: Add counter metric for blocks that are not available at query time (#12968) (d6374bc)
• blooms: Add in-memory LRU cache for meta files (#12862) (fcd544c)
• blooms: Blooms/v2 encoding multipart series (#13093) (fbe7c55)
• blooms: compute chunks once (#12664) (bc78d13)
• blooms: ignore individual bloom-gw failures (#12863) (4c9b22f)
• blooms: ingester aware bounded impl (#12840) (7bbd8b5)
• bloom: Skip attempts to filter chunks for which blooms have not been built (#12961) (a1b1eeb)
• blooms: limit bloom size during creation (#12796) (eac5622)
• blooms: record time spent resolving shards (#12636) (9c25985)
• blooms: Separate page buffer pools for series pages and bloom pages (#12992) (75ccf21)
• Boilerplate for new bloom build planner and worker components. (#12989) (8978ecf)
• Bootstrap metastore for wal segments (#13550) (0b47498)
• cache: Add Cache-Control: no-cache support for Loki instant queries. (#12896) (88e545f)
• canary: Add test to check query results with and without cache. (#13104) (71507a2)
• chunks-inspect: support structured metadata (#11506) (1834065)
• collect and serve pre-aggregated bytes and counts (#13020) (467eb1b)
• Collect duplicate log line metrics (#13084) (40ee766)
• convert WalSegmentWriter to io.ReadSeeker (#13340) (19c0509)
• Detected labels from store (#12441) (587a6d2)
• detected-labels: include labels with cardinality > 1 (#13128) (8be8364)
• detectedFields: add parser to response (#12872) (2b3ae48)
• detectedFields: Support multiple parsers to be returned for a single field (#12899) (19fef93)
• downsample aggregated metrics (#13449) (2c053ee)
• Drain uses different tokenizer based on log format (#13384) (bc01e6f)
• Enable log volume endpoint by default (#12628) (397aa56)
• Enable log volume endpoint by default in helm (#12690) (e39677f)
• exclude and from creating new tokens in patterns (#13395) (8c8454b)
• exclude in pattern tokens (#13397) (e612dd3)
• exporter: include boolean values in limit exporter (#13466) (4220737)
• flush not owned streams (#13254) (2ca1ac6)
• Generic logline placeholder replacement and tokenization (#12799) (4047902)
• helm: Allow extraObject items as multiline strings (#12397) (af5be90)
• helm: Make gateway container port configurable. (#13294) (05176e4)
• helm: Support alibabacloud oss in helm chart (#13441) (3ebab6f)
• helm: Support for PVC Annotations for Non-Distributed Modes (#12023) (efdae3d)
• improve Owned Streams feature observability (#13232) (ce86459)
• improve performance of first_over_time and last_over_time queries by sharding them (#11605) (f66172e)
• improve placeholder replacement of byte sizes (#13508) (ac284ca)
• improve syntax parser for pattern (#12489) (48dae44)
• include the stream we failed to create in the stream limit error message (#12437) (ec81991)
• Increase drain max depth from 8 -> 30 (#13063) (d0a2859)
• Ingester RF-1 (#13365) (7f35179)
• ingester: Smooth out chunk flush operations (#9994) (82fbb2f)
• Introduce index audit to lokitool (#13008) (47f0236)
• Introduce a new Object Storage WAL format. (#13253) (1d6f8d5)
• lambda-promtail; ensure messages to Kinesis are usable by refactoring parsing of KinesisEvent to match parsing of CWEvents + code cleanup (#13098) (dbfb19b)
• loki/main.go: Log which config file path is used on startup (#12985) (7a3338e)
• loki: Do not enforce max-query-bytes-read and max-querier-bytes-read in limited tripperware (#13406) (47f6ea5)
• new stream count limiter (#13006) (1111595)
• operator: Add support for the volume API (#13369) (d451e23)
• operator: Update Loki operand to v3.1.0 (#13422) (cf5f52d)
• Optimize log parsing performance by using unsafe package (#13223) (9f31b25)
• parameterise the MaximumEventAgeInSeconds, LogGroupName, and IAMRoleName for lambda-promtail CloudFormation template (#12728) (8892dc8)
• Pattern ingesters add a limiter for high eviction rate (#13464) (e08b4a7)
• promtail: Support of RFC3164 aka BSD Syslog (#12810) (be41525)
• Querier: Split gRPC client into two. (#12726) (7b6f057)
• Remove flush loop and queue from Ingester RF-1 (#13538) (8ca03a2)
• reporting: Report cpu usage (#12970) (87288d3)
• RF1 improves replication stategy to support 1 replica (#13469) (790135b)
• split detected fields queries (#12491) (6c33809)
• Support negative numbers in LogQL (#13091) (6df81db)
• Track when builders are doing work with a gauge (#13570) (0029d46)
• Tune Patterns query drain instance (#13137) (30df31e)
• Update doc-validator version (#13558) (a88a0d3)
• Update getting started demo to Loki 3.0 (#12723) (282e385)
• update helm chart to support distributed mode and 3.0 (#12067) (79b876b)
• Update Loki monitoring docs to new meta monitoring helm (#13176) (b4d44f8)
• Updated best practises for labels (#12749) (6ebfbe6)
• Updated SS and microservices deployment docs (#13083) (1b80458)
• Use prefixed WAL storage path in Object Storage (#13377) (973aa2d)
• Use WAL Manager (#13491) (8f1d12f)
• WAL Manager (#13428) (15c8b45)
• WAL Manager configuration options (#13531) (c4405fe)
• wal: Add sizing information to writer and reader. (#13267) (41fbacd)
• wal: Benchmark and improve WAL writes using Reset. (#13272) (debb5f2)

Bug Fixes

• codec not initialized in downstream roundtripper (#12873) (b6049f6)
• Add a missing continue in fuse which may cause incorrect bloom test result (#12650) (0d1ebeb)
• add a retry middleware to all the stats handlers (#13584) (7232795)
• Add copyString function to symbolizer to avoid retaining memory (#13146) (86b119a)
• add detected_level info when the info word appears on log message (#13218) (c9bfa3e)
• add logging to empty bloom (#13502) (c263a68)
• Add missing Helm helper loki.hpa.apiVersion (#12755) (3070ea7)
• Add missing OTLP endpoint to nginx config (#12709) (8096748)
• add missing parentheses in meta monitoring dashboards (#12802) (151d0a5)
• add retry middleware to the "limited" query roundtripper (#13161) (bb864b3)
• allow cluster label override in bloom dashboards (#13012) (987e551)
• blooms: bloomshipper no longer returns empty metas on fetch (#13130) (ad279e5)
• blooms: Clean block directories recursively on startup (#12895) (7b77e31)
• blooms: Correctly return unfiltered chunks for series that are not mapped to any block (#12774) (c36b114)
• blooms: Deduplicate filtered series and chunks (#12791) (3bf2d1f)
• blooms: Delete outdated metas during planning (#13363) (11e1976)
• blooms: Disable metas cache on bloom gateway (#12959) (00bdd2f)
• blooms: Do not fail requests when fetching metas from cache fails (#12838) (667076d)
• blooms: dont break iterator conventions (#12808) (1665e85)
• blooms: ensure tokenizer cache is reset between series (#13370) (04bc3a4)
• blooms: Fix partitionSeriesByDay function (#12900) (738c274)
• blooms: Fix a regression introduced with #12774 (#12776) (ecefb49)
• blooms: Fix eviction of multiple blockcache items (#13573) (c9950e3)
• blooms: Fix findGaps when ownership goes to MaxUInt64 and that is covered by existing meta (#12558) (0ee2a61)
• blooms: Fix panic in bloom gateway (#13303) (66f97b2)
• blooms: Fully deduplicate chunks from FilterChunkRef responses (#12807) (a0f358f)
• blooms: Handle not found metas gracefully (#12853) (37c8822)
• blooms: ignores bloom filtering errors in bounded shard query planning (#13285) (ede6941)
• blooms: improves mempool metrics (#13283) (d36e1d5)
• blooms: Minor fixes and improvements for testing in dev (#13341) (d0f56ee)
• blooms: Remove backoff from notify planner (#13506) (e506995)
• blooms: Remove unused arg (#13343) (fcb9b28)
• blooms: Reset error on LazyBloomIter.Seek (#12806) (76ba24e)
• blooms: skip empty blooms on reads (#13500) (bfa6955)
• blooms: Suppress error from resolving server addresses for blocks (#13385) (3ac2317)
• change log level since this is a known case (#13029) (ca030a5)
• close res body (#12444) (616977a)
• Correctly encode step when translating proto to http internally (#13171) (740551b)
• crrect initialization of a few slices (#12674) (0eba448)
• Dedup pattern tokens on output (#13534) (e23598d)
• Defer closing blocks iter after checking error from loadWorkForGap (#12934) (cb1f5d9)
• deps: update module github.com/docker/docker to v25.0.5+incompatible [security] (main) (#12279) (960c034)
• detected fields incorrect type bug (#13515) (f6a94d3)
• Do not filter out chunks for store when From==Through and From==start (#13117) (d9cc513)
• docs: broken link in getting started readme (#12736) (425a2d6)
• docs: Move promtail configuration to the correct doc (#12737) (1161846)
• Ensure Drain patterns are valid for LogQL pattern match filter (#12815) (fd2301f)
• errors reported by the race detector (#13174) (2b19dac), closes #8586
• Fix bloom deleter PR after merge (#13167) (c996349)
• Fix compactor matcher in the loki-deletion dashboard (#12790) (a03846b)
• Fix duplicate enqueue item problem in bloom download queue when do sync download (#13114) (f98ff7f)
• Fix for how the loop sync is done (#12941) (5cd850e)
• Fix HPA ingester typo (#13158) (4ca9785)
• Fix incorrect sorting of chunks in bloom-filtered response since ChunkRef.Cmp method is used in reverse (#12999) (670cd89)
• Fix indentation of query range values in helm (#12577) (9823f20)
• Fix log level detection (#12651) (6904a65)
• Fix log line for fingerprint not found (#13555) (aeb23bb)
• Fix panic in BloomStore initialisation (#13457) (5f4b8fc)
• Fix panic in ingester.go (#13557) (dbff69a)
• Fix panic on requesting out-of-order Pattern samples (#13010) (2171f64)
• fix parsing of default per tenant otlp config (#12836) (7cc9a93)
• fix retry code to handle grpc status codes. updated newer stats retries to be wrapped with spans (#13592) (d3e1edb)
• fix setting of info log level when trying to detect level from log lines (#12635) (0831802)
• Fix the lokitool imports (#12673) (6dce988)
• Fixes pattern pruning stability (#13429) (7c86e65)
• Fixes read & backend replicas settings (#12828) (d751134)
• Fixes span name of serializeRounTripper (#13541) (4451d56)
• gateway client: Fix nil pointer dereference panic when using ruler in ring mode (#13436) (304db10)
• helm: Set compactor addr for distributed mode. (#12748) (521d40a)
• helm: fix extraObjects (#13107) (b7fcf2b)
• helm: Fix GEL image tag, bucket name and proxy URLs (#12878) (67ed2f7)
• helm: fix query-frontend and ruler targetPort 'http-metrics' in Service template (#13024) (1ab9d27)
• helm: fix queryScheduler servicemonitor (#12753) (8101e21)
• helm: fixed ingress paths mapping (#12932) (5ada92b)
• helm: only default bucket names when using minio (#12548) (2e32ec5)
• helm: Removed duplicate bucketNames from documentation and fixed key name deploymentMode (#12641) (0d8ff9e)
• helm: Update yaml file ./production/helm/loki/Chart.yaml (+1 other) (#13392) (b5b861c)
• helm: Update yaml file ./production/helm/loki/values.yaml (+1 other) (#13426) (fc3904e)
• incorrect compactor matcher in loki-deletion dashboard mixin (#12567) (006f88c)
• incorrect pod matcher for compactor in mixin when using ssd mode (#12846) (515e13c)
• indexstats: do not collect stats from "IndexStats" lookups for other query types (#12978) (1f5291a)
• Ingester zoneAwareReplication (#12659) (9edb0ce)
• Init internal server log along with loki's server instance (#13221) (66b8c9b)
• Introduce feature flag for [last|first]_over_time sharding. (#13067) (6e45550)
• Invalidate caches when pipeline wrappers are disabled (#12903) (a772ed7)
• ksonnet: Do not generate rbac for consul if you are using memberlist (#12688) (2d62fca)
• lambda-promtail, update s3 filename regex to allow finding of log files from AWS GovCloud regions (#12482) (7a81d26)
• Lint issues after merge to main (#13326) (7e19cc7)
• log results cache: include pipeline wrapper disabled in cache key (#13328) (221491c)
• loki version prefix in Makefile (#12514) (dff72d2)
• loki-operational.libsonnet (#12789) (51a841f)
• make detected fields work for both json and proto (#12682) (f68d1f7)
• make the tsdb filenames correctly reproducible from the identifier (#12536) (ec888ec)
• Missing password for Loki-Canary when loki.auth_enabled is true (#12411) (68b23dc)
• mixin generation when cluster label is changed (#12613) (1ba7a30)
• mixin: dashboards $__auto fix (#12707) (91ef72f)
• Mixins - Add missing log datasource on loki-deletion (#13011) (1948899)
• mixins: Align loki-writes mixins with loki-reads (#13022) (757b776)
• mixin: various latency panels in operational dashboard should have ms unit type instead of seconds (#13260) (f5a9905)
• nix: lambda-promtail vendor hash (#12763) (ae180d6)
• not owned stream count (#13030) (4901a5c)
• operator: add alertmanager client config to ruler template (#13182) (6148c37)
• operator: Allow structured metadata only if V13 schema provided (#13463) (3ac130b)
• operator: Bump golang builder to 1.21.9 (#12503) (f680ee0)
• operator: Configure Loki to use virtual-host-style URLs for S3 AWS endpoints (#12469) (0084262)
• operator: Improve API documentation for schema version (#13122) (3a9f50f)
• operator: Remove duplicate conditions from status (#13497) (527510d)
• operator: Set object storage for delete requests when using retention (#13562) (46de4c1)
• operator: Skip updating annotations for serviceaccounts (#13450) (1b9b111)
• operator: Support v3.1.0 in OpenShift dashboards (#13430) (8279d59)
• operator: Use a minimum value for replay memory ceiling (#13066) (4f3ed77)
• operator: Watch for CredentialsRequests on CCOAuthEnv only (#13299) (7fc926e)
• Optimize regular initialization (#12926) (a46d14f)
• orFilters: fix multiple or filters would get wrong filtertype (#13169) (9981e9e)
• otel: Map 500 errors to 503 (#13173) (b31e04e)
• packaging: Require online network in systemd unit file for Loki and Promtail (#12741) (57f78b5)
• panics when ingester response is nil (#12946) (3cc28aa)
• promtail race fixes (#12656) (4e04d07)
• promtail; clean up metrics generated from logs after a config reload. (#11882) (39a7181)
• promtail: Fix bug with Promtail config reloading getting stuck indefinitely (#12795) (4d761ac)
• promtail: Fix UDP receiver on syslog transport (#10708) (a00f1f1)
• promtail: Handle docker logs when a log is split in multiple frames (#12374) (c0113db)
• Properly reset wal segment writer (#13468) (6ea83b4)
• properly return http status codes from ingester to querier for RPC function calls (#13134) (691b174)
• querier address in SingleBinary mode (#13297) (29f1ea9)
• query engine: Include lines with ts equal to end timestamp of the query range when executing range aggregations (#13448) (e0ca67d)
• query sharding: Generalize avg -> sum/count sharding using existing binop mapper (#12599) (11e7687)
• Redo ingester profile tagging (#13239) (32097c8)
• regression: reverts grafana/loki#13039 to prevent use-after-free corruptions (#13162) (41c5ee2)
• Remove Hardcoded Bucket Name from EventBridge Example CloudFormation Template (#12609) (8c18463)
• Remove query size limit for detected fields (#13423) (1fa5127)
• remove tenant label tagging from profiles to reduce cardinality (#13270) (f897758)
• remove unneccessary disk panels for ssd read path (#13014) (8d9fb68)
• remove unused parameter causing lint error (#12801) (33e82ec)
• Retain original values in logfmt pattern tokenizer (#13535) (5fa9c4b)
• retry: fix retries when using protobuf encoding (#13316) (a457c5d)
• Return empty vector instead of nil for empty evaluator. (#13485) (08615bf)
• separates directory creation from permission checks (#13248) (1086783)
• sharding: use without() grouping when merging avg_over_time shard results (#12176) (eb8a363)
• spans: corrects early-close for a few spans (#12887) (93aaf29)
• stream ownership check (#13314) (5ae5b31)
• support multi-zone ingesters when converting global to local limits for streams in limiter.go (#13321) (e28c15f)
• temporarily moving from alloy -> alloy dev (#13062) (7ffe0fb)
• Track bytes discarded by ingester. (#12981) (88c6711)
• Update expected patterns when pruning (#13079) (2923a7d)
• update fixed limit once streams ownership re-checked (#13231) (7ac19f0)
• Update Loki v3 Helm statefulset-ingester.yaml template (#13118) (5b4e576)
• update to build image 0.33.2, fixes bug with promtail windows DNS resolution (#12732) (759f42d)
• updated all dockerfiles go1.22 (#12708) (71a8f2c)
• Updated Loki Otlp Ingest Configuration (#12648) (ff88f3c)
• upgrade old plugin for the loki-operational dashboard. (#13016) (d3c9cec)
• Use an intermediate env variable in GH workflow (#12905) (772616c)
• Use to the proper config names in warning messages (#12114) (4a05964)
• workflows: don't run metric collector on forks (#12687) (7253444)

Performance Improvements

• blooms: always return bloom pages to allocator (#13288) (0cb3ff1)
• blooms: Avoid tiny string allocations for insert cache (#13487) (652ad24)
• blooms: mempool no longer zeroes out buffers unnecessarily (#13282) (eb1cd4c)
• blooms: Resolve bloom blocks on index gateway and shard by block address (#12720) (5540c92)
• Gather aggregate per-line and per-tenant metrics for Drain patterns (#13368) (bf1d6e3)
• Improve Detected labels API (#12816) (e7fdeb9)
• Introduce fixed size memory pool for bloom querier (#13039) (fc26431)
• Limit tokens per pattern to 128 (#13376) (a1efafd)
• mempool: Replace sync.Mutex with sync.Once (#13293) (61a9854)
• Prune unused pattern branches (#13329) (5ef18cf)
• Re-introduce fixed size memory pool for bloom querier (#13172) (4117b6c)
• Replace channel check with atomic bool in tailer.send() (#12976) (4a5edf1)
• TSDB: Add fast-path to inversePostingsForMatcher (#12679) (402d1d7)

---

Merging this PR will release the artifacts of 182165a

[grafanabot]

This PR must be merged before a backport PR will be created.

[grafanabot]

This PR must be merged before a backport PR will be created.

[grafanabot]

This PR must be merged before a backport PR will be created.