Unable to get Badger to work

[nickmarkfurness]

Hi,

I am running Jaeger all-in-one just fine using all the defaults without the need of a config file.
I now want to persist the data coming in to disk so it is not lost and want to setup Badger but i am unable to get it to work as i expect.

Info:
This is windows 10
The latest version to date of writing this
I have run:
set SPAN_STORAGE_TYPE=badger
in the same dos window i use to start Jaeger to set the environment variable.
I have created a config.yaml file along side the .exe
I have created the folder structure within the folder the all-in-one .exe is running
The dos window is running as admin too.

The command to start Jaeger is:
jaeger-all-in-one.exe --config-file=config.yaml

config.yaml:

service:
  extensions: [jaeger_storage, jaeger_query, healthcheckv2]
  pipelines:
    traces:
      receivers: [otlp]
      processors: [batch]
      exporters: [jaeger_storage_exporter]
  telemetry:
    resource:
      service.name: jaeger
    metrics:
      level: detailed
      address: 0.0.0.0:8888
    logs:
      level: info
    # TODO Initialize telemetry tracer once OTEL released new feature.
    # https://github.com/open-telemetry/opentelemetry-collector/issues/10663

extensions:
  healthcheckv2:
    use_v2: true
    http:

  jaeger_query:
    storage:
      traces: some_store
      traces_archive: another_store
    #ui:
      #config_file: ./cmd/jaeger/config-ui.json

  jaeger_storage:
    backends:
      some_store:
        badger:
          directories:
            keys: "/tmp/jaeger/"
            values: "/tmp/jaeger/"
          ephemeral: false
      another_store:
        badger:
          directories:
            keys: "/tmp/jaeger_archive/"
            values: "/tmp/jaeger_archive/"
          ephemeral: false

receivers:
  otlp:
    protocols:
      grpc:
      http:

processors:
  batch:

exporters:
  jaeger_storage_exporter:
    trace_storage: some_store

console output:

2024/12/03 12:28:14 maxprocs: Leaving GOMAXPROCS=2: CPU quota undefined
2024/12/03 12:28:14 application version: git-commit=3f752c2f434fabde20694816c264cfe8b546c575, git-version=v1.63.0, build-date=2024-11-10T18:53:12Z
{"level":"info","ts":1733228894.1549802,"caller":"flags/service.go:99","msg":"Mounting metrics handler on admin server","route":"/metrics"}
{"level":"info","ts":1733228894.1549802,"caller":"flags/service.go:105","msg":"Mounting expvar handler on admin server","route":"/debug/vars"}
{"level":"info","ts":1733228894.1561146,"caller":"flags/admin.go:119","msg":"Mounting health check on admin server","route":"/"}
{"level":"info","ts":1733228894.1561146,"caller":"flags/admin.go:133","msg":"Starting admin HTTP server","http-addr":":14269"}
{"level":"info","ts":1733228894.1561146,"caller":"flags/admin.go:111","msg":"Admin server started","http.host-port":"[::]:14269","health-status":"unavailable"}
{"level":"info","ts":1733228894.1561146,"caller":"[email protected]/clientconn.go:162","msg":"[core] original dial target is: \"localhost:4317\""}
{"level":"info","ts":1733228894.1569471,"caller":"[email protected]/clientconn.go:440","msg":"[core] [Channel #1]Channel created"}
{"level":"info","ts":1733228894.1569471,"caller":"[email protected]/clientconn.go:193","msg":"[core] [Channel #1]parsed dial target is: resolver.Target{URL:url.URL{Scheme:\"dns\", Opaque:\"\", User:(*url.Userinfo)(nil), Host:\"\", Path:\"/localhost:4317\", RawPath:\"\", OmitHost:false, ForceQuery:false, RawQuery:\"\", Fragment:\"\", RawFragment:\"\"}}"}
{"level":"info","ts":1733228894.1569471,"caller":"[email protected]/clientconn.go:194","msg":"[core] [Channel #1]Channel authority set to \"localhost:4317\""}
badger 2024/12/03 12:28:14 INFO: All 0 tables opened in 0s
badger 2024/12/03 12:28:14 INFO: Discard stats nextEmptySlot: 0
badger 2024/12/03 12:28:14 INFO: Set nextTxnTs to 0
{"level":"info","ts":1733228894.1893773,"caller":"badger/factory.go:162","msg":"Badger storage configuration","configuration":{"Dir":"C:\\Users\\OTEL\\AppData\\Local\\Temp\\badger1881882557","ValueDir":"C:\\Users\\OTEL\\AppData\\Local\\Temp\\badger1881882557","SyncWrites":false,"NumVersionsToKeep":1,"ReadOnly":false,"Logger":{},"Compression":1,"InMemory":false,"MetricsEnabled":true,"NumGoroutines":8,"MemTableSize":67108864,"BaseTableSize":2097152,"BaseLevelSize":10485760,"LevelSizeMultiplier":10,"TableSizeMultiplier":2,"MaxLevels":7,"VLogPercentile":0,"ValueThreshold":1048576,"NumMemtables":5,"BlockSize":4096,"BloomFalsePositive":0.01,"BlockCacheSize":268435456,"IndexCacheSize":0,"NumLevelZeroTables":5,"NumLevelZeroTablesStall":15,"ValueLogFileSize":1073741823,"ValueLogMaxEntries":1000000,"NumCompactors":4,"CompactL0OnClose":false,"LmaxCompaction":false,"ZSTDCompressionLevel":1,"VerifyValueChecksum":false,"EncryptionKey":"","EncryptionKeyRotationDuration":864000000000000,"BypassLockGuard":false,"ChecksumVerificationMode":0,"DetectConflicts":true,"NamespaceOffset":-1,"ExternalMagicVersion":0}}
{"level":"info","ts":1733228894.1893773,"caller":"static/provider.go:57","msg":"No sampling strategies source provided, using defaults"}
{"level":"info","ts":1733228894.1923037,"caller":"[email protected]/server.go:684","msg":"[core] [Server #2]Server created"}
{"level":"info","ts":1733228894.1935275,"caller":"server/grpc.go:93","msg":"Starting jaeger-collector gRPC server","grpc.host-port":"[::]:14250"}
{"level":"info","ts":1733228894.1935275,"caller":"server/http.go:45","msg":"Starting jaeger-collector HTTP server","http host-port":":14268"}
{"level":"info","ts":1733228894.1942978,"caller":"app/collector.go:139","msg":"Not listening for Zipkin HTTP traffic, port not configured"}
{"level":"info","ts":1733228894.1949322,"caller":"handler/otlp_receiver.go:69","msg":"OTLP receiver status change","status":"StatusStarting"}
{"level":"warn","ts":1733228894.1949322,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
{"level":"info","ts":1733228894.1949322,"caller":"[email protected]/server.go:684","msg":"[core] [Server #4]Server created"}
{"level":"info","ts":1733228894.1949322,"caller":"[email protected]/otlp.go:112","msg":"Starting GRPC server","endpoint":":4317"}
{"level":"warn","ts":1733228894.1958146,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
{"level":"info","ts":1733228894.1942978,"caller":"[email protected]/server.go:880","msg":"[core] [Server #2 ListenSocket #3]ListenSocket created"}
{"level":"info","ts":1733228894.1958146,"caller":"[email protected]/server.go:880","msg":"[core] [Server #4 ListenSocket #5]ListenSocket created"}
{"level":"info","ts":1733228894.1958146,"caller":"[email protected]/otlp.go:169","msg":"Starting HTTP server","endpoint":":4318"}
{"level":"info","ts":1733228894.1958146,"caller":"querysvc/query_service.go:138","msg":"Archive storage not created","reason":"archive storage not supported"}
{"level":"info","ts":1733228894.1967673,"caller":"app/flags.go:142","msg":"Archive storage not initialized"}
{"level":"warn","ts":1733228894.1967673,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
{"level":"info","ts":1733228894.1967673,"caller":"[email protected]/server.go:684","msg":"[core] [Server #6]Server created"}
{"level":"info","ts":1733228894.1977606,"caller":"app/static_handler.go:92","msg":"Using UI configuration","path":""}
{"level":"warn","ts":1733228894.1977606,"caller":"[email protected]/warning.go:40","msg":"Using the 0.0.0.0 address exposes this server to every network interface, which may facilitate Denial of Service attacks.","documentation":"https://github.com/open-telemetry/opentelemetry-collector/blob/main/docs/security-best-practices.md#safeguards-against-denial-of-service-attacks"}
{"level":"info","ts":1733228894.1977606,"caller":"app/server.go:270","msg":"Query server started","http_addr":"[::]:16686","grpc_addr":"[::]:16685"}
{"level":"info","ts":1733228894.1977606,"caller":"healthcheck/handler.go:118","msg":"Health Check state change","status":"ready"}
{"level":"info","ts":1733228894.1977606,"caller":"app/server.go:355","msg":"Starting GRPC server","port":16685,"addr":":16685"}
{"level":"info","ts":1733228894.198773,"caller":"[email protected]/server.go:880","msg":"[core] [Server #6 ListenSocket #7]ListenSocket created"}
{"level":"info","ts":1733228894.198773,"caller":"app/server.go:336","msg":"Starting HTTP server","port":16686,"addr":":16686"}

The problem:

It seems to be starting badger but it keeps using and creating a new folder within the local user tmp folder ... so it does not retain data between restarts.
I don't need a powerful distributed option, simple and local is fine.

Can anyone tell me what i am missing?

[dosubot[bot]]

Hey @nickmarkfurness! I'm here to help you out with any bugs, questions, or contributions you have. Let's dive into this together!

It looks like you're trying to configure Badger to persist data to a specific directory on Windows 10, but it's defaulting to a temporary folder. To resolve this, you need to ensure that the directory paths in your config.yaml are correctly set to your desired location on disk. Here's how you can adjust your configuration:

1. Update the directories paths in your config.yaml to point to a specific directory on your disk, such as C:\jaeger_data. Here's an example configuration:

extensions:
  jaeger_storage:
    backends:
      some_store:
        badger:
          directories:
            keys: "C:\\jaeger_data\\"
            values: "C:\\jaeger_data\\"
          ephemeral: false
      another_store:
        badger:
          directories:
            keys: "C:\\jaeger_data_archive\\"
            values: "C:\\jaeger_data_archive\\"
          ephemeral: false

2. Ensure that the ephemeral flag is set to false to allow data persistence across restarts ^[1].

By setting the directories to a specific path on your disk and ensuring ephemeral is false, Badger should store data in the specified location and retain it between restarts.

To continue talking to Dosu, mention @dosu.

[nickmarkfurness]

I have created the folders you used in your example and changed my config.
This has not worked for me sorry, i get the same output when i start Jaeger

[nickmarkfurness]

Resolved. I dropped the use of a config file and specified my overrides via startup command only:

jaeger-all-in-one.exe --badger.ephemeral=false --badger.directory-key="C:\jaeger_data\keys" --badger.directory-value="C:\jaeger_data\values"

To get a list of all the overrides run:

jaeger-all-in-one.exe --help

[yurishkuro]

note - jaeger-all-in-one.exe is a v1 binary. The config file you showed is a config for Jaeger v2 (jaeger.exe binary).