Integrate into ZIO Logging & Metrics

[jdegoes]

The goal of ZIO Telemetry should be to seamlessly add OpenTelemetry support via ZIO's own built-in logging and metrics, adding integration points as necessary and useful.

TODO

• How to map ZIO logging + annotations + other context to OpenTelemetry traces
• How to map ZIO logging + annotations + other context to OpenTelemetry logs
• How to map ZIO metrics to OpenTelemetry metrics
• How to map ZIO context + annotations to OpenTelemetry baggage

[easel]

These goals sound incredible. What's less clear to me is how complete the coverage is in native ZIO capabilities. I know ZIO has some "tracing" capabilities but am much less clear what they are and how well they map to the equivalent Otel concepts. @jdegoes any suggestions as to how we can get starting mapping these capabilities?

[grouzen]

I'm integrating Baggage with ZIO log annotations in this PR: #662

[omidb]

is there any update on this?

[grouzen]

Hey! I'm still working on adding metrics support in general. After this, I believe mapping ZIO metrics to OTEL metrics should be possible.

The only part that was done long ago is mapping ZIO annotations to OpenTelemetry baggage. It is not reflected in docs, though, but you can check out the source code at https://github.com/zio/zio-telemetry/blob/series/2.x/opentelemetry/src/main/scala/zio/telemetry/opentelemetry/baggage/Baggage.scala and find some examples in https://github.com/zio/zio-telemetry/blob/series/2.x/opentelemetry/src/test/scala/zio/telemetry/opentelemetry/baggage/BaggageTest.scala#L89.

I was researching how to map ZIO logging and I don't think it is doable since there is no way to extract the original log message. I hope I am wrong about it.
Mapping annotations to traces and logs is also possible.

[omidb]

@grouzen thanks for the update. I'm trying to understand how does this work with something like myzio.logError(). Should I tapError and then create a Baggage? something like this: .tapError(e => baggage.set("some", e.getMessage))

[grouzen]

@omidb Yes, your solution is fine. As I said in the message above, there is no integration with ZIO logging at the moment and I don't think it is possible to make it happen due to technical reasons.

[grouzen]

Also, I've got another idea that we can potentially implement some sort of otel-bridge as a module of zio-logging library, which will use zio-telemetry as a backend, so the end user would be able to call ZIO.log* methods that will be sending logs to OTEL collector.

[grouzen]

@omidb Hello! The support for the Logs signal has been added recently, you can find it in the latest 3.0.0-RC-18 release version.

[grouzen]

Support for metrics has been added at #790

[omidb]

@grouzen thanks for the update. we will check this out. is there an example or doc?

[grouzen]

@omidb Of course! The docs were updated as well: https://zio.dev/zio-telemetry/opentelemetry/#logging
If you have any questions after reading the docs, please do not hesitate to ask them in the Discord channel.

[omidb]

@grouzen thanks. not related to this issue, I was wondering what do u use for visualizing all the traces, logs, metrics? do u have a favorite all in one ecosystem?

[grouzen]

@omidb I'm planning to add an example for Honeycomb. They have a nice free plan. It is enough to run the example and see good-looking visualized results in one place. Related issue: #791

[omidb]

@grouzen we use honeycomb internally. I want to find a good open source platform that does logging, tracing and metric all together. https://skywalking.apache.org/ looks interesting but it's not OTEL first and seems I have to go through hoops to get it to work.

[grouzen]

@omidb Sorry, do you mean Honeycomb doesn't provide a UI for all metrics? Yeah, Apache Skywalking could be a good option, but I also decided not to spend time on it for the same reason as yours.

[omidb]

@grouzen Honeycomb is limited to tracing and not a good log explorer. We can continue this discussion on Discord :)

[grouzen]

@omidb, As far as I know, DD provides a free plan, and it fits these requirements well. Moreover, there were many requests in the Discord channel to add an example for it :). The issue for adding a DD example: #690

[grouzen]

Closing this one since all goals have been achieved.