Add Open Telememetry in Python backend

triton-inference-server · Jun 19, 2024 · f51b272 · f51b272
1 parent 0966bf2
commit f51b272
Show file tree

Hide file tree

Showing 21 changed files with 866 additions and 98 deletions.
diff --git a/CHANGELOG.md b/CHANGELOG.md
@@ -16,7 +16,9 @@ limitations under the License.
 
 # Changelog
 
-## Unreleased
+## 0.5.7 (2024-06-19)
+
+- New: Open Telemetry propagation support for tracing
 
 [//]: <> (put here on external component update with short summary what change or link to changelog)
 

diff --git a/README.md b/README.md
@@ -28,7 +28,7 @@ The distinct capabilities of PyTriton are summarized in the feature matrix:
 | ------- | ----------- |
 | Native Python support | You can create any [Python function](https://triton-inference-server.github.io/pytriton/latest/inference_callables/) and expose it as an HTTP/gRPC API. |
 | Framework-agnostic | You can run any Python code with any framework of your choice, such as: PyTorch, TensorFlow, or JAX. |
-| Performance optimization | You can benefit from [dynamic batching](https://triton-inference-server.github.io/pytriton/latest/inference_callables/decorators/#batch), response cache, model pipelining, [clusters](https://triton-inference-server.github.io/pytriton/latest/guides/deploying_in_clusters/), and GPU/CPU inference. |
+| Performance optimization | You can benefit from [dynamic batching](https://triton-inference-server.github.io/pytriton/latest/inference_callables/decorators/#batch), response cache, model pipelining, [clusters](https://triton-inference-server.github.io/pytriton/latest/guides/deploying_in_clusters/), performance [tracing](https://triton-inference-server.github.io/pytriton/latest/guides/distributed_tracing/), and GPU/CPU inference.
 | Decorators | You can use batching [decorators](https://triton-inference-server.github.io/pytriton/latest/inference_callables/decorators/) to handle batching and other pre-processing tasks for your inference function. |
 | Easy [installation](https://triton-inference-server.github.io/pytriton/latest/installation/) and setup | You can use a simple and familiar interface based on Flask/FastAPI for easy installation and [setup](https://triton-inference-server.github.io/pytriton/latest/binding_models/).  |
 | [Model clients](https://triton-inference-server.github.io/pytriton/latest/clients)   | You can access high-level model clients for HTTP/gRPC requests with configurable options and both synchronous and [asynchronous](https://triton-inference-server.github.io/pytriton/latest/clients/#asynciomodelclient)  API. |

diff --git a/docs/README.md b/docs/README.md
@@ -202,6 +202,8 @@ used to profile models served through PyTriton. We have prepared an example of
 using the Perf Analyzer to profile the BART PyTorch model. The example code can be found
 in [examples/perf_analyzer](../examples/perf_analyzer).
 
+Open Telemetry is a set of APIs, libraries, agents, and instrumentation to provide observability for cloud-native software. We have prepared an
+[guide](guides/distributed_tracing.md) on how to use Open Telemetry with PyTriton.
 
 ## What next?
 

diff --git a/docs/assets/jaeger_traces_list.png b/docs/assets/jaeger_traces_list.png
diff --git a/docs/assets/jaeger_traces_list_only_triton.png b/docs/assets/jaeger_traces_list_only_triton.png
diff --git a/docs/assets/jaeger_traces_list_propagation.png b/docs/assets/jaeger_traces_list_propagation.png
diff --git a/docs/guides/assets/jaeger_trace_details.png b/docs/guides/assets/jaeger_trace_details.png
diff --git a/docs/guides/assets/jaeger_traces_list.png b/docs/guides/assets/jaeger_traces_list.png
diff --git a/docs/guides/assets/jaegger_context_propagation_request_details.png b/docs/guides/assets/jaegger_context_propagation_request_details.png
diff --git a/docs/guides/assets/jaegger_context_propagation_traces_list.png b/docs/guides/assets/jaegger_context_propagation_traces_list.png