Adds Prometheus metrics for API and client #14783
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This adds the `prometheus_client` library as a client-side dependency. This
is a very lightweight (<500KB compiled) library with no transitive dependencies.
I'm choosing Prometheus here as it is the current standard for metrics from the
CNCF, and because its Python client library is very light and thus suitable for
inclusion on the client-side of Prefect.
There are two "sides" that expose a Prometheus metrics endpoint:
* The Prefect server will expose it from startup if `PREFECT_API_ENABLE_METRICS`
is on. This endpoint is at `/api/metrics` (to avoid any potential collision
with future UI routes).
* The Prefect client SDK will expose these from any of the following points if
`PREFECT_CLIENT_ENABLE_METRICS` is on:
* Serving one or more flow via `serve(...)`
* Serving one or more tasks via `serve(...)`
* Entering a flow run context for a flow (e.g. entering the engine for a flow
as a Kubernetes job)
The client metrics will be served via `localhost` at the port defined in
`PREFECT_CLIENT_METRICS_PORT`
Note that I haven't included any Prefect-specific metrics here, so we will start
with the stock Python-oriented metrics that come out of the box with
`prometheus_client`.
chrisguidry
requested review from
discdiver,
daniel-prefect and
cicdw
as code owners
CodSpeed Performance ReportMerging #14783 will not alter performanceComparing Summary
|
jakekaplan
approved these changes
Jul 29, 2024
chrisguidry
added a commit
that referenced
this pull request
Jul 29, 2024
Following the addition of the `prometheus_client` in #14783, this adds counters for measuring the performance of Prefect event emissions and subscription. A note about unit tests: I don't traditionally add unit tests for "leaf-level" instrumentation like this. Leaf-level here means it is measuring something about the system and doesn't form part of a measurement API (like if there were middleware for measuring HTTP latency, for example). Unless it is particularly complex to calculate, I generally skip extra unit tests and let the standard test suite and coverage inform me if the instrumentation might be a problem or not executed.
4 tasks
|
When can we expect this PR changes to be included in the release ? @jakekaplan @chrisguidry |
|
@akashtyagi08 these changes are in Prefect's 3.0 series, which we released on September 3! Give it a try and let us know how it goes, with either issues or a discussion. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This adds the
prometheus_clientlibrary as a client-side dependency. Thisis a very lightweight (<500KB compiled) library with no transitive dependencies.
I'm choosing Prometheus here as it is the current standard for metrics from the
CNCF, and because its Python client library is very light and thus suitable for
inclusion on the client-side of Prefect.
There are two "sides" that expose a Prometheus metrics endpoint:
The Prefect server will expose it from startup if
PREFECT_API_ENABLE_METRICSis on. This endpoint is at
/api/metrics(to avoid any potential collisionwith future UI routes).
The Prefect client SDK will expose these from any of the following points if
PREFECT_CLIENT_ENABLE_METRICSis on:serve(...)serve(...)as a Kubernetes job)
The client metrics will be served via
localhostat the port defined inPREFECT_CLIENT_METRICS_PORTNote that I haven't included any Prefect-specific metrics here, so we will start
with the stock Python-oriented metrics that come out of the box with
prometheus_client.Checklist
<link to issue>"mint.json.