Skip to content

Commit

Permalink
Add faro.receiver component (#5314)
Browse files Browse the repository at this point in the history
This commit adds a component for faro.reciever, an equivalent to the app_agent_receiver integration from static mode. 

This is not 100% a straight port from the existing integration. I have done some minor refactoring to make it possible for this component to be dynamically updateable at runtime. However, the implementation remains mostly the same. 

During my porting, I have noticed bugs in integration, particularly around sourcemaps. I have not fixed these, but I have added TODO comments in the code for them to be resolved down the line. 

Unfortunately, the resulting code is quite complex. Future changes to refactor it to make it easier to read are warranted, but it would likely require more code and would drift further from the original integration. In the spirit of this being a "port," such changes are not included here. 

Supersedes #5243 
Closes #2320

Co-authored-by: CRA Keishi Kawada <[email protected]>
Co-authored-by: mattdurham <[email protected]>
Co-authored-by: Clayton Cornell <[email protected]>
Co-authored-by: Paschalis Tsilias <[email protected]>
  • Loading branch information
5 people authored Oct 4, 2023
1 parent 2b4e368 commit 0aeca59
Show file tree
Hide file tree
Showing 20 changed files with 3,896 additions and 8 deletions.
19 changes: 11 additions & 8 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,9 @@ Main (unreleased)
- `discovery.serverset` discovers Serversets stored in Zookeeper. (@thampiotr)
- `discovery.scaleway` discovers scrape targets from Scaleway virtual
instances and bare-metal machines. (@rfratto)
- `faro.receiver` accepts Grafana Faro-formatted telemetry data over the
network and forwards it to other components. (@megumish, @rfratto)
- `prometheus.exporter.azure` collects metrics from Azure. (@wildum)
- `discovery.dockerswarm` discovers scrape targets from Docker Swarm. (@wildum)
- `otelcol.connector.servicegraph` creates service graph metrics from spans. It is the
flow mode equivalent to static mode's `service_graphs` processor. (@ptodev)
Expand All @@ -48,8 +51,8 @@ Main (unreleased)
- `otelcol.processor.k8sattributes` adds Kubernetes metadata as resource attributes
to spans, logs, and metrics. (@acr92)
- `otelcol.processor.probabilistic_sampler` samples logs and traces based on configuration options. (@mar4uk)
- `otelcol.processor.transform` transforms OTLP telemetry data using the
OpenTelemetry Transformation Language (OTTL). It is most commonly used
- `otelcol.processor.transform` transforms OTLP telemetry data using the
OpenTelemetry Transformation Language (OTTL). It is most commonly used
for transformations on attributes.
- `remote.kubernetes.configmap` loads a configmap's data for use in other components (@captncraig)
- `remote.kubernetes.secret` loads a secret's data for use in other components (@captncraig)
Expand Down Expand Up @@ -105,7 +108,7 @@ Main (unreleased)

- Flow: add `randomization_factor` and `multiplier` to retry settings in
`otelcol` components. (@rfratto)

- Add support for `windows_certificate_filter` under http tls config block. (@mattdurham)

- Add `openstack` config converter to convert OpenStack yaml config (static mode) to river config (flow mode). (@wildum)
Expand Down Expand Up @@ -133,15 +136,15 @@ Main (unreleased)

- Promtail converter will now treat `global positions configuration is not supported` as a Warning instead of Error. (@erikbaranowski)

- Add new `agent_component_dependencies_wait_seconds` histogram metric and a dashboard panel
- Add new `agent_component_dependencies_wait_seconds` histogram metric and a dashboard panel
that measures how long components wait to be evaluated after their dependency is updated (@thampiotr)

- Add additional endpoint to debug scrape configs generated inside `prometheus.operator.*` components (@captncraig)

- Components evaluation is now performed in parallel, reducing the impact of
slow components potentially blocking the entire telemetry pipeline.
The `agent_component_evaluation_seconds` metric now measures evaluation time
of each node separately, instead of all the directly and indirectly
- Components evaluation is now performed in parallel, reducing the impact of
slow components potentially blocking the entire telemetry pipeline.
The `agent_component_evaluation_seconds` metric now measures evaluation time
of each node separately, instead of all the directly and indirectly
dependant nodes. (@thampiotr)

### Bugfixes
Expand Down
1 change: 1 addition & 0 deletions component/all/all.go
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,7 @@ import (
_ "github.com/grafana/agent/component/discovery/serverset" // Import discovery.serverset
_ "github.com/grafana/agent/component/discovery/triton" // Import discovery.triton
_ "github.com/grafana/agent/component/discovery/uyuni" // Import discovery.uyuni
_ "github.com/grafana/agent/component/faro/receiver" // Import faro.receiver
_ "github.com/grafana/agent/component/local/file" // Import local.file
_ "github.com/grafana/agent/component/local/file_match" // Import local.file_match
_ "github.com/grafana/agent/component/loki/echo" // Import loki.echo
Expand Down
93 changes: 93 additions & 0 deletions component/faro/receiver/arguments.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
package receiver

import (
"time"

"github.com/alecthomas/units"
"github.com/grafana/agent/component/common/loki"
"github.com/grafana/agent/component/otelcol"
"github.com/grafana/river"
"github.com/grafana/river/rivertypes"
)

// Defaults for various arguments.
var (
DefaultArguments = Arguments{
Server: DefaultServerArguments,
SourceMaps: DefaultSourceMapsArguments,
}

DefaultServerArguments = ServerArguments{
Host: "127.0.0.1",
Port: 12347,
RateLimiting: DefaultRateLimitingArguments,
MaxAllowedPayloadSize: 5 * units.MiB,
}

DefaultRateLimitingArguments = RateLimitingArguments{
Enabled: true,
Rate: 50,
BurstSize: 100,
}

DefaultSourceMapsArguments = SourceMapsArguments{
Download: true,
DownloadFromOrigins: []string{"*"},
DownloadTimeout: time.Second,
}
)

// Arguments configures the app_agent_receiver component.
type Arguments struct {
LogLabels map[string]string `river:"extra_log_labels,attr,optional"`

Server ServerArguments `river:"server,block,optional"`
SourceMaps SourceMapsArguments `river:"sourcemaps,block,optional"`
Output OutputArguments `river:"output,block"`
}

var _ river.Defaulter = (*Arguments)(nil)

// SetToDefault applies default settings.
func (args *Arguments) SetToDefault() { *args = DefaultArguments }

// ServerArguments configures the HTTP server where telemetry information will
// be sent from Faro clients.
type ServerArguments struct {
Host string `river:"listen_address,attr,optional"`
Port int `river:"listen_port,attr,optional"`
CORSAllowedOrigins []string `river:"cors_allowed_origins,attr,optional"`
APIKey rivertypes.Secret `river:"api_key,attr,optional"`
MaxAllowedPayloadSize units.Base2Bytes `river:"max_allowed_payload_size,attr,optional"`

RateLimiting RateLimitingArguments `river:"rate_limiting,block,optional"`
}

// RateLimitingArguments configures rate limiting for the HTTP server.
type RateLimitingArguments struct {
Enabled bool `river:"enabled,attr,optional"`
Rate float64 `river:"rate,attr,optional"`
BurstSize float64 `river:"burst_size,attr,optional"`
}

// SourceMapsArguments configures how app_agent_receiver will retrieve source
// maps for transforming stack traces.
type SourceMapsArguments struct {
Download bool `river:"download,attr,optional"`
DownloadFromOrigins []string `river:"download_from_origins,attr,optional"`
DownloadTimeout time.Duration `river:"download_timeout,attr,optional"`
Locations []LocationArguments `river:"location,block,optional"`
}

// LocationArguments specifies an individual location where source maps will be loaded.
type LocationArguments struct {
Path string `river:"path,attr"`
MinifiedPathPrefix string `river:"minified_path_prefix,attr"`
}

// OutputArguments configures where to send emitted logs and traces. Metrics
// emitted by app_agent_receiver are exported as targets to be scraped.
type OutputArguments struct {
Logs []loki.LogsReceiver `river:"logs,attr,optional"`
Traces []otelcol.Consumer `river:"traces,attr,optional"`
}
Loading

0 comments on commit 0aeca59

Please sign in to comment.