Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Templatize version updates #5333

Merged
merged 26 commits into from
Oct 10, 2023
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
26 commits
Select commit Hold shift + click to select a range
82adb42
Add template generation script and template files for each relevant n…
spartan0x117 Sep 28, 2023
3e2ced8
Add pattern matching and use sed
spartan0x117 Sep 28, 2023
956205c
Rename .templ to .t
spartan0x117 Sep 28, 2023
ca3d415
Make POSIX compliant
spartan0x117 Sep 28, 2023
f0d1078
Fix condition
spartan0x117 Sep 28, 2023
7dcc797
Add newline to end of file
spartan0x117 Sep 28, 2023
fcd3c99
Update release instructions
spartan0x117 Sep 28, 2023
ceaecf0
Add agent-version.txt file and use that to make it easier to use GitH…
spartan0x117 Sep 29, 2023
eeb3633
Add generate-versioned-files make target
spartan0x117 Sep 29, 2023
772eff2
Add workflow check to check that templates and generated files are in…
spartan0x117 Sep 29, 2023
b1d5683
Update developer docs
spartan0x117 Sep 29, 2023
0489c47
Merge branch 'main' into spartan0x117/templatize-version-updates
spartan0x117 Sep 29, 2023
fd46e01
Update templates
spartan0x117 Sep 29, 2023
b6868c4
Rename files to be FILENAME.t.EXTENSION so that editors will still de…
spartan0x117 Sep 30, 2023
250ba4a
Revert "Rename files to be FILENAME.t.EXTENSION so that editors will …
spartan0x117 Oct 2, 2023
d161b91
Remove unnecessary templates and update docs pages
spartan0x117 Oct 4, 2023
a40a2c9
Merge branch 'main' into spartan0x117/templatize-version-updates
spartan0x117 Oct 4, 2023
24e5df1
Update docs/sources/_index.md.t
spartan0x117 Oct 5, 2023
26372d0
Fix missing cascade
spartan0x117 Oct 5, 2023
7897cf2
Merge branch 'main' into spartan0x117/templatize-version-updates
spartan0x117 Oct 5, 2023
a21004f
Remove templates from already generated or unnecessary files
spartan0x117 Oct 5, 2023
84665d8
Update version and regenerate
spartan0x117 Oct 5, 2023
e230a1f
Merge branch 'main' into spartan0x117/templatize-version-updates
spartan0x117 Oct 6, 2023
ea3533b
Update version in gen-versioned-files and regenerate
spartan0x117 Oct 6, 2023
61001ac
Merge branch 'main' into spartan0x117/templatize-version-updates
spartan0x117 Oct 10, 2023
a25e6c4
Fix missing cascade params and update to latest version
spartan0x117 Oct 10, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions .github/workflows/check-versioned-files.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
name: Test Versioned Files
on: pull_request
jobs:
regenerate-docs:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v4

- name: Regenerate versioned files
run: |
make generate-versioned-files
if ! git diff --exit-code; then
echo "Newly generated versioned files differ from those checked in. Make sure to only update the templates manually and run 'make generate-versioned-files'!" >&2
exit 1
fi
32 changes: 20 additions & 12 deletions Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -50,16 +50,17 @@
##
## Targets for generating assets:
##
## generate Generate everything.
## generate-crds Generate Grafana Agent Operator CRDs ands its documentation.
## generate-drone Generate the Drone YAML from Jsonnet.
## generate-helm-docs Generate Helm chart documentation.
## generate-helm-tests Generate Helm chart tests.
## generate-manifests Generate production/kubernetes YAML manifests.
## generate-dashboards Generate dashboards in example/docker-compose after
## changing Jsonnet.
## generate-protos Generate protobuf files.
## generate-ui Generate the UI assets.
## generate Generate everything.
## generate-crds Generate Grafana Agent Operator CRDs ands its documentation.
## generate-drone Generate the Drone YAML from Jsonnet.
## generate-helm-docs Generate Helm chart documentation.
## generate-helm-tests Generate Helm chart tests.
## generate-manifests Generate production/kubernetes YAML manifests.
## generate-dashboards Generate dashboards in example/docker-compose after
## changing Jsonnet.
## generate-protos Generate protobuf files.
## generate-ui Generate the UI assets.
## generate-versioned-files Generate versioned files.
##
## Other targets:
##
Expand Down Expand Up @@ -280,8 +281,8 @@ smoke-image:
# Targets for generating assets
#

.PHONY: generate generate-crds generate-drone generate-helm-docs generate-helm-tests generate-manifests generate-dashboards generate-protos generate-ui
generate: generate-crds generate-drone generate-helm-docs generate-helm-tests generate-manifests generate-dashboards generate-protos generate-ui
.PHONY: generate generate-crds generate-drone generate-helm-docs generate-helm-tests generate-manifests generate-dashboards generate-protos generate-ui generate-versioned-files
generate: generate-crds generate-drone generate-helm-docs generate-helm-tests generate-manifests generate-dashboards generate-protos generate-ui generate-versioned-files

generate-crds:
ifeq ($(USE_CONTAINER),1)
Expand Down Expand Up @@ -337,6 +338,13 @@ else
cd ./web/ui && yarn --network-timeout=1200000 && yarn run build
endif

generate-versioned-files:
ifeq ($(USE_CONTAINER),1)
$(RERUN_IN_CONTAINER)
else
sh ./tools/gen-versioned-files/gen-versioned-files.sh
endif

#
# Other targets
#
Expand Down
8 changes: 8 additions & 0 deletions docs/developer/release/3-update-version-in-code.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,14 @@ The project must be updated to reference the upcoming release tag whenever a new

3. Update appropriate places in the codebase that have the previous version with the new version determined above.

First update `tools/gen-versioned-files/agent-version.txt` with the new `VERSION` and run:

```
make generate-versioned-files
```

Next, commit the changes (including those to `tools/gen-versioned-files/agent-version.txt`, as a workflow will use this version to ensure that the templates and generated files are in sync).

* Do **not** update the `operations/helm` directory. It is updated independently from Agent releases.

3. Create a PR to merge to main (must be merged before continuing).
Expand Down
2 changes: 2 additions & 0 deletions docs/sources/_index.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,8 @@ canonical: https://grafana.com/docs/agent/latest/
title: Grafana Agent
description: Grafana Agent is a flexible, performant, vendor-neutral, telemetry collector
weight: 350
cascade:
AGENT_RELEASE: v0.37.1
---

# Grafana Agent
Expand Down
120 changes: 120 additions & 0 deletions docs/sources/_index.md.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,120 @@
---
aliases:
- /docs/grafana-cloud/agent/
- /docs/grafana-cloud/monitor-infrastructure/agent/
- /docs/grafana-cloud/monitor-infrastructure/integrations/agent/
canonical: https://grafana.com/docs/agent/latest/
title: Grafana Agent
description: Grafana Agent is a flexible, performant, vendor-neutral, telemetry collector
weight: 350
cascade:
AGENT_RELEASE: $AGENT_VERSION
---

# Grafana Agent

Grafana Agent is a vendor-neutral, batteries-included telemetry collector with
configuration inspired by [Terraform][]. It is designed to be flexible,
performant, and compatible with multiple ecosystems such as Prometheus and
OpenTelemetry.

Grafana Agent is based around **components**. Components are wired together to
form programmable observability **pipelines** for telemetry collection,
processing, and delivery.

{{% admonition type="note" %}}
This page focuses mainly on [Flow mode][], the Terraform-inspired variant of Grafana Agent.

For information on other variants of Grafana Agent, refer to [Introduction to Grafana Agent]({{< relref "./about.md" >}}).
{{% /admonition %}}

Grafana Agent can collect, transform, and send data to:

* The [Prometheus][] ecosystem
* The [OpenTelemetry][] ecosystem
* The Grafana open source ecosystem ([Loki][], [Grafana][], [Tempo][], [Mimir][], [Pyroscope][])

[Terraform]: https://terraform.io
[Prometheus]: https://prometheus.io
[OpenTelemetry]: https://opentelemetry.io
[Loki]: https://github.com/grafana/loki
[Grafana]: https://github.com/grafana/grafana
[Tempo]: https://github.com/grafana/tempo
[Mimir]: https://github.com/grafana/mimir
[Pyroscope]: https://github.com/grafana/pyroscope

## Why use Grafana Agent?

* **Vendor-neutral**: Fully compatible with the Prometheus, OpenTelemetry, and
Grafana open source ecosystems.
* **Every signal**: Collect telemetry data for metrics, logs, traces, and
continuous profiles.
* **Scalable**: Deploy on any number of machines to collect millions of active
series and terabytes of logs.
* **Battle-tested**: Grafana Agent extends the existing battle-tested code from
the Prometheus and OpenTelemetry Collector projects.
* **Powerful**: Write programmable pipelines with ease, and debug them using a
[built-in UI][UI].
* **Batteries included**: Integrate with systems like MySQL, Kubernetes, and
Apache to get telemetry that's immediately useful.

## Getting started

* Choose a [variant][variants] of Grafana Agent to run.
* Refer to the documentation for the variant to use:
* [Static mode][]
* [Static mode Kubernetes operator][]
* [Flow mode][]

## Supported platforms

* Linux

* Minimum version: kernel 2.6.32 or later
* Architectures: AMD64, ARM64

* Windows

* Minimum version: Windows Server 2012 or later, or Windows 10 or later.
* Architectures: AMD64

* macOS

* Minimum version: macOS 10.13 or later
* Architectures: AMD64 (Intel), ARM64 (Apple Silicon)

* FreeBSD

* Minimum version: FreeBSD 10 or later
* Architectures: AMD64

## Release cadence

A new minor release is planned every six weeks for the entire Grafana Agent
project, including Static mode, the Static mode Kubernetes operator, and Flow
mode.

The release cadence is best-effort: releases may be moved forwards or backwards
if needed. The planned release dates for future minor releases do not change if
one minor release is moved.

Patch and security releases may be created at any time.

[Milestones]: https://github.com/grafana/agent/milestones

{{% docs/reference %}}
[variants]: "/docs/agent/ -> /docs/agent/<AGENT VERSION>/about"
[variants]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/monitor-infrastructure/agent/about"

[Static mode]: "/docs/agent/ -> /docs/agent/<AGENT VERSION>/static"
[Static mode]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/monitor-infrastructure/agent/static"

[Static mode Kubernetes operator]: "/docs/agent/ -> /docs/agent/<AGENT VERSION>/operator"
[Static mode Kubernetes operator]: "/docs/grafana-cloud/ -> /docs/grafana-cloud/monitor-infrastructure/agent/operator"

[Flow mode]: "/docs/agent/ -> /docs/agent/<AGENT VERSION>/flow"
[Flow mode]: "/docs/grafana-cloud/ -> /docs/agent/<AGENT VERSION>/flow"

[UI]: "/docs/agent/ -> /docs/agent/<AGENT VERSION>/flow/monitoring/debugging.md#grafana-agent-flow-ui"
[UI]: "/docs/grafana-cloud/ -> /docs/agent/<AGENT VERSION>/flow/monitoring/debugging.md#grafana-agent-flow-ui"
{{% /docs/reference %}}
2 changes: 1 addition & 1 deletion docs/sources/operator/deploy-agent-operator-resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -62,7 +62,7 @@ To deploy the `GrafanaAgent` resource:
labels:
app: grafana-agent
spec:
image: grafana/agent:v0.37.1
image: grafana/agent:{{< param "AGENT_RELEASE" >}}
integrations:
selector:
matchLabels:
Expand Down
2 changes: 1 addition & 1 deletion docs/sources/operator/getting-started.md
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ To install Agent Operator:
serviceAccountName: grafana-agent-operator
containers:
- name: operator
image: grafana/agent-operator:v0.37.1
image: grafana/agent-operator:{{< param "AGENT_RELEASE" >}}
args:
- --kubelet-service=default/kubelet
---
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ docker run \
-v "/proc:/host/proc:ro,rslave" \
-v /tmp/agent:/etc/agent \
-v /path/to/config.yaml:/etc/agent-config/agent.yaml \
grafana/agent:v0.37.1 \
grafana/agent:{{< param "AGENT_RELEASE" >}} \
--config.file=/etc/agent-config/agent.yaml
```

Expand Down Expand Up @@ -70,7 +70,7 @@ metadata:
name: agent
spec:
containers:
- image: grafana/agent:v0.37.0
- image: {{< param "AGENT_RELEASE" >}}
name: agent
args:
- --config.file=/etc/agent-config/agent.yaml
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ docker run \
-v "/proc:/proc:ro" \
-v /tmp/agent:/etc/agent \
-v /path/to/config.yaml:/etc/agent-config/agent.yaml \
grafana/agent:v0.37.1 \
grafana/agent:{{< param "AGENT_RELEASE" >}} \
--config.file=/etc/agent-config/agent.yaml
```

Expand All @@ -39,7 +39,7 @@ metadata:
name: agent
spec:
containers:
- image: grafana/agent:v0.37.1
- image: grafana/agent:{{< param "AGENT_RELEASE" >}}
name: agent
args:
- --config.file=/etc/agent-config/agent.yaml
Expand Down
4 changes: 2 additions & 2 deletions docs/sources/static/set-up/install/install-agent-docker.md
Original file line number Diff line number Diff line change
Expand Up @@ -34,7 +34,7 @@ To run a Grafana Agent Docker container on Linux, run the following command in a
docker run \
-v WAL_DATA_DIRECTORY:/etc/agent/data \
-v CONFIG_FILE_PATH:/etc/agent/agent.yaml \
grafana/agent:v0.37.1
grafana/agent:{{< param "AGENT_RELEASE" >}}
```

Replace `CONFIG_FILE_PATH` with the configuration file path on your Linux host system.
Expand All @@ -51,7 +51,7 @@ To run a Grafana Agent Docker container on Windows, run the following command in
docker run ^
-v WAL_DATA_DIRECTORY:C:\etc\grafana-agent\data ^
-v CONFIG_FILE_PATH:C:\etc\grafana-agent ^
grafana/agent:v0.37.1-windows
grafana/agent:{{< param "AGENT_RELEASE" >}}-windows
```

Replace the following:
Expand Down
15 changes: 15 additions & 0 deletions pkg/operator/defaults.go.t
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
package operator

// Supported versions of the Grafana Agent.
var (
DefaultAgentVersion = "$AGENT_VERSION"
DefaultAgentBaseImage = "grafana/agent"
DefaultAgentImage = DefaultAgentBaseImage + ":" + DefaultAgentVersion
)

// Defaults for Prometheus Config Reloader.
var (
DefaultConfigReloaderVersion = "v0.67.1"
DefaultConfigReloaderBaseImage = "quay.io/prometheus-operator/prometheus-config-reloader"
DefaultConfigReloaderImage = DefaultConfigReloaderBaseImage + ":" + DefaultConfigReloaderVersion
)
1 change: 1 addition & 0 deletions tools/gen-versioned-files/agent-version.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
v0.37.1
20 changes: 20 additions & 0 deletions tools/gen-versioned-files/gen-versioned-files.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
#!/bin/sh
AGENT_VERSION=$(cat ./tools/gen-versioned-files/agent-version.txt | tr -d '\n')

if [ -z "$AGENT_VERSION" ]; then
echo "AGENT_VERSION can't be found. Are you running this from the repo root?"
exit 1
fi

versionMatcher='^v[0-9]+\.[0-9]+\.[0-9]+(-rc[0-9]+)?$'

if ! echo "$AGENT_VERSION" | grep -Eq "$versionMatcher"; then
echo "AGENT_VERSION env var is not in the correct format. It should be in the format of vX.Y.Z or vX.Y.Z-rcN"
exit 1
fi

templates=$(find . -type f -name "*.t" -not -path "./.git/*")
for template in $templates; do
echo "Generating ${template%.t}"
sed -e "s/\$AGENT_VERSION/$AGENT_VERSION/g" < "$template" > "${template%.t}"
done