-
Notifications
You must be signed in to change notification settings - Fork 59
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add telemetry support #158
Merged
Merged
Changes from all commits
Commits
Show all changes
13 commits
Select commit
Hold shift + click to select a range
6aae4ac
Refactor workflows label selectors to support telemetry logs
ryanartecona 9f295de
give pod containers semantic names for telemetry
ryanartecona b6bfb3f
Add telemetry support
ryanartecona d2f8077
Bump version
ryanartecona 3e06284
PR feedback
ryanartecona 3e08c5f
Add telemetry.extraEnv
ryanartecona a080818
Set RTEL_ENABLED=true when telemetry container enabled
ryanartecona 12c35f1
Make telemetry deployment configurable
ryanartecona ba0982f
Make CI kubeconform check all values files
ryanartecona 119f8ab
Make default values files line up
ryanartecona 977fff6
Fix CI
ryanartecona 6581b2e
Better fix CI
ryanartecona 9afccbd
Make .helmignore respected again
ryanartecona File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,6 +1,6 @@ | ||
charts/**/*.tgz | ||
.prettierignore | ||
|
||
kubeconform | ||
/kubeconform | ||
|
||
.DS_Store |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -2,7 +2,7 @@ apiVersion: v2 | |
name: retool | ||
description: A Helm chart for Kubernetes | ||
type: application | ||
version: 6.1.6 | ||
version: 6.2.0 | ||
maintainers: | ||
- name: Retool Engineering | ||
email: [email protected] | ||
|
59 changes: 59 additions & 0 deletions
59
charts/retool/ci/kubeconform/telemetry-enabled-basic-values.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
# ================================================ | ||
# === boilerplate to get a bootable deployment === | ||
# ================================================ | ||
config: | ||
encryptionKey: '000000000' | ||
jwtSecret: '000000000' | ||
|
||
licenseKey: "EXPIRED-LICENSE-KEY-TRIAL" | ||
|
||
image: | ||
tag: 'latest' | ||
|
||
resources: | ||
requests: | ||
cpu: 500m | ||
memory: 1024Mi | ||
limits: | ||
cpu: 1000m | ||
memory: 4096Mi | ||
|
||
workflows: | ||
resources: | ||
requests: | ||
cpu: 500m | ||
memory: 1024Mi | ||
limits: | ||
cpu: 1000m | ||
memory: 2048Mi | ||
|
||
env: | ||
NUM_WORKERS: '1' | ||
# NODE_DEBUG: 'module' | ||
|
||
ingress: | ||
kubernetes.io/ingress.class: nginx | ||
hosts: | ||
- host: retool.example.com | ||
paths: | ||
- path: / | ||
|
||
livenessProbe: | ||
timeoutSeconds: 60 | ||
periodSeconds: 2 | ||
failureThreshold: 60 | ||
readinessProbe: | ||
timeoutSeconds: 30 | ||
periodSeconds: 5 | ||
|
||
replicaCount: 1 | ||
|
||
persistentVolumeClaim: | ||
size: '3Gi' | ||
# ================================================ | ||
|
||
# === New telemetry stuff === | ||
telemetry: | ||
enabled: true | ||
image: | ||
tag: 3.42.0-edge |
84 changes: 84 additions & 0 deletions
84
charts/retool/ci/kubeconform/telemetry-enabled-full-values.yaml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,84 @@ | ||
# ================================================ | ||
# === boilerplate to get a bootable deployment === | ||
# ================================================ | ||
config: | ||
encryptionKey: '000000000' | ||
jwtSecret: '000000000' | ||
|
||
licenseKey: "EXPIRED-LICENSE-KEY-TRIAL" | ||
|
||
image: | ||
tag: 'latest' | ||
|
||
resources: | ||
requests: | ||
cpu: 500m | ||
memory: 1024Mi | ||
limits: | ||
cpu: 1000m | ||
memory: 4096Mi | ||
|
||
workflows: | ||
resources: | ||
requests: | ||
cpu: 500m | ||
memory: 1024Mi | ||
limits: | ||
cpu: 1000m | ||
memory: 2048Mi | ||
|
||
env: | ||
NUM_WORKERS: '1' | ||
# NODE_DEBUG: 'module' | ||
|
||
ingress: | ||
kubernetes.io/ingress.class: nginx | ||
hosts: | ||
- host: retool.example.com | ||
paths: | ||
- path: / | ||
|
||
livenessProbe: | ||
timeoutSeconds: 60 | ||
periodSeconds: 2 | ||
failureThreshold: 60 | ||
readinessProbe: | ||
timeoutSeconds: 30 | ||
periodSeconds: 5 | ||
|
||
replicaCount: 1 | ||
|
||
persistentVolumeClaim: | ||
size: '3Gi' | ||
# ================================================ | ||
|
||
# === New telemetry stuff === | ||
telemetry: | ||
enabled: true | ||
image: | ||
repository: 'tryretool/telemetry' | ||
tag: 3.42.0-edge | ||
sendToRetool: | ||
enabled: true | ||
customVectorConfig: | ||
foo: bar | ||
customGrafanaAgentConfig: | | ||
// wee woo custom text | ||
extraAnnotations: | ||
foo.com/bar: baz | ||
extraPodSpec: | ||
hostname: bleepbloop | ||
extraVolumes: | ||
- name: varlog | ||
hostPath: | ||
path: "/var/log" | ||
extraVolumeMounts: | ||
- name: varlog | ||
mountPath: "/hostvarlog" | ||
readOnly: true | ||
extraPorts: | ||
- containerPort: 9999 | ||
name: yaaaaa | ||
protocol: TCP | ||
extraContainerSpec: | ||
args: ["this should be ignored"] |
File renamed without changes.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -44,21 +44,71 @@ Common labels | |
*/}} | ||
{{- define "retool.labels" -}} | ||
helm.sh/chart: {{ include "retool.chart" . }} | ||
{{ include "retool.selectorLabels" . }} | ||
{{- if .Chart.AppVersion }} | ||
app.kubernetes.io/version: {{ .Chart.AppVersion | quote }} | ||
{{- end }} | ||
app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
{{- end }} | ||
|
||
{{/* | ||
Selector labels | ||
Selector labels for main backend. Note changes here will require deployment | ||
recreation and incur downtime. The "app.kubernetes.io/instance" label should | ||
also be included in all deployments, so telemetry knows how to find logs. | ||
*/}} | ||
{{- define "retool.selectorLabels" -}} | ||
app.kubernetes.io/name: {{ include "retool.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
{{- end }} | ||
|
||
{{/* | ||
Selector labels for workflow backend. Note changes here will require manual | ||
deployment recreation and incur downtime, so should be avoided. | ||
*/}} | ||
{{- define "retool.workflowBackend.selectorLabels" -}} | ||
retoolService: {{ include "retool.workflowBackend.name" . }} | ||
{{- end }} | ||
|
||
{{/* | ||
Extra (non-selector) labels for workflow backend. | ||
*/}} | ||
{{- define "retool.workflowBackend.labels" -}} | ||
app.kubernetes.io/name: {{ include "retool.workflowBackend.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
telemetry.retool.com/service-name: workflow-backend | ||
{{- end }} | ||
|
||
{{/* | ||
Selector labels for workflow worker. Note changes here will require manual | ||
deployment recreation and incur downtime, so should be avoided. | ||
*/}} | ||
{{- define "retool.workflowWorker.selectorLabels" -}} | ||
retoolService: {{ include "retool.workflowWorker.name" . }} | ||
{{- end }} | ||
|
||
{{/* | ||
Extra (non-selector) labels for workflow worker. | ||
*/}} | ||
{{- define "retool.workflowWorker.labels" -}} | ||
app.kubernetes.io/name: {{ include "retool.workflowWorker.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
telemetry.retool.com/service-name: workflow-worker | ||
{{- end }} | ||
|
||
{{/* | ||
Selector labels for code executor. Note changes here will require manual | ||
deployment recreation and incur downtime, so should be avoided. | ||
*/}} | ||
{{- define "retool.codeExecutor.selectorLabels" -}} | ||
retoolService: {{ include "retool.codeExecutor.name" . }} | ||
{{- end }} | ||
|
||
{{/* | ||
Extra (non-selector) labels for code executor. | ||
*/}} | ||
{{- define "retool.codeExecutor.labels" -}} | ||
app.kubernetes.io/name: {{ include "retool.codeExecutor.name" . }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
telemetry.retool.com/service-name: code-executor | ||
{{- end }} | ||
|
||
|
||
{{/* | ||
Create the name of the service account to use | ||
*/}} | ||
|
@@ -234,6 +284,20 @@ Set Temporal namespace | |
{{- end -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Set workflow backend service name | ||
*/}} | ||
{{- define "retool.workflowBackend.name" -}} | ||
{{ template "retool.fullname" . }}-workflow-backend | ||
{{- end -}} | ||
|
||
{{/* | ||
Set workflow worker service name | ||
*/}} | ||
{{- define "retool.workflowWorker.name" -}} | ||
{{ template "retool.fullname" . }}-workflow-worker | ||
{{- end -}} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. These result in the exact same names as before, just a template code refactor for reuse. |
||
|
||
{{/* | ||
Set code executor service name | ||
*/}} | ||
|
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,72 @@ | ||
{{/* | ||
Reusable name for telemetry-related chart resources. | ||
*/}} | ||
{{- define "retool.telemetry.fullname" -}} | ||
{{- $name := default "telemetry" .Values.telemetry.nameOverride -}} | ||
{{- printf "%s-%s" .Release.Name $name | trunc 63 | trimSuffix "-" -}} | ||
{{- end -}} | ||
|
||
{{/* | ||
Labels to include on telemetry pods. | ||
*/}} | ||
{{- define "retool.telemetry.labels" -}} | ||
helm.sh/chart: {{ include "retool.chart" . }} | ||
app.kubernetes.io/managed-by: {{ .Release.Service }} | ||
{{- end -}} | ||
|
||
{{/* | ||
Labels to use as selector for telemetry pods and deployment. Note these become | ||
immutable once deployed, so changes here will require recreating the deployment. | ||
*/}} | ||
{{- define "retool.telemetry.selectorLabels" -}} | ||
app.kubernetes.io/name: {{ default "telemetry" .Values.telemetry.nameOverride }} | ||
app.kubernetes.io/instance: {{ .Release.Name }} | ||
{{- end -}} | ||
|
||
|
||
{{/* | ||
The name of the service account to use. | ||
*/}} | ||
{{- define "retool.telemetry.serviceAccountName" -}} | ||
{{- if .Values.telemetry.serviceAccount.create }} | ||
{{- default (include "retool.telemetry.fullname" .) .Values.telemetry.serviceAccount.name }} | ||
{{- else }} | ||
{{- default "default" .Values.telemetry.serviceAccount.name }} | ||
{{- end }} | ||
{{- end }} | ||
|
||
|
||
{{/* | ||
The labels to use for scoping log collection to only the pods in the same | ||
release, as a single comma-separated string. The label(s) below should be | ||
present on all relevant pods, or else logs won't be collected. */}} | ||
{{- define "retool.telemetry.logSourcePodLabels" -}} | ||
app.kubernetes.io/instance={{ .Release.Name }} | ||
{{- end }} | ||
|
||
|
||
{{/* | ||
Env vars to include on retool pods to collect telemetry via telemetry pod. | ||
*/}} | ||
{{- define "retool.telemetry.includeEnvVars" -}} | ||
{{- if .Values.telemetry.enabled }} | ||
- name: RTEL_ENABLED | ||
value: 'true' | ||
- name: RTEL_SERVICE_NAME | ||
valueFrom: | ||
fieldRef: | ||
fieldPath: metadata.labels['telemetry.retool.com/service-name'] | ||
- name: K8S_POD_NAME | ||
valueFrom: | ||
fieldRef: | ||
fieldPath: metadata.name | ||
- name: K8S_NODE_NAME | ||
valueFrom: | ||
fieldRef: | ||
fieldPath: spec.nodeName | ||
- name: STATSD_HOST | ||
value: {{ printf "%s.%s" (include "retool.telemetry.fullname" .) .Release.Namespace | quote }} | ||
- name: STATSD_PORT | ||
value: "9125" | ||
{{- end }} | ||
{{- end }} |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This and all the
retool.*.selectorLabels
partials below don't actually change the selector labels that were already used, they're just refactoring the template code.