chore: add mdox formating and binary flag doc autogen (#1053)

Rationales:
* Flags are nice to have for OSS users to use, but it's extremely useful
for us to double check if we don't accidently break some flag names/drop
some flags (API surface). This is to help with
#1001 and
co (:
* Mdox can help us with link check in future

.bingo/Variables.mk

@@ -35,6 +35,12 @@ $(HELM): $(BINGO_DIR)/helm.mod
 	@echo "(re)installing $(GOBIN)/helm-v3.14.0"
 	@cd $(BINGO_DIR) && GOWORK=off $(GO) build -mod=mod -modfile=helm.mod -o=$(GOBIN)/helm-v3.14.0 "helm.sh/helm/v3/cmd/helm"
 
+MDOX := $(GOBIN)/mdox-v0.9.0
+$(MDOX): $(BINGO_DIR)/mdox.mod
+	@# Install binary/ries using Go 1.14+ build command. This is using bwplotka/bingo-controlled, separate go module with pinned dependencies.
+	@echo "(re)installing $(GOBIN)/mdox-v0.9.0"
+	@cd $(BINGO_DIR) && GOWORK=off $(GO) build -mod=mod -modfile=mdox.mod -o=$(GOBIN)/mdox-v0.9.0 "github.com/bwplotka/mdox"
+
 YQ := $(GOBIN)/yq-v4.40.7
 $(YQ): $(BINGO_DIR)/yq.mod
 	@# Install binary/ries using Go 1.14+ build command. This is using bwplotka/bingo-controlled, separate go module with pinned dependencies.

.bingo/mdox.mod

@@ -0,0 +1,5 @@
+module _ // Auto generated by https://github.com/bwplotka/bingo. DO NOT EDIT
+
+go 1.21.4
+
+require github.com/bwplotka/mdox v0.9.0

.bingo/variables.env

@@ -14,5 +14,7 @@ GOLANGCI_LINT="${GOBIN}/golangci-lint-v1.59.0"
 
 HELM="${GOBIN}/helm-v3.14.0"
 
+MDOX="${GOBIN}/mdox-v0.9.0"
+
 YQ="${GOBIN}/yq-v4.40.7"
 

.dockerignore

@@ -1,4 +1,5 @@
 *
+!*.md
 !go.mod
 !go.sum
 !.bingo

CHANGELOG.md

@@ -2,12 +2,10 @@
 
 ## [0.11.0-rc.1](https://github.com/GoogleCloudPlatform/prometheus-engine/compare/v0.10.0-rc.1...v0.11.0-rc.1) (2024-02-16)
 
-
 ### Features
 
 * add secretless OAuth 2 support ([8194baa](https://github.com/GoogleCloudPlatform/prometheus-engine/commit/8194baae3e30dcec6b88d66ae2cc361dcb33aeef))
 
-
 ### Bug Fixes
 
 * **deps:** bump golang in /cmd/frontend ([ff3902a](https://github.com/GoogleCloudPlatform/prometheus-engine/commit/ff3902aeddb41eab6ac36bc58bb577118dd0a56d))

CONTRIBUTING.md

@@ -9,7 +9,7 @@ Contributions to this project must be accompanied by a Contributor License
 Agreement (CLA). You (or your employer) retain the copyright to your
 contribution; this simply gives us permission to use and redistribute your
 contributions as part of the project. Head over to
-<https://cla.developers.google.com/> to see your current agreements on file or
+https://cla.developers.google.com/ to see your current agreements on file or
 to sign a new one.
 
 You generally only need to submit a CLA once, so if you've already submitted one

README.md

@@ -6,7 +6,6 @@
 [![License](https://img.shields.io/badge/License-Apache_2.0-blue.svg)](https://opensource.org/licenses/Apache-2.0)
 ![Build status](https://github.com/GoogleCloudPlatform/prometheus-engine/actions/workflows/presubmit.yml/badge.svg)
 
-
 This repository contains various binaries and packages for client-side usage
 of Google Cloud Managed Service for Prometheus (GMP), a managed Prometheus offering on top
 of Google Cloud Monitoring (GCM).
@@ -19,13 +18,14 @@ For more documentation and to get started, go to [g.co/cloud/managedprometheus](
 * **[frontend](cmd/frontend)**: An authorizing proxy for the Prometheus-compatible query API of GMP. It additionally hosts a query UI.
 * **[operator](cmd/operator)**: A Kubernetes operator for managed metric collection for GMP.
 * **[rule-evaluator](cmd/rule-evaluator)**: A Prometheus rule evaluation engine that evaluates against GMP.
+* **[datasource-syncer](cmd/datasource-syncer)**: A cron job for periodic Oauth2 token injection to Grafana Prometheus datasource.
 
 For the fully Prometheus-compatible binary that writes ingested data into GMP/GCM,
 see [GoogleCloudPlatform/prometheus](https://github.com/GoogleCloudPlatform/prometheus).
 
 ## Docker Images
 
-Images for this repo are regularly released [in the GKE release GCR](https://console.cloud.google.com/gcr/images/gke-release/global/prometheus-engine).
+Images for this repo are regularly released [in the GKE release GCR](https://console.cloud.google.com/gcr/images/gke-release/global[main.go](cmd%2Fdatasource-syncer%2Fmain.go)/prometheus-engine).
 
 ## Development
 
@@ -39,19 +39,20 @@ recommended:
 
 Can be also installed via:
 
-  ```bash
-  gcloud components install kubectl
-  ```
+```bash
+gcloud components install kubectl
+```
 4. [`Docker`](https://docs.docker.com/get-docker/) with
    [`buildx`](https://docs.docker.com/build/architecture/#install-buildx) plugin.
 
 If you want to execute docker containers on remote machine you can run:
-  ```bash
-  gcloud alpha cloud-shell ssh --authorize-session -- -nNT -L `pwd`/docker.sock:/var/run/docker.sock
-
-  # Then in separate terminal.
-  export DOCKER_HOST=unix://docker.sock
-  ```
+
+```bash
+gcloud alpha cloud-shell ssh --authorize-session -- -nNT -L `pwd`/docker.sock:/var/run/docker.sock
+
+# Then in separate terminal.
+export DOCKER_HOST=unix://docker.sock
+```
 
 5. For UI development or update (e.g. to resolve UI security issue), `npm` is
    required. See [pkg/ui documentation](pkg/ui/README.md) for details.
@@ -88,7 +89,7 @@ make go-synthetic
 ```
 
 Running `make bin` will build all of the above go binaries.
-  * Setting `NO_DOCKER=1` here will build all the binaries natively on the host machine.
+* Setting `NO_DOCKER=1` here will build all the binaries natively on the host machine.
 
 ### Testing
 
@@ -101,17 +102,18 @@ To run unit tests from docker container run `make test`
 #### Kubernetes End-to-end tests
 
 Running `make e2e` will run e2e tests against Kubernetes cluster:
-  * By default, it run in hermetic docker container, downloads kind, recreates
-a single node kind cluster and runs [e2e](./e2e) tests against it.
-  * To run a single test, use the `TEST_RUN` environment variable. For example, to run all collector tests, pass `TEST_RUN=TestCollector`:
-  ```bash
-  TEST_RUN=TestCollector make e2e
-  ```
+* By default, it run in hermetic docker container, downloads kind, recreates
+  a single node kind cluster and runs [e2e](./e2e) tests against it.
+* To run a single test, use the `TEST_RUN` environment variable. For example, to run all collector tests, pass `TEST_RUN=TestCollector`:
+
+```bash
+TEST_RUN=TestCollector make e2e
+```
 
 ##### Debugging
 
 In docker mode, to run a single test or debug a cluster during or after failed
-test, you can try entering shell of the `kindtest` container. Before doing so, 
+test, you can try entering shell of the `kindtest` container. Before doing so,
 run `make e2e` to setup `kind` and start a cluster.
 
 To enter shell with kind Kubernetes context, (ensure your docker socket is on

cmd/config-reloader/README.md

@@ -0,0 +1,30 @@
+# Config Reloader
+
+Small binary, a wrapper on top of github.com/thanos-io/thanos/pkg/reloader for extra checks and tuning.
+Meant to be run as a sidecar.
+
+## Flags
+
+```bash mdox-exec="bash hack/format_help.sh config-reloader"
+Usage of config-reloader:
+  -config-dir string
+    	config directory to watch for changes
+  -config-dir-output string
+    	config directory to write with interpolated environment variables
+  -config-file string
+    	config file to watch for changes
+  -config-file-output string
+    	config file to write with interpolated environment variables
+  -listen-address string
+    	address on which to expose metrics (default ":19091")
+  -ready-startup-probing-interval duration
+    	how often to poll ready endpoint during startup (default 1s)
+  -ready-startup-probing-no-conn-threshold int
+    	how many times ready endpoint can fail due to no connection failure. This can happen if the config-reloader starts faster than the config target endpoint readiness server. (default 5)
+  -ready-url string
+    	ready endpoint of the configuration target that returns a 200 when ready to serve traffic. If set, the config-reloader will probe it on startup (default "http://127.0.0.1:19090/-/ready")
+  -reload-url string
+    	reload endpoint of the configuration target that triggers a reload of the configuration file (default "http://127.0.0.1:19090/-/reload")
+  -watched-dir value
+    	directory to watch for file changes (for rule and secret files, may be repeated)
+```

cmd/datasource-syncer/README.md

@@ -12,3 +12,29 @@ By regularly refreshing the oAuth2 access token, you can configure Grafana to di
 [Google access tokens have a lifetime of 1 hour.](https://cloud.google.com/docs/authentication/token-types#at-lifetime) This script runs every 30 minutes to ensure you have an uninterrupted connection between Grafana and Google Cloud Monitoring.
 
 For instructions, see the [Google Cloud documentation for configuring Grafana to use Managed Service for Prometheus](https://cloud.google.com/stackdriver/docs/managed-prometheus/query).
+
+## Flags
+
+```bash mdox-exec="bash hack/format_help.sh datasource-syncer"
+Usage of datasource-syncer:
+  -datasource-uids string
+    	datasource-uids is a comma separated list of data source UIDs to update.
+  -gcm-endpoint-override string
+    	gcm-endpoint-override is the URL where queries should be sent to from Grafana. This should be left blank in almost all circumstances.
+  -grafana-api-endpoint string
+    	grafana-api-endpoint is the endpoint of the Grafana instance that contains the data sources to update.
+  -grafana-api-token string
+    	grafana-api-token used to access Grafana. Can be created using: https://grafana.com/docs/grafana/latest/administration/service-accounts/#create-a-service-account-in-grafana
+  -insecure-skip-verify
+    	Skip TLS certificate verification
+  -project-id string
+    	Project ID of the Google Cloud Monitoring scoping project to query. Queries sent to this project will union results from all projects within the scope.
+  -query.credentials-file string
+    	JSON-encoded credentials (service account or refresh token). Can be left empty if default credentials have sufficient permission.
+  -tls-ca-cert string
+    	Path to the server certificate authority
+  -tls-cert string
+    	Path to the server TLS certificate.
+  -tls-key string
+    	Path to the server TLS key.
+```

cmd/frontend/README.md

@@ -1,5 +1,7 @@
 # Frontend
 
+> NOTE: Consider using [datasource-syncer](../datasource-syncer) for Grafana integration.
+
 The frontend binary is a thin query frontend for Google Cloud Managed Service
 for Prometheus (GMP) that looks and feels like a regular Prometheus server.
 It primarily serves as a target URL for a Prometheus datasource in Grafana
@@ -37,6 +39,22 @@ go run main.go \
 
 Access the frontend UI in your browser at http://localhost:19090.
 
+## Flags
+
+```bash mdox-exec="bash hack/format_help.sh frontend"
+Usage of frontend:
+  -query.credentials-file string
+    	JSON-encoded credentials (service account or refresh token). Can be left empty if default credentials have sufficient permission.
+  -query.project-id string
+    	Project ID of the Google Cloud Monitoring workspace project to query.
+  -query.target-url string
+    	The URL to forward authenticated requests to. (PROJECT_ID is replaced with the --query.project-id flag.) (default "https://monitoring.googleapis.com/v1/projects/PROJECT_ID/location/global/prometheus")
+  -web.external-url string
+    	The URL under which the frontend is externally reachable (for example, if it is served via a reverse proxy). Used for generating relative and absolute links back to the frontend itself. If the URL has a path portion, it will be used to prefix served HTTP endpoints. If omitted, relevant URL components will be derived automatically.
+  -web.listen-address string
+    	Address on which to expose metrics and the query UI. (default ":19090")
+```
+
 ## Docker
 
 You can also build a docker image from source using `make frontend`.

cmd/operator/README.md

@@ -96,6 +96,42 @@ kubectl -n gmp-system port-forward --address 0.0.0.0 collector 19090
 
 Go to `http://localhost:19090/targets`.
 
+## Flags
+
+```bash mdox-exec="bash hack/format_help.sh operator"
+Usage of operator:
+  -ca-cert-base64 string
+    	The base64-encoded certificate authority.
+  -cert-dir string
+    	The directory which contains TLS certificates for webhook server. (default "/etc/tls/private")
+  -cleanup-unless-annotation-key string
+    	Clean up operator-managed workloads without the provided annotation key.
+  -cluster string
+    	Name of the cluster the operator acts on. May be left empty on GKE.
+  -kubeconfig string
+    	Paths to a kubeconfig. Only required if out-of-cluster.
+  -location string
+    	Google Cloud region or zone where your data will be stored. May be left empty on GKE.
+  -metrics-addr string
+    	Address to emit metrics on. (default ":18080")
+  -operator-namespace string
+    	Namespace in which the operator manages its resources. (default "gmp-system")
+  -probe-addr string
+    	Address to outputs probe statuses (e.g. /readyz and /healthz) (default ":18081")
+  -project-id string
+    	Project ID of the cluster. May be left empty on GKE.
+  -public-namespace string
+    	Namespace in which the operator reads user-provided resources. (default "gmp-public")
+  -tls-cert-base64 string
+    	The base64-encoded TLS certificate.
+  -tls-key-base64 string
+    	The base64-encoded TLS key.
+  -v int
+    	Logging verbosity
+  -webhook-addr string
+    	Address to listen to for incoming kube admission webhook connections. (default ":10250")
+```
+
 ## Teardown
 
 Simply stop running the operator locally and remove all manifests in the cluster