diff --git a/mkdocs.yml b/mkdocs.yml index 5f4e1040a..098afdcbb 100644 --- a/mkdocs.yml +++ b/mkdocs.yml @@ -50,7 +50,7 @@ nav: - 'Concepts': - 'Feature Flagging': 'concepts/feature-flagging.md' - 'OpenFeature Compliance': 'concepts/openfeature-compliance.md' - - 'Flag Sources': 'concepts/flag-sources.md' + - 'Syncs': 'concepts/syncs.md' - 'Architecture': 'architecture.md' - 'Deployment': 'deployment.md' - 'Reference': @@ -58,6 +58,7 @@ nav: - 'flagd Commands': 'reference/flagd-cli/flagd.md' - '"start" Command': 'reference/flagd-cli/flagd_start.md' - '"version" Command': 'reference/flagd-cli/flagd/flagd_version.md' + - 'Sync Configuration': 'reference/sync-configuration.md' - 'Flag Configuration': - 'Configuration Overview': 'reference/flag-configuration.md' - 'Custom Operations': diff --git a/web-docs/concepts/flag-sources.md b/web-docs/concepts/flag-sources.md deleted file mode 100644 index e69de29bb..000000000 diff --git a/web-docs/concepts/syncs.md b/web-docs/concepts/syncs.md new file mode 100644 index 000000000..4b612ca17 --- /dev/null +++ b/web-docs/concepts/syncs.md @@ -0,0 +1,114 @@ +# Syncs + +Syncs are a core part of flagd; they are the abstraction that enables different sources for feature flag definitions. +flagd can connect to one or more sync sources to + +## Available syncs + +### Filepath sync + +The file path sync provider reads and watch the source file for updates(ex: changes and deletions). + +```shell +flagd start --uri file:etc/featureflags.json +``` + +In this example, `etc/featureflags.json` is a valid feature flag configuration file accessible by the flagd process. +See [sync source](../reference/sync-configuration.md#source-configuration) configuration for details. + +--- + +### HTTP sync + +The HTTP sync provider fetch flags from a remote source and periodically poll the source for flag configuration updates. + +```shell +flagd start --uri https://my-flag-source.json +``` + +In this example, `https://my-flag-source.json` is a remote endpoint responding valid feature flag configurations when +invoked with **HTTP GET** request. +The polling interval, port, TLS settings, and authentication information can be configured. +See [sync source](../reference/sync-configuration.md#source-configuration) configuration for details. + +--- + +### gRPC sync + +The gRPC sync provider streams flag configurations from a gRPC sync provider implementation. This stream connection is ruled +by +the [sync service protobuf definition](https://github.com/open-feature/schemas/blob/main/protobuf/sync/v1/sync_service.proto). + +```shell +flagd start --uri grpc://grpc-sync-source +``` + +In this example, `grpc-sync-source` is a grpc target implementing [sync.proto](../reference/flag-sync-protocol.md) definition. +See [sync source](../reference/sync-configuration.md#source-configuration) configuration for details. + +--- + +### Kubernetes sync + +The Kubernetes sync provider allows flagd to connect to a Kubernetes cluster and evaluate flags against a specified +FeatureFlagConfiguration resource as defined within +the [open-feature-operator](https://github.com/open-feature/open-feature-operator/blob/main/apis/core/v1alpha1/featureflagconfiguration_types.go) +spec. +This configuration is best used in conjunction with the [OpenFeature Operator](https://github.com/open-feature/open-feature-operator). + +To use an existing FeatureFlagConfiguration custom resource, start flagD with the following command: + +```shell +flagd start --uri core.openfeature.dev/default/my_example +``` + +In this example, `default/my_example` expected to be a valid FeatureFlagConfiguration resource, where `default` is the +namespace and `my_example` being the resource name. +See [sync source](../reference/sync-configuration.md#source-configuration) configuration for details. + +## Merging + +Flagd can be configured to read from multiple sources at once, when this is the case flagd will merge all flag configurations into a single +merged state. + +For example: + +![flag merge 1](../images/flag-merge-1.svg) + +In this example, `source-A` and `source-B` provide a single flag configuration, the `foo` flag and the `bar` flag respectively. +The merge logic for this configuration is simple, both flag configurations are added to the `store`. + +In most scenarios, these flag sources will be supplying `n` number of configurations, using a unique flag key for each configuration. + +However, as multiple sources are being used, there is the opportunity for keys to be duplicated, intentionally or not, between flag sources. +In these situations `flagd` uses a merge priority order to ensure that its behavior is consistent. + +Merge order is dictated by the order that `sync-providers` and `uris` are defined, with the latest defined source taking precedence over those defined before it, as an example: + +```sh +./bin/flagd start --uri file:source-A.json --uri file:source-B.json --uri file:source-C.json +``` + +When `flagd` is started with the command defined above, `source-B` takes priority over `source-A`, whilst `source-C` takes priority over both `source-B` and `source-A`. + +Using the above example, if a flag key is duplicated across all 3 sources, then the configuration from `source-C` would be the only one stored in the merged state. + +![flag merge 2](../images/flag-merge-2.svg) + +### State Resync Events + +Given the above example, the `source-A` and `source-B` 'versions' of flag configuration the `foo` have been discarded, so if a delete event in `source-C` results in the removal of the `foo`flag, there will no longer be any reference of `foo` in flagd's store. + +As a result of this flagd will return `FLAG_NOT_FOUND` errors, and the OpenFeature SDK will always return the default value. + +To prevent flagd falling out of sync with its flag sources during delete events, resync events are used. +When a delete event results in a flag configuration being removed from the merged state, the full set of configurations is requested from all flag sources, and the merged state is rebuilt. +As a result, the value of the `foo` flag from `source-B` will be stored in the merged state, preventing flagd from returning `FLAG_NOT_FOUND` errors. + +![flag merge 3](../images/flag-merge-3.svg) + +In the example above, a delete event results in a resync event being fired, as `source-C` has deleted its 'version' of the `foo`, this results in a new merge state being formed from the remaining configurations. + +![flag merge 4](../images/flag-merge-4.svg) + +Resync events may lead to further resync events if the returned flag configurations result in further delete events, however the state will eventually be resolved correctly. diff --git a/web-docs/reference/sync-configuration.md b/web-docs/reference/sync-configuration.md new file mode 100644 index 000000000..089cf8f4d --- /dev/null +++ b/web-docs/reference/sync-configuration.md @@ -0,0 +1,82 @@ +# Sync configuration + +See [syncs](../concepts/syncs.md) for a conceptual overview. + +## URI patterns + +Any URI passed to flagd via the `--uri` (`-f`) flag must follow one of the 4 following patterns with prefixes to ensure that +it is passed to the correct implementation: + +| Implied Sync Provider | Prefix | Example | +| --------------------- | ---------------------- | ------------------------------------- | +| `kubernetes` | `core.openfeature.dev` | `core.openfeature.dev/default/my-crd` | +| `file` | `file:` | `file:etc/flagd/my-flags.json` | +| `http` | `http(s)://` | `https://my-flags.com/flags` | +| `grpc` | `grpc(s)://` | `grpc://my-flags-server` | + +## Source Configuration + +While a URI may be passed to flagd via the `--uri` (`-f`) flag, some implementations may require further configurations. +In these cases the `--sources` flag should be used. + +The flagd accepts a string argument, which should be a JSON representation of an array of `SourceConfig` objects. + +Alternatively, these configurations can be passed to flagd via config file, specified using the `--config` flag. + +| Field | Type | Note | +| ----------- | ------------------ | ------------------------------------------------------------------------------------------------------------------------------------------------ | +| uri | required `string` | Flag configuration source of the sync | +| provider | required `string` | Provider type - `file`, `kubernetes`, `http` or `grpc` | +| bearerToken | optional `string` | Used for http sync; token gets appended to `Authorization` header with [bearer schema](https://www.rfc-editor.org/rfc/rfc6750#section-2.1) | +| interval | optional `uint32` | Used for http sync; requests will be made at this interval. Defaults to 5 seconds. | +| tls | optional `boolean` | Enable/Disable secure TLS connectivity. Currently used only by gRPC sync. Default(ex:- if unset) is false, which will use an insecure connection | +| providerID | optional `string` | Value binds to grpc connection's providerID field. gRPC server implementations may use this to identify connecting flagd instance | +| selector | optional `string` | Value binds to grpc connection's selector field. gRPC server implementations may use this to filter flag configurations | +| certPath | optional `string` | Used for grpcs sync when TLS certificate is needed. If not provided, system certificates will be used for TLS connection | + +The `uri` field values **do not** follow the [URI patterns](#uri-patterns). The provider type is instead derived +from the `provider` field. Only exception is the remote provider where `http(s)://` is expected by default. Incorrect +URIs will result in a flagd start-up failure with errors from the respective sync provider implementation. + +Given below are example sync providers, startup command and equivalent config file definition: + +Sync providers: + +- `file` - config/samples/example_flags.json +- `http` - +- `https` - +- `kubernetes` - default/my-flag-config +- `grpc`(insecure) - grpc-source:8080 +- `grpcs`(secure) - my-flag-source:8080 + +Startup command: + +```sh +./bin/flagd start +--sources='[{"uri":"config/samples/example_flags.json","provider":"file"}, + {"uri":"http://my-flag-source.json","provider":"http","bearerToken":"bearer-dji34ld2l"}, + {"uri":"default/my-flag-config","provider":"kubernetes"}, + {"uri":"grpc-source:8080","provider":"grpc"}, + {"uri":"my-flag-source:8080","provider":"grpc", "certPath": "/certs/ca.cert", "tls": true, "providerID": "flagd-weatherapp-sidecar", "selector": "source=database,app=weatherapp"}]' +``` + +Configuration file, + +```yaml +sources: + - uri: config/samples/example_flags.json + provider: file + - uri: http://my-flag-source.json + provider: http + bearerToken: bearer-dji34ld2l + - uri: default/my-flag-config + provider: kubernetes + - uri: my-flag-source:8080 + provider: grpc + - uri: my-flag-source:8080 + provider: grpc + certPath: /certs/ca.cert + tls: true + providerID: flagd-weatherapp-sidecar + selector: 'source=database,app=weatherapp' +```