Skip to content

Latest commit

 

History

History
 
 

azure

Folders and files

NameName
Last commit message
Last commit date

parent directory

..
src
src
 
 
LICENSE
LICENSE
 
 
README.md
README.md
 
 
pom.xml
pom.xml
 
 

README.md

Microsoft Azure Key Vault Config Provider

confluent-hub install confluentinc/kafka-config-provider-azure:latest

Note: if this command does not work and the package is not online available to download, then you need to download the zip file for azure and include it manually in the plugins path, for example for Azure Key Vault 1.0.5 you would get confluent-csid-secrets-provider-azure-1.0.5.zip from https://github.com/confluentinc/csid-secrets-providers/releases/tag/csid-secrets-providers-1.0.5.

This plugin provides integration with the Microsoft Azure Key Vault service.

KeyVaultConfigProvider

This config provider is used to retrieve secrets from the Microsoft Azure Key Vault service.

Secret Value

The value for the secret must be formatted as a JSON object. This allows multiple keys of data to be stored in a single secret. The name of the secret in Microsoft Azure Key Vault will correspond to the path that is requested by the config provider.

{
  "username" : "db101",
  "password" : "superSecretPassword"
}

Secret Retrieval

The ConfigProvider will use the name of the secret to build the request to the Key Vault service. This behavior can be overridden by setting config.providers.keyVault.param.prefix=staging- and requested the secret with ${keyVault:test-secret}, the ConfigProvider will build a request for staging-test-secret. Some behaviors can be overridden by query string parameters. More than one query string parameter can be used. For example ${keyVault:test-secret?ttl=30000&version=1} would return the secret named test-secret version 1 with a TTL of 30 seconds. After the TTL has expired the ConfigProvider will request an updated credential. If you're using this with Kafka Connect, your tasks will be reconfigured if one of the values have changed.

+-----------+------------------------------------------------+--------------------------------------------------------------------+------------------------------------------+ | Parameter | Description | Default | Example | +===========+================================================+====================================================================+==========================================+ | ttl | Used to override the TTL for the secret. | Value specified by config.providers.keyVault.param.secret.ttl.ms | ${keyVault:test-secret?ttl=60000} | +-----------+------------------------------------------------+--------------------------------------------------------------------+------------------------------------------+ | version | Used to override the version of the secret. | latest | ${keyVault:test-secret?version=1} | +-----------+------------------------------------------------+--------------------------------------------------------------------+------------------------------------------+

If the secret in Azure has been created using json format for both username and password for example, then you can retrieve it using "${keyVault:test-secret:username}" and "${keyVault:test-secret:password}"

Configuration

Client Certificate

client.certificate.path

Location on the local filesystem for the client certificate that will be used to authenticate to Azure.

  • Type: STRING
  • Default:
  • Valid Values:
  • Importance: HIGH
client.certificate.pfx.password

The password protecting the PFX file.

  • Type: PASSWORD
  • Default: [hidden]
  • Valid Values:
  • Importance: HIGH
client.certificate.send.certificate.chain.enabled

Flag to indicate if certificate chain should be sent as part of authentication request.

  • Type: BOOLEAN
  • Default: false
  • Valid Values:
  • Importance: HIGH
client.certificate.type

The type of encoding used on the file specified in client.certificate.path. PEM - Certificate is formatted using PEM encoding., PFX - Certificate is formatted using PFX encoding. client.certificate.pfx.password is required.

  • Type: STRING
  • Default: PEM
  • Valid Values: Matches: PEM, PFX
  • Importance: HIGH

General

client.id

The client ID of the application.

  • Type: STRING
  • Default:
  • Valid Values:
  • Importance: HIGH
credential.type

The type of credentials to use. ClientCertificate - Uses the ClientCertificateCredential., ClientSecret - Uses the ClientSecretCredential., DefaultAzure - Uses the DefaultAzureCredential., UsernamePassword - Uses the UsernamePasswordCredential.

  • Type: STRING
  • Default: DefaultAzure
  • Valid Values: Matches: DefaultAzure, ClientSecret, ClientCertificate, UsernamePassword
  • Importance: HIGH
retry.count

The number of attempts to retrieve a secret from the upstream secret store.

  • Type: INT
  • Default: 3
  • Valid Values:
  • Importance: LOW
retry.interval.seconds

The amount of time in seconds to wait between each attempt to retrieve a secret form the upstream secret store.

  • Type: LONG
  • Default: 10
  • Valid Values:
  • Importance: LOW
secret.prefix

Sets a prefix that will be added to all paths. For example you can use staging or production and all of the calls to Secrets Manager will be prefixed with that path. This allows the same configuration settings to be used across multiple environments.

  • Type: STRING
  • Default:
  • Valid Values:
  • Importance: LOW
thread.count

The number of threads to use when retrieving secrets and executing subscription callbacks.

  • Type: INT
  • Default: 3
  • Valid Values:
  • Importance: LOW
timeout.seconds

The amount of time in seconds to wait before timing out a call to retrieve a secret from the upstream secret store. The total timeout of get(path) and get(path, keys) will be retry.count * timeout.seconds. For example if timeout.seconds = 30 and retry.count = 3 then get(path) and get(path, keys) will block for 90 seconds.

  • Type: LONG
  • Default: 30
  • Valid Values:
  • Importance: LOW
polling.enabled

Determines if the config provider supports polling the upstream secret stores for changes. If disabled the methods subscribe, unsubscribe, and unsubscribeAll will throw a UnsupportedOperationException.

  • Type: BOOLEAN
  • Default: true
  • Valid Values:
  • Importance: MEDIUM
polling.interval.seconds

The number of seconds to wait between polling intervals.

  • Type: LONG
  • Default: 300
  • Valid Values:
  • Importance: MEDIUM
tenant.id

The tenant ID of the application.

  • Type: STRING
  • Default:
  • Valid Values:
  • Importance: HIGH
vault.url

The vault url to connect to. For example https://example.vault.azure.net/

  • Type: STRING
  • Default: java.lang.Object@65a4798f
  • Valid Values:
  • Importance: HIGH

Username and Password

password

The password to authenticate with.

  • Type: PASSWORD
  • Default: [hidden]
  • Valid Values:
  • Importance: HIGH
username

The username to authenticate with.

  • Type: STRING
  • Default:
  • Valid Values:
  • Importance: HIGH

Client Secret

client.secret

The client secret for the authentication.

  • Type: PASSWORD
  • Default: [hidden]
  • Valid Values:
  • Importance: HIGH

Examples

Default Credentials

The following example uses the DefaultAzureCredential to load the credentials.

config.providers=keyVault
config.providers.keyVault.class=io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
config.providers.keyVault.param.vault.url=https://example.vault.azure.net/

Client Certificate - PEM

The following example uses the ClientCertificateCredential to load the credentials.

config.providers=keyVault
config.providers.keyVault.class=io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
config.providers.keyVault.param.vault.url=https://example.vault.azure.net/
config.providers.keyVault.param.client.certificate.type=PEM
config.providers.keyVault.param.credential.type=ClientCertificate
config.providers.keyVault.param.client.certificate.path=/path/to/certificate.pem
config.providers.keyVault.param.tenant.id=27e831e4-5cff-4143-b612-64de151b2f3e

Client Secret

The following example uses the ClientSecretCredential to load the credentials.

config.providers=keyVault
config.providers.keyVault.class=io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
config.providers.keyVault.param.vault.url=https://example.vault.azure.net/
config.providers.keyVault.param.credential.type=ClientSecret
config.providers.keyVault.param.client.secret=asdonfasodfasd
config.providers.keyVault.param.tenant.id=27e831e4-5cff-4143-b612-64de151b2f3e

Client Certificate - PFX

The following example uses the ClientCertificateCredential to load the credentials.

config.providers=keyVault
config.providers.keyVault.class=io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
config.providers.keyVault.param.vault.url=https://example.vault.azure.net/
config.providers.keyVault.param.client.certificate.type=PFX
config.providers.keyVault.param.credential.type=ClientCertificate
config.providers.keyVault.param.client.certificate.path=/path/to/certificate.pfx
config.providers.keyVault.param.tenant.id=27e831e4-5cff-4143-b612-64de151b2f3e

Username and Password

The following example uses the UsernamePasswordCredential to load the credentials.

config.providers=keyVault
config.providers.keyVault.class=io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
config.providers.keyVault.param.vault.url=https://example.vault.azure.net/
config.providers.keyVault.param.credential.type=UsernamePassword
config.providers.keyVault.param.username=foo
config.providers.keyVault.param.password=bar
config.providers.keyVault.param.tenant.id=27e831e4-5cff-4143-b612-64de151b2f3e
config.providers.keyVault.param.client.id=qwerqwte-qwteqwtqwet

Username and Password - Example using Ansible hosts.yaml

The following example uses the UsernamePasswordCredential to load the credentials. Make sure to make a few changes described below to work with cp-ansible

all:
  vars:
    .....
    kafka_connect_custom_properties:
      config.providers: keyVault
      config.providers.keyVault.class: io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
      config.providers.keyVault.param.vault.url: 'https://example.vault.azure.net'
      config.providers.keyVault.param.credential.type: UsernamePassword
      config.providers.keyVault.param.username: foo
      config.providers.keyVault.param.password: bar
      config.providers.keyVault.param.tenant.id: 27e831e4-5cff-4143-b612-64de151b2f3e
      config.providers.keyVault.param.client.id: qwerqwte-qwteqwtqwet

Client Secret - Example using Ansible hosts.yaml

The following example uses the ClientSecretCredential to load the credentials. Make sure to make a few changes described below to work with cp-ansible

all:
  vars:
    .....
    kafka_connect_custom_properties:
      config.providers: keyVault
      config.providers.keyVault.class: io.confluent.csid.config.provider.azure.KeyVaultConfigProvider
      config.providers.keyVault.param.vault.url: 'https://example.vault.azure.net'
      config.providers.keyVault.param.credential.type: ClientSecret
      config.providers.keyVault.param.client.secret: asdonfasodfasd
      config.providers.keyVault.param.tenant.id: 27e831e4-5cff-4143-b612-64de151b2f3e
      config.providers.keyVault.param.client.id: qwerqwte-qwteqwtqwet