Event Streams module

This module implements IBM Event Streams for IBM Cloud with topics, partitions, throughput, storage size, cleanup policy, retention time, retention size, segment size, and schema.

About KMS encryption

The Event Streams service supports payload data encryption that uses a root key CRN of a key management service, such as Key Protect or Hyper Protect Crypto Services. You specify the root key CRN with the kms_key_crn input. For more information, see Managing encryption in Event Streams.

Overview

• terraform-ibm-event-streams
• Submodules
  • fscloud
• Examples
  • Basic example
  • Complete example with topics and schema creation.
  • Financial Services Cloud profile example
• Contributing

terraform-ibm-event-streams

Usage

module "event_streams" {
  source  = "terraform-ibm-modules/event-streams/ibm"
  version = "latest" # Replace "latest" with a release version to lock into a specific release
  resource_group    = "event-streams-rg"
  plan                 = "standard"
  topics           = [
    {
      name       = "topic-1"
      partitions = 1
      config = {
        "cleanup.policy"  = "delete"
        "retention.ms"    = "86400000"
        "retention.bytes" = "10485760"
        "segment.bytes"   = "10485760"
      }
    },
    {
      name       = "topic-2"
      partitions = 1
      config = {
        "cleanup.policy"  = "compact,delete"
        "retention.ms"    = "86400000"
        "retention.bytes" = "1073741824"
        "segment.bytes"   = "536870912"
      }
    }
  ]
  schema_id            = [{
    schema_id = "my-es-schema_1"
    schema = {
      type = "string"
      name = "name_1"
    }
    },
    {
      schema_id = "my-es-schema_2"
      schema = {
        type = "string"
        name = "name_2"
      }
    },
    {
      schema_id = "my-es-schema_3"
      schema = {
        type = "string"
        name = "name_3"
      }
    }
  ]
}

Required IAM access policies

You need the following permissions to run this module.

• Account Management
  • Resource Group service
    • Viewer platform access
• IAM Services
  • Event Streams service
    • Editor platform access
    • Manager service access

Requirements

Name	Version
terraform	>= 1.3.0
ibm	>= 1.71.0, <2.0.0
time	>= 0.9.1

Modules

Name	Source	Version
cbr_rule	terraform-ibm-modules/cbr/ibm//modules/cbr-rule-module	1.29.0
es_guid_crn_parser	terraform-ibm-modules/common-utilities/ibm//modules/crn-parser	1.1.0

Resources

Name	Type
ibm_event_streams_mirroring_config.es_mirroring_config	resource
ibm_event_streams_quota.eventstreams_quotas	resource
ibm_event_streams_schema.es_schema	resource
ibm_event_streams_schema_global_rule.es_globalrule	resource
ibm_event_streams_topic.es_topic	resource
ibm_iam_authorization_policy.es_s2s_policy	resource
ibm_iam_authorization_policy.kms_policy	resource
ibm_resource_instance.es_instance	resource
ibm_resource_key.service_credentials	resource
ibm_resource_tag.es_access_tag	resource
time_sleep.wait_for_authorization_policy	resource
time_sleep.wait_for_es_s2s_policy	resource
time_sleep.wait_for_kms_authorization_policy	resource

Inputs

Name	Description	Type	Default	Required
access_tags	The list of access tags associated with the Event Streams instance.	list(string)	[]	no
cbr_rules	The list of context-based restriction rules to create.	list(object({ description = string account_id = string rule_contexts = list(object({ attributes = optional(list(object({ name = string value = string }))) })) enforcement_mode = string }))	[]	no
create_timeout	The timeout value for creating an Event Streams instance. Specify 3h for an Enterprise plan instance. Add 1 h for each level of non-default throughput. Add 30 min for each level of non-default storage size.	string	"3h"	no
delete_timeout	The timeout value for deleting an Event Streams instance.	string	"15m"	no
es_name	The name to give the Event Streams instance created by this module.	string	n/a	yes
existing_kms_instance_guid	The GUID of the Hyper Protect Crypto Services or Key Protect instance in which the key specified in var.kms_key_crn is coming from. Required only if var.kms_encryption_enabled is set to true, var.skip_kms_iam_authorization_policy is set to false, and you pass a value for var.kms_key_crn.	string	null	no
kms_encryption_enabled	Set this to true to control the encryption keys used to encrypt the data that you store in IBM Cloud® Databases. If set to false, the data is encrypted by using randomly generated keys. For more info on Key Protect integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-key-protect. For more info on HPCS integration, see https://cloud.ibm.com/docs/cloud-databases?topic=cloud-databases-hpcs	bool	false	no
kms_key_crn	The root key CRN of the key management service (Key Protect or Hyper Protect Crypto Services) to use to encrypt the payload data. Learn more about integrating Hyper Protect Crypto Services with Event Streams.	string	null	no
metrics	Enhanced metrics to activate, as list of strings. Only allowed for enterprise plans. Allowed values: 'topic', 'partition', 'consumers'.	list(string)	[]	no
mirroring	Event Streams mirroring configuration. Required only if creating mirroring instance. For more information on mirroring, see https://cloud.ibm.com/docs/EventStreams?topic=EventStreams-mirroring.	object({ source_crn = string source_alias = string target_alias = string options = optional(object({ topic_name_transform = object({ type = string rename = optional(object({ add_prefix = optional(string) add_suffix = optional(string) remove_prefix = optional(string) remove_suffix = optional(string) })) }) group_id_transform = object({ type = string rename = optional(object({ add_prefix = optional(string) add_suffix = optional(string) remove_prefix = optional(string) remove_suffix = optional(string) })) }) })) })	null	no
mirroring_topic_patterns	The list of the topics to set in instance. Required only if creating mirroring instance.	list(string)	null	no
plan	The plan for the Event Streams instance. Possible values: lite, standard, enterprise-3nodes-2tb.	string	"standard"	no
quotas	Quotas to be applied to the Event Streams instance. Entity may be 'default' to apply to all users, or an IAM ServiceID for a specific user. Rates are bytes/second, with -1 meaning no quota.	list(object({ entity = string producer_byte_rate = optional(number, -1) consumer_byte_rate = optional(number, -1) }))	[]	no
region	The region where the Event Streams are created.	string	"us-south"	no
resource_group_id	The resource group ID where the Event Streams instance is created.	string	n/a	yes
schema_global_rule	Schema global compatibility rule. Allowed values are 'NONE', 'FULL', 'FULL_TRANSITIVE', 'FORWARD', 'FORWARD_TRANSITIVE', 'BACKWARD', 'BACKWARD_TRANSITIVE'.	string	null	no
schemas	The list of schema objects. Include the schema_id and the type and name of the schema in the schema object.	list(object( { schema_id = string schema = object({ type = string name = string fields = optional(list(object({ name = string type = string }))) }) } ))	[]	no
service_credential_names	The mapping of names and roles for service credentials that you want to create for the Event streams.	map(string)	{}	no
service_endpoints	The type of service endpoints. Possible values: 'public', 'private', 'public-and-private'.	string	"public"	no
skip_es_s2s_iam_authorization_policy	Set to true to skip the creation of an IAM authorization policy that will allow all Event Streams instances in the given resource group access to read from the mirror source instance. This policy is required when creating a mirroring instance, and will only be created if a value is passed in the mirroring input.	bool	false	no
skip_kms_iam_authorization_policy	Set to true to skip the creation of an IAM authorization policy that permits all Event Streams database instances in the resource group to read the encryption key from the KMS instance. If set to false, pass in a value for the KMS instance in the existing_kms_instance_guid variable. In addition, no policy is created if var.kms_encryption_enabled is set to false.	bool	false	no
storage_size	Storage size of the Event Streams in GB. Applies only to Enterprise plan instances. Possible values: 2048, 4096, 6144, 8192, 10240, 12288. Storage capacity cannot be reduced after the instance is created. When the throughput input variable is set to 300, storage size starts at 4096. When throughput is 450, storage size starts starts at 6144.	number	"2048"	no
tags	The list of tags associated with the Event Steams instance.	list(string)	[]	no
throughput	Throughput capacity in MB per second. Applies only to Enterprise plan instances. Possible values: 150, 300, 450.	number	"150"	no
topics	The list of topics to apply to resources. Only one topic is allowed for Lite plan instances.	list(object( { name = string partitions = number config = map(string) } ))	[]	no
update_timeout	The timeout value for updating an Event Streams instance. Specify 1h for an Enterprise plan instance. Add 1 h for each level of non-default throughput. A 30 min for each level of non-default storage size.	string	"1h"	no

Outputs

Name	Description
crn	Event Streams crn
guid	Event Streams guid
id	Event Streams instance id
kafka_broker_version	The Kafka version
kafka_brokers_sasl	(Array of Strings) Kafka brokers use for interacting with Kafka native API
kafka_http_url	The API endpoint to interact with Event Streams REST API
mirroring_config_id	The ID of the mirroring config in CRN format
mirroring_topic_patterns	Mirroring topic patterns
service_credentials_json	The service credentials JSON map.
service_credentials_object	The service credentials object.

Contributing

You can report issues and request features for this module in GitHub issues in the module repo. See Report an issue or request a feature.

To set up your local development environment, see Local development setup in the project documentation.