Skip to content

Latest commit

 

History

History
217 lines (187 loc) · 23.4 KB

README.md

File metadata and controls

217 lines (187 loc) · 23.4 KB
Raw

AWS OpenSearch Terraform Module

Terraform module to provision an OpenSearch cluster with SAML authentication.

Prerequisites

Features

Usage

This example is using Azure AD as SAML identity provider.

locals {
  cluster_name      = "opensearch"
  cluster_domain    = "example.com"
  saml_entity_id    = "https://sts.windows.net/XXX-XXX-XXX-XXX-XXX/"
  saml_metadata_url = "https://login.microsoftonline.com/XXX-XXX-XXX-XXX-XXX/federationmetadata/2007-06/federationmetadata.xml?appid=YYY-YYY-YYY-YYY-YYY"
}

data "aws_region" "current" {}

data "http" "saml_metadata" {
  url = local.saml_metadata_url
}

provider "opensearch" {
  url                   = module.opensearch.cluster_endpoint
  aws_region            = data.aws_region.current.name
  healthcheck           = false
}

module "opensearch" {
  source  = "idealo/opensearch/aws"
  version = "~> 2.0"

  cluster_name    = local.cluster_name
  cluster_domain  = local.cluster_domain
  cluster_version = "2.11"

  saml_entity_id        = local.saml_entity_id
  saml_metadata_content = data.http.saml_metadata.body

  indices = {
    example-index-1 = {
      number_of_shards   = 2
      number_of_replicas = 1
    }
    example-index-2 = {
      number_of_shards   = 2
      number_of_replicas = 1
      mappings = {
        "properties" : {
          "id" : {
            "type" : "text"
          },
          "name" : {
            "type" : "text"
          },
          "containerType" : {
            "type" : "text"
          },
          "containerIds" : {
            "type" : "text"
          },
          "synonyms" : {
            "type" : "text"
          },
          "parentEvents" : {
            "type" : "text"
          },
          "valueType" : {
            "type" : "text"
          }
        }
      }
    }
  }
}

Examples

Here is a working example of using this Terraform module:

  • Complete - Create an AWS OpenSearch cluster with all necessary resources.
  • Minimal - Create an empty AWS OpenSearch cluster without saml.

Requirements

Name Version
terraform >= 1.3.0
aws >= 4.12.0
opensearch >= 2.0.0

Modules

Name Source Version
acm terraform-aws-modules/acm/aws ~> 5.0.0

Resources

Name Type
aws_cloudwatch_log_group.opensearch resource
aws_cloudwatch_log_resource_policy.allow_logging resource
aws_elasticsearch_domain.opensearch resource
aws_elasticsearch_domain_saml_options.opensearch resource
aws_iam_service_linked_role.es resource
aws_route53_record.opensearch resource
opensearch_composable_index_template.composable_index_template resource
opensearch_index.index resource
opensearch_index_template.index_template resource
opensearch_ism_policy.ism_policy resource
opensearch_role.role resource
opensearch_roles_mapping.master_user_arn resource
opensearch_roles_mapping.master_user_name resource
opensearch_roles_mapping.role_mapping resource
aws_caller_identity.current data source
aws_iam_policy_document.access_policy data source
aws_iam_policy_document.allow_logging data source
aws_region.current data source
aws_route53_zone.opensearch data source

Inputs

Name Description Type Default Required
access_policies IAM policy document specifying the access policies for the domain. string null no
advanced_options Key-value string pairs to specify advanced configuration options. map(string) null no
advanced_security_options_enabled Whether advanced security is enabled. bool true no
advanced_security_options_internal_user_database_enabled Whether to enable or not internal Kibana user database for ELK OpenDistro security plugin bool false no
advanced_security_options_master_user_name Master user username (applicable if advanced_security_options_internal_user_database_enabled set to true) string null no
advanced_security_options_master_user_password Master user password (applicable if advanced_security_options_internal_user_database_enabled set to true) string null no
auto_tune_enabled Whether to enable/disable auto-tune bool true no
auto_tune_options Configuration block for auto-tune options. The maintenance schedule block is required if rollback_on_disable is set to DEFAULT_ROLLBACK. The start_at field must be a time and date in RFC3339 format 
object({
    maintenance_schedule = optional(list(object({
      start_at = string
      duration = object({
        value = number
      })
      cron_expression_for_recurrence = optional(string)
    })))
    rollback_on_disable = string
  })
 
{
  "maintenance_schedule": [],
  "rollback_on_disable": "NO_ROLLBACK"
}
 no
availability_zones The number of availability zones for the OpenSearch cluster. Valid values: 1, 2 or 3. number 3 no
cluster_domain The hosted zone name of the OpenSearch cluster. string n/a yes
cluster_domain_private Indicates whether to create records in a private (true) or public (false) zone bool false no
cluster_name The name of the OpenSearch cluster. string "opensearch" no
cluster_version The version of OpenSearch to deploy. string "2.11" no
cognito_options Configuration block for authenticating Kibana with Cognito. map(string) {} no
cognito_options_enabled Whether Amazon Cognito authentication with Kibana is enabled or not. bool false no
composable_index_template_files A set of all composable index template files to create. set(string) [] no
composable_index_templates A map of all composable index templates to create. map(any) {} no
create_service_role Indicates whether to create the service-linked role. See https://docs.aws.amazon.com/opensearch-service/latest/developerguide/slr.html bool true no
custom_endpoint Fully qualified domain for your custom endpoint. If not specified, then it defaults to <cluster_name>.<cluster_domain> string null no
custom_endpoint_certificate_arn The ARN of the custom ACM certificate. string "" no
ebs_enabled Indicates whether attach EBS volumes to the data nodes. bool false no
ebs_iops The baseline input/output (I/O) performance of EBS volumes attached to data nodes. number 3000 no
ebs_throughput The throughput (in MiB/s) of the EBS volumes attached to data nodes. Valid values are between 125 and 1000. number 125 no
ebs_volume_size The size of EBS volumes attached to data nodes (in GiB). number 10 no
ebs_volume_type The type of EBS volumes attached to data nodes. string "gp3" no
encrypt_at_rest_enabled Configuration block for encrypt at rest options bool true no
encrypt_kms_key_id The KMS key ID to encrypt the OpenSearch cluster with. If not specified, then it defaults to using the AWS OpenSearch Service KMS key. string "" no
hot_instance_count The number of dedicated hot nodes in the cluster. number 3 no
hot_instance_type The type of EC2 instances to run for each hot node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing string "r6gd.4xlarge.elasticsearch" no
index_files A set of all index files to create. set(string) [] no
index_template_files A set of all index template files to create. set(string) [] no
index_templates A map of all index templates to create. map(any) {} no
indices A map of all indices to create. 
map(object({
    number_of_shards                       = optional(number)
    number_of_replicas                     = optional(number)
    refresh_interval                       = optional(string)
    mappings                               = optional(any, {})
    aliases                                = optional(any, {})
    analysis_analyzer                      = optional(string)
    analysis_char_filter                   = optional(string)
    analysis_filter                        = optional(string)
    analysis_normalizer                    = optional(string)
    analysis_tokenizer                     = optional(string)
    analyze_max_token_count                = optional(string)
    auto_expand_replicas                   = optional(string)
    blocks_metadata                        = optional(bool)
    blocks_read                            = optional(bool)
    blocks_read_only                       = optional(bool)
    blocks_read_only_allow_delete          = optional(bool)
    blocks_write                           = optional(bool)
    codec                                  = optional(string)
    default_pipeline                       = optional(string)
    gc_deletes                             = optional(string)
    highlight_max_analyzed_offset          = optional(string)
    include_type_name                      = optional(string)
    index_similarity_default               = optional(string)
    indexing_slowlog_level                 = optional(string)
    indexing_slowlog_source                = optional(string)
    indexing_slowlog_threshold_index_debug = optional(string)
    indexing_slowlog_threshold_index_info  = optional(string)
    indexing_slowlog_threshold_index_trace = optional(string)
    indexing_slowlog_threshold_index_warn  = optional(string)
    load_fixed_bitset_filters_eagerly      = optional(bool)
    max_docvalue_fields_search             = optional(string)
    max_inner_result_window                = optional(string)
    max_ngram_diff                         = optional(string)
    max_refresh_listeners                  = optional(string)
    max_regex_length                       = optional(string)
    max_rescore_window                     = optional(string)
    max_result_window                      = optional(string)
    max_script_fields                      = optional(string)
    max_shingle_diff                       = optional(string)
    max_terms_count                        = optional(string)
    number_of_routing_shards               = optional(string)
    rollover_alias                         = optional(string)
    routing_allocation_enable              = optional(string)
    routing_partition_size                 = optional(string)
    routing_rebalance_enable               = optional(string)
    search_idle_after                      = optional(string)
    search_slowlog_level                   = optional(string)
    search_slowlog_threshold_fetch_info    = optional(string)
    search_slowlog_threshold_fetch_debug   = optional(string)
    search_slowlog_threshold_fetch_trace   = optional(string)
    search_slowlog_threshold_fetch_warn    = optional(string)
    search_slowlog_threshold_query_debug   = optional(string)
    search_slowlog_threshold_query_info    = optional(string)
    search_slowlog_threshold_query_trace   = optional(string)
    search_slowlog_threshold_query_warn    = optional(string)
    shard_check_on_startup                 = optional(string)
    sort_field                             = optional(string)
    sort_order                             = optional(string)
  }))
 {} no
ism_policies A map of all ISM policies to create. map(any) {} no
ism_policy_files A set of all ISM policy files to create. set(string) [] no
log_streams_enabled Configuration for which log streams to enable sending logs to CloudWatch. map(string) 
{
  "AUDIT_LOGS": "false",
  "ES_APPLICATION_LOGS": "false",
  "INDEX_SLOW_LOGS": "false",
  "SEARCH_SLOW_LOGS": "false"
}
 no
master_instance_count The number of dedicated master nodes in the cluster. number 3 no
master_instance_enabled Indicates whether dedicated master nodes are enabled for the cluster. bool true no
master_instance_type The type of EC2 instances to run for each master node. A list of available instance types can you find at https://aws.amazon.com/en/opensearch-service/pricing/#On-Demand_instance_pricing string "r6gd.large.elasticsearch" no
master_user_arn The ARN for the master user of the cluster. If not specified, then it defaults to using the IAM user that is making the request. string "" no
node_to_node_encryption_enabled Configuration block for node-to-node encryption options bool true no
role_files A set of all role files to create. set(string) [] no
role_mapping_files A set of all role mapping files to create. set(string) [] no
role_mappings A map of all role mappings to create. map(any) {} no
roles A map of all roles to create. map(any) {} no
saml_enabled Indicates whether to configure SAML for the OpenSearch dashboard. bool true no
saml_entity_id The unique Entity ID of the application in SAML Identity Provider. string "" no
saml_master_backend_role This backend role receives full permissions to the cluster, equivalent to a new master role, but can only use those permissions within Dashboards. string null no
saml_master_user_name This username receives full permissions to the cluster, equivalent to a new master user, but can only use those permissions within Dashboards. string null no
saml_metadata_content The metadata of the SAML application in xml format. string "" no
saml_roles_key Element of the SAML assertion to use for backend roles. string "http://schemas.microsoft.com/ws/2008/06/identity/claims/role" no
saml_session_timeout Duration of a session in minutes after a user logs in. Default is 60. Maximum value is 1,440. number 60 no
saml_subject_key Element of the SAML assertion to use for username. string "http://schemas.xmlsoap.org/ws/2005/05/identity/claims/name" no
security_group_ids The list of VPC security groups IDs to attach. list(string) [] no
subnet_ids The list of VPC subnet IDs to use. list(string) [] no
tags A map of tags to add to all resources. map(string) {} no
vpc_enabled Indicates whether the cluster is running inside a VPC. bool false no
warm_instance_count The number of dedicated warm nodes in the cluster. number 3 no
warm_instance_enabled Indicates whether ultrawarm nodes are enabled for the cluster. bool true no
warm_instance_type The type of EC2 instances to run for each warm node. A list of available instance types can you find at https://aws.amazon.com/en/elasticsearch-service/pricing/#UltraWarm_pricing string "ultrawarm1.large.elasticsearch" no

Outputs

Name Description
cluster_endpoint The endpoint URL of the OpenSearch cluster.
cluster_name The name of the OpenSearch cluster.
cluster_version The version of the OpenSearch cluster.
kibana_endpoint The endpoint URL of the OpenSearch dashboards.

License

Apache 2 Licensed. See LICENSE for full details.