Unable to read state for resource with composite identifier

[tmccombs]

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave "+1" or other comments that do not add relevant new information or questions, they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment
• The resources and data sources in this provider are generated from the CloudFormation schema, so they can only support the actions that the underlying schema supports. For this reason submitted bugs should be limited to defects in the generation and runtime code of the provider. Customizing behavior of the resource, or noting a gap in behavior are not valid bugs and should be submitted as enhancements to AWS via the CloudFormation Open Coverage Roadmap.

Terraform CLI and Terraform AWS Cloud Control Provider Version

Terraform v1.5.0
on linux_amd64
+ provider registry.terraform.io/hashicorp/awscc v0.62.0

Affected Resource(s)

• awscc_kendra_data_source
• probably others as well

Terraform Configuration Files

Please include all Terraform configurations required to reproduce the bug. Bug reports without a functional reproduction may be closed without investigation.

resource "awscc_kendra_data_source" "confluence" {
  index_id    = local.kendra_index_id
  name        = "${var.index_name}-confluence"
  type        = "CONFLUENCE"
  description = "Confluence data source for our kendra index"
  role_arn    = aws_iam_role.kendra_data_source_role.arn
  schedule    = "cron(0 0 ? * FRI *)" # weekly indexing Fridays at midnight
  data_source_configuration = {
    confluence_configuration = {
      secret_arn = aws_secretsmanager_secret.confluence.arn
      server_url = "https://lucidatlassian.atlassian.net/"
      version    = "CLOUD"
    }
  }

  depends_on = [aws_iam_role_policy_attachment.kendra_data_source_role_policy_attachment]
}

Expected Behavior

Reading the state of a resource should work.

Actual Behavior

The primaryIdentifier for the kendra data source resource is [/properties/id, /poperties/indexId]. It is possible to create the resource,
and import the resource using the id format "|". However future reads and refreshes fail. With the error:

 ValidationException: Identifier <id> is not valid for identifier [/properties/Id, /properties/IndexId]

Steps to Reproduce

1. terraform apply -refresh-only plan also works

Important Factoids

I believe this is because the identifier used to make the request to AWS just uses the value of the "id" property. But it needs to be the composite identifier that also includes the index id.

But when it is created/imported, or read for the first time, it sets the "id" property in terraform to the value of the "id" property in the response from Cloud Control.

Essentially terraform is using the "id" property for two conflicting purposes: As the primary identifier used to uniquely identify the resource when communicating with Cloud Control, and as the equivalent of the "id" property on the resource itself, which is itself a component of the primary identifier.

I suspect there are other resources that have a similar problem. Specifically any resource that has a composite primary identifier AND an "id" property defined on the resource type.

References

• Cannot import awscc_quicksight_data_set #688 may be related.
• https://docs.aws.amazon.com/cloudcontrolapi/latest/userguide/resource-identifier.html

Proposed solution

Instead of using the "id" property as the primary identifier used to make requests to Cloud Control. Create a new property that is less likely to cause collisions (for example: "__primaryIdentifier"). So for this resource id would correspond to the normal id property on the resource, but the new property would contain "|".

Alternatively, if the resource has an id property that isn't the sole component of the primaryIdentifier, that id could be mangled somehow, but that wouldn't be backwards compatible, since other resources may refer to the id output of the resource.

[kadrach]

This resource/datasource is affected by #125.

[corey-cole]

I realize this is closed, but since I came here via web search for a related problem, I thought it would be useful to leave a resolution note.

If you're hitting this problem while trying to instantiate a data source, you may need to pass a JSON string that includes the properties referenced in your error message rather than the plain string value given by the CloudControl API.

As a specific example, I had an AWS::Bedrock::DataSource for which the CloudControl Identifier was "BWE8J93OJZ|CAI8TBABJG". However, this value did not work for instantiating a data source. Instead, I had to use HCL like this:

data "awscc_bedrock_data_source" "ds" {
    id = jsonencode({KnowledgeBaseId=module.bedrock.default_kb_identifier, DataSourceId=module.bedrock.datasource_identifier})
}