[terraform evaluator] Failed to expand block. Invalid "for-each" argument.

[robbert-nlo]

Description

Terraform evaluator shows language errors for valid configuration:
ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable. block="data.azurerm_subnet.spoke" value="cty.NilVal"

Desired Behavior

No errors are shown for valid configuration.

Actual Behavior

An error is shown.

Reproduction Steps

1. Example code:

locals {
  spoke_subnets = flatten([for subnet in data.azurerm_virtual_network.spoke.subnets :
    [
      {
        key = subnet
        name = subnet
      }
    ]
  ])
}

data "azurerm_virtual_network" "spoke" {
  name                = "vnet-app-spoke-tst-we"
  resource_group_name = "rg-app-spoke-tst-we"
}

data "azurerm_subnet" "spoke" {
  for_each = { for subnet in local.spoke_subnets : subnet.key => subnet }

  name                 = each.value.name
  virtual_network_name = data.azurerm_virtual_network.spoke.name
  resource_group_name  = data.azurerm_virtual_network.spoke.resource_group_name
}

2. terraform init
3. trivy fs --scanners misconfig .

Target

Filesystem

Scanner

Misconfiguration

Output Format

Table

Mode

Standalone

Debug Output

2024-11-07T12:41:35+01:00 DEBUG No plugins loaded
2024-11-07T12:41:35+01:00 DEBUG Default config file "file_path=trivy.yaml" not found, using built in values
2024-11-07T12:41:35+01:00 DEBUG Cache dir dir="/Users/user1/Library/Caches/trivy"
2024-11-07T12:41:35+01:00 DEBUG Cache dir dir="/Users/user1/Library/Caches/trivy"
2024-11-07T12:41:35+01:00 DEBUG Parsed severities severities=[UNKNOWN LOW MEDIUM HIGH CRITICAL]
2024-11-07T12:41:35+01:00 DEBUG Ignore statuses statuses=[]
2024-11-07T12:41:35+01:00 INFO  [misconfig] Misconfiguration scanning is enabled
2024-11-07T12:41:35+01:00 DEBUG [misconfig] Checks successfully loaded from disk
2024-11-07T12:41:35+01:00 DEBUG Enabling misconfiguration scanners  scanners=[azure-arm cloudformation dockerfile helm kubernetes terraform terraformplan-json terraformplan-snapshot]
2024-11-07T12:41:35+01:00 DEBUG Initializing scan cache...  type="memory"
2024-11-07T12:41:35+01:00 DEBUG [misconfig] Scanning files for misconfigurations... scanner="Terraform"
2024-11-07T12:41:35+01:00 DEBUG [terraform scanner] Scanning directory  file_path="."
2024-11-07T12:41:36+01:00 DEBUG [rego] Overriding filesystem for checks
2024-11-07T12:41:36+01:00 DEBUG [rego] Embedded libraries are loaded  count=15
2024-11-07T12:41:36+01:00 DEBUG [rego] Embedded checks are loaded count=509
2024-11-07T12:41:38+01:00 DEBUG [rego] Checks from disk are loaded  count=524
2024-11-07T12:41:38+01:00 DEBUG [rego] Overriding filesystem for data
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Setting project/module root  module="root" file_path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing FS module="root" file_path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing  module="root" file_path="data.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Added file module="root" file_path="data.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing  module="root" file_path="providers.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Added file module="root" file_path="providers.tf"
2024-11-07T12:41:39+01:00 INFO  [terraform scanner] Scanning root module  file_path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Setting project/module root  module="root" file_path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing FS module="root" file_path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing  module="root" file_path="data.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Added file module="root" file_path="data.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Parsing  module="root" file_path="providers.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Added file module="root" file_path="providers.tf"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Loading module module="root" module="root"
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Read block(s) and ignore(s)  module="root" blocks=5 ignores=0
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Added input variables from tfvars  module="root" count=0
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Working directory for module evaluation  module="root" file_path="/Users/user1/git/TrivyTest/tst2"
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting module evaluation... path="."
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting iteration  iteration=0
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting iteration  iteration=1
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting iteration  iteration=2
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Context unchanged iteration=2
2024-11-07T12:41:39+01:00 ERROR [terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.  block="data.azurerm_subnet.spoke" value="cty.NilVal"
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting post-submodules evaluation...
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting iteration  iteration=0
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Starting iteration  iteration=1
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Context unchanged iteration=1
2024-11-07T12:41:39+01:00 DEBUG [terraform evaluator] Module evaluation complete.
2024-11-07T12:41:39+01:00 DEBUG [terraform parser] Finished parsing module  module="root"
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Adapting modules...
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Adapted module(s) into state data. count=1
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Using max routines count=9
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Initialized Go check(s). count=482
2024-11-07T12:41:39+01:00 DEBUG [rego] Scanning inputs  count=1
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Finished applying rules.
2024-11-07T12:41:39+01:00 DEBUG [terraform executor] Applying ignores...
2024-11-07T12:41:39+01:00 DEBUG OS is not detected.
2024-11-07T12:41:39+01:00 INFO  Detected config files num=1
2024-11-07T12:41:39+01:00 DEBUG Scanned config file file_path="."
2024-11-07T12:41:39+01:00 DEBUG [vex] VEX filtering is disabled

Operating System

macOS Sonoma

Version

Version: 0.57.0
Vulnerability DB:
  Version: 2
  UpdatedAt: 2024-11-07 06:17:13.658691211 +0000 UTC
  NextUpdate: 2024-11-08 06:17:13.65869063 +0000 UTC
  DownloadedAt: 2024-11-07 09:41:15.14762 +0000 UTC
Check Bundle:
  Digest: sha256:9cc30e6eb1c0dc0b4a4791b61c3dbff8799d08daeac893c08317e7b054ecab14
  DownloadedAt: 2024-11-07 08:54:36.493542 +0000 UTC

Checklist

• Run trivy clean --all
• Read the troubleshooting

[nikpivkin]

Hi @robbert-nlo !

Trivy performs static scanning and does not know anything about the attributes of data blocks.

You can find the answer regarding data blocks here.

[robbert-nlo]

Right, thanks! Will look into that.

[robbert-nlo]

@nikpivkin I understand that Trivy cannot know these attributes due to its static scanning, but isn't throwing and ERROR and stating "Invalid for-each argument" in this case a bit harsh? I mean, this configuration is perfectly valid and it's not a misconfiguration. IMO it would be better to just send an INFO level message to the user that the validity of the for-each argument couldn't be verified.

[DimashOne]

@nikpivkin
I agree with @robbert-nlo. Trivy shouldn't throw ERROR for that cases. 0.54.1 and fewer versions work as expected

[rtizzy]

I was about to start my own discussion @nikpivkin but it appears there are a number of similar issues and this one seems the closest.

Here is a (relatively) simple example in my case

# main.tf 

terraform {
  backend "s3" {}
  required_version = ">= 1.0"
}

locals {

}

resource "aws_acm_certificate" "main" {
  domain_name               = var.domain
  validation_method         = "DNS"
  subject_alternative_names = var.additional_sans != null ? concat(var.additional_sans, ["*.${var.domain}"]) : ["*.${var.domain}"]
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_route53_record" "main" {
  for_each = {
    for dvo in aws_acm_certificate.main.domain_validation_options : dvo.domain_name => {
      name    = dvo.resource_record_name
      record  = dvo.resource_record_value
      type    = dvo.resource_record_type
      zone_id = dvo.domain_name == var.domain ? data.aws_route53_zone.selected.zone_id : data.aws_route53_zone.san.zone_id
    }
  }

  allow_overwrite = true
  name            = each.value.name
  records         = [each.value.record]
  ttl             = 60
  type            = each.value.type
  zone_id         = each.value.zone_id
}

resource "aws_acm_certificate_validation" "main" {
  certificate_arn         = aws_acm_certificate.main.arn
  validation_record_fqdns = [for record in aws_route53_record.main : record.fqdn]
}

# variables.tf
variable "domain" {
  description = "The domain to generate the ACM certificate for"
  type        = string
}

variable "additional_sans" {
  description = "Additional Subject Alternatives Names to add to the cert"
  type        = list(string)
  default     = null
}
# datasources.tf

data "aws_route53_zone" "selected" {
  name = var.domain
}

data "aws_route53_zone" "san" {
  name = var.additional_sans[0]
}

As others have reported, for this module I receive the following error

2024-11-27T13:04:45Z	ERROR	[terraform evaluator] Failed to expand block. Invalid "for-each" argument. Must be known and iterable.	block="aws_route53_record.main" value="cty.NilVal"

I would expect Trivy to successfully scan this module out of the box or at minimum, allow someone to provide values, or at absolute minimum not error out.

[simar7]

It's been fixed via #7964 - will be out in the next release.