internal.md

Internal documentation

This documentation is meant for the maintainers and contributors of this project.

Updating cfn-lint

• Supporting a new region: aws-cloudformation#1496
• Supporting a new availability zone: aws-cloudformation#1447
• Supporting a new Lambda runtime: aws-cloudformation#1469
• Supporting new EC2 instance types: aws-cloudformation#1535
• Supporting new Python versions: aws-cloudformation#1334
• Updating the SAM translator dependency: aws-cloudformation#1536
• Releasing a new version of cfn-lint: aws-cloudformation#1530
• Releasing a new version of the companion VS Code plugin: aws-cloudformation/cfn-lint-visual-studio-code#76

Property data

The precision of the linter depends on having up-to-date resource specifications that model the properties accurately. The rules use this property data for all the validations.

Updating it

The official resource specification is updated on a weekly basis (every Friday), so every week we update the property data by:

pip3 install -e .
scripts/update_specs_from_pricing.py # requires Boto3 and Credentials
scripts/update_specs_services_from_ssm.py # requires Boto3 and Credentials
cfn-lint --update-specs
cfn-lint --update-iam-policies
cfn-lint --update-documentation

Folder structure

The official resource specifications are one source of data, the other two are the "extended specs" which are "patches" to the spec that enforce more constraints, and the "additional specs" which are rules written in JSON format that are then picked up by their respective Python class.

CloudSpecs

The command cfn-lint --update-specs pulls down the official resource specifications into folder CloudSpecs and patches the JSON files with the contents of the files in ExtendedSpecs. The merged results are stored in CloudSpecs.

ExtendedSpecs

These files follow the JsonPatch format and are merged with the official specs. They support the following syntax:

• Allowed patterns. Example:

  {
      "op": "add",
      "path": "/ValueTypes",
      "value": {
          "CidrIp": {
              "AllowedPattern": "x.x.x.x/y",
              "AllowedPatternRegex": "^(([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])\\.){3}([0-9]|[1-9][0-9]|1[0-9]{2}|2[0-4][0-9]|25[0-5])(/([0-9]|[1-2][0-9]|3[0-2]))$"
          }
      }
  }

• Allowed values. One example: ExtendedSpecs/$REGION/05_pricing_property_values.json validates EMR instance types and is generated by scripts/update_specs_from_pricing.py. Another example:

  {
      "op": "add",
      "path": "/ValueTypes/AWS::CodeBuild::Project.Artifacts.Packaging",
      "value": {
          "AllowedValues": [
              "NONE",
              "ZIP"
          ]
      }
  }

• List size constraints. Example:

  {
      "op": "add",
      "path": "/ValueTypes/AWS::IAM::Group.Names",
      "value": {
          "ListMax": 10,
          "ListMin": 0
      }
  }

• Number size constraints. Example:

  {
      "op": "add",
      "path": "/ValueTypes/AWS::SQS::Queue.MaximumMessageSize",
      "value": {
          "NumberMax": 262144,
          "NumberMin": 1024
      }
  }

• String size constraints. Example:

  {
      "op": "add",
      "path": "/ValueTypes/AWS::Logs::LogGroup.LogGroupName",
      "value": {
          "StringMax": 512,
          "StringMin": 1
      }
  }

There should be no functional difference, but src/cfnlint/data/ExtendedSpecs/all/03_value_types and src/cfnlint/data/ExtendedSpecs/all/04_property_values are more organized than src/cfnlint/data/ExtendedSpecs/all/03_value_types.json and src/cfnlint/data/ExtendedSpecs/all/04_property_values.json, so they should be preferred locations for new constraints.

ExtendedSpecs/$REGION/06_ssm_service_removal.json is written by scripts/update_specs_services_from_ssm.py and ExtendedSpecs/$REGION/07_ssm_service_addition.json is written by scripts/update_specs_services_from_ssm.py.

If we push changes to these files, customers will have to update cfn-lint. The person changing the file(s) can also see the changes by running the following:

pip3 install -e .
cfn-lint --update-specs # https://github.com/aws-cloudformation/cfn-python-lint/pull/1383#issuecomment-629891506

AdditionalSpecs

If we push changes to these files, customers will have to update their version of cfn-lint. They support the following syntax:

• At least one of these properties must be specified, used by rule E2522. Example:

  "AWS::EC2::Instance": [ 
      [ 
          "ImageId", 
          "LaunchTemplate" 
      ] 
  ]

• Only one of these properties may be specified, used by rule E2523. Example:

  "AWS::CloudWatch::Alarm": [ 
      [ 
          "MetricName", 
          "Metrics" 
      ] 
  ]

• If this property is specified, these properties must be excluded, used by rule E2520. Example:

  "AWS::RDS::DBCluster": { 
      "SnapshotIdentifier": [ 
          "MasterUsername", 
          "MasterUserPassword" 
      ] 
  }

• If this property is specified, these properties must be included, used by rule E2521. Example:

  "AWS::OpsWorks::Stack": { 
      "VpcId": [ 
          "DefaultSubnetId" 
      ] 
  }

AdditionalSpecs/RdsProperties.json is written by scripts/update_specs_from_pricing.py and used by rule E3025 and AdditionalSpecs/Policies.json is written by cfn-lint --update-iam-policies and used by rule W3037.