synapse_linked_service_resource

[druckman1]

Is there an existing issue for this?

• I have searched the existing issues

Community Note

• Please vote on this issue by adding a 👍 reaction to the original issue to help the community and maintainers prioritize this request
• Please do not leave comments along the lines of "+1", "me too" or "any updates", they generate extra noise for issue followers and do not help prioritize the request
• If you are interested in working on this issue or have submitted a pull request, please leave a comment and review the contribution guide to help.

Terraform Version

1.4.5

AzureRM Provider Version

3.116.0

Affected Resource(s)/Data Source(s)

synapse_linked_service_resource, azurerm_synapse_workspace

Terraform Configuration Files

provider "azurerm" {
  subscription_id = var.subscription_id
  tenant_id       = var.tenant_id
  features {
  }
}
resource "azurerm_resource_group" "example" {
  name     = "example98186519845"
  location = "Eastus"
}

resource "azurerm_storage_account" "example" {
  name                     = "example98186519845"
  resource_group_name      = azurerm_resource_group.example.name
  location                 = azurerm_resource_group.example.location
  account_kind             = "BlobStorage"
  account_tier             = "Standard"
  account_replication_type = "LRS"
}

resource "azurerm_storage_data_lake_gen2_filesystem" "example" {
  name               = "example98186519845"
  storage_account_id = azurerm_storage_account.example.id
}

resource "azurerm_synapse_workspace" "example" {
  name                                 = "example98186519845"
  resource_group_name                  = azurerm_resource_group.example.name
  location                             = azurerm_resource_group.example.location
  storage_data_lake_gen2_filesystem_id = azurerm_storage_data_lake_gen2_filesystem.example.id
  sql_administrator_login              = "sqladminuser"
  sql_administrator_login_password     = "H@Sh1CoR3!"
  managed_virtual_network_enabled      = true

  identity {
    type = "SystemAssigned"
  }
}

resource "azurerm_synapse_firewall_rule" "example" {
  name                 = "allowAll"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  start_ip_address     = "0.0.0.0"
  end_ip_address       = "255.255.255.255"
}

resource "azurerm_synapse_integration_runtime_azure" "example" {
  name                 = "example98186519845"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  location             = azurerm_resource_group.example.location
}


resource "azurerm_synapse_linked_service" "example" {
  name                 = "example98186519845"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  type                 = "AzureSqlDatabase"
  # SecretName needs to match the secret placed in the Key Vault
  type_properties_json = <<JSON
{
    "server": "db-apps-dev.database.windows.net",
    "database": "dbOfInterest",
    "encrypt": "mandatory",
    "trustServerCertificate": false,
    "authenticationType": "SystemAssignedManagedIdentity"}
JSON

  depends_on = [
    azurerm_synapse_firewall_rule.example,
  ]
}

Debug Output/Panic Output

azuser::vm-123sfe{.../synwLSTest}$ terraform apply -input=false -auto-approve=true
azurerm_resource_group.example: Refreshing state... [id=/subscriptions/<subscription_id>/resourceGroups/example98186519845]
azurerm_storage_account.example: Refreshing state... [id=/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Storage/storageAccounts/example98186519845]
azurerm_storage_data_lake_gen2_filesystem.example: Refreshing state... [id=https://example98186519845.dfs.core.windows.net/example98186519845]
azurerm_synapse_workspace.example: Refreshing state... [id=/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Synapse/workspaces/example98186519845]
azurerm_synapse_firewall_rule.example: Refreshing state... [id=/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Synapse/workspaces/example98186519845/firewallRules/allowAll]
azurerm_synapse_integration_runtime_azure.example: Refreshing state... [id=/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Synapse/workspaces/example98186519845/integrationRuntimes/example98186519845]

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_synapse_linked_service.example will be created
  + resource "azurerm_synapse_linked_service" "example" {
      + id                   = (known after apply)
      + name                 = "example98186519845"
      + synapse_workspace_id = "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Synapse/workspaces/example98186519845"
      + type                 = "AzureSqlDatabase"
      + type_properties_json = jsonencode(
            {
              + authenticationType     = "SystemAssignedManagedIdentity"
              + database               = "dbOfInterest"
              + encrypt                = "mandatory"
              + server                 = "db-apps-dev.database.windows.net"
              + trustServerCertificate = false
            }
        )
    }

Plan: 1 to add, 0 to change, 0 to destroy.
azurerm_synapse_linked_service.example: Creating...
azurerm_synapse_linked_service.example: Still creating... [10s elapsed]
azurerm_synapse_linked_service.example: Still creating... [20s elapsed]
2024-08-21T16:09:05.806-0400 [ERROR] provider.terraform-provider-azurerm_v3.116.0_x5.exe: Response contains error diagnostic: @module=sdk.proto diagnostic_detail= tf_proto_version=5.6 tf_rpc=ApplyResourceChange @caller=github.com/hashicorp/[email protected]/tfprotov5/internal/diag/diagnostics.go:58 diagnostic_summary="creating/updating Linked Service: Invalid payload {
  "Id": "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.DataFactory/factories/example98186519845/linkedservices/example98186519845",
  "Name": "example98186519845",
  "Properties": {
    "type": "AzureSqlDatabase",
    "typeProperties": {}
  }
} and error is Invalid linked service payload, the 'typeProperties' nested in payload is null.." tf_provider_addr=provider tf_req_id=47239517-aefb-761f-4e1f-d1c47c9ffc08 tf_resource_type=azurerm_synapse_linked_service diagnostic_severity=ERROR timestamp=2024-08-21T16:09:05.806-0400
2024-08-21T16:09:05.809-0400 [ERROR] vertex "azurerm_synapse_linked_service.example" error: creating/updating Linked Service: Invalid payload {
  "Id": "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.DataFactory/factories/example98186519845/linkedservices/example98186519845",
  "Name": "example98186519845",
  "Properties": {
    "type": "AzureSqlDatabase",
    "typeProperties": {}
  }
} and error is Invalid linked service payload, the 'typeProperties' nested in payload is null..
╷
│ Error: creating/updating Linked Service: Invalid payload {
│   "Id": "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.DataFactory/factories/example98186519845/linkedservices/example98186519845",
│   "Name": "example98186519845",
│   "Properties": {
│     "type": "AzureSqlDatabase",
│     "typeProperties": {}
│   }
│ } and error is Invalid linked service payload, the 'typeProperties' nested in payload is null..
│
│   with azurerm_synapse_linked_service.example,
│   on main.tf line 54, in resource "azurerm_synapse_linked_service" "example":
│   54: resource "azurerm_synapse_linked_service" "example" {
│
╵

Expected Behaviour

The values of type_properties_json should have been populated and resulted in the creation of an AzureSqlDatabase linked service

The behavior should reflect this terraform plan

Terraform used the selected providers to generate the following execution
plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # azurerm_synapse_linked_service.example will be created
  + resource "azurerm_synapse_linked_service" "example" {
      + id                   = (known after apply)
      + name                 = "example98186519845"
      + synapse_workspace_id = "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.Synapse/workspaces/example98186519845"
      + type                 = "AzureSqlDatabase"
      + type_properties_json = jsonencode(
            {
              + authenticationType     = "SystemAssignedManagedIdentity"
              + database               = "dbOfInterest"
              + encrypt                = "mandatory"
              + server                 = "db-apps-dev.database.windows.net"
              + trustServerCertificate = false
            }
        )
    }

Actual Behaviour

│ Error: creating/updating Linked Service: Invalid payload {
│   "Id": "/subscriptions/<subscription_id>/resourceGroups/example98186519845/providers/Microsoft.DataFactory/factories/example98186519845/linkedservices/example98186519845",
│   "Name": "example98186519845",
│   "Properties": {
│     "type": "AzureSqlDatabase",
│     "typeProperties": {}
│   }
│ } and error is Invalid linked service payload, the 'typeProperties' nested in payload is null..
│
│   with azurerm_synapse_linked_service.example,
│   on main.tf line 54, in resource "azurerm_synapse_linked_service" "example":
│   54: resource "azurerm_synapse_linked_service" "example" {
│

Steps to Reproduce

terraform apply

Important Factoids

No response

References

The behavior seems related to this

[druckman1]

I can create a non-functional linked service if I provide the resource module any accepted value (I don't know the complete list of acceptable values for AzureSqlDatabase)

For example, if I were to add tenant to the type_properties_json field, the linked service would be created. The values I have found that will be populated in the linked service are case-sensitive.

Here is a code chunk that applies successfully.

resource "azurerm_synapse_linked_service" "example" {
  name                 = "example98186519845"
  synapse_workspace_id = azurerm_synapse_workspace.example.id
  type                 = "AzureSqlDatabase"
  # SecretName needs to match the secret placed in the Key Vault
  type_properties_json = <<JSON
{
    "server": "db-apps-dev.database.windows.net",
    "database": "dbOfInterest",
    "encrypt": "mandatory",
    "trustServerCertificate": false,
    "authenticationType": "SystemAssignedManagedIdentity",
    "tenant": "<tenant_id>"
}
JSON

Here is the resulting linked service json.

{
    "name": "example98186519845",
    "type": "Microsoft.Synapse/workspaces/linkedservices",
    "properties": {
        "type": "AzureSqlDatabase",
        "typeProperties": {
            "tenant": "<tenant_id>"
        },
        "annotations": []
    }
}

[druckman1]

Most of the values listed in this table, are not applied to the final linked service for AzureSqlDatabase

I haven't tested every value, but I was only able to get 3 or 4 values through into the linked service.

[gerrytan]

I can reproduce the problem via main.tf.

Will check the resource config / state management logic next to determine if there is a bug here: synapse_linked_service_resource.go

[gerrytan]

I just ran the code thru debugger and looks like everything is fine up to linkedService.UnmarshalJSON(jsonData). After this unmarshal happen the typeProperties provided are gone. Need to dig deeper on root causes.

Code ref:

terraform-provider-azurerm/internal/services/synapse/synapse_linked_service_resource.go

Line 304 in 8649171

if err := linkedService.UnmarshalJSON(jsonData); err != nil {