New-HNSNetwork changes the MTU size from 1500 to 1450 in AKS Windows VMs

[AbelHu]

It seems like that someone has observed it in https://raw.githubusercontent.com/pjh/SDN/mtu-with-new-hnsnetwork/mtu-with-new-hnsnetwork.txt.

#####################################################################
## The just-created L2Bridge type HNS network subsumes the previous
## Ethernet interface with the "vEthernet (Ethernet) 2" interface
## which for some reason has an MTU of 1450 rather than 1432 or 1500.
## Also, the MTU of the existing "vEthernet (nat)" interface has been
## decreased by 50 bytes!
#####################################################################

Reference: TCP/IP performance tuning for Azure VMs | Microsoft Docs

Azure and VM MTU
The default MTU for Azure VMs is 1,500 bytes. The Azure Virtual Network stack will attempt to fragment a packet at 1,400 bytes.
Note that the Virtual Network stack isn't inherently inefficient because it fragments packets at 1,400 bytes even though VMs have an MTU of 1,500. A large percentage of network packets are much smaller than 1,400 or 1,500 bytes.

Below is the command in provisioning AKS Windows nodes.

    $externalNetwork = "ext"
    $nas = @(Get-NetAdapter -Physical)
    
    foreach($na in $nas)
    {
        $netIP = Get-NetIPAddress -ifIndex $na.ifIndex -AddressFamily IPv4 -ErrorAction SilentlyContinue -ErrorVariable netIPErr
        if ($netIP)
        {
            $managementIP = $netIP.IPAddress
            $adapterName = $na.Name
            break
        }
        else {
            Write-Error "No IPv4 found on the network adapter $($na.Name); trying the next adapter ..."
            if ($netIPErr) {
                Write-Host "error when retrieving IPAddress: $netIPErr"
                $netIPErr.Clear()
            }
        }
    }
    if(-Not $managementIP)
    {
        throw "None of the physical network adapters has an IP address"
    }
    Write-Host "Using adapter $adapterName with IP address $managementIP"    
    New-HNSNetwork -Type "L2Bridge" -AddressPrefix "192.168.255.0/30" -Gateway "192.168.255.1" -AdapterName $adapterName -Name $externalNetwork -Verbose

Logs:

Getting MTU size

  MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0          0  Loopback Pseudo-Interface 1
 1500                1          0       1988  vEthernet (nat)
 1500                1   35399353    1419013  Ethernet 2
 
2022-03-24T04:46:56.1914010+00:00: Creating new HNS network "ext"
2022-03-24T04:46:59.6000519+00:00: No IPv4 found on the network adapter Ethernet 3; trying the next adapter ...
2022-03-24T04:46:59.6000519+00:00: error when retrieving IPAddress: No matching MSFT_NetIPAddress objects found by CIM query for instances of the ROOT/StandardCimv2/MSFT_NetIPAddress class on the  CIM server: SELECT * FROM MSFT_NetIPAddress  WHERE ((InterfaceIndex = 17)) AND ((AddressFamily = 2)). Verify query parameters and retry.
2022-03-24T04:46:59.6156744+00:00: Using adapter Ethernet 2 with IP address 10.240.1.151
VERBOSE: Invoke-HNSRequest Method[POST] Path[/networks] Data[{

   "Subnets":  [

                   {

                       "GatewayAddress":  "192.168.255.1",

                       "AddressPrefix":  "192.168.255.0/30"

                   }

               ],

   "NetworkAdapterName":  "Ethernet 2",

   "Name":  "ext",

   "Type":  "L2Bridge"

}]
VERBOSE: Result :
{"Output":{"ActivityId":"F8905258-1CFD-4B88-95F8-072C9B663F39","AdditionalParams":{},"CurrentEndpointCount":0,"Extensio
ns":[{"Id":"E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A","IsEnabled":false,"Name":"Microsoft Windows Filtering
Platform"},{"Id":"E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017","IsEnabled":true,"Name":"Microsoft Azure VFP Switch
Extension"},{"Id":"EA24CD6C-D17A-4348-9190-09F0D5BE83DD","IsEnabled":true,"Name":"Microsoft NDIS
Capture"}],"Flags":0,"Health":{"LastErrorCode":0,"LastUpdateTime":132925708223917418},"ID":"7F4003A8-5E19-4414-990F-72B
19796E78E","IPv6":false,"LayeredOn":"798783E7-1A28-4CDD-9CDC-F438A4E30B46","MacPools":[{"EndMacAddress":"00-15-5D-9D-DF
-FF","StartMacAddress":"00-15-5D-9D-D0-00"}],"ManagementIP":"10.240.1.151","MaxConcurrentEndpoints":0,"Name":"ext","Net
workAdapterName":"Ethernet
2","Policies":[],"Resources":{"AdditionalParams":{},"AllocationOrder":0,"Health":{"LastErrorCode":0,"LastUpdateTime":13
2925708267108672},"ID":"F8905258-1CFD-4B88-95F8-072C9B663F39","PortOperationTime":0,"State":1,"SwitchOperationTime":0,"
VfpOperationTime":0,"parentId":"6C5D500F-C4C8-45CA-8BBB-730C5E13D338"},"State":1,"Subnets":[{"AdditionalParams":{},"Add
ressPrefix":"192.168.255.0/30","GatewayAddress":"192.168.255.1","Health":{"LastErrorCode":0,"LastUpdateTime":1329257082
23917418},"ID":"76E15ADC-3AE9-4DF7-90C4-5FF342E88901","ObjectType":5,"Policies":[],"State":0}],"TotalEndpoints":0,"Type
":"L2Bridge","Version":38654705669},"Success":true}

ActivityId             : F8905258-1CFD-4B88-95F8-072C9B663F39
AdditionalParams       :
CurrentEndpointCount   : 0
Extensions             : {@{Id=E7C3B2F0-F3C5-48DF-AF2B-10FED6D72E7A; IsEnabled=False; Name=Microsoft Windows Filtering
                        Platform}, @{Id=E9B59CFA-2BE1-4B21-828F-B6FBDBDDC017; IsEnabled=True; Name=Microsoft Azure
                        VFP Switch Extension}, @{Id=EA24CD6C-D17A-4348-9190-09F0D5BE83DD; IsEnabled=True;
                        Name=Microsoft NDIS Capture}}
Flags                  : 0
Health                 : @{LastErrorCode=0; LastUpdateTime=132925708223917418}
ID                     : 7F4003A8-5E19-4414-990F-72B19796E78E
IPv6                   : False
LayeredOn              : 798783E7-1A28-4CDD-9CDC-F438A4E30B46
MacPools               : {@{EndMacAddress=00-15-5D-9D-DF-FF; StartMacAddress=00-15-5D-9D-D0-00}}
ManagementIP           : 10.240.1.151
MaxConcurrentEndpoints : 0
Name                   : ext
NetworkAdapterName     : Ethernet 2
Policies               : {}
Resources              : @{AdditionalParams=; AllocationOrder=0; Health=; ID=F8905258-1CFD-4B88-95F8-072C9B663F39;
                        PortOperationTime=0; State=1; SwitchOperationTime=0; VfpOperationTime=0;
                        parentId=6C5D500F-C4C8-45CA-8BBB-730C5E13D338}
State                  : 1
Subnets                : {@{AdditionalParams=; AddressPrefix=192.168.255.0/30; GatewayAddress=192.168.255.1; Health=;
                        ID=76E15ADC-3AE9-4DF7-90C4-5FF342E88901; ObjectType=5; Policies=System.Object[]; State=0}}
TotalEndpoints         : 0
Type                   : L2Bridge
Version                : 38654705669

2022-03-24T04:47:06.7421215+00:00: It took 7 seconds to create the ext network.
Getting MTU size

  MTU  MediaSenseState   Bytes In  Bytes Out  Interface
------  ---------------  ---------  ---------  -------------
4294967295                1          0          0  Loopback Pseudo-Interface 1
 1450                1          0       1988  vEthernet (nat)
 1450                1       2676       3066  vEthernet (Ethernet 2)

[AbelHu]

We have closed the issue in AKS with "It is by design that HNS sets the MTU from 1500 to 1450 for process-isolated containers". It should be better if HNS can share more details for this design. Thanks.