TLS retry_middlebox_assert issue after upgrading to erlang 26.0.2

[nulian]

After I upgraded our project to erlang 26.0.2 i'm sometimes randomly getting hello_retry_middlebox_assert errors when sending mails through smtp client that uses hackney

The error doesn't happen every time it seems very random if it works or not. Even got 2 different errors while trying to send mails and other times it works without any error.

Did see this being talked about in #6807 but not sure how I can configure or change something that it works consistent again like it did in erlang 25.3.2.5

The mailing domain is of an external service we use so we don't have control on their ssl configuration.

Using domain smtp.flowmailer.net and see in the result it get's converted to front.flowmailer.net

Error

[notice] TLS :client: In state :hello_retry_middlebox_assert at ssl_gen_statem.erl:807 generated CLIENT ALERT: Fatal - Unexpected Message
  - {:unexpected_msg,
  {:internal,
   {:server_hello, {3, 3},
    <<102, 142, 172, 37, 33, 66, 144, 99, 163, 77, 88, 13, 149, 80, 9, 91, 203,
      19, 203, 86, 92, 72, 159, 173, 249, 71, 170, 255, 179, 246, 78, 237>>,
    <<216, 127, 10, 162, 43, 134, 143, 46, 79, 122, 44, 98, 62, 240, 44, 220,
      150, 220, 86, 18, 161, 174, 72, 237, 76, 13, 207, 22, 52, 172, 190, 201>>,
    <<19, 2>>, 0,
    %{
      server_hello_selected_version: {:server_hello_selected_version, {3, 4}},
      key_share: {:key_share_server_hello,
       {:key_share_entry, :secp256r1,
        <<4, 166, 1, 55, 56, 87, 244, 17, 150, 154, 47, 234, 86, 249, 89, 189,
          216, 24, 208, 96, 14, 137, 146, 80, 76, 187, 254, 120, 35, 97, 135, 46,
          ...>>}},
      pre_shared_key: :undefined
    }}}}
 12:58:50.957 [warning] Description: ~c"Failed to assert middlebox server message"
      Reason: [missing: {:change_cipher_spec, 1}]
 
 12:58:50.958 [notice] TLS :client: In state :hello_retry_middlebox_assert at ssl_gen_statem.erl:807 generated CLIENT ALERT: Fatal - Unexpected Message
  - {:unexpected_msg,
  {:internal,
   {:server_hello, {3, 3},
    <<74, 149, 190, 159, 255, 55, 143, 79, 51, 187, 60, 235, 227, 156, 173, 152,
      4, 199, 1, 124, 110, 59, 163, 190, 84, 61, 188, 99, 232, 198, 152, 9>>,
    <<216, 199, 168, 59, 78, 40, 179, 187, 167, 173, 108, 192, 17, 175, 182, 98,
      249, 106, 90, 140, 243, 154, 87, 236, 234, 43, 99, 150, 34, 6, 125, 206>>,
    <<19, 2>>, 0,
    %{
      server_hello_selected_version: {:server_hello_selected_version, {3, 4}},
      key_share: {:key_share_server_hello,
       {:key_share_entry, :secp256r1,
        <<4, 254, 3, 61, 151, 111, 56, 96, 55, 204, 219, 198, 72, 55, 169, 181,
          89, 106, 162, 229, 218, 149, 165, 136, 111, 111, 32, 109, 117, 229,
          218, 90, ...>>}},
      pre_shared_key: :undefined
    }}}}

{:network_failure, ~c"front.flowmailer.net", {:error, :einval}}

[IngelaAndin]

Have you tried to disable middle_box mode ? {middlebox_comp_mode, false}. I do not know hackney but it should let you pass this ssl option to the client. You probably do not need this mode.

[IngelaAndin]

@nulian You should also be able to supply the {versions, ['tlsv1.3']} to the client as you know the server knows TLS-1.3 and that should also make your problem go away. As this is not a bug in our code but in the server that was previously undetected. Of course it could be possible to have a flag to ignore this bug (as it seems harmless at the moment), but that is a "dirty" solution that we prefer not implement.

[nulian]

Do you have to supply only tlsv1.3 then to version because I tried adding it to the existing ones like versions: [:tlsv1, :"tlsv1.1", :"tlsv1.2", :"tlsv1.3"] and I still encountered the issue?
The middlebox disabling works though I need to fork and edit that package then because it doesn't allow me to set the option myself.

[IngelaAndin]

Well yes if it does help with the version you must specify only TLS-1.3 ( {versions, ['tlsv1.3']} ) . Does it not allow you to put {middlebox_comp_mode, false}, you should be able to supply it in the same list as the versions option? But it might be the case, that versions, will not be sufficient as a work around for this specific version of middlebox compatibility mode issues (so far it seems to have caused more problems than it solved).

[IngelaAndin]

It of course could also work to downgrade to run tlsv-1.2 ( {versions, ['tlsv1.2']} ) , but probably that would be less desirable. Maybe you can make PR to allow you to set middle_box mode in hackny?

[nulian]

Tried also with tlsv1.3 still had issue so forked the bamboo_smtp and added the middlebox_comp_mode to it's tls settings so I could disable it.

bettyblocks/bamboo_smtp@105528d