app_message mismatch error when stress test is performed

[PrithviAPai]

Hi Team,

I performed stress test with my device as Requester and i2c simulator as SPDM Responder.
SPDM emulator(https://github.com/DMTF/spdm-emu/tree/main) runs on i2c simulator.
With that test, I run into following issue.

1. Application message is given to libspdm for encryption
   msg code 0x80 not found!!! libspdm_send_spdm_request[ffffffff] msg <unknown>(0x80), size (0x4): 0000: 01 80 00 02 smbus: >TX> 0F 42 13 01 09 08 C8 06 FF FF FF FF BD 05 34 00 11 D4 BA 97 ED 27 40 D0 1D 4D 91 4B 67 D3 5E 5E 3A ED 08 70 A8 13 B5 F8 4A 1D 92 98 17 50 77 07 DF DB 8D 88 9D 94 A1 7E 85 C8 4B A8 19 71 2B E1 38 EB 69 7F D2
2. We dont wait for response meanwhile another application request is given to libspdm for encryption
   msg code 0x80 not found!!! libspdm_send_spdm_request[ffffffff] msg <unknown>(0x80), size (0x4): 0000: 01 80 00 02 smbus: >TX> 0F 30 13 01 09 08 C9 06 FF FF FF FF BE 05 22 00 2E 43 F9 2C 99 85 A9 E4 BE C8 10 AE B1 1E 09 72 04 0C 29 9D 40 10 B6 FC 4E FD 2B 7D 4E 20 70 F2 30 85 8D
3. At this point we get a series of response from SPDM Emulator for previous requests sent.
   smbus: <RX< 12 0F 45 00 01 08 09 80 06 FF FF FF FF BD 05 38 00 D9 88 80 6A 99 87 EA 7C AB 92 46 F9 55 DC 0B 8A DE 57 E8 48 19 2E 69 26 5E 30 4D 1B 90 17 21 43 2D 40 D5 B7 BE AA 54 BF 6D 73 C2 D1 1E 51 EF F6 8C F4 3B 3D F2 DB F2 02 smbus: <RX< 12 0F 06 00 01 08 09 50 B4 BB smbus: <RX< 12 0F 2D 00 01 08 09 C1 06 FF FF FF FF BE 05 1F 00 92 A2 FF C1 D3 D8 2F 06 00 FB A4 44 75 D0 2F 95 AF 0D D2 59 01 59 26 33 B7 0F A0 A3 5B A2 34 93 libspdm_decode_secured_message - 0x80010005 libspdm_receive_spdm_response[ffffffff] app_message mismatch
4. From this point for all the response messages app_message mismatch error continues. However, the SPDM Emulator continues to take the request and provide the encrypted response.

Questions:

1. As discussed earlier in other issue (Performance impact with libspdm #2544) we don't have compulsion for synchronous request/response yet why we see app_message mismatch error?
2. Once any response decryption fails should requester be starting session again ? or will it be able to continue to operate using same session ? We dont have any documentation on this provided.

Thanks,
Prithvi A Pai

[PrithviAPai]

Can you please add your inputs ? @steven-bellock @xiangfeima @jyao1

[steven-bellock]

0x80010005 / LIBSPDM_STATUS_INVALID_MSG_FIELD is probably coming from

libspdm/library/spdm_secured_message_lib/libspdm_secmes_encode_decode.c

Lines 489 to 494 in f6558cc

	if (!libspdm_consttime_is_mem_equal(record_header1 + 1, &sequence_num_in_header,
	sequence_num_in_header_size) != 0) {
	libspdm_secured_message_set_last_spdm_error_struct(
	spdm_secured_message_context, &spdm_error);
	return LIBSPDM_STATUS_INVALID_MSG_FIELD;
	}

Are you using MCTP and possibly sending messages out-of-order? Note that, currently, libspdm_decode_secured_message does not support out-of-order decryption. Ie, the internal sequence number is incremented by 1 and if the encrypting endpoint does not use the same sequence number in the message then that is flagged as an error.

or will it be able to continue to operate using same session ?

According to the SPDM specification

The application phase ends when the HEARTBEAT requirements fail, or with an END_SESSION message, or with an ERROR message of ErrorCode=DecryptError.

So if a Responder sends DecryptError then the session is terminated. However if the Requester fails to decrypt the Responder's message then it's up to the Requester whether it considers the session terminated or not.

[PrithviAPai]

@steven-bellock Thanks for your reply.
Yes, I am using MCTP and sending messages in async way. However, even if message is sent sequentially and we don't get the response for one of the message as a Requester what is the way to handle it ?
What are the options available for Requester in this case ? As you mentioned one method is to terminate and restart new session.

[steven-bellock]

Yes, in this case the Requester can send END_SESSION for the current session and try and start a new one. It can also restart the SPDM connection with GET_VERSION. It can reset / reboot the Responder if it has that capability.

[PrithviAPai]

0x80010005 / LIBSPDM_STATUS_INVALID_MSG_FIELD is probably coming from

libspdm/library/spdm_secured_message_lib/libspdm_secmes_encode_decode.c

Lines 489 to 494 in f6558cc

	if (!libspdm_consttime_is_mem_equal(record_header1 + 1, &sequence_num_in_header,
	sequence_num_in_header_size) != 0) {
	libspdm_secured_message_set_last_spdm_error_struct(
	spdm_secured_message_context, &spdm_error);
	return LIBSPDM_STATUS_INVALID_MSG_FIELD;
	}

Are you using MCTP and possibly sending messages out-of-order? Note that, currently, libspdm_decode_secured_message does not support out-of-order decryption. Ie, the internal sequence number is incremented by 1 and if the encrypting endpoint does not use the same sequence number in the message then that is flagged as an error.

or will it be able to continue to operate using same session ?

According to the SPDM specification

The application phase ends when the HEARTBEAT requirements fail, or with an END_SESSION message, or with an ERROR message of ErrorCode=DecryptError.

So if a Responder sends DecryptError then the session is terminated. However if the Requester fails to decrypt the Responder's message then it's up to the Requester whether it considers the session terminated or not.

@steven-bellock is this limitation of the library ?
Spec does not mention NOT supporting out of order decryption.

Is there any plan to support out of order decryption ? We may run into packet drop/delay in response for a particular packet while receiving the next packets. In real environment this is bound to happen. Re-establishing session each time would be a costly operation.

[steven-bellock]

Is there any plan to support out of order decryption ?

Maybe? I see an issue filed so it's on the books. If it is supported then it would probably be disabled by default and then libspdm would allow the Integrator to enable it and specify the size of the window in which sequence numbers can be out-of-order. In addition there would probably be an option on whether libspdm would tolerate completely unsent sequence numbers.

[PrithviAPai]

Closing the issue as we have issue created.
Thanks a lot @steven-bellock for your timely response and suggestions