BSD tcp socket - send error with large data size

[ATrocan]

Hi,
I use Thread X with RA6M3 (Cortex M4).
FSP version 5.3

I want to send via tcp 119k byte using BSD wrapper, but:

If i try to send with a single call to send() function i get an error.

bytes_to_send = 118784;

status = send(accepted_socket, ((const char*)p_SCmos), bytes_to_send, 0);

I thought it was a problem with the number of packet pool and their size but even setting the size to 1568 with 32 packets even trying to send only 15k bytes at a time I get an error.

The only wait to make it to go is to split with small send().

#define MAX_ONE_TIME_SEND_BYTES 1500
       while(remaining_bytes > 0){
           if(remaining_bytes > MAX_ONE_TIME_SEND_BYTES){
               bytes_to_send = MAX_ONE_TIME_SEND_BYTES;
           } else {
               bytes_to_send = remaining_bytes;
           }
           status = send(accepted_socket, ((const char*)p_SCmos + (118784 - remaining_bytes)), bytes_to_send, 0); //bytes to send= 118784 
           if(status == -1){
               SetLed(RED);
               D(printf("Error sending body GetRawImage!\n"));
           }
           remaining_bytes -= status;
           bytes_SENT += status;
           n_send++;

           D(printf("BYTES SENT: %d, SEND_NUM = %d\n", status, n_send));
       }

What is the connection with the BSD send() and packet pools?
I thought that BSD send() took care of dividing my data to send it and not having to do it myself. (BSD should take care of this itself, If it can't send all the bytes it should send only those it can but not get an error)
The only limit i think may be space due to the number of packets * packet size, but even if i use 1568 * 32 = 50176 a send with only 15k at a time it should work but it doesn't.
Can you explain to me why?
I think it's a bug, can you check?
Thanks

[Hnz2]

Hi @ATrocan,
Just a quick question. Do you have disabled packet chaining feature by macro NX_DISABLE_PACKET_CHAIN? I would expect exact same error as you see when packet chaining is disabled.
Jan

[ATrocan]

Hi @Hnz2,
No, I haven't defined NX_DISABLE_PACKET_CHAIN.
Alex

[Hnz2]

Hi @ATrocan,
OK. Unfortunately I don't have time to deeply debug this issue. Please wait for hints from other users.
btw ... I saw already that you discussed this topic at Renesas community forum here.
Jan

[ATrocan]

Yes, you’re right, but they told me things I had already guessed. Nothing new.
The behavior is unpredictable, and this worries me and my work colleagues.
I say this because:
with 32 packets of 1568 bytes (50176 bytes for the packet pool), sends of 40k do not always complete. If I set TCP Maximum TX Queue to 5 everything works (default is 20), otherwise no.
The BSD wrapper should behave in a certain way, but it doesn’t, so we wanted to ask for a check to see if it might be a bug.
Thanks for your replies. @Hnz2
Alex

[Hnz2]

Hi @ATrocan,
Do you use separate packet pool for TX of your TCP socket or do you share packet pool with other sockets or even RX? Maybe you can check how looks your TX packet pool via packet pool property nx_packet_pool_available. Sharing packet pool may to introduce some unpredictability. Also make sure that your IP instance have higher priority (lower number) that your task handling TCP socket.
Jan

[ATrocan]

Hi @Hnz2,
Yes, I use a dedicated packet pool only for my tcp socket.
Configuration:
packet pool for tcp = 32 packets * 1568 bytes
IP Helper Thread Priority = 1
BSD internal thread priority = 2
My Thread handling tcp socket priority = 3

My output is after asking in loop for the image to receive(118784 bytes):

SONO DENTRO IL WHILE (secondo read_next_bytes_count)
Message received: GetRawImage
COMANDO GetRawImage Rilevato
Image capture Successful
BYTES SENT: 40000, SEND_NUM = 1
BYTES SENT: 40000, SEND_NUM = 2
BYTES SENT: 38784, SEND_NUM = 3
---> BYTES SENT: 118784 in 3 send()
SONO DENTRO IL WHILE (primo read_next_bytes_count)
Message received: 000011:
bodyLen: 11
SONO DENTRO IL WHILE (secondo read_next_bytes_count)
Message received: GetRawImage
COMANDO GetRawImage Rilevato
Image capture Successful
BYTES SENT: 40000, SEND_NUM = 1
BYTES SENT: 40000, SEND_NUM = 2
BYTES SENT: 38784, SEND_NUM = 3
---> BYTES SENT: 118784 in 3 send()
SONO DENTRO IL WHILE (primo read_next_bytes_count)
Message received: 000011:
bodyLen: 11
SONO DENTRO IL WHILE (secondo read_next_bytes_count)
Message received: GetRawImage
COMANDO GetRawImage Rilevato
Image capture Successful
BYTES SENT: 40000, SEND_NUM = 1
Error sending body GetRawImage!
errno = 22

As you can see on the 3th send fails, this is the unreliability I was talking about. why did it work the first 2 times and then not?
Sometimes it fails directly from the first send, sometimes from the second and so on.
Thanks
Alex

[Hnz2]

Hi @ATrocan,
Hard to say what is going on at your case (I never used BSD wrapper at NetX Duo). I used BSD sockets at my previous project, but I ported my code into native NetX APIs, because these APIs are much more flexible than BSD and allow to save tons of memory.
Maybe you can check nx_packet_pool_available and nx_packet_pool_total. I think you have exhausted packet pool when sending failed.
Jan

[ATrocan]

Hi @Hnz2 ,
yes, you might be right but it's strange because after each send all packets are released... so they should all be available again.
I don't know, anyway thanks for your help.

Alex

[yuxinzhou5]

@ATrocan It is possible that your packet pool needs to be much bigger than 32 packets. The BSD layer creates a chain of packets to store the original frame of 119K of data and then passes the chain to the TCP layer. TCP layer attempts to fragment the 119K data into MSS size (assume it is 1460 bytes) and sends each segment to the IP and MAC layer for transmission. Assuming each packet is 1536 bytes, in the worst case you are looking at 158 packets to transmit the 119K payload.

[yuxinzhou5]

@ATrocan Not quite... Transmitted TCP frames are queued in the socket, in case the frame needs to be re-transmitted. The frame is released only after an ACK covers the entire frame.

Hi @Hnz2 , yes, you might be right but it's strange because after each send all packets are released... so they should all be available again. I don't know, anyway thanks for your help.

Alex