sys/shell: reduce overhead of XFA shell commands #20958

maribu · 2024-11-06T17:30:13Z

Contribution description

We do not need to add an array of pointers to the shell commands, just an array of shell commands is sufficient. This reduced the overhead of XFA by sizeof(void *) per command.

Testing procedure

$ make BOARD=arduino-uno flash term -C examples/default
[...]

2024-11-06 18:27:11,666 # main(): This is RIOT! (Version: 2024.10-devel-388-gf4b2ef-sys/shell/xfa-reduce-overhead)
2024-11-06 18:27:11,686 # Welcome to RIOT!
help
2024-11-06 18:27:25,324 # help
2024-11-06 18:27:25,361 # Command              Description
2024-11-06 18:27:25,402 # ---------------------------------------
2024-11-06 18:27:25,459 # pm                   interact with layered PM subsystem
2024-11-06 18:27:25,525 # ps                   Prints information about running threads.
2024-11-06 18:27:25,594 # saul                 interact with sensors and actuators using SAUL
2024-11-06 18:27:25,648 # version              Prints current RIOT_VERSION
2024-11-06 18:27:25,684 # reboot               Reboot the node
> ps
2024-11-06 18:27:27,196 # ps
2024-11-06 18:27:27,302 # 	pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2024-11-06 18:27:27,405 # 	 1 | idle                 | pending  Q |  15 |    128 (   90) (   38) |      0x3f2 |      0x41a 
2024-11-06 18:27:27,507 # 	 2 | main                 | running  Q |   7 |    644 (  296) (  348) |      0x472 |      0x5fc 
2024-11-06 18:27:27,581 # 	   | SUM                  |            |     |    772 (  386) (  386)
> 2024-11-06 18:27:28,874 # Exiting Pyterm

$ make BOARD=same54-xpro flash term -C examples/default
[...]
> help
2024-11-06 18:29:29,540 # help
2024-11-06 18:29:29,543 # Command              Description
2024-11-06 18:29:29,546 # ---------------------------------------
2024-11-06 18:29:29,551 # ifconfig             Configure network interfaces
2024-11-06 18:29:29,557 # txtsnd               Sends a custom string as is over the link layer
2024-11-06 18:29:29,562 # pm                   interact with layered PM subsystem
2024-11-06 18:29:29,567 # ps                   Prints information about running threads.
2024-11-06 18:29:29,572 # rtc                  control RTC peripheral interface
2024-11-06 18:29:29,578 # saul                 interact with sensors and actuators using SAUL
2024-11-06 18:29:29,582 # version              Prints current RIOT_VERSION
2024-11-06 18:29:29,585 # reboot               Reboot the node
> ps
2024-11-06 18:29:31,316 # ps
2024-11-06 18:29:31,324 # 	pid | name                 | state    Q | pri | stack  ( used) ( free) | base addr  | current     
2024-11-06 18:29:31,333 # 	 - | isr_stack            | -        - |   - |    512 (  172) (  340) | 0x20000000 | 0x200001c8
2024-11-06 18:29:31,342 # 	 1 | main                 | running  Q |   7 |   1536 (  700) (  836) | 0x200002c0 | 0x20000724 
2024-11-06 18:29:31,350 # 	 2 | pktdump              | bl rx    _ |   6 |   1472 (  468) ( 1004) | 0x20001048 | 0x2000155c 
2024-11-06 18:29:31,359 # 	 3 | sam0_eth             | bl anyfl _ |   2 |    896 (  276) (  620) | 0x20000a5c | 0x20000d1c

And native will be tested by the CI.

Issues/PRs references

Depends on and includes build_system/xfa: change API to fix alignment #20960

maribu · 2024-11-06T17:32:47Z

Ah, I should also show that this does actually bring down .text.

This PR:

$ make BOARD=samr21-xpro RIOT_CI_BUILD=1 -C examples/default 
make: Entering directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'
Building application "default" for "samr21-xpro" with CPU "samd21".

   text	  data	   bss	   dec	   hex	filename
  49180	   168	  6196	 55544	  d8f8	/home/[email protected]/Repos/software/RIOT/master/examples/default/bin/samr21-xpro/default.elf
make: Leaving directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'

And in master:

make BOARD=samr21-xpro RIOT_CI_BUILD=1 -C examples/default
make: Entering directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'
Building application "default" for "samr21-xpro" with CPU "samd21".

   text	  data	   bss	   dec	   hex	filename
  49220	   168	  6196	 55584	  d920	/home/[email protected]/Repos/software/RIOT/master/examples/default/bin/samr21-xpro/default.elf
make: Leaving directory '/home/[email protected]/Repos/software/RIOT/master/examples/default'

riot-ci · 2024-11-06T17:32:58Z

Murdock results

✔️ PASSED

a7e2515 sys/shell: reduce overhead of XFA shell commands

Success	Failures	Total	Runtime
10248	0	10249	18m:38s

Artifacts

Documentation preview

maribu · 2024-11-06T17:54:55Z

Interesting: On native64 an 8 byte gap is added between XFA members. Looks like some padding gone wrong in the linker script or so.

maribu · 2024-11-06T20:07:47Z

Turned out that the failure of native64 are a bug in XFA. #20960 fixes that bug. This now depends on and includes #20960.

mguetschow

(comments moved to #20964 (review), which is now part of this PR)

Teufelchen1 · 2024-11-12T12:20:32Z

There are two static issues, one introduced by this PR, the other already in the file tests/rust_libs/tests/01-run.py - could you fix both, please?
In the config.toml, you are switching the wrappers to your own github profile, is that a temporary work-around?

maribu · 2024-11-12T14:53:14Z

There are two static issues, one introduced by this PR, the other already in the file tests/rust_libs/tests/01-run.py - could you fix both, please?

Those should be gone when rebasing on top of #20964

In the config.toml, you are switching the wrappers to your own github profile, is that a temporary work-around?

Indeed. The idea is to proof here that RIOT-OS/rust-riot-wrappers#134 would work (via Murdock/CI), get RIOT-OS/rust-riot-wrappers#134 merged, then replace the commit with updating the rust-riot-wrappers instead of bending the repo to my PR.

benpicco · 2024-11-21T14:27:05Z

This needs a rebase

See-Also: RIOT-OS/RIOT#20958 Merges: #134

See-Also: RIOT-OS/RIOT#20958 See-Also: #134

We do not need to add an array of pointers to the shell commands, just an array of shell commands is sufficient. This reduced the overhead of XFA by `sizeof(void *)` per command.

maribu · 2024-11-22T10:16:51Z

In the config.toml, you are switching the wrappers to your own github profile, is that a temporary work-around?

I dropped the work around now that the change are upstream, and instead just bump rust-riot-wrappers.

This needs a rebase

Done

chrysn

LGTM. That it was the way it was confused me once already; the padding issues you've seen that apparently now work fine might be why it was this way originally. Not tested locally; trusting your and CI tests.

mguetschow

Code changes look good, thanks! Trusting CI for correctness as well.

maribu · 2024-11-23T07:44:15Z

Thx!

maribu requested a review from Teufelchen1 November 6, 2024 17:30

github-actions bot added the Area: sys Area: System label Nov 6, 2024

maribu added Type: cleanup The issue proposes a clean-up / The PR cleans-up parts of the codebase / documentation CI: ready for build If set, CI server will compile all applications for all available boards for the labeled PR labels Nov 6, 2024

maribu requested a review from benpicco November 6, 2024 17:33

maribu force-pushed the sys/shell/xfa-reduce-overhead branch from f4b2ef1 to 9434532 Compare November 6, 2024 20:07

maribu requested review from miri64, leandrolanzieri, aabadie, MichelRottleuthner, kaspar030 and bergzand as code owners November 6, 2024 20:07

maribu added the State: waiting for other PR State: The PR requires another PR to be merged first label Nov 6, 2024

maribu force-pushed the sys/shell/xfa-reduce-overhead branch from 9434532 to c9240ac Compare November 6, 2024 20:21

maribu requested review from vincent-d and gschorcht as code owners November 6, 2024 20:21

github-actions bot added Platform: ARM Platform: This PR/issue effects ARM-based platforms Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: cpu Area: CPU/MCU ports labels Nov 6, 2024

maribu mentioned this pull request Nov 6, 2024

tests/net/nanocoap_cli: make use of XFA for shell commands #20961

Merged

github-actions bot added Area: build system Area: Build system and removed Platform: ESP Platform: This PR/issue effects ESP-based platforms Area: cpu Area: CPU/MCU ports Area: examples Area: Example Applications labels Nov 7, 2024

maribu force-pushed the sys/shell/xfa-reduce-overhead branch from 55a1a7a to 7d2a457 Compare November 8, 2024 08:04

mguetschow requested changes Nov 8, 2024

View reviewed changes

maribu force-pushed the sys/shell/xfa-reduce-overhead branch from 7d2a457 to ce07e7e Compare November 8, 2024 11:41

benpicco removed the State: waiting for other PR State: The PR requires another PR to be merged first label Nov 21, 2024

chrysn added a commit to RIOT-OS/rust-riot-wrappers that referenced this pull request Nov 22, 2024

shell: change XFA format to v2

cdf3fc2

See-Also: RIOT-OS/RIOT#20958 Merges: #134

chrysn added a commit to RIOT-OS/rust-riot-wrappers that referenced this pull request Nov 22, 2024

shell: Drop old XFA mechanism

8dcad3e

See-Also: RIOT-OS/RIOT#20958 See-Also: #134

chrysn mentioned this pull request Nov 22, 2024

shell: Drop old XFA mechanism RIOT-OS/rust-riot-wrappers#135

Draft

maribu added 2 commits November 22, 2024 11:14

rust: bump rust-riot-wrappers

2be92b5

sys/shell: reduce overhead of XFA shell commands

a7e2515

We do not need to add an array of pointers to the shell commands, just an array of shell commands is sufficient. This reduced the overhead of XFA by `sizeof(void *)` per command.

maribu force-pushed the sys/shell/xfa-reduce-overhead branch from ce07e7e to a7e2515 Compare November 22, 2024 10:14

maribu requested a review from chrysn as a code owner November 22, 2024 10:14

github-actions bot added Area: examples Area: Example Applications and removed Area: build system Area: Build system labels Nov 22, 2024

chrysn approved these changes Nov 22, 2024

View reviewed changes

mguetschow approved these changes Nov 22, 2024

View reviewed changes

mguetschow enabled auto-merge November 22, 2024 10:28

mguetschow added this pull request to the merge queue Nov 22, 2024

Merged via the queue into RIOT-OS:master with commit a9108a4 Nov 23, 2024
27 checks passed

maribu deleted the sys/shell/xfa-reduce-overhead branch November 23, 2024 07:44

maribu mentioned this pull request Nov 25, 2024

tests/sys/shell: make test script more robust #21036

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

sys/shell: reduce overhead of XFA shell commands #20958

sys/shell: reduce overhead of XFA shell commands #20958

maribu commented Nov 6, 2024 •

edited by benpicco

Loading

maribu commented Nov 6, 2024

riot-ci commented Nov 6, 2024 •

edited

Loading

maribu commented Nov 6, 2024

maribu commented Nov 6, 2024

mguetschow left a comment •

edited

Loading

Teufelchen1 commented Nov 12, 2024 •

edited

Loading

maribu commented Nov 12, 2024

benpicco commented Nov 21, 2024

maribu commented Nov 22, 2024

chrysn left a comment

mguetschow left a comment

maribu commented Nov 23, 2024

sys/shell: reduce overhead of XFA shell commands #20958

sys/shell: reduce overhead of XFA shell commands #20958

Conversation

maribu commented Nov 6, 2024 • edited by benpicco Loading

Contribution description

Testing procedure

Issues/PRs references

maribu commented Nov 6, 2024

riot-ci commented Nov 6, 2024 • edited Loading

Murdock results

Artifacts

maribu commented Nov 6, 2024

maribu commented Nov 6, 2024

mguetschow left a comment • edited Loading

Choose a reason for hiding this comment

Teufelchen1 commented Nov 12, 2024 • edited Loading

maribu commented Nov 12, 2024

benpicco commented Nov 21, 2024

maribu commented Nov 22, 2024

chrysn left a comment

Choose a reason for hiding this comment

mguetschow left a comment

Choose a reason for hiding this comment

maribu commented Nov 23, 2024

maribu commented Nov 6, 2024 •

edited by benpicco

Loading

riot-ci commented Nov 6, 2024 •

edited

Loading

mguetschow left a comment •

edited

Loading

Teufelchen1 commented Nov 12, 2024 •

edited

Loading