Rework documentation generation functionality of xtask

[jessebraham]

This will be the first or 2 or 3 PRs for #2967.

We can now generate documentation for self-hosting for all packages in the repository, and generate the indexes for those packages which require them (i.e. esp-hal, esp-lp-hal, esp-wifi):

$ cargo xtask build-documentation
$ cargo xtask build-documentation-index

Each package will have its documentation generated, and for those which have API differences depending on which chip features is enabled, we will generate docs for each specific chip. Otherwise, just a single version is generated.

There's probably still some room for improvement here, but given that it's just automation I didn't spend too much time refining it. Happy to make any changes if requested, though.

I will still need to rework the documentation.yml workflow to publish to the Espressif documentation server, but need to figure that all out still (and get deployment credentials for the production server), so that will be handled separately.

[bjoernQ]

For esp-alloc I get

❯ cargo xtask build-documentation --packages=esp-alloc
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.04s
     Running `target\debug\xtask.exe build-documentation --packages=esp-alloc`
[2025-01-24T12:43:48Z WARN  xtask] Package 'esp-alloc' does not have chip features, ignoring argument
[2025-01-24T12:43:48Z INFO  xtask] Building 'esp-alloc' documentation
error: -Zbuild-std requires --target
Error: Failed to execute cargo subcommand

[jessebraham]

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.13s
     Running `target/debug/xtask build-documentation --packages=esp-alloc`
[2025-01-24T12:45:53Z WARN  xtask] Package 'esp-alloc' does not have chip features, ignoring argument
[2025-01-24T12:45:53Z INFO  xtask] Building 'esp-alloc' documentation
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.07s
   Generated /Users/jesse/Work/esp-hal/esp-alloc/target/doc/esp_alloc/index.html

🤔

[bugadani]

❯ cargo xtask build-documentation --packages=esp-alloc        
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.15s
     Running `target\debug\xtask.exe build-documentation --packages=esp-alloc`
[2025-01-24T12:49:45Z WARN  xtask] Package 'esp-alloc' does not have chip features, ignoring argument
[2025-01-24T12:49:45Z INFO  xtask] Building 'esp-alloc' documentation
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.11s
   Generated C:\_Espressif\esp-hal\esp-alloc\target\doc\esp_alloc\index.html

Works for me, too

[bjoernQ]

ok - updated and use nightly and now it works 🤷‍♂️

[bjoernQ]

Thanks

[bugadani]

For me, build-index gives a nebulous "Error: The system cannot find the path specified. (os error 3)" when I don't specify and packages, which doesn't really tell me why it failed. Also, calling it for esp-alloc,esp-hal,esp-wifi I get an index that only has esp-hal.

[bjoernQ]

For me, build-index gives a nebulous "Error: The system cannot find the path specified. (os error 3)" when I don't specify and packages, which doesn't really tell me why it failed.

That at least worked for me 🤷‍♂️

[jessebraham]

For me, build-index gives a nebulous "Error: The system cannot find the path specified. (os error 3)" when I don't specify and packages, which doesn't really tell me why it failed.

Not sure, Works On My System™️ 🤷🏻‍♂️ Potentially a Windows-related issue, but seems it works for Bjoern so again, no idea.

Also, calling it for esp-alloc,esp-hal,esp-wifi I get an index that only has esp-hal.

Since we're moving to our own self-hosted documentation server everything doesn't need to be namespaced under esp-hal like it is on GH Pages. The index is only generated for packages which have chip-specific docs, this is intentional.

Works as expected for me, i.e. esp-alloc does not get a generated index but esp-hal/esp-wifi do:

➜  esp-hal git:(feature/docs) ✗ ll docs/esp-alloc/0.6.0
total 40
-rw-r--r--@  1 jesse  staff    73B Jan 24 13:49 crates.js
drwxr-xr-x@ 15 jesse  staff   480B Jan 24 09:14 esp_alloc
-rw-r--r--@  1 jesse  staff   2.8K Jan 24 13:49 help.html
-rw-r--r--@  1 jesse  staff   2.4K Jan 24 13:49 search-index.js
drwxr-xr-x@  3 jesse  staff    96B Jan 24 09:14 search.desc
-rw-r--r--@  1 jesse  staff   2.7K Jan 24 13:49 settings.html
drwxr-xr-x@  3 jesse  staff    96B Jan 24 09:14 src
-rw-r--r--@  1 jesse  staff   144B Jan 24 13:49 src-files.js
drwxr-xr-x@ 32 jesse  staff   1.0K Jan 24 13:49 static.files
drwxr-xr-x@  4 jesse  staff   128B Jan 24 09:14 trait.impl
➜  esp-hal git:(feature/docs) ✗ ll docs/esp-hal/0.23.1
total 32
-rw-r--r--@  1 jesse  staff    11K Jan 16 13:06 esp-rs.svg
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:14 esp32
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:14 esp32c2
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:14 esp32c3
drwxr-xr-x@ 51 jesse  staff   1.6K Jan 24 09:15 esp32c6
drwxr-xr-x@ 51 jesse  staff   1.6K Jan 24 09:15 esp32h2
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32s2
drwxr-xr-x@ 56 jesse  staff   1.8K Jan 24 09:15 esp32s3
-rw-r--r--@  1 jesse  staff   3.9K Jan 24 10:30 index.html
➜  esp-hal git:(feature/docs) ✗ ll docs/esp-wifi/0.12.0
total 32
-rw-r--r--@  1 jesse  staff    11K Jan 16 13:06 esp-rs.svg
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32c2
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32c3
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32c6
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32h2
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32s2
drwxr-xr-x@ 14 jesse  staff   448B Jan 24 09:15 esp32s3
-rw-r--r--@  1 jesse  staff   3.9K Jan 24 10:30 index.html

[MabezDev]

For me, build-index gives a nebulous "Error: The system cannot find the path specified. (os error 3)"

I believe this happens if you don't build the docs first, or delete the docs folder

[bugadani]

I'm sure it's a me-issue, but:

❯ cargo xtask build-documentation --packages=esp-hal --packages esp-wifi --chips esp32
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
     Running `target\debug\xtask.exe build-documentation --packages=esp-hal --packages esp-wifi --chips esp32`
[2025-01-24T13:42:58Z INFO  xtask] Building 'esp-hal' documentation targeting 'esp32'
   Compiling esp-config v0.3.0 (C:\_Espressif\esp-hal\esp-config)
   Compiling esp-hal v0.23.1 (C:\_Espressif\esp-hal\esp-hal)
 Documenting esp-hal v0.23.1 (C:\_Espressif\esp-hal\esp-hal)


WARNING: use --release
  We *strongly* recommend using release profile when building esp-hal.
  The dev profile can potentially be one or more orders of magnitude
  slower than release, and may cause issues with timing-senstive
  peripherals and/or devices.

    Finished `dev` profile [unoptimized + debuginfo] target(s) in 4.67s
   Generated C:\_Espressif\esp-hal\esp-hal\target\xtensa-esp32-none-elf\doc\esp_hal\index.html
[2025-01-24T13:43:03Z INFO  xtask] Building 'esp-wifi' documentation targeting 'esp32'
    Updating crates.io index
   Compiling esp-config v0.3.0 (C:\_Espressif\esp-hal\esp-config)
   Compiling esp-hal v0.23.1 (C:\_Espressif\esp-hal\esp-hal)
   Compiling esp-wifi v0.12.0 (C:\_Espressif\esp-hal\esp-wifi)
warning: [email protected]: esp-wifi should be built with optimization level 2 or 3 - yours is 0.


WARNING: use --release
  We *strongly* recommend using release profile when building esp-hal.
  The dev profile can potentially be one or more orders of magnitude
  slower than release, and may cause issues with timing-senstive
  peripherals and/or devices.

 Documenting esp-wifi v0.12.0 (C:\_Espressif\esp-hal\esp-wifi)
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 9.04s
   Generated C:\_Espressif\esp-hal\esp-wifi\target\xtensa-esp32-none-elf\doc\esp_wifi\index.html

❯ cargo xtask build-documentation-index --packages=esp-hal --packages esp-wifi
    Finished `dev` profile [unoptimized + debuginfo] target(s) in 0.10s
     Running `target\debug\xtask.exe build-documentation-index --packages=esp-hal --packages esp-wifi`

After building hal/wifi docs, and index, I don't have a docs/index.html

[jessebraham]

After building hal/wifi docs, and index, I don't have a docs/index.html

Yes this is intentional. I discussed adding this with @MabezDev but we are not sure what other information will be included other than just listing the packages, so I have left it for a future PR.

[bugadani]

So what does build-documentation-index do now?