Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Setup a local ci sstate-mirror #13

Open
quaresmajose opened this issue Sep 11, 2024 · 3 comments
Open

Setup a local ci sstate-mirror #13

quaresmajose opened this issue Sep 11, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request
Milestone

Comments

@quaresmajose
Copy link
Contributor

Running more than one machine using the same local sstate-cache is not efficient and we need to setup a local sstate mirror to be used in CI builds.

The PR #12 build a change that only add a new job but however the local sstate-cache is not being used fully, despite the fact that this is persistent between builds.

This can be seen in the job where only about 52% is being used:

2024-09-11 14:13:17 - INFO     - Initialising tasks...Sstate summary: Wanted 2180 Local 1141 Mirrors 0 Missed 1039 Current 0 (52% match, 0% complete)
@quaresmajose quaresmajose self-assigned this Sep 11, 2024
@ricardosalveti
Copy link
Contributor

This hit in particular is probably because some of the base layers changed (since the run happens against master all the time).

Triggered a re-run and got:

2024-09-11 17:06:37 - INFO     - Initialising tasks...Sstate summary: Wanted 2180 Local 2176 Mirrors 0 Missed 4 Current 0 (99% match, 0% complete)

We could of course investigate ways to improve this, hashserv, etc.

@EmbeddedAndroid EmbeddedAndroid added the enhancement New feature or request label Sep 11, 2024
@quaresmajose
Copy link
Contributor Author

Right, we need some type of lock on the layers to improve the build. Maybe in this case the sstate mirror is not needed because all the machines are pretty similar and from the same arch.

About the lock:
running kas with --lock can improve this situation but for that it might be better to have an external repository (like quic-yocto/ci-manifest.git) just to save the hashes on the kas yaml. this way we don't pollute the local kas config of the bsp layer and not force the users to a specific version.

@ricardosalveti
Copy link
Contributor

running kas with --lock can improve this situation but for that it might be better to have an external repository (like quic-yocto/ci-manifest.git) just to save the hashes on the kas yaml. this way we don't pollute the local kas config of the bsp layer and not force the users to a specific version.

Yeah, that is my thinking as well. We can probably always use the latest in PRs and generate a lock file after the merge happens, publishing into another external repo.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
Development

No branches or pull requests

3 participants