🔭 Ignore(or rollback) the missing caches when executing rugby use

[cragod]

Is your feature request related to a problem? Please describe.

Yes, but this may not be a common problem.

We are using Rugby to improve our project's development workflow. Here is some background information about the project:

Due to the size of our business modules (utilizing development pods for dependencies), we aim to cache on the CI server during branch merging and use the cache on developers' machines.

CI Server Caching Workflow:

1. Branch merged
2. Triggers job
3. rugby warmup —analyse ... (acquires a complete cache list)
4. rugby cache …
5. Determines and saves uncached packages to the file server

Local Developer Workflow (executed via build pre-action in Xcode):

1. rugby warmup —analyse …
2. Pulls the corresponding cache package from the file server
3. rugby use … (avoids using build or cache because module compilation is slow, and developers prefer compiling uncached modules in the regular compilation phase in Xcode)

As mentioned, we use development pods, allowing developers to make changes to their assigned modules at any time. In such cases, the cache becomes invalid, and a new path is used to search for the framework (after the build pre-action), leading to compilation failures.

Describe the solution you'd like

Ignore(or rollback) the missing caches when executing rugby use

Describe alternatives you've considered

We can add an additional step in the project to exclude targets without a cache. However, a challenge is that the target name is not reflected in the cache path. We need to maintain a mapping between target names and framework names.

Additional context

nil :)

[cragod]

Additionally, while debugging this process and https://github.com/swiftyfinch/Rugby/issues/279, I found a need for a more comprehensive analysis command, an enhanced version of warmup --analyse. This could include:

• Outputting information on all binaries or those not found.
• Generating hash YAMLs to debug hash-related issues, such as understanding why someone did not hit the cache.

Currently, I've added some temporary debugging code within the warmup logic to achieve these goals. However, it might be beneficial to consider a new standalone command for these expansions in the future. This is just a suggestion.

[swiftyfinch]

@cragod It's an interesting idea. Thank you!

I'm not sure about auto-rollback in use command. I've tried to design a such feature, but it's not always what we want.

But I think that it can be handy to have a flag in use command to patch only found binaries.
Something like rugby use --found. This logic has been already used in build command. It reuses found binaries including downloaded ones from CI.

---

About warmup --analyse:

• Yeah, feel free to suggest new design. I think it can be useful for the whole Rugby community 👍
• We should remember of backward compatibility based on this guide:
  https://github.com/swiftyfinch/Rugby/blob/main/Docs/remote-cache.md
• FYI: And recently, I've added docs to env command:
  https://github.com/swiftyfinch/Rugby/blob/main/Docs/commands-help/env.md#keep-hash-yml-files

[cragod]

Local Developer Workflow (executed via build pre-action in Xcode):

1. rugby warmup —analyse …
2. Pulls the corresponding cache package from the file server
3. rugby use … (avoids using build or cache because module compilation is slow, and developers prefer compiling uncached modules in the regular compilation phase in Xcode)

Update:
I've used a script that worked with warmup —analyse to retrieve the currently downloaded cache targets (with the architecture, SDK, and configuration of the current compilation).
After that, in the use command, I added a --ignore-dependencies flag, which sets includingDependencies to false in 'UseBinariesManager', then only pass the cached targets to use. This way, only the targets for which we have cache will be installed.

[swiftyfinch]

@cragod Hello! Is it still actual to you? I understood about suggestion to add an analyse command, but what about the use command with an additional flag?

I thought a little bit about an interface:

# We can reuse binariesManager.findBinaries here and skip patching of missing binaries
rugby use --skip-missing-binaries

# It means that we should skip the build command
# and should pass --skip-missing-binaries flag to the use command
rugby cache --skip-build 

What do you think about this interface? And I need to know if it necessary to you, because any extension of the current interface make 🏈 Rugby more complicated.

[cragod]

@swiftyfinch Thank you for your reply. rugby use --skip-missing-binaries is useful for our current use case.

If cross-device cache sharing is an important use case, it can filter out targets that have not yet been built in the remote cache.

Of course, this would make rugby more complex, so you should carefully consider it.