Provide a way to exclude packages/slices when running cut

[lengau]

It would be nice to be able to exclude certain packages or slices when running cut, even if those are dependencies of the requested chisel slices. This would allow snaps and charms to be able to use chiselled packages without pulling everything all the way back to libc.

Ideally for my particular use case (integrating chisel into more craft* apps), we'd be able to pass an exclusions file (one package/slice name per line in a text file), and if a slice is explicitly listed it overrides that exclusions file.

Examples for why I'd like this

A core24 snap that only stages `curl_bins` is 8.6 MiB, whereas using craft-parts's ignore list but extracting the full debs is still only 1.1 MiB. Here's the `snapcraft.yaml` file:

name: my-snap-name # you probably want to 'snapcraft register <name>'
base: core24 # the base snap is the execution environment for this snap
version: '0.1' # just for humans, typically '1.2+git' or '1.3.2'
summary: Single-line elevator pitch for your amazing snap # 79 char long summary
description: |
  This is my-snap's description. You have a paragraph or two to tell the
  most important story about your snap. Keep it under 100 words though,
  we live in tweetspace and your description wants to look good in the snap
  store.

grade: devel # must be 'stable' to release into candidate/stable channels
confinement: devmode # use 'strict' once you have the right plugs and slots

parts:
  my-part:
    # See 'snapcraft plugins'
    plugin: nil
    stage-packages:
      - curl_bins  # Remove _bins here to extract the full packages

A real-world example that could use this feature are the postgresql charms. Currently they're using a workaround because Charmcraft doesn't stage packages correctly. However, an "ideal" version of this would have this part instead:

libpq:
  plugin: nil
  stage-packages: [libpq5_libs]

[letFunny]

@lengau Apologies for taking so long to reply. I think I am missing some context here, are you saying that you want to install curl_bins without libc? How would that work if curl is linked against libc?

[lengau]

Thanks for the response! The use case here would be for systems where we're going to have a different base than rocks - e.g. snaps and charms. Currently Rockcraft does this by using overlays and deduplicating files, but snaps and charms can't do this for a variety of reasons, one of which is that it's not feasible for snaps to tell whether a file was staged because it's in a stage-package dependency (can be removed because the base snap provides it) or because it's output of the part.

If chisel implements this feature, Snapcraft could parse e.g. /snap/core24/current/usr/share/snappy/dpkg.yaml (or similar for other bases) to get a list of packages to ask chisel to exclude. This would provide parity for using chiselled packages in snaps to staging full packages. Right now, users who want to reduce what they stage from a package will use the stage or prime keywords to only include specific files, which can be problematic as they seldom include files such as those included in copyright slices. This will also encourage more users to contribute slices, as they will be incentivized to use a slice rather than duplicating their stage filters across many snaps.

This is similarly true for charms, which right now do not support staging packages at all, causing users to do workarounds in Charmcraft using build-packages and cp, which isn't ideal either. Rather than including everything used in both Rockcraft and Snapcraft to allow staging full packages, I'd prefer to include only chisel and require users to use slices. A real-world example there is the postgresql-operator charm, which dumps libpq.so.5 (but not the copyright file) into the charm so that psycopg2 can talk to postgres.

[cjdcordeiro]

This is an interesting feature request and we should definitely talk more about it.

This sounds too craft-specific, which makes me wonder if Chisel is the right place to handle it. But maybe there are other useful cases for such a feature.

In fact, even if we were to implement this feature, we must first consider the ability for Chisel to cut on top of an existing rootfs. This is not yet addressed, and it is fundamentally what you are asking for (kind of) - don't install what's not needed.

Let's discuss it. Would you be able to join one of our Rocks Community monthly meetings or even the Rockcraft weekly sync? We can also talk about it during the Clinic hours.

[letFunny]

I also want to add that now that Chisel is able to generate a manifest you can deal much more easily with:

Right now, users who want to reduce what they stage from a package will use the stage or prime keywords to only include specific files, which can be problematic as they seldom include files such as those included in copyright slices. This will also encourage more users to contribute slices, as they will be incentivized to use a slice rather than duplicating their stage filters across many snaps.

You can include base-files_manifest slice (example definition) which will generate a compressed file containing an inventory of all the installed packages and slices with their corresponding files. You can use it to query for all the installed files per slice that you want to remove and remove them, or alternatively query for all the slices that you want to keep.

We need better documentation for the manifest and we are working on it but if you have any questions about your use-case we'll be happy to help.