(Archived) Roadmap - SingularityCE 4.0

[dtrudg]

SingularityCE 4.0 is targeted for release in Jul-Aug 2023.

Please use this thread to comment and suggest features for inclusion in 4.0. We'll update this top post as items are suggested and discussed.

4.0 Planned Features

• SIF Encapsulated OCI Images ✅
  Support encapsulation of native OCI data and configs in SIF. This will permit native OCI containers (not a translation to singularity images) to benefit from the performance (metadata) and portability advantages of the SIF single file format.

  • Pull from OCI sources into OCI-SIF. ✅
  • Push / Pull of these OCI-SIFs for library and oras destinations. ✅
  • Execution of OCI-SIFs with OCI engine. ✅

• Mainstream execution of native singularity images through OCI runtime 🔨
  Cover remaining run/shell/exec flags/features that would make it practical for majority of users to switch between singularity's native runtime, and the OCI runtime, for execution of native singularity images (not encapsulated OCI).

• Container Device Interface Support ✅
  Support setup of NVIDIA, Intel GPUs and other devices using CDI - the Container Device Interface. CDI will be the preferred GPU integration for --oci mode, and considered for support with the native singularity runtime. Co-ordinated with / collaborating with NVIDIA - Add CDI --device support to --oci mode #1394

• Rework the remote Command ✅
  Reworking the remote Command #78
  The remote command configures access to Sylabs cloud services, alternative keyservers, and OCI registries. It is complex as there is overlap between these targets, a concept of priorities and global/exclusive keyservers etc. This is likely a good area for a comprehensive rework. See dedicated Discussions topic at Revising `remote` functionality #1639

• Consider Overhaul of the key command ✅
  Overhaul key command #79
  The implementation of the key command has some significant technical debt, with portions of code at a different level than they should be. In addition, management of keys is not trivial for those not clear about the public/private nature, fingerprints etc. of GPG. More expressive CLI output, and re-examining of the verbs and flags would be useful.

• Complete Removal of Code Supporting Legacy Distros ✅
  Removal of Code Supporting Legacy Distros (3.10 tasks) #82
  SingulartyCE contains various workarounds for RHEL6 / 2.6 kernel, old versions of invoked external programs etc. Special cases supporting these distributions can be removed gradually through 3.10 and beyond. This will reduce code and testing complexity.

• internal vs pkg ✅
  Review and correct internal/ vs pkg/ code location #80
  Various portions of code in public pkg/ areas still use internal/ packages. This should be worked out so that it is possible to have...

• Go Module Conformance
  Go Module Conformance #81
  A major version offers an opportunity to revise the versioning approach, so that SingularityCE pkg/ code can be called from other projects as expected of a go module.

• Rework fakeroot engine for nvccli compatibility ❌
  The hybrid fakeroot workflow creates / enters namespaces and performs container setup in an order that is not compatible with using the nvidia-container-cli binary for NVIDIA GPU setup. Rework the engine to permit this. May include removing Singularity's own subuid/gid mapping setup, to depend on the standard newuidmap / newgidmap tooling employed by other rootless runtimes, and SingularityCE in no-setuid mode.

This has been abandoned due to NVIDIA's container support transitioning from nvidia-container-cli to CDI

Consider #1736 for 4.1?

4.0 Under Consideration

[preminger]

There's now a dedicated Discussions topic for the proposed reworking of the remote command: #1639

[dtrudg]

SingularityCE 4.0.0 has now been released!