Automate plugin installation

[mostafa]

Ticket(s)

• Automate plugin installation #402

Description

This PR introduces a notable enhancement for creating stateless pods in Kubernetes (k8s) by automating the plugin installation process. At present, installing plugins in GatewayD requires manual intervention, where each plugin is installed using the GatewayD CLI with a GitHub URL or a local file path of a plugin archive. This method requires executing gatewayd plugin install individually for each plugin, which is inefficient (in general and also) for Kubernetes environments. A more automated approach is needed for installing and configuring plugins in k8s. Currently, the only feasible method for plugin installation involves using a volume to store the plugins and a script to execute the plugin installation commands. This approach ensures that restarting a pod does not erase the plugin configurations and binaries.

To address these challenges, this PR automates the plugin installation process by introducing a url parameter in the gatewayd_plugins.yaml configuration file. Consequently, executing gatewayd plugin install without specifying CLI arguments (such as a GitHub URL or file path) will trigger the following automated steps:

1. Parse the gatewayd_plugins.yaml file to extract all the plugin URLs.
2. Download and unpack the plugin archives from the specified URLs.
3. Install the plugins, with the installation details reflected in the updated gatewayd_plugins.yaml file.

It also adds support for the --overwrite-config flag that can be used to disable overwriting the plugins config file after the plugins are installed.

Note

Since the automated plugin installation relies on the gatewayd_plugins.yaml file to be up to date with all the configuration, local paths and checksums for the latest plugins, and overwriting the plugins config file might be disabled by the user (for example, since they use configmaps), one should make sure to use the correct local paths and checksums.

Related PRs

N/A

Development Checklist

• I have added a descriptive title to this PR.
• I have squashed related commits together.
• I have rebased my branch on top of the latest main branch.
• I have performed a self-review of my own code.
• I have commented on my code, particularly in hard-to-understand areas.
• I have added docstring(s) to my code.
• I have made corresponding changes to the documentation (docs).
• I have added tests for my changes.
• I have signed all the commits.

Legal Checklist

• I have read the CONTRIBUTING document.
• I have read and understood the Code of Conduct.
• I have read and agreed to the Apache CLA (required).
• I have read and agreed to the LICENSE (required).

[mostafa]

This is to increase the rate limit of the GitHub API, so that consecutive CI runs don't hit the limit.

[mostafa]

This is to prevent GitHub API rate limit to be reached by pulling the plugin once and reusing it (almost) everywhere.