Move publish to tag-publish

.github/publish.yaml

@@ -0,0 +1,8 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/camptocamp/tag-publish/0.7.1/tag_publish/schema.json
+
+pypi:
+  versions:
+    - version_tag
+    - version_branch
+  packages:
+    - {}

.github/spell-ignore-words.txt

@@ -2,7 +2,6 @@ dpkg
 repology.org
 pypi
 Snyk
-c2cciutils-publish
 fixup
 codespell
 pipenv

.github/workflows/main.yaml

@@ -87,9 +87,10 @@ jobs:
         run: make build
 
       - name: Publish
-        run: c2cciutils-publish
-        if: |
-          env.HAS_SECRETS == 'HAS_SECRETS'
+        run: tag-publish
+        if: env.HAS_SECRETS == 'HAS_SECRETS'
+        env:
+          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
 
       - run: git diff --exit-code --patch > /tmp/dpkg-versions.patch || true
         if: failure()

Dockerfile

@@ -9,7 +9,7 @@ RUN --mount=type=cache,target=/var/lib/apt/lists \
     && apt-get upgrade --yes \
     && apt-get install --yes --no-install-recommends apt-utils \
     && DEBIAN_FRONTEND=noninteractive TZ=Etc/UTC apt-get install --yes tzdata \
-    && apt-get install --yes --no-install-recommends binutils python3-pip python3-venv \
+    && apt-get install --yes --no-install-recommends binutils git python3-pip python3-venv \
     && python3 -m venv /venv
 
 ENV PATH=/venv/bin:$PATH
@@ -27,7 +27,7 @@ RUN --mount=type=cache,target=/root/.cache \
 # Do the conversion
 COPY poetry.lock pyproject.toml ./
 ENV POETRY_DYNAMIC_VERSIONING_BYPASS=0.0.0
-RUN poetry export --extras=checks --extras=publish --extras=audit --extras=version --output=requirements.txt \
+RUN poetry export --output=requirements.txt \
     && poetry export --with=dev --output=requirements-dev.txt
 
 # Base, the biggest thing is to install the Python packages
@@ -49,25 +49,12 @@ FROM base AS run
 
 SHELL ["/bin/bash", "-o", "pipefail", "-c"]
 
-COPY .nvmrc /tmp
-RUN --mount=type=cache,target=/var/lib/apt/lists --mount=type=cache,target=/var/cache \
-    apt-get update \
-    && apt-get --assume-yes upgrade \
-    && apt-get install --assume-yes --no-install-recommends apt-transport-https gnupg curl \
-    && NODE_MAJOR="$(cat /tmp/.nvmrc)" \
-    && echo "deb [signed-by=/etc/apt/keyrings/nodesource.gpg] https://deb.nodesource.com/node_${NODE_MAJOR}.x nodistro main" > /etc/apt/sources.list.d/nodesource.list \
-    && curl --silent https://deb.nodesource.com/gpgkey/nodesource-repo.gpg.key | gpg --dearmor --output=/etc/apt/keyrings/nodesource.gpg \
-    && apt-get update \
-    && apt-get install --assume-yes --no-install-recommends "nodejs=${NODE_MAJOR}.*" libmagic1 git python3-dev libpq-dev gcc python-is-python3
-
 RUN python3 -m compileall -q -- *
 
 COPY . ./
 ARG VERSION=dev
 RUN --mount=type=cache,target=/root/.cache \
-    --mount=type=cache,target=/root/.npm \
-    cd c2cciutils && npm install && cd - \
-    && POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=. \
+    POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=. \
     && python3 -m pip freeze > /requirements.txt \
     && python3 -m compileall -q /app/c2cciutils
 

README.md

@@ -46,16 +46,13 @@ C2cciutils make easier to have those workflows in a project:
 
 - `auto-review.yaml`: Auto review the Renovate pull requests
 - `backport.yaml`: Trigger the backports (work with labels)
-- `clean.yaml`: Clean the Docker images related on a deleted feature branch
 - `main.yaml`: Main workflow especially with the c2cciutils-checks command
 
 All the provided commands used in the workflow:
 
 - `c2cciutils`: some generic tools.
 - `c2cciutils-version`: Create a new version of the project.
 - `c2cciutils-env`: Print some environment information.
-- `c2cciutils-publish`: Publish the project.
-- `c2cciutils-clean`: Delete Docker images on Docker Hub after corresponding branch have been deleted.
 
 ## Utilities
 
@@ -102,7 +99,6 @@ In the CI we need to have the following secrets::
 ## Use locally, in the projects that use c2cciutils
 
 Install it: `python3 -m pip install --user --requirement ci/requirements.txt`
-Dry run publish: `GITHUB_REF=... c2cciutils-publish --dry-run ...`
 
 ## Configuration
 
@@ -159,183 +155,6 @@ Select a formatter:
 - Configure Default Formatter...
 - Select the formatter
 
-## Publishing
-
-### To pypi
-
-The config is like this:
-
-```yaml
-versions:
-  # List of kinds of versions you want to publish, that can be:
-  # rebuild (specified with --type),
-  # version_tag, version_branch, feature_branch, feature_tag (for pull request)
-```
-
-It we have a `setup.py` file, we will be in legacy mode:
-When publishing, the version computed from arguments or `GITHUB_REF` is put in environment variable `VERSION`, thus you should use it in `setup.py`, example:
-
-```python
-VERSION = os.environ.get("VERSION", "1.0.0")
-```
-
-Also we consider that we use `poetry` with [poetry-dynamic-versioning](https://pypi.org/project/poetry-dynamic-versioning/) to manage the version, and [poetry-plugin-tweak-dependencies-version](https://pypi.org/project/poetry-plugin-tweak-dependencies-version/) to manage the dependencies versions.
-
-Example of configuration:
-
-```toml
-[tool.poetry-dynamic-versioning]
-enable = true
-vcs = "git"
-pattern = "^(?P<base>\\d+(\\.\\d+)*)"
-format-jinja = """
-{%- if env.get("VERSION_TYPE") == "version_branch" -%}
-{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
-{%- elif distance == 0 -%}
-{{serialize_pep440(base)}}
-{%- else -%}
-{{serialize_pep440(bump_version(base), dev=distance)}}
-{%- endif -%}
-"""
-
-```
-
-Note that we can access to the environment variables `VERSION`,`VERSION_TYPE` and `IS_MASTER`.
-
-Then by default:
-
-- Tag with `1.2.3` => release `1.2.3`
-- Commit on feature branch just do a validation
-- Commit on `master` branch after the tag 1.3.0 => release `1.4.0.dev1`
-- Commit on `1.3` branch after the tag 1.3.0 => release `1.3.1.dev1`
-
-#### Authentication
-
-If the file `~/.pypirc` exists we consider that we ar already logged in also
-we will do the login with the `pypi` server with OpenID Connect (OIDC).
-
-The OIDC login is recommended because it didn't needs any additional secrets,
-but it need some configuration on pypi in the package,
-see the [GitHub Documentation](https://docs.github.com/en/actions/security-for-github-actions/security-hardening-your-deployments/configuring-openid-connect-in-pypi#adding-the-identity-provider-to-pypi).
-
-#### Integration if the package directly in a Docker image
-
-To make it working in the `Dockerfile` you should have in the `poetry` stage:
-
-```Dockerfile
-ENV POETRY_DYNAMIC_VERSIONING_BYPASS=dev
-RUN poetry export --extras=checks --extras=publish --output=requirements.txt \
-    && poetry export --with=dev --output=requirements-dev.txt
-```
-
-And in the `run` stage
-
-```Dockerfile
-ARG VERSION=dev
-RUN --mount=type=cache,target=/root/.cache \
-    POETRY_DYNAMIC_VERSIONING_BYPASS=${VERSION} python3 -m pip install --disable-pip-version-check --no-deps --editable=.
-```
-
-And in the `Makefile`:
-
-```Makefile
-VERSION = $(strip $(shell poetry version --short))
-
-.PHONY: build
-build: ## Build the Docker images
-    docker build --build-arg=VERSION=$(VERSION) --tag=$(GITHUB_REPOSITORY) .
-```
-
-### To Docker registry
-
-The config is like this:
-
-```yaml
-latest: True
-images:
-  - # The base name of the image we want to publish
-    name:
-repository:
-  <internal_name>:
-    # The fqdn name of the server if not Docker hub
-    server:
-    # List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
-    # version_tag, version_branch, feature_branch, feature_tag (for pull request)
-    version:
-    # List of tags we want to publish interpreted with `format(version=version)`
-    # e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
-    # (that should be built by the application build) is `latest-lite`, and it will be published
-    # with the tag `1.2.3-lite`.
-    tags:
-    # If your images are published by different jobs you can separate them in different groups
-    # and publish them with `c2cciutils-publish --group=<group>`
-    group:
-```
-
-By default, the last line of the `SECURITY.md` file will be published (`docker`) with the tag
-`latest`. Set `latest` to `False` to disable it.
-
-With the `c2cciutils-clean` the images on Docker hub for `feature_branch` will be removed on branch removing.
-
-## Download applications
-
-In case some executables or applications from GitHub releases or any other URLs are required on the CI host
-and are not handled by any dependency manager, we provide a set of tools to install them and manage upgrades
-through Renovate.
-
-Create an application file (e.-g. `applications.yaml`) with:
-
-```yaml
-# yaml-language-server: $schema=https://raw.githubusercontent.com/camptocamp/c2cciutils/master/c2cciutils/schema-applications.json
-
-# Application from GitHub release
-<organization>/<project>:
-  get-file-name: <file name present in the release>
-  to-file-name: <The file name you want to create in ~/.local/bin>
-  finish-command: # The command you want to run after the file is downloaded
-    - - chmod # To be executable (usually required)
-      - +x
-      - <to-file-name>
-    - - <to-file-name> # Print the version of the application
-      - --version
-# Application from GitHub release in a tar file (or tar.gz)
-<organization>/<project>:
-  get-file-name: <file name present in the release>
-  type: tar
-  tar-file-name: <The file name available in the tar file>
-  to-file-name: <The file name you want to create in ~/.local/bin>
-  finish-command: [...] # The command you want to run after the file is downloaded
-# Application from an URL
-<application reference name>:
-  url-pattern: <The URL used to download the application>
-  to-file-name: <The file name you want to create in ~/.local/bin>
-  finish-command: [...] # The command you want to run after the file is downloaded
-```
-
-In the attributes `url-pattern`, `get-file-name` you can use the following variables:
-
-- `{version}`: The version of the application present in the version file.
-- `{version_quote}`: The URL encoded version.
-- `{short_version}`: The version without the `v` prefix.
-
-The `applications-versions.yaml` file is a map of applications and their versions.
-
-Add in your Renovate configuration:
-
-```json5
-  regexManagers: [
-    {
-      fileMatch: ['^applications-versions.yaml$'],
-      matchStrings: [
-        '(?<depName>[^\\s]+): (?<currentValue>[^\\s]+) # (?<datasource>[^\\s]+)',
-      ],
-    },
-  ],
-```
-
-Now you need to call `c2cciutils-download-applications --applications-file=applications.yaml --versions-file=applications-version.yaml`
-to install required applications on CI host before using them (an already installed application is installed only if needed).
-
 ## Use Renovate to trigger a new build instead of the legacy rebuild
 
 Run the command `c2cciutils-docker-versions-gen camptocamp/image[:tag]` to generate a file that is a kind of package lock of the Debian packages in the file `ci/dpkg-versions.yaml`.