Skip to content

Commit

Permalink
Refactor code
Browse files Browse the repository at this point in the history
  • Loading branch information
@sbrunner
sbrunner committed Nov 21, 2024
1 parent 14c0960 commit bf4fbe9
Show file tree
Hide file tree
Showing 10 changed files with 339 additions and 368 deletions.
6 changes: 3 additions & 3 deletions .github/publish.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -2,12 +2,12 @@

pypi:
versions:
- version_tag
- version_branch
- tag
- stabilization_branch
- default_branch
packages:
- {}
docker:
auto_login: true
images:
- name: camptocamp/tag-publish
helm:
Expand Down
4 changes: 4 additions & 0 deletions .github/workflows/main.yaml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ jobs:

- run: docker build --tag camptocamp/tag-publish tests

- name: Publish dry-run
run: poetry run tag-publish --dry-run
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
- name: Publish
run: poetry run tag-publish
env:
Expand Down
25 changes: 14 additions & 11 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
The main goals of Tag Publish offer the commands to publish the project,
Using a tag, a stabilization branch, a feature branch or a pull request.

When possible it can do a secret-less publishing, if it's not possible the login should be done before the publishing.
When possible it can do a secret-less publishing (privileged in defaults), if it's not possible the login should be done before the publishing.

See the [documentation](https://github.com/camptocamp/c2cciutils/wiki/Publishing).

Expand Down Expand Up @@ -62,8 +62,9 @@ To create a patch version you should just create tag.

This tool can publish on different kind of versions:

- `version_tag`: Related to a Git tag.
- `version_branch`: Related to a stabilization branch (including the default branch).
- `tag`: Related to a Git tag.
- `default_branch`: Related to the default branch.
- `stabilization_branch`: Related to a stabilization branch (including the default branch).
- `feature_branch`: Related to a feature branch or a pull request.

We can also publish on different kind of versions like `rebuild` by using the `--type` argument.
Expand Down Expand Up @@ -108,7 +109,7 @@ https://docs.github.com/en/actions/security-for-github-actions/security-hardenin

By default the package will be published only on tag, if you want to publish on stabilization branch you should add
a `versions` key with the list of versions you want to publish, that can be:
`rebuild` (specified with --type), `version_tag`, `version_branch`, `feature_branch`, `feature_tag` (for pull request)
`rebuild` (specified with --type), `tag`, `default_branch`, `stabilization_branch`, `feature_branch`, `pull_request` (for pull request merge: number)

It we have a `setup.py` file, we will be in legacy mode:
When publishing, the version computed from arguments or `GITHUB_REF` is put in environment variable `VERSION`, thus you should use it in `setup.py`, example:
Expand All @@ -127,8 +128,10 @@ enable = true
vcs = "git"
pattern = "^(?P<base>\\d+(\\.\\d+)*)"
format-jinja = """
{%- if env.get("VERSION_TYPE") == "version_branch" -%}
{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
{%- if env.get("VERSION_TYPE") == "default_branch" -%}
{{serialize_pep440(bump_version(base, 1), dev=distance)}}
{%- elif env.get("VERSION_TYPE") == "stabilization_branch" -%}
{{serialize_pep440(bump_version(base, 2), dev=distance)}}
{%- elif distance == 0 -%}
{{serialize_pep440(base)}}
{%- else -%}
Expand All @@ -138,7 +141,7 @@ format-jinja = """
```

Note that we can access to the environment variables `VERSION`,`VERSION_TYPE` and `IS_MASTER`.
Note that we can access to the environment variables `VERSION`,`VERSION_TYPE`.

Then by default:

Expand Down Expand Up @@ -199,13 +202,13 @@ docker:
If you want to use the GitHub token to be logged in on ghcr you should set `auto_login` to `True`, the
requires the permissions are `packages: write`.

With that the image initially named `camptocamp/tag-publish:latest` will be published on GitHub CHCR and on Docker hub.
With that the image initially named `camptocamp/tag-publish:latest` will be published on GitHub GHCR and on Docker hub.

The full config is like this:

```yaml
docker:
auto_login: False
github_oidc_login: True
latest: True
images:
- # The base name of the image we want to publish
Expand All @@ -215,7 +218,7 @@ docker:
# The fqdn name of the server if not Docker hub
server:
# List of kinds of versions you want to publish, that can be: rebuild (specified using --type),
# version_tag, version_branch, feature_branch, feature_tag (for pull request)
# tag, stabilization_branch, feature_branch, pull_request (for pull request merge: number)
version:
# List of tags we want to publish interpreted with `format(version=version)`
# e.g. if you use `{version}-lite` when you publish the version `1.2.3` the source tag
Expand Down Expand Up @@ -293,7 +296,7 @@ This will create a repository dispatch of type `published` on own repository wit
```json
{
"version": "1.2.3",
"version_type": "version_tag",
"version_type": "tag",
"repository": "camptocamp/tag-publish",
"items": [
{
Expand Down
33 changes: 17 additions & 16 deletions config.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -4,17 +4,18 @@ _Tag Publish configuration file (.github/publish.yaml)_

## Properties

- **`version`** _(object)_: The version configurations.
- **`branch_to_version_re`**: Refer to _[#/definitions/version_transform](#definitions/version_transform)_.
- **`tag_to_version_re`**: Refer to _[#/definitions/version_transform](#definitions/version_transform)_.
- **`transformers`** _(object)_: The version transform configurations. Default: `{"pull_request_to_version_re": [{"to": "pr-\\1"}]}`.
- **`branch_to_version`**: Refer to _[#/definitions/transform](#definitions/transform)_.
- **`tag_to_version`**: Refer to _[#/definitions/transform](#definitions/transform)_.
- **`pull_request_to_version`**: Refer to _[#/definitions/transform](#definitions/transform)_.
- **`docker`**: Refer to _[#/definitions/docker](#definitions/docker)_.
- **`pypi`**: Refer to _[#/definitions/pypi](#definitions/pypi)_.
- **`node`**: Refer to _[#/definitions/node](#definitions/node)_.
- **`helm`**: Refer to _[#/definitions/helm](#definitions/helm)_.
- **`dispatch`** _(array)_: Default: `[]`.
- **Items** _(object)_: Send a dispatch event to an other repository. Default: `{}`.
- **`repository`** _(string)_: The repository name to be triggered. Default: `"camptocamp/argocd-gs-gmf-apps"`.
- **`event-type`** _(string)_: The event type to be triggered. Default: `"published"`.
- **`event_type`** _(string)_: The event type to be triggered. Default: `"published"`.

## Definitions

Expand All @@ -26,41 +27,41 @@ _Tag Publish configuration file (.github/publish.yaml)_
- **`name`** _(string)_: The image name.
- **`tags`** _(array)_: The tag name, will be formatted with the version=<the version>, the image with version=latest should be present when we call the tag-publish script. Default: `["{version}"]`.
- **Items** _(string)_
- **`repository`** _(object)_: The repository where we should publish the images. Can contain additional properties. Default: `{"github": {"server": "ghcr.io", "versions": ["version_tag", "version_branch", "rebuild"]}}`.
- **`repository`** _(object)_: The repository where we should publish the images. Can contain additional properties. Default: `{"github": {"host": "ghcr.io", "versions_type": ["tag", "default_branch", "stabilization_branch", "rebuild"]}}`.
- **Additional properties** _(object)_
- **`server`** _(string)_: The server URL.
- **`versions`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["version_tag", "version_branch", "rebuild", "feature_branch"]`.
- **`host`** _(string)_: The host of the repository URL.
- **`versions_type`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["tag", "default_branch", "stabilization_branch", "rebuild", "feature_branch", "pull_request"]`.
- **Items** _(string)_
- **`auto_login`** _(boolean)_: Auto login to the GitHub Docker registry. Default: `false`.
- **`github_oidc_login`** _(boolean)_: Auto login to the GitHub Docker registry. Default: `true`.
- <a id="definitions/pypi"></a>**`pypi`** _(object)_: Configuration to publish on pypi.
- **`packages`** _(array)_: The configuration of packages that will be published.
- **Items** _(object)_: The configuration of package that will be published.
- **`group`** _(string)_: The image is in the group, should be used with the --group option of tag-publish script. Default: `"default"`.
- **`folder`** _(string)_: The folder of the pypi package. Default: `"."`.
- **`build_command`** _(array)_: The command used to do the build.
- **Items** _(string)_
- **`versions`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["version_tag"]`.
- **`versions_type`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["tag"]`.
- **Items** _(string)_
- <a id="definitions/node"></a>**`node`** _(object)_: Configuration to publish on node.
- **`packages`** _(array)_: The configuration of packages that will be published.
- **Items** _(object)_: The configuration of package that will be published.
- **`group`** _(string)_: The image is in the group, should be used with the --group option of tag-publish script. Default: `"default"`.
- **`folder`** _(string)_: The folder of the node package. Default: `"."`.
- **`versions`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["version_tag"]`.
- **`versions_type`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["tag"]`.
- **Items** _(string)_
- **`repository`** _(object)_: The packages repository where we should publish the packages. Can contain additional properties. Default: `{"github": {"server": "npm.pkg.github.com"}}`.
- **`repository`** _(object)_: The packages repository where we should publish the packages. Can contain additional properties. Default: `{"github": {"host": "npm.pkg.github.com"}}`.
- **Additional properties** _(object)_
- **`server`** _(string)_: The server URL.
- **`host`** _(string)_: The host of the repository URL.
- **`args`** _(array)_: The arguments to pass to the publish command. Default: `["--provenance"]`.
- **Items** _(string)_
- <a id="definitions/helm"></a>**`helm`** _(object)_: Configuration to publish Helm charts on GitHub release.
- **`packages`** _(array)_: The configuration of packages that will be published.
- **Items** _(object)_: The configuration of package that will be published.
- **`group`** _(string)_: The image is in the group, should be used with the --group option of tag-publish script. Default: `"default"`.
- **`folder`** _(string)_: The folder of the pypi package. Default: `"."`.
- **`versions`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["version_tag"]`.
- **`versions_type`** _(array)_: The kind or version that should be published, tag, branch or value of the --version argument of the tag-publish script. Default: `["tag"]`.
- **Items** _(string)_
- <a id="definitions/version_transform"></a>**`version_transform`** _(array)_: A version transformer definition.
- <a id="definitions/transform"></a>**`transform`** _(array)_: A version transformer definition. Default: `[]`.
- **Items** _(object)_
- **`from`** _(string)_: The from regular expression.
- **`to`** _(string)_: The expand regular expression: https://docs.python.org/3/library/re.html#re.Match.expand.
- **`from_re`** _(string)_: The from regular expression. Default: `"(.+)"`.
- **`to`** _(string)_: The expand regular expression: https://docs.python.org/3/library/re.html#re.Match.expand. Default: `"\\1"`.
6 changes: 4 additions & 2 deletions pyproject.toml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -93,8 +93,10 @@ enable = true
vcs = "git"
pattern = "^(?P<base>\\d+(\\.\\d+)*)"
format-jinja = """
{%- if env.get("VERSION_TYPE") == "version_branch" -%}
{{serialize_pep440(bump_version(base, 1 if env.get("IS_MASTER") == "TRUE" else 2), dev=distance)}}
{%- if env.get("VERSION_TYPE") == "default_branch" -%}
{{serialize_pep440(bump_version(base, 1), dev=distance)}}
{%- elif env.get("VERSION_TYPE") == "stabilization_branch" -%}
{{serialize_pep440(bump_version(base, 2), dev=distance)}}
{%- elif distance == 0 -%}
{{serialize_pep440(base)}}
{%- else -%}
Expand Down
115 changes: 20 additions & 95 deletions tag_publish/__init__.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
import applications_download
import github
import jsonschema_validator
import requests
import ruamel.yaml
import security_md
import yaml
Expand Down Expand Up @@ -59,17 +58,27 @@ def __init__(self) -> None:
self.default_branch = self.repo.default_branch


def get_security_md(gh: GH) -> security_md.Security:
def get_security_md(gh: GH, local: bool) -> security_md.Security:
"""
Get the SECURITY.md file.
Arguments:
gh: The GitHub helper
local: If we should use the local file
"""
if local:
if os.path.exists("SECURITY.md"):
print("Using the local SECURITY.md file")
with open("SECURITY.md", encoding="utf-8") as open_file:
return security_md.Security(open_file.read())
print("No local SECURITY.md file")
return security_md.Security("")

try:
security_file = gh.repo.get_contents("SECURITY.md")
assert isinstance(security_file, github.ContentFile.ContentFile)
print("Using SECURITY.md file from the default branch")
return security_md.Security(security_file.decoded_content.decode("utf-8"))
except github.GithubException as exception:
if exception.status == 404:
Expand All @@ -79,27 +88,7 @@ def get_security_md(gh: GH) -> security_md.Security:
raise exception


def merge(default_config: Any, config: Any) -> Any:
"""
Deep merge the dictionaries (on dictionaries only, not on arrays).
Arguments:
default_config: The default config that will be applied
config: The base config, will be modified
"""
if not isinstance(default_config, dict) or not isinstance(config, dict):
return config

for key in default_config:
if key not in config:
config[key] = default_config[key]
else:
merge(default_config[key], config[key])
return config


def get_config(gh: GH) -> tag_publish.configuration.Configuration:
def get_config() -> tag_publish.configuration.Configuration:
"""
Get the configuration, with project and auto detections.
"""
Expand All @@ -114,21 +103,6 @@ def get_config(gh: GH) -> tag_publish.configuration.Configuration:
config = yaml_.load(open_file)
jsonschema_validator.validate(".github/publish.yaml", cast(dict[str, Any], config), schema)

merge(
{
"version": {
"tag_to_version_re": [
{"from": r"([0-9]+.[0-9]+.[0-9]+)", "to": r"\1"},
],
"branch_to_version_re": [
{"from": r"([0-9]+.[0-9]+)", "to": r"\1"},
{"from": gh.default_branch, "to": gh.default_branch},
],
}
},
config,
)

return config


Expand All @@ -148,34 +122,32 @@ def get_value(matched: Optional[Match[str]], config: Optional[VersionTransform],
Return the value
"""
return matched.expand(config.get("to", r"\1")) if matched is not None and config is not None else value
return matched.expand(config["to"]) if matched is not None and config is not None else value


def compile_re(
config: tag_publish.configuration.VersionTransform, prefix: str = ""
) -> list[VersionTransform]:
def compile_re(config: tag_publish.configuration.Transform) -> list[VersionTransform]:
"""
Compile the from as a regular expression of a dictionary of the config list.
to be used with convert and match
Arguments:
config: The transform config
prefix: The version prefix
Return the compiled transform config.
"""
result = []
for conf in config:
new_conf = cast(VersionTransform, dict(conf))
new_conf = cast(
VersionTransform, {"to": conf.get("to", tag_publish.configuration.TRANSFORM_TO_DEFAULT)}
)

from_re = conf.get("from", r"(.*)")
if from_re[0] == "^":
from_re = from_re[1:]
from_re = conf.get("from_re", tag_publish.configuration.TRANSFORM_FROM_DEFAULT)
if from_re[0] != "^":
from_re = f"^{from_re}"
if from_re[-1] != "$":
from_re += "$"
from_re = f"^{re.escape(prefix)}{from_re}"

new_conf["from"] = re.compile(from_re)
result.append(new_conf)
Expand Down Expand Up @@ -204,53 +176,6 @@ def match(
return None, None, value


def does_match(value: str, config: list[VersionTransform]) -> bool:
"""
Check if the version match with the config patterns.
Arguments:
---------
value: That we want to match with
config: The result of `compile`
Returns True it it does match else False
"""
matched, _, _ = match(value, config)
return matched is not None


def check_response(response: requests.Response, raise_for_status: bool = True) -> Any:
"""
Check the response and raise an exception if it's not ok.
Also print the X-Ratelimit- headers to get information about the rate limiting.
"""
for header in response.headers:
if header.lower().startswith("x-ratelimit-"):
print(f"{header}: {response.headers[header]}")
if raise_for_status:
response.raise_for_status()


def add_authorization_header(headers: dict[str, str]) -> dict[str, str]:
"""
Add the Authorization header needed to be authenticated on GitHub.
Arguments:
headers: The headers
Return the headers (to be chained)
"""
try:
token = os.environ["GITHUB_TOKEN"].strip()
headers["Authorization"] = f"Bearer {token}"
return headers
except FileNotFoundError:
return headers


@overload
def download_application(application_name: str, binary_filename: str) -> str: ...

Expand Down
Loading

0 comments on commit bf4fbe9

Please sign in to comment.