Skip to content

Commit

Permalink
feat(vm): refine pipeline from jenkins to tekton
Browse files Browse the repository at this point in the history
  • Loading branch information
alomerry authored Dec 22, 2023
1 parent 1bbdac5 commit 09aba58
Show file tree
Hide file tree
Showing 13 changed files with 469 additions and 2 deletions.
26 changes: 26 additions & 0 deletions .github/workflows/main.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
name: sync to gitee

on:
push:
branches:
- master

jobs:
mirror:
env:
dst_key: ${{ secrets.GITEE_PRIVATE_KEY }}
dst_token: ${{ secrets.GITEE_TOKEN }}
runs-on: ubuntu-latest
timeout-minutes: 10

steps:
- name: Mirror the Github organization repos to Gitee.
uses: Yikun/hub-mirror-action@master
if: env.dst_key && env.dst_token
with:
src: github/alomerry
dst: gitee/alomerry
force_update: true
dst_key: ${{ secrets.GITEE_PRIVATE_KEY }}
dst_token: ${{ secrets.GITEE_TOKEN }}
static_list: mix
15 changes: 15 additions & 0 deletions vm/ansible/playbooks/roles/k8s-master/pv.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,15 @@
apiVersion: v1
kind: PersistentVolume
metadata:
name: tekton-pv
labels:
service: tekton
spec:
capacity:
storage: 15Gi
accessModes:
- ReadWriteOnce
nfs:
path: /mnt/nvme0n1p1/tekton
server: 192.168.31.2
persistentVolumeReclaimPolicy: Retain
4 changes: 3 additions & 1 deletion vm/ansible/playbooks/roles/vm-admin/files/nfs-server/exports
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,6 @@
# /srv/nfs4/homes gss/krb5i(rw,sync,no_subtree_check)
#
/mnt/sda1 192.168.31.0/255(rw,sync,no_root_squash,no_subtree_check)
/mnt/nvme1n1 192.168.31.0/255(rw,sync,no_root_squash,no_subtree_check)
/mnt/nvme0n1p1 192.168.31.0/255(rw,sync,no_root_squash,no_subtree_check)
# mkdir /mnt/nvme0n1p1
# mount 192.168.31.2:/mnt/nvme0n1p1 /mnt/nvme0n1p1/
5 changes: 4 additions & 1 deletion vm/ansible/playbooks/roles/vm-admin/tasks/nfs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,9 @@
mode: 0644
- name: 启动 nfs server
shell: |
exportfs -arv
systemctl stop nfs-kernel-server
sleep 1
systemctl start nfs-kernel-server
systemctl start nfs-kernel-server
# https://www.linuxprobe.com/ubuntu-configure-nfs.html
# https://access.redhat.com/documentation/zh-cn/openshift_container_platform/4.3/html/storage/persistent-storage-using-nfs
1 change: 1 addition & 0 deletions vm/ansible/playbooks/roles/vm-init/files/vm-init.sh
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ install_depandence() {
socat \
lsof \
qemu-guest-agent
# nfs
}

install_qemu_guest() {
Expand Down
4 changes: 4 additions & 0 deletions vm/k8s/tekton/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,10 @@
192.168.31.0/24 dev vmbr0 proto kernel scope link src 192.168.31.2
```

## 存储

- pve admin nfs

## reference

- https://www.lixueduan.com/posts/tekton/01-deploy-tekton/
Expand Down
Empty file.
242 changes: 242 additions & 0 deletions vm/k8s/tekton/pipeline/base-task/git-clone.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,242 @@
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: git-clone
labels:
app.kubernetes.io/version: "0.9"
annotations:
tekton.dev/pipelines.minVersion: "0.38.0"
tekton.dev/categories: Git
tekton.dev/tags: git
tekton.dev/displayName: "git clone"
tekton.dev/platforms: "linux/amd64,linux/s390x,linux/ppc64le,linux/arm64"
spec:
description: >-
These Tasks are Git tasks to work with repositories used by other tasks
in your Pipeline.
The git-clone Task will clone a repo from the provided url into the
output Workspace. By default the repo will be cloned into the root of
your Workspace. You can clone into a subdirectory by setting this Task's
subdirectory param. This Task also supports sparse checkouts. To perform
a sparse checkout, pass a list of comma separated directory patterns to
this Task's sparseCheckoutDirectories param.
workspaces:
- name: output
description: The git repo will be cloned onto the volume backing this Workspace.
- name: ssh-directory
optional: true
description: |
A .ssh directory with private key, known_hosts, config, etc. Copied to
the user's home before git commands are executed. Used to authenticate
with the git remote when performing the clone. Binding a Secret to this
Workspace is strongly recommended over other volume types.
- name: basic-auth
optional: true
description: |
A Workspace containing a .gitconfig and .git-credentials file. These
will be copied to the user's home before any git commands are run. Any
other files in this Workspace are ignored. It is strongly recommended
to use ssh-directory over basic-auth whenever possible and to bind a
Secret to this Workspace over other volume types.
- name: ssl-ca-directory
optional: true
description: |
A workspace containing CA certificates, this will be used by Git to
verify the peer with when fetching or pushing over HTTPS.
params:
- name: url
description: Repository URL to clone from.
type: string
- name: revision
description: Revision to checkout. (branch, tag, sha, ref, etc...)
type: string
default: ""
- name: refspec
description: Refspec to fetch before checking out revision.
default: ""
- name: submodules
description: Initialize and fetch git submodules.
type: string
default: "true"
- name: depth
description: Perform a shallow clone, fetching only the most recent N commits.
type: string
default: "1"
- name: sslVerify
description: Set the `http.sslVerify` global git config. Setting this to `false` is not advised unless you are sure that you trust your git remote.
type: string
default: "true"
- name: crtFileName
description: file name of mounted crt using ssl-ca-directory workspace. default value is ca-bundle.crt.
type: string
default: "ca-bundle.crt"
- name: subdirectory
description: Subdirectory inside the `output` Workspace to clone the repo into.
type: string
default: ""
- name: sparseCheckoutDirectories
description: Define the directory patterns to match or exclude when performing a sparse checkout.
type: string
default: ""
- name: deleteExisting
description: Clean out the contents of the destination directory if it already exists before cloning.
type: string
default: "true"
- name: httpProxy
description: HTTP proxy server for non-SSL requests.
type: string
default: ""
- name: httpsProxy
description: HTTPS proxy server for SSL requests.
type: string
default: ""
- name: noProxy
description: Opt out of proxying HTTP/HTTPS requests.
type: string
default: ""
- name: verbose
description: Log the commands that are executed during `git-clone`'s operation.
type: string
default: "true"
- name: gitInitImage
description: The image providing the git-init binary that this Task runs.
type: string
default: "gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/git-init:v0.40.2"
- name: userHome
description: |
Absolute path to the user's home directory.
type: string
default: "/home/git"
results:
- name: commit
description: The precise commit SHA that was fetched by this Task.
- name: url
description: The precise URL that was fetched by this Task.
- name: committer-date
description: The epoch timestamp of the commit that was fetched by this Task.
steps:
- name: clone
image: "$(params.gitInitImage)"
env:
- name: HOME
value: "$(params.userHome)"
- name: PARAM_URL
value: $(params.url)
- name: PARAM_REVISION
value: $(params.revision)
- name: PARAM_REFSPEC
value: $(params.refspec)
- name: PARAM_SUBMODULES
value: $(params.submodules)
- name: PARAM_DEPTH
value: $(params.depth)
- name: PARAM_SSL_VERIFY
value: $(params.sslVerify)
- name: PARAM_CRT_FILENAME
value: $(params.crtFileName)
- name: PARAM_SUBDIRECTORY
value: $(params.subdirectory)
- name: PARAM_DELETE_EXISTING
value: $(params.deleteExisting)
- name: PARAM_HTTP_PROXY
value: $(params.httpProxy)
- name: PARAM_HTTPS_PROXY
value: $(params.httpsProxy)
- name: PARAM_NO_PROXY
value: $(params.noProxy)
- name: PARAM_VERBOSE
value: $(params.verbose)
- name: PARAM_SPARSE_CHECKOUT_DIRECTORIES
value: $(params.sparseCheckoutDirectories)
- name: PARAM_USER_HOME
value: $(params.userHome)
- name: WORKSPACE_OUTPUT_PATH
value: $(workspaces.output.path)
- name: WORKSPACE_SSH_DIRECTORY_BOUND
value: $(workspaces.ssh-directory.bound)
- name: WORKSPACE_SSH_DIRECTORY_PATH
value: $(workspaces.ssh-directory.path)
- name: WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND
value: $(workspaces.basic-auth.bound)
- name: WORKSPACE_BASIC_AUTH_DIRECTORY_PATH
value: $(workspaces.basic-auth.path)
- name: WORKSPACE_SSL_CA_DIRECTORY_BOUND
value: $(workspaces.ssl-ca-directory.bound)
- name: WORKSPACE_SSL_CA_DIRECTORY_PATH
value: $(workspaces.ssl-ca-directory.path)
securityContext:
runAsNonRoot: true
runAsUser: 65532
script: |
#!/usr/bin/env sh
set -eu
if [ "${PARAM_VERBOSE}" = "true" ] ; then
set -x
fi
if [ "${WORKSPACE_BASIC_AUTH_DIRECTORY_BOUND}" = "true" ] ; then
cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.git-credentials" "${PARAM_USER_HOME}/.git-credentials"
cp "${WORKSPACE_BASIC_AUTH_DIRECTORY_PATH}/.gitconfig" "${PARAM_USER_HOME}/.gitconfig"
chmod 400 "${PARAM_USER_HOME}/.git-credentials"
chmod 400 "${PARAM_USER_HOME}/.gitconfig"
fi
if [ "${WORKSPACE_SSH_DIRECTORY_BOUND}" = "true" ] ; then
cp -R "${WORKSPACE_SSH_DIRECTORY_PATH}" "${PARAM_USER_HOME}"/.ssh
chmod 700 "${PARAM_USER_HOME}"/.ssh
chmod -R 400 "${PARAM_USER_HOME}"/.ssh/*
fi
if [ "${WORKSPACE_SSL_CA_DIRECTORY_BOUND}" = "true" ] ; then
export GIT_SSL_CAPATH="${WORKSPACE_SSL_CA_DIRECTORY_PATH}"
if [ "${PARAM_CRT_FILENAME}" != "" ] ; then
export GIT_SSL_CAINFO="${WORKSPACE_SSL_CA_DIRECTORY_PATH}/${PARAM_CRT_FILENAME}"
fi
fi
CHECKOUT_DIR="${WORKSPACE_OUTPUT_PATH}/${PARAM_SUBDIRECTORY}"
cleandir() {
# Delete any existing contents of the repo directory if it exists.
#
# We don't just "rm -rf ${CHECKOUT_DIR}" because ${CHECKOUT_DIR} might be "/"
# or the root of a mounted volume.
if [ -d "${CHECKOUT_DIR}" ] ; then
# Delete non-hidden files and directories
rm -rf "${CHECKOUT_DIR:?}"/*
# Delete files and directories starting with . but excluding ..
rm -rf "${CHECKOUT_DIR}"/.[!.]*
# Delete files and directories starting with .. plus any other character
rm -rf "${CHECKOUT_DIR}"/..?*
fi
}
if [ "${PARAM_DELETE_EXISTING}" = "true" ] ; then
cleandir || true
fi
test -z "${PARAM_HTTP_PROXY}" || export HTTP_PROXY="${PARAM_HTTP_PROXY}"
test -z "${PARAM_HTTPS_PROXY}" || export HTTPS_PROXY="${PARAM_HTTPS_PROXY}"
test -z "${PARAM_NO_PROXY}" || export NO_PROXY="${PARAM_NO_PROXY}"
git config --global --add safe.directory "${WORKSPACE_OUTPUT_PATH}"
/ko-app/git-init \
-url="${PARAM_URL}" \
-revision="${PARAM_REVISION}" \
-refspec="${PARAM_REFSPEC}" \
-path="${CHECKOUT_DIR}" \
-sslVerify="${PARAM_SSL_VERIFY}" \
-submodules="${PARAM_SUBMODULES}" \
-depth="${PARAM_DEPTH}" \
-sparseCheckoutDirectories="${PARAM_SPARSE_CHECKOUT_DIRECTORIES}"
cd "${CHECKOUT_DIR}"
RESULT_SHA="$(git rev-parse HEAD)"
EXIT_CODE="$?"
if [ "${EXIT_CODE}" != 0 ] ; then
exit "${EXIT_CODE}"
fi
RESULT_COMMITTER_DATE="$(git log -1 --pretty=%ct)"
printf "%s" "${RESULT_COMMITTER_DATE}" > "$(results.committer-date.path)"
printf "%s" "${RESULT_SHA}" > "$(results.commit.path)"
printf "%s" "${PARAM_URL}" > "$(results.url.path)"
57 changes: 57 additions & 0 deletions vm/k8s/tekton/pipeline/base-task/rsync.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,57 @@
---
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: rsync
labels:
app.kubernetes.io/version: "0.1"
annotations:
tekton.dev/pipelines.minVersion: "0.17.0"
tekton.dev/tags: rsync, files sync
tekton.dev/categories: CLI
tekton.dev/displayName: "rsync"
tekton.dev/platforms: "linux/amd64,linux/s390x,linux/ppc64le,linux/arm64"
spec:
description: >-
This task can be used to synchronize local and remote files.
workspaces:
- name: source
description: The directory you want to sync.
- name: ssh-directory
optional: true
description: ssh directory with private key, known_hosts, config.
params:
- name: direction
type: string
default: "local-to-remote"
- name: local-path
type: string
default: "./"
- name: remote-path
type: string
default: "~/"
- name: remote-ip
type: string
default: "0.0.0.0"
- name: remote-username
type: string
default: "root"
steps:
- name: rsync
image: docker.io/shaowenchen/rsync-sshpass:v1@sha256:934985f9a955f416c7378e22d847ac1320fcae1495ee70ad53c9f9fc08b99323
env:
- name: SSH_DIRECTORY_PATH
value: $(workspaces.ssh-directory.path)
workingDir: $(workspaces.source.path)
script: |
cp -R "${SSH_DIRECTORY_PATH}" "$HOME"/.ssh
chmod 700 "$HOME"/.ssh
chmod -R 400 "$HOME"/.ssh/*
if [ "$(params.direction)" = "remote-to-local" ] ; then
echo "remote to local"
rsync -ratlzv -e "ssh -o StrictHostKeyChecking=no -i $HOME/.ssh/id_rsa" "$(params.remote-username)"@"$(params.remote-ip)":"$(params.remote-path)"/ "$(params.local-path)"
else
echo "local to remote"
rsync -ratlzv -e "ssh -o StrictHostKeyChecking=no -i $HOME/.ssh/id_rsa" "$(params.local-path)" "$(params.remote-username)"@"$(params.remote-ip)":"$(params.remote-path)"/
fi
Loading

0 comments on commit 09aba58

Please sign in to comment.