-
Notifications
You must be signed in to change notification settings - Fork 67
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #131 from soyowind0/main
feat: add mapper of windows execution virtual device
- Loading branch information
Showing
22 changed files
with
1,481 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,33 @@ | ||
| SHELL := /bin/bash | ||
|
|
||
| curr_dir := $(patsubst %/,%,$(dir $(abspath $(lastword $(MAKEFILE_LIST))))) | ||
| rest_args := $(wordlist 2, $(words $(MAKECMDGOALS)), $(MAKECMDGOALS)) | ||
| $(eval $(rest_args):;@:) | ||
|
|
||
| help: | ||
| # | ||
| # Usage: | ||
| # make generate : generate a mapper based on a template. | ||
| # make mapper {mapper-name} <action> <parameter>: execute mapper building process. | ||
| # | ||
| # Actions: | ||
| # - mod, m : download code dependencies. | ||
| # - lint, l : verify code via go fmt and `golangci-lint`. | ||
| # - build, b : compile code. | ||
| # - package, p : package docker image. | ||
| # - clean, c : clean output binary. | ||
| # | ||
| # Parameters: | ||
| # ARM : true or undefined | ||
| # ARM64 : true or undefined | ||
| # | ||
| # Example: | ||
| # - make mapper windows-virtual-exec : execute `build` "windows-virtual-exec" mapper. | ||
| @echo | ||
|
|
||
| make_rules := $(shell ls $(curr_dir)/hack/make-rules | sed 's/.sh//g') | ||
| $(make_rules): | ||
| @$(curr_dir)/hack/make-rules/$@.sh $(rest_args) | ||
|
|
||
| .DEFAULT_GOAL := help | ||
| .PHONY: $(make_rules) build test package |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,114 @@ | ||
| # Windows Execution Virtual Device | ||
|
|
||
| ## Overall design | ||
|
|
||
| Overall, based on the modularity of Kubeedge, this project utilizes the independent data communication capability provided by DeviceTwin module to expand the usage scenarios of Kubeedge and adapt the problems encountered by the actual landing of Kubeedge in the automotive industry: | ||
|
|
||
| 1. In order to explore the possibility of Kubeedge cloud-native solution in the automotive industry, Kubeedge is introduced for multi-region- and multi-state edge node management, so that automotive software testing can be automated and the testing environment can go to the cloud. | ||
|
|
||
| 2. With the increase of automotive testing and simulation needs, some traditional industrial software relies on native Windows platform, even though Kubeedge has provided support for Windows Server with container capability, upgrading the original Windows Enterprise Edition machine to Windows Server is unacceptable to the staff and cost department. | ||
|
|
||
| 3. Automated testing uses third-party simulation and debugging software based on the Windows platform. In order to analyze and model the test data, it is necessary to upload the test process and result data from the test host computer to the cloud. | ||
|
|
||
| For this reason, the following design is made: | ||
|
|
||
|  | ||
|
|
||
| The overall design is shown in the figure, the software test host, regardless of Windows or Linux, is unified and managed by Kubeedge as a node. Users use K8s API to issue commands to operate it and create corresponding test tasks, and the DeviceTwin module of Kubeedge provides additional data communication capability to realize the triggering and reporting of non-container environment Windows nodes. | ||
|
|
||
| ## CRDs | ||
|
|
||
| Non-containerized Windows environments execute automotive testing tasks that rely on third-party closed-source tools in a bare metal form, where the environment is relatively static and the tasks performed are variable. Therefore, we believe that Windows nodes need a unified executor to execute test tasks (scripts) issued by the cloud, scheduling, running commands and returning data to the cloud. | ||
|
|
||
| In this project, automotive software testing has the following characteristics: | ||
|
|
||
| 1. The execution environment is decentralized and fixed, and the automotive software runs on a collection of hardware called a “pedestal”, which is physically fixed and configured with a PC to access it as a “host computer”. The dais and the host computer are dispersed across the country and around the world, depending on the location of the business unit. | ||
| 2. Test tasks are dynamically plugged in and out, and their operational status needs to be monitored to prevent the desktop from failing. 3. | ||
| 3. During the execution of tasks, test logs are constantly generated and need to be aggregated and analyzed in the cloud. | ||
|
|
||
| Kubeedge has designed DeviceTwin to support device and data management, and defined CRD resources for Device and DeviceModel. CRD declarative yaml design is simple for users to understand, easy to operate, and can be promoted to testers; Kubeedge CloudCore has implemented DeviceController, a controller for Device resource objects. Kubeedge CloudCore has already implemented DeviceController for Device resource.Considering a test task as a virtual device, according to the design of Kubeedge Device and the requirements of automotive software testing, the Device and DeviceModel are defined as follows: | ||
|
|
||
| ### Device Model | ||
|
|
||
| ```yaml | ||
| apiVersion: devices.kubeedge.io/v1alpha2 | ||
| kind: DeviceModel | ||
| metadata: | ||
| name: win-exec-model | ||
| namespace: default | ||
| spec: | ||
| properties: | ||
| - name: exec-file-content | ||
| description: custom content to execute | ||
| type: | ||
| string: | ||
| accessMode: ReadOnly | ||
| - name: exec-file-name | ||
| description: save custom content as filename | ||
| type: | ||
| string: | ||
| accessMode: ReadOnly | ||
| - name: exec-command | ||
| description: entrypoint of target | ||
| type: | ||
| string: | ||
| accessMode: ReadOnly | ||
| - name: status | ||
| description: status of current executation | ||
| type: | ||
| string: | ||
| accessMode: ReadWrite | ||
| - name: output | ||
| description: console output of current executation | ||
| type: | ||
| string: | ||
| accessMode: ReadWrite | ||
| ``` | ||
| ### Device Instance | ||
| ```yaml | ||
| apiVersion: devices.kubeedge.io/v1alpha2 | ||
| kind: Device | ||
| metadata: | ||
| name: exec-instance-001 | ||
| labels: | ||
| description: "test" | ||
| model: win-exec-model | ||
| spec: | ||
| deviceModelRef: | ||
| name: win-exec-model | ||
| nodeSelector: | ||
| nodeSelectorTerms: | ||
| - matchExpressions: | ||
| - key: '' | ||
| operator: In | ||
| values: | ||
| - win11-node | ||
| protocol: | ||
| customizedProtocal: | ||
| protocolName: winExec | ||
| status: | ||
| twins: | ||
| - propertyName: status | ||
| - propertyName: output | ||
| - propertyName: exec-file-content | ||
| desired: | ||
| value: 'echo "hello,world"' | ||
| - propertyName: exec-file-name | ||
| desired: | ||
| value: 'run.bat' | ||
| - propertyName: exec-command | ||
| desired: | ||
| value: 'run.bat' | ||
| ``` | ||
| The Device Instance is defined based on the Device Model and instantiates the contents of exec-file-content, exec-file-name and exec-command. The status and output fields are reflected in the actual field after the execution is completed, and the result is reported to the cloud as the test task script. The result of the execution is reported to the cloud. | ||
| ## Mapper | ||
| According to the docs/proposals/device-crd.md document, the lifecycle of an IoT device consists of six parts: registration, configuration, upgrade, monitoring, logout, and destruction. Among them, registration, upgrade, logout, and destruction are not considered in the device-crd.md document. Therefore, device is designed for device configuration and monitoring. Configuration is designed to reconfigure the device multiple times without adding new functionality, setting the expected expectation in the CRD, i.e., a declarative configuration of the behavior that the device should have. Monitoring is designed to constantly update the state of the device so that the cloud can be informed of the state of the device in time for the next step. | ||
| Considering the test task as an abstract device, configuring the device means setting the parameters of the test task; monitoring the status of the device means collecting and monitoring the execution logs of the test task. | ||
| The execution of test tasks and reporting of results uses only a small portion of the mapper framework, which supports device creation, query and deletion, and does not require the introduction of complete framework logic. The operations are triggered as callback functions, controlled by the cloud, and in addition to the cloud commands, messages are actively pushed to the local MQTT to trigger the next operation. mapper loads the local database into memory at startup, and then triggers the execution of the task and the reporting of the status based on the type of message. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,64 @@ | ||
| /* | ||
| Copyright 2024 The KubeEdge Authors. | ||
| Licensed under the Apache License, Version 2.0 (the "License"); | ||
| you may not use this file except in compliance with the License. | ||
| You may obtain a copy of the License at | ||
| http://www.apache.org/licenses/LICENSE-2.0 | ||
| Unless required by applicable law or agreed to in writing, software | ||
| distributed under the License is distributed on an "AS IS" BASIS, | ||
| WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| See the License for the specific language governing permissions and | ||
| limitations under the License. | ||
| */ | ||
|
|
||
| package main | ||
|
|
||
| import ( | ||
| "os" | ||
| "os/signal" | ||
|
|
||
| klog "k8s.io/klog/v2" | ||
|
|
||
| "github.com/kubeedge/mappers-go/mappers/windows-virtual-exec/internal/config" | ||
| "github.com/kubeedge/mappers-go/mappers/windows-virtual-exec/internal/core/model" | ||
| "github.com/kubeedge/mappers-go/mappers/windows-virtual-exec/internal/core/mqtt" | ||
| "github.com/kubeedge/mappers-go/mappers/windows-virtual-exec/internal/core/store" | ||
| "github.com/kubeedge/mappers-go/mappers/windows-virtual-exec/internal/missions" | ||
| ) | ||
|
|
||
| func main() { | ||
| var err error | ||
| var c config.Config | ||
|
|
||
| klog.InitFlags(nil) | ||
| defer klog.Flush() | ||
|
|
||
| if err = c.Parse(); err != nil { | ||
| klog.Fatal(err) | ||
| } | ||
|
|
||
| store.InitDB("internal.db") | ||
| if err := store.DB.AutoMigrate(&model.Mission{}); err != nil { | ||
| klog.Errorf("Failed to init db: %v", err) | ||
| } | ||
|
|
||
| if err := mqtt.InitClient( | ||
| c.Mqtt.ServerAddress, | ||
| c.Mqtt.Username, | ||
| c.Mqtt.Password, | ||
| c.Mqtt.Cert, | ||
| c.Mqtt.PrivateKey, | ||
| ); err != nil { | ||
| klog.Fatal(err) | ||
| } | ||
|
|
||
| missions.InitCallback(c.NodeName) | ||
| klog.Info("Start to subscribe") | ||
| missions.InitMissions(c.NodeName) | ||
|
|
||
| // waiting kill signal | ||
| var ch = make(chan os.Signal, 1) | ||
| signal.Notify(ch, os.Interrupt) | ||
| <-ch | ||
| klog.Info("Exit") | ||
| } |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,125 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -o errexit | ||
| set -o nounset | ||
| set -o pipefail | ||
|
|
||
| CURR_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd -P)" | ||
| ROOT_DIR="$(cd "${CURR_DIR}/../.." && pwd -P)" | ||
| source "${ROOT_DIR}/hack/lib/init.sh" | ||
|
|
||
| mkdir -p "${CURR_DIR}/bin" | ||
| mkdir -p "${CURR_DIR}/dist" | ||
|
|
||
| function mod() { | ||
| [[ "${2:-}" != "only" ]] | ||
| local mapper="${1}" | ||
|
|
||
| # the mapper is sharing the vendor with root | ||
| pushd "${ROOT_DIR}" >/dev/null || exist 1 | ||
| echo "downloading dependencies for mapper ${mapper}..." | ||
|
|
||
| if [[ "$(go env GO111MODULE)" == "off" ]]; then | ||
| echo "go mod has been disabled by GO111MODULE=off" | ||
| else | ||
| echo "tidying" | ||
| go mod tidy | ||
| echo "vending" | ||
| go mod vendor | ||
| fi | ||
|
|
||
| echo "...done" | ||
| popd >/dev/null || return | ||
| } | ||
|
|
||
| function lint() { | ||
| [[ "${2:-}" != "only" ]] && mod "$@" | ||
| local mapper="${1}" | ||
|
|
||
| echo "fmt and linting mapper ${mapper}..." | ||
|
|
||
| gofmt -s -w "${CURR_DIR}/" | ||
| golangci-lint run "${CURR_DIR}/..." | ||
|
|
||
| echo "...done" | ||
| } | ||
|
|
||
| function build() { | ||
| [[ "${2:-}" != "only" ]] && lint "$@" | ||
| local mapper="${1}" | ||
|
|
||
| local flags=" -w -s " | ||
| local ext_flags=" -extldflags '-static' " | ||
| local os="${OS:-$(go env GOOS)}" | ||
| local arch="${ARCH:-$(go env GOARCH)}" | ||
|
|
||
| local platform | ||
| if [[ "${ARM:-false}" == "true" ]]; then | ||
| echo "crossed packaging for windows/arm" | ||
| platform=("windows/arm") | ||
| elif [[ "${ARM64:-false}" == "true" ]]; then | ||
| echo "crossed packaging for windows/arm64" | ||
| platform=("windows/arm64") | ||
| else | ||
| local os="windows" | ||
| local arch="${ARCH:-$(go env GOARCH)}" | ||
| platform=("${os}/${arch}") | ||
| fi | ||
|
|
||
| echo "building ${platform}" | ||
|
|
||
| local os_arch | ||
| IFS="/" read -r -a os_arch <<<"${platform}" | ||
| local os=${os_arch[0]} | ||
| local arch=${os_arch[1]} | ||
| GOOS=${os} GOARCH=${arch} CGO_ENABLED=0 go build \ | ||
| -ldflags "${flags} ${ext_flags}" \ | ||
| -o "${CURR_DIR}/bin/${mapper}_${os}_${arch}.exe" \ | ||
| "${CURR_DIR}/cmd/main.go" | ||
|
|
||
| echo "...done" | ||
| } | ||
|
|
||
| function package() { | ||
| echo "docker package not support for windows virtual exec driver" | ||
| } | ||
|
|
||
| function clean() { | ||
| local mapper="${1}" | ||
|
|
||
| echo "cleanup mapper ${mapper}..." | ||
|
|
||
| rm -rf "${CURR_DIR}/bin/*" | ||
|
|
||
| echo "...done" | ||
| } | ||
|
|
||
| function entry() { | ||
| local mapper="${1:-}" | ||
| shift 1 | ||
|
|
||
| local stages="${1:-build}" | ||
| shift $(($# > 0 ? 1 : 0)) | ||
|
|
||
| IFS="," read -r -a stages <<<"${stages}" | ||
| local commands=$* | ||
| if [[ ${#stages[@]} -ne 1 ]]; then | ||
| commands="only" | ||
| fi | ||
|
|
||
| for stage in "${stages[@]}"; do | ||
| echo "# make mapper ${mapper} ${stage} ${commands}" | ||
| case ${stage} in | ||
| m | mod) mod "${mapper}" "${commands}" ;; | ||
| l | lint) lint "${mapper}" "${commands}" ;; | ||
| b | build) build "${mapper}" "${commands}" ;; | ||
| p | pkg | package) package "${mapper}" "${commands}" ;; | ||
| t | test) test "${mapper}" "${commands}" ;; | ||
| c | clean) clean "${mapper}" "${commands}" ;; | ||
| *) echo "unknown action '${stage}', select from mod,lint,build,test,clean" ;; | ||
| esac | ||
| done | ||
| } | ||
|
|
||
| echo $@ | ||
| entry "$@" |
Oops, something went wrong.