Guest test framework initial commit

guest-test/README.md

@@ -0,0 +1,119 @@
+# KVM/QEMU based Guest VM auto test framework
+
+## Description
+A simple auto guest VM test framework for several types of VM launched by KVM/QEMU.
+Types of VM supported includes: legacy, tdx, tdxio,
+may extend in future for specific type of VMs,
+in fact, the main differences among above VM types are QEMU config parameters.
+
+As QEMU config parameters may vary from version to version,
+current implementation is QEMU version 7.2.0 based with tdx, tdxio special feature support.
+
+The qemu.config.json and common test framework code need to change along with new QEMU update
+
+## Limitaion
+Each test execution will launch a new VM and run specific test scripts/binaries in guest VM,
+test log will be captured from QEMU launching VM to test scripts/binaries execution in guest VM,
+untill VM launched being shutdown properly or pkilled on purpose in abnormal test status.
+
+No multi-VMs test scenarios covered/supported.
+
+In any case of issue debugging, please refer to above test log with VM QEMU config info
+and launch VM and debug issues manually running test scripts/binaries.
+
+A prepared Guest OS Image (qcow2 or raw image format) is requried with preset root account and password,
+several values/parameters in qemu.config.json highly depend on Guest OS Image, please accomodate accordingly.
+
+## Usage
+### qemu.config.json description
+with QEMU emulator version 7.2 support, qemu.config.json parameters are fully aligned to it,
+and grouped in 4 by 1st-level-keys: "common", "vm", "tdx", "tdxio"
+
+group "common" includes all configurable values to be passed to group "vm", "tdx", "tdxio"
+2nd-level-keys info:
+"kernel_img": [mandatory] /abs/path/to/vmlinuz file or bzImage file of target VM guest kernel
+"initrd_img": [optional] /abs/path/to/initrd file or initramfs file of target VM guest kernel
+"bios_img": [legacy vm optional, tdx/tdxio vm mandatory] /abs/path/to/ovmf file or other bios file of target VM guest bios
+"qemu_img": [mandatory] /abs/path/to/qemu-kvm or qemu-system-x86_64 file to boot target VM guest
+"guest_img": [mandatory] /abs/path/to/VM guest OS image with qcow2 format or raw image format
+"guest_img_format": [mandatory] value range in [qcow2/raw], guest os image file type qcow2 or raw image
+"boot_pattern": [mandatory] Guest OS booting pattern shows bootup completed, depends on Guest OS image provided
+"guest_root_passwd": [mandatory] Guest OS root account password
+"vm_type": [mandatory] value range in [legacy/tdx/tdxio], VM type to test includes legacy vm, tdx vm or tdxio vm
+"pmu": [mandatory] value range in [on/off], qemu config -cpu pmu=on or -cpu pmu=off
+"cpus": [mandatory] value range in [1 ~ maximum vcpu number], qemu config -smp cpus=$VCPU
+"sockets": [mandatory] value range in [1 ~ maximum sockets number], qemu config -smp sockets=$SOCKETS
+"mem": [mandatory] value range in [1 ~ maximum mem size in GB], qemu config -m memory size in GB
+"cmdline": [optional] value range in [guest kernel paramter extra string], qemu config -append extra command line parameters to pass to VM guest kernel
+"debug": [mandatory] value range in [on/off], qemu config -object tdx-guest,debug=on or -object tdx-guest,debug=off
+
+group "vm" includes all legacy vm launch qemu config options, which can be used to launch legacy vm standalone or as base part to launch tdx vm
+group "vm" 2nd-level-keys could be bypassed if not provided (file not exists)
+"cfg_var_6" & "cfg_var_10"
+
+group "tdx" includes tdx vm specific qemu config options, which is used to launch tdx vm (with group "vm" as base) or as a based part to launch tdxio vm
+
+group "tdxio" includes tdxio vm specific qemu config options, which is used to launch tdxio vm (with group "vm" + "tdx" as base)
+
+note about qemu.config.json:
+- no changes allowed on 1st-level-keys hierarchy
+- cfg_x part is allowed to extend freely based on needs
+- cfg_var_x part is allowed to revised/extended too, please remember to keep changes specifically aligned in qemu_get_config.py
+
+### guest.test_launcher description
+main test entrance, with following key args can be passed to override the values in qemu.config.json
+  -v $VCPU number of vcpus
+  -s $SOCKETS number of sockets
+  -m $MEM memory size in GB
+  -d $DEBUG debug on/off
+  -t $VM_TYPE vm_type legacy/tdx/tdxio
+  -x $TESTCASE testcase pass to test_executor
+  -c $CMDLINE guest kernel extra commandline
+  -p $PMU guest pmu off/on
+  -g $GCOV code coverage test mode off/on
+
+above key args will be recorded in a fresh new test_params.py for further import/source purpose accross scripts
+
+by enter each test, qemu_get_config.py will be called to get following pre-set parameters from qemu.config.json
+  $KERNEL_IMG values passed by group "common" key "kernel_img"
+  $INITRD_IMG values passed by group "common" key "initrd_img"
+  $BIOS_IMG values passed by group "common" key "bios_img"
+  $QEMU_IMG values passed by group "common" key "qemu_img"
+  $GUEST_IMG values passed by group "common" key "guest_img"
+  $GUEST_IMG_FORMAT values passed by group "common" key "guest_img_format"
+  $BOOT_PATTERN values passed by group "common" key "boot_pattern"
+  $SSHPASS values passed by group "common" key "guest_root_passwd"
+
+call guest.qemu_runner.sh and wait for $BOOT_PATTERN (shows VM boot up completed and ready for login) during VM boot
+  $BOOT_PATTERN selected based on following CentOS Stream 8/9 boot log example: "*Kernel*on*x86_64*"
+  boot log quoted:
+    CentOS Stream 9
+    Kernel 6.5.0-rc5-next-20230809-next-20230809 on an x86_64
+    Activate the web console with: systemctl enable --now cockpit.socket
+    CentOS-9 login:
+
+if $BOOT_PATTERN found, call guest.test_executor.sh with proper $TESTCASE to be executed in VM Guest, and shutdown VM after test compelted
+if $BOOT_PATTERN not found, several VM life-cycles management logic applied to handle boot failure in different stages
+if $ERR_STRx found, handle the error info accordingly (err_handlers)
+no matter what, in the end, pkill VM process to avoid any potential test step failures above
+
+Note: bydeault, $GCOV is off, if $GCOV is on, above VM life-cycles management logic will be bypassed to keep VM process alive for gcov code coverage data collection
+
+### guest.qemu_runner description
+VM boot engine, with parames exported from qemu_get_config.py and test scenario config sourced from test_params.py
+
+before VM boot, for $VM_TYPE tdx or tdxio, tdx_pre_check will be called to make sure basic environment is ready for TDX/TDXIO launching
+VM boot is triggered by qemu_runner.py based on $VM_TYPE, with proper qemu config options applied
+
+### guest.test_executor description
+guest VM test execution basic framework implemented in guerst.test_executor.sh, such as
+  guest_test_prepare, function based on sshpass to scp common.sh and test_script.sh to Guest VM
+  guest_test_source_code, function based on sshpass to scp source_code_dir and compile test_binary in Guest VM
+  guest_test_entry, function based on sshpass to execute test_script.sh and potential script params in Guest VM
+  guest_test_close, function based on sshpass to close VM
+
+## How to add new feature test
+as described above, if simply add new TCs to run based on current qemu.config.json format, just need to implement it in test_executor with new $TESTCASE branch,
+common functions of test_executor should be good enough to prepare/run/close new $TESTCASE
+
+if qemu.config.json format will be revised due to feature changes on QEMU implementation, please update qemu.config.json and qemu_get_config.py accordingly

guest-test/guest.qemu_runner.sh

@@ -0,0 +1,88 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (c) 2023 Intel Corporation
+
+# Author:   Hongyu Ning <[email protected]>
+#
+# History:  24, Aug., 2023 - Hongyu Ning - creation
+
+
+# @desc This script boots VM thru $QEMU_IMG (called by qemu_runner.py)
+# @ params source 1: general params exported from qemu_get_config.py
+# @ params source 2: test scenario config sourced from test_params.py
+
+###################### Variables ######################
+SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
+echo "$SCRIPT_DIR"
+
+###################### Functions ######################
+
+# function to do TDX/TDXIO VM launching basic pre-check
+# list all the variables value
+tdx_pre_check() {
+  if [[ ! -f $KERNEL_IMG ]]; then
+    test_print_wrg "In qemu.config.json file, need to set guest kernel properly"
+    die "TDX guest kernel does not exist..."
+  else
+    test_print_trc "TDX guest kernel to test: $KERNEL_IMG"
+  fi
+
+  if [[ ! -f $BIOS_IMG ]]; then
+    test_print_wrg "In qemu.config.json file, need to set ovmf properly"
+    die "Virtual BIOS does not exist..."
+  else
+    test_print_trc "BIOS to test: $BIOS_IMG"
+  fi
+
+  if [[ ! -f $QEMU_IMG ]]; then
+    test_print_wrg "In qemu.config.json file, need to set qemu properly"
+    die "QEMU does not exist..."
+  else
+    test_print_trc "QEMU to test: $QEMU_IMG"
+  fi
+
+  if [[ ! -f $GUEST_IMG ]]; then
+    test_print_wrg "In qemu.config.json file, need to set guest OS img properly"
+    die "Guest OS does not exist..."
+  else
+    test_print_trc "Guest OS image to test: $GUEST_IMG"
+  fi
+
+  if [[ $GUEST_IMG_FORMAT = "qcow2" ]]; then
+    test_print_trc "Guest OS image format: qcow2"
+  else
+    test_print_trc "Guest OS image format: raw"
+  fi
+
+  test_print_trc "Guest OS root password: $SSHPASS"
+  test_print_trc "TDX guest config: vcpu $VCPU, socket $SOCKETS, memory ${MEM}GB"
+  test_print_trc "TDX guest extra config: debug $DEBUG, extra commandline: $CMDLINE"
+  test_print_trc "TDX guest ssh forward port: $PORT"
+
+  TDX_SYSFS_FILE="/sys/module/kvm_intel/parameters/tdx"
+  if [[ -f "$TDX_SYSFS_FILE" ]]; then
+    if [ "Y" != "$(cat $TDX_SYSFS_FILE)" ] ;then
+      die "TDX not enabled as expected, please check"
+    else
+      test_print_trc "TDX enabled, try to launch TD VM now......"
+    fi
+  else
+     die "kvm_intel module tdx params does not exist, plase check"
+  fi
+}
+
+###################### Do Works ######################
+cd "$(dirname "$0")" 2>/dev/null || exit 1
+source ../.env
+
+# get test scenario config for qemu_runner
+source $SCRIPT_DIR/test_params.py
+
+# do basic pre-check for TDX/TDXIO VM launching
+if [[ $VM_TYPE == "tdx" ]] || [[ $VM_TYPE == "tdxio" ]]; then
+  tdx_pre_check
+fi
+
+# launch VM by qemu via qemu_runner.py
+test_print_trc "qemu_runner start to launch $VM_TYPE VM"
+python3 $SCRIPT_DIR/qemu_runner.py

guest-test/guest.test_executor.sh

@@ -0,0 +1,111 @@
+#!/bin/bash
+# SPDX-License-Identifier: GPL-2.0-only
+# Copyright (c) 2023 Intel Corporation
+
+# Author:   Hongyu Ning <[email protected]>
+#
+# History:  24, Aug., 2023 - Hongyu Ning - creation
+
+
+# @desc This script prepare and run $TESTCASE in Guest VM
+
+###################### Variables ######################
+SCRIPT_DIR="$( cd "$( dirname "$0" )" && pwd )"
+echo "$SCRIPT_DIR"
+GUEST_TEST_DIR="/root/guest_test/"
+
+###################### Functions ######################
+
+# function based on sshpass to scp common.sh and $1 test_script.sh to Guest VM
+guest_test_prepare() {
+  rm -rf common.sh
+  wget https://raw.githubusercontent.com/intel/lkvs/main/common/common.sh
+  sshpass -e ssh -p "$PORT" -o StrictHostKeyChecking=no root@localhost << EOF
+    rm -rf $GUEST_TEST_DIR
+    mkdir $GUEST_TEST_DIR
+EOF
+  sshpass -e scp -P "$PORT" -o StrictHostKeyChecking=no common.sh root@localhost:$GUEST_TEST_DIR
+  sshpass -e scp -P "$PORT" -o StrictHostKeyChecking=no $1 root@localhost:$GUEST_TEST_DIR
+  test_print_trc "Guest VM test script prepare complete"
+}
+
+# function based on sshpass to scp $1 source_code_dir and compile $2 test_binary in Guest VM
+guest_test_source_code() {
+  sshpass -e scp -P "$PORT" -o StrictHostKeyChecking=no -r $1 root@localhost:$GUEST_TEST_DIR
+  sshpass -e ssh -p "$PORT" -o StrictHostKeyChecking=no root@localhost << EOF
+    source $GUEST_TEST_DIR/common.sh
+    cd $GUEST_TEST_DIR/$1
+    make || die "Failed to compile source code $1"
+    if [ -f $2 ]; then
+      chmod a+x $2
+      cp $2 $GUEST_TEST_DIR
+    else
+      die "Can't find test binary $2"
+    fi
+EOF
+  test_print_trc "Guest VM test source code and binary prepare complete"
+}
+
+# function based on sshpass to execute $1 test_script.sh and potential $2 script params in Guest VM
+guest_test_entry() {
+  sshpass -e ssh -p "$PORT" -o StrictHostKeyChecking=no root@localhost << EOF
+    source $GUEST_TEST_DIR/common.sh
+    cd $GUEST_TEST_DIR
+    test_print_trc "guest_test_entry args 1: $1"
+    test_print_trc "guest_test_entry args 2: $2"
+    ./$1 $2
+EOF
+ERR_NUM=$?
+if [ $ERR_NUM -eq 0 ] || [ $ERR_NUM -eq 255 ]; then
+  return 0
+else
+  return 1
+fi
+}
+
+# function based on sshpass to close VM
+guest_test_close() {
+  sshpass -e ssh -p "$PORT" -o StrictHostKeyChecking=no root@localhost << EOF
+    source $GUEST_TEST_DIR/common.sh
+    test_print_trc "guest test complete, close VM now"
+    systemctl reboot --reboot-argument=now
+EOF
+  test_print_trc "Guest VM closed properly after test"
+}
+
+###################### Do Works ######################
+cd "$(dirname "$0")" 2>/dev/null || exit 1
+source ../.env
+
+# get test scenario config for test_executor
+source $SCRIPT_DIR/test_params.py
+
+cd $SCRIPT_DIR
+# select test_functions by $TEST_SCENARIO
+case "$TESTCASE" in
+  TD_BOOT)
+    guest_test_prepare tdx_guest_boot_check.sh
+    guest_test_entry tdx_guest_boot_check.sh "-v $VCPU -s $SOCKETS -m $MEM" || \
+    die "Failed on TD_BOOT test tdx_guest_boot_check.sh -v $VCPU -s $SOCKETS -m $MEM"
+    if [[ $GCOV == "off" ]]; then
+      guest_test_close
+    fi
+    ;;
+  GUEST_TESTCASE_EXAMPLE)
+    guest_test_prepare guest_test.sh
+    guest_test_source_code test_source_code_dir_example test_binary_example
+    guest_test_entry guest_test.sh "-t $TESTCASE" || \
+    die "Failed on GUEST_TESTCASE_EXAMPLE guest_test.sh -t $TESTCASE"
+    if [[ $GCOV == "off" ]]; then
+      guest_test_close
+    fi
+    ;;
+  :)
+    test_print_err "Must specify the test scenario option by [-t]"
+    usage && exit 1
+    ;;
+  \?)
+    test_print_err "Input test case option $TESTCASE is not supported"
+    usage && exit 1
+    ;;
+esac