Skip to content

Latest commit

 

History

History
246 lines (173 loc) · 12.4 KB

README.md

File metadata and controls

246 lines (173 loc) · 12.4 KB

The pyvesync testing library

This is the testing suite for the pyvesync library. Each device that is added must include tests. This helps to maintain the consistency of the API as new devices are added and the backend is refactored.

I've built a relatively simple framework to ease the burden of writing tests. There are some old tests that I had previously written that I've kept as I build the new framework but these tests were not comprehensive or portable. The files that begin with test_x_ are these previous tests and can safely be ignored.

The tests primarily run each API call for each device and record the request details in YAML files in the tests\API directory. These files are then used to verify the request when the test is run again.

The structure of the framework is as follows:

  1. call_json.py - This file contains general functions and the device list builder. This file does not need to be edited when adding a device.
  2. call_json_DEVICE.py - This file contains device specific responses such as the get_details() response and specific method responses. This file pulls in the device type list from each module. The minimum addition is to add the appropriate response to the DeviceDetails class and the device type associated with that response in the DETAILS_RESPONSES dictionary. This file also contains the DeviceDefaults class which are specific to the device.
  3. test_DEVICE.py - Each module in pyvesync has it's own test module, typically with one class that inherits the utils.BaseTest class. The class has two methods - test_details() and test_methods() that are parametrized by utils.pytest_generate_tests
  4. utils.py - Contains the general default values for all devices in the Defaults class and the TestBase class that contains the fixture that instantiates the VS object and patches the call_api() method.
  5. conftest.py - Contains the pytest_generate_tests function that is used to parametrize the tests based on all device types listed in the respective modules.

Running the tests

There are two pytest command line arguments built into the tests to specify when to write the api data to YAML files or when to overwrite the existing API calls in the YAML files.

To run a tests for development on existing devices or if you are not ready to write the api calls yet:

# Through pytest
pytest

# or through tox
tox -e testenv # you can also use the environments lint, pylint, mypy

If developing a new device and it is completed and thoroughly tested, pass the --write_api to pytest. Be sure to include the -- before the argument in the tox command.

pytest --write_api

tox -e testenv -- --write_api

If fixing an existing device where the API call was incorrect or the api has changed, pass --write_api and overwrite to pytest. Both arguments need to be provided to overwrite existing API data already in the YAML files.

pytest --write_api --overwrite

tox -e testenv -- --write_api --overwrite

Testing Process

The first test run verifies that all of the devices defined in each pyvesync module have a corresponding response in each call_json_DEVICE module. This verifies that when a new device is added, a corresponding response is added to be tested.

The testing framework takes the approach of verifying the response and request of each API call separately. The request side of the call is verified by recording the request for a mocked call. The requests are recorded into YAML files in the api folder of the tests directory, grouped in folders by module and file by device type.

The response side of the API is tested through the use of responses that have been documented in the call_json files and the values specified in the Defaults and DeviceDefaults classes.

File Structure

Device Responses and Details

The call_json files contain all of the response data for each device type. The following call_json files are included in the test directory:

  • call_json.py - general API responses, including login() and get_devices(). The device list from the get_devices() can be used to create the device list response for all devices.
  • call_json_outlets.py - API responses for the outlets
  • call_json_switches.py - API responses for the switches
  • call_json_fans.py - API responses for the fans
  • call_json_bulbs.py - API responses for the bulbs

call_json.py

The call_json.py file contains the functions to build the get_devices() response containing the device list and each item on the device list. The DeviceList class contains the device_list_response(devices_types=None, _types=None) method which returns the full device list response based on the defined device types (model number(s)) or types (outlets, fans, etc.). The device_list_item(model) builds the individual device list item that is used to instantiate the device object. The default values for device configuration values are pulled from the Defaults class in the utils.py module for consistency.

call_json_DEVICE.py

Each device module has it's own call_json file. The structure of the files maintains a consistency for easy test definition. The DeviceDetails (SwitchDetails, OutletDetails) class contains the get_details() responses for each device as a class attribute. The name of the class attribute does not matter.

The DETAILS_RESPONSES dictionary contains the device type as the key and references the DeviceDetails class attribute as the value. The DETAILS_RESPONSES dictionary is used to lookup the appropriate response for each device type.

The responses for device methods are also defined in the call_json_DEVICE module. The METHOD_RESPONSES dictionary uses a defaultdict imported from utils.py with a simple {"code": 0, "message": ""} as the default value. The METHOD_RESPONSES dictionary is created with keys of device type and values as the defaultdict object. From here the method responses can be added to the defaultdict object for specific scenarios.

from utils import FunctionResponses 
from copy import deepcopy

device_types = ['dev1', 'dev2']

# defaultdict with default value - ({"code": 0, "msg": None}, 200)
method_response = FunctionResponses 

# Use deepcopy to build the device response dictionary used to test the get_details() method
device_responses = {dev_type: deepcopy(method_response) for dev_type in device_types}

# Define response for specific device & method
# All response must be tuples with (json response, 200)
device_responses['dev1']['special_method'] = ({'response': 'special response', 'msg': 'special method'}, 200)

# The default factory can be change for a single device type since deepcopy is used.
device_responses['dev2'].default_factory = lambda: ({'new_code': 0, 'msg': 'success', {'payload': {}}}, 200)

The method responses can also be a function that accept one argument that contains the kwargs used in the method call. This allows for more complex responses based on the method call.

The test will know whether it is a straight value or function and call it accordingly.

For example, this is the set status response of the valceno bulb:

METHOD_RESPONSES = {k: deepcopy(FunctionResponses) for k in BULBS}

def valceno_set_status_response(kwargs=None):
    default_resp = {
        "traceId": Defaults.trace_id,
        "code": 0,
        "msg": "request success",
        "result": {
            "traceId": Defaults.trace_id,
            "code": 0,
            "result": {
                "enabled": "on",
                "colorMode": "hsv",
                "brightness": Defaults.brightness,
                "colorTemp": Defaults.color_temp,
                "hue": Defaults.color.hsv.hue*27.7778,
                "saturation": Defaults.color.hsv.saturation*100,
                "value": Defaults.color.hsv.value
            }
        }
    }
    if kwargs is not None and isinstance(kwargs, dict):
        if kwargs.get('hue') is not None:
            default_resp['result']['result']['hue'] = kwargs['hue'] * 27.7778
        if kwargs.get('saturation') is not None:
            default_resp['result']['result']['saturation'] = kwargs['saturation'] * 100
        if kwargs.get('value') is not None:
            default_resp['result']['result']['value'] = kwargs['value']
    return default_resp, 200


XYD0001_RESP = {
    'set_brightness': valceno_set_status_response,
    'set_color_temp': valceno_set_status_response,
    'set_hsv': valceno_set_status_response,
    'set_rgb': valceno_set_status_response,
}

METHOD_RESPONSES['XYD0001'].update(XYD0001_RESP)

api directory with YAML files

API requests recorded from the mocked call_api() method. The api directory contains folders for each module and files for each device_type. The structure of the YAML files is:

File tests/api/switches/esl100.yaml

turn_off:
  headers:
    accept-language: en
    accountId: sample_id
    appVersion: 2.8.6
    content-type: application/json
    tk: sample_tk
    tz: America/New_York
  json_object:
    acceptLanguage: en
    accountID: sample_id
    status: 'off'
    switchNo: 3
    timeZone: America/New_York
    token: sample_tk
    uuid: ESO15-TB-UUID
  method: put
  url: /outdoorsocket15a/v1/device/devicestatus

utils.py - utility functions and default value factory for tests.

The utils.py file contains several helper functions and classes:

Default values for API responses and requests

The recorded requests are automatically scrubbed with these default values to remove sensitive information and normalize the data. Any new API calls added to call_json_ files should use the defaults values wherever possible.

from utils import Defaults

# Default Class variables
token = Defaults.token
account_id = Defaults.account_id
trace_id = Defaults.trace_id
active_time = Defaults.active_time
# The default Color dataclass contains the attributes red, green, blue, hue saturation & value. Conversion is automatically done regardless of the input color model. This is to normalize any API calls that involve changing color
color = Color(red=50, green=100, blue=255)
brightness = Defaults.brightness
color_temp = Defaults.color_temp

# Default values that use methods
device_name = Defaults.name(dev_type="ESL100") # returns 'ESL100-NAME'
device_cid = Defaults.cid(dev_type="ESL100") # returns 'ESL100-CID'
device_uuid = Defaults.uuid(dev_type="ESL100") # returns 'ESL100-UUID'
device_mac = Defaults.macid(dev_type="ESL100") # returns 'ESL100-MACID'

The utils module contains the base class with a fixture that instantiates the VeSync object and patches call_api() automatically, allowing a return value to be set..

from utils import TestBase, FunctionResponses


class TestDevice(TestBase):

    def test_details(self):
        vesync_instance = self.manager
        mock_api_object = self.mock_api # patch('pyvesync.helpers.call_api', autopspec=True)
        mock_api_object.return_value = FunctionResponses['default']
        caplog = self.caplog
        assert vesync_instance.enabled is True

Test Structure

Each module in the pyvesync library has an associated testing module, for example, vesyncswitches and test_switches. Most testing modules have one class, except for the test_fans module, which has separate classes for humidifiers and air purifiers.

The class inherits from the TestBase class in utils.py and is parametrized by pytest_generate_tests based on the method. The parameters are defined by the class attributes. The base_methods and device_methods class attributes define the method and arguments in a list of lists with the first item, the method name and the second optional item, the method kwargs. The base_methods class attribute defines methods that are common to all devices. The device_methods class attribute defines methods that are specific to the device type.

This is an examples of the class definition:

from utils import TestBase

class TestBulbs(TestBase):
    device = 'bulbs'
    bulbs = call_json_bulbs.BULBS
      base_methods = [['turn_on'], ['turn_off'],
                      ['set_brightness',   {'brightness': 50}]]
    device_methods = {
        'ESL100CW': [['set_color_temp', {'color_temp': 50}]]
    }

The methods are then parametrized based on those values. For most device additions, the only thing that needs to be added is the device type in the DETAILS_RESPONSES and possibly a response in the METHOD_RESPONSES dictionary.

See the docstrings in the modules for more information.