Skip to content

UI testing

Jump to bottom Edit New page
Kazuki Nakanishi edited this page Aug 23, 2018 · 4 revisions

This is a draft version to be reviewed.

Overview

This is an overview of UI testing framework.

overview

Target reader

  • Node-red editor developers
  • Node-red core node developers

Prerequisites

  • OS: mac OS/ Linux / Windows
  • You cannot test on Raspberry Pi.
  • Browser: The latest version of Chrome

Usage

Install

Follow the steps of how to get started for developers so that the components for running UI test are installed.

Run

  • Build and run tests without opening a browser.
grunt test-ui
  • Run tests with opening a browser.
grunt test-ui --non-headless
  • Runs tests without building (this is useful from the second time).
grunt webdriver [--non-headless]

Components

Separation of test specifications and object manipulations

If a test script contains the XPATH of HTML elements here and there, once an HTML element of Node-RED editor was changed, all test scripts using such element will be affected. To narrow the range of influence, UI testing framework separates test specifications and object manipulations.

Test specifications only calls the APIs provided by Node-RED and does not contain the code of object manipulations such as XPATH and WebDriver IO API. Those details appear only in page objects. When developers modified Node-RED editor, they needs to modify only page objects.

Folder structure

Path Description
test/editor/pageobjects page objects that were affected by the change of elements
test/editor/pageobjects/nodes page objects of edit window of nodes
test/editor/pageobjects/util utilities for operating a browser
test/editor/pageobjects/editor page objects of Node-RED editor
test/editor/specs test specifications that were not affected by the change of elements
test/editor/specs/scenario scripts based on test scenarios
test/editor/specs/editor scripts for testing Node-RED UI
test/resources resources that are necessary for testing
test/nodes node specific test scenarios (Currently there are no UI tests for nodes though)

API

Editor

Contains the page objects for manipulating the objects on the Node-RED editor.

Workspace

  • test/editor/pageobjects/editor/workspace_page.js

Provides the functionality that operates on a workspace.

Function Description
addNode(type, x, y) Adds a new node on workspace and return its object. type is the type of nodes which is defined in palette_page.js. x and y is a coordinate of node from the top-left of workspace.
deleteAllNodes() Deletes all nodes on a workspace. This function is basically called before each test case.
deploy() Hits a deploy button.

Debug tab

  • test/editor/pageobjects/editor/debugTab_page.js

Provides the functionality that operates on a debug tab.

Function Description
open() Opens a debug tab.
getMessage(index) Returns an index-th message on the debug window. If index is omitted, returns the first message.
clearMessage() Clears all messages on debug tab.

Nodes

Contains the page objects related to manipulate nodes. When you set arguments of each node, follow the following steps.

  1. Add a node on the Node-RED.
  2. Edit the node as you want to run on the UI test.
  3. See the information tab of the node.
  4. Check the key and value and set them to the argument.

The following picture indicates how to check the argument value of setPayload function of 20_inject_page.js.

overview

Nodes

  • test/editor/pageobjects/nodes/node_page.js

Provides the functionality that contains the common operation among all nodes.

Function Description
edit() Opens an edit window.
clickOk() Hits a Done button on edit tab.
connect(targetNode) Connects a wire to a targetNode. targetNode is the object that Workspace.addNode returns.

Inject node

  • test/editor/pageobjects/nodes/core/core/20-inject_page.js

Provides the functionality that operates an inject node.

Function Description
setPayload(payloadType, payload) Sets a payload type and its value.
setTopic(topic) Sets a Topic.
setOnce(once) Sets a flag of whether running only once. Only true and false are acceptable.
setRepeat(repeatType) Sets a repeat type. repeatType is one of none, interval, intervalBetweenTimes, or atASpecificTime (note: repeatType is not a key of inject node.)
setRepeatInterval(repeat) Sets the interval of repeat (sec). This function can be called only when interval was specified for setRepeatType function.

Debug node

  • test/editor/pageobjects/nodes/core/core/58-debug_page.js

Provides the functionality that operates a debug node.

Function Description
setOutput(complete) Sets an output type. If complete is true, outputs complete message. Otherwise, specifies the path of msg.

Function node

  • test/editor/pageobjects/nodes/core/core/80-function_page.js

Provides the functionality that operates a function node.

Function Description
setFunction(func) Sets a JavaScript code.

Template node

  • test/editor/pageobjects/nodes/core/core/80-template_page.js

Provides the functionality that operates a template node.

Function Description
setSyntax(syntax) Sets a format type (not syntax! The key name and its value is mismatched on Node-RED).
setFormat(format) Sets a syntax type (not format!).
setTemplate(template) Sets a template text.

Httpin node

  • test/editor/pageobjects/nodes/core/io/21-httpin_page.js

Provides the functionality that operates an httpin node.

Function Description
setMethod(method) Sets a method type.
setUrl(url) Sets a Url.

Httprequest node

  • test/editor/pageobjects/nodes/core/io/21-httprequest_page.js

Provides the functionality that operates an httprequest node.

Function Description
setMethod(method) Sets a method type.
setUrl(url) Sets a Url.
setReturn(ret) Sets a return type of response.

Change node

  • test/editor/pageobjects/nodes/core/logic/15-change_page.js

Provides the functionality that operates a change node.

Function Description
ruleSet(p, pt, to, tot, index) Sets a rule of change node. You can see the first four arguments in the rules object on the information tab. index indicates the order of rules. If index is omitted, the first rule is operated.
ruleDelete(index) Deletes a rule.
ruleMove(p, to, index) Moves or rename a index-th rule.
addRule() Adds a rule.

Range node

  • test/editor/pageobjects/nodes/core/logic/16-range_page.js

Provides the functionality that operates a range node.

Function Description
setAction(action) Sets a action type.
setRange(minin, maxin, minout, maxout) Sets a range.

HTML node

  • test/editor/pageobjects/nodes/core/parsers/70-HTML_page.js

Provides the functionality that operates an html node.

Function Description
setSelector(tag) Sets a selector to extract a specific element.

Filein node

  • test/editor/pageobjects/nodes/core/storage/50-filein_page.js

Provides the functionality that operates a filein node.

Function Description
setFilename(filename) Sets an input file name.
setOutput(format) Sets a format type.

Utility

Contains the utilities for testing.

util

  • test/editor/pageobjects/util/spec_util_page.js
Function Description
pause(msec) Pause for a specified number of milliseconds.

Helper

Helper script contains the API to operate Node-RED server.

  • test/editor/editor_helper.js
Function Description
startServer() Starts a node-red server. Flow files will be created and deleted for each test script. As a debug purpose, you can get the flow file that the test script generates by setting isDeleteFlow to false. Then you can find a flow file under test/resources/home.
stopServer() Stops a node-red server.
url() Returns the URL that the Node-RED is running. Since the port number is ephemeral, you have to call this API when you need URL of Node-RED server.

Development

Settings

The setting file for UI test is test/editor/wdio.conf.js. The following settings affects the behavior of UI test.

  • capabilities.maxInstances: This number of test scripts are executed in parallel. As the UI test runs much slower than the runtime test, UI test scripts should be executed in parallel. But if the number is too large, the machine may not be able to handle all of them and runs much slower as a result.
  • mochaOpts.timeout: This value affects all UI test scripts. If you write the test case that takes more than this timeout value, it would be good to specify this.timeout([milliseconds]); at the beginning of the test case instead of changing this timeout value.

Write test code in a synchronous manner

UI test scripts are executed in a synchronous manner (see sync parameter in wdio.conf.js). It means that you cannot use callback during the test basically.

  • If you want to wait for a while:
    • Use pause function in spec_util_page.
  • If you still need to process asynchronously:
    • Use browser.call of WebDriver IO (see editor_helper.js).

Create a test case

How to create a test script

  • The file name of UI test script must ends with _uispecs.js.
  • As multiple test scenarios are executed at the same time, the total testing time of each script should be nearly equal like three minutes. (Otherwise, you need to wait for a single long test scenario.)

Type of test scripts

  • UI test scripts are classified broadly into three categories so far, scenario based script, editor based script, and node based script.
    • Scenario based script: Test scripts based on a scenario such as cookbook.
    • Editor based script: Test scripts for Node-RED UI such as workspace, menu, etc.
    • Node based script: Test scripts for the edit window of a node.

Since they might be intersected, you need to consider which category is appropriate for the each script.

Example

Here's an example of test scenario:

    it('should output a timestamp', function() {
        var injectNode = workspace.addNode("inject"); // Add an Inject node.
        var debugNode = workspace.addNode("debug", nodeWidth); // Add an debug node.
        injectNode.connect(debugNode);  // Wire the two together.

        workspace.deploy(); // Deploy

        debugTab.open(); // Open a debug tab.
        debugTab.clearMessage(); // Click a delete button.
        injectNode.clickLeftButton(); // Click a left button of an Inject node.
        debugTab.getMessage().should.within(1500000000000, 3000000000000); // Check the message.
    });

Create a page object

  • The file name of a page object must ends with _page.js.

Naming rule

Basic rule is as follows:

  • Function name: The title shown on the left on edit node window (e.g. setPayload, setTopic for inject node).
  • Argument name: The key name shown on the information tab. If you cannot see it, click show more link. (e.g. payload, payloadType for inject node)

Manipulate a browser

To check the CSS selector or XPath of the target element that you want to manipulate:

  1. Access to Node-RED with chrome.
  2. Show the target element.
  3. Hit f12 key. Then DevTools window will appear.
  4. On the DevTools window, Hit Ctrl + Shift + c.
  5. Click the target element. Then the HTML tag of the target element will be focused on the Elements tab of DevTools.
  6. Right click -> [Copy] -> [Copy selector] or [Copy XPath]. Then the CSS selector or XPATH is copied to the clipboard.

Tips

  • When running UI test with --non-headless option, if you move your mouse on the on-going testing window, it may affect the test and will be failed.
Add a custom footer
Add a custom sidebar
Clone this wiki locally