Skip to content

Commit

Permalink
Branches (#960)
Browse files Browse the repository at this point in the history 
* Drop py2.7 tests from travis config and setup.py.

* diagnoses_lib: Serialize diagnoses result values, not names

Diagnosis results should values should be serialized out rather than the enum constant names.

PiperOrigin-RevId: 318469885

* Attachments: Handle case where NamedTemporaryFile object stops working.

We have noticed times where we get bad file descriptor exceptions trying to
reload data from the temporary files.  For now, just return empty byte strings
while we investigate more.

Also, adding a callback to close the Attachments' temporary files.

PiperOrigin-RevId: 319879865

* test_record: Fix lint issues.

PiperOrigin-RevId: 320025482

* output callbacks: Fix CloseAttachments

Attachments are stored in a dictionary under the phase record, not a list.

PiperOrigin-RevId: 321406529

* OpenHTF: Fix lint issues and run autoformatter.

PiperOrigin-RevId: 323661267

* Initial pytype-based type annotations.

PiperOrigin-RevId: 324720299

* conf: Use inspect.getfullargspec.

PiperOrigin-RevId: 324754078

* Internal change

PiperOrigin-RevId: 325271171

* Only have the attachment's temporary file open while reading/writing.

If there are a lot of attachments, the program can exceed Linux's max allowed open files per process.

PiperOrigin-RevId: 325490340

* Add core annotations and replace mutablerecords and namedtuple with attr.

PhaseDescriptor will be in the next commit.

PiperOrigin-RevId: 327075946

* Rearrange and add comments for the Measurement fields.

PiperOrigin-RevId: 327140593

* Internal change

PiperOrigin-RevId: 327735936

* Convert PhaseOptions and PhaseDescriptor to attr.

PiperOrigin-RevId: 328778682

* callbacks: Add type annotations

Add type annotations to the callbacks library and break apart some complex
types.  Also breaking out the JSON conversion logic to an independent function
for easier use by other modules.

PiperOrigin-RevId: 328972148

* Internal change

PiperOrigin-RevId: 329550447

* Internal change

PiperOrigin-RevId: 331590164

* Internal change

PiperOrigin-RevId: 331890539

* Remove TestPhase alias.

The TestPhase alias for PhaseOptions has long been deprecated.  Removing it.

PiperOrigin-RevId: 332291802

* Add more type annotations.

PiperOrigin-RevId: 332519023

* Remove plugs-phase_descriptor circular dependency

Remove the plugs to phase_descriptor circular dependency by moving the pieces
phase_descriptor depends on to a new core/base_plugs.py file.

PiperOrigin-RevId: 332527874

* PhaseDescriptor: with_plugs and with_args now ignore unknowns

Change with_plugs and with_args to use their with_known_plugs and
with_known_args implementations instead.

PiperOrigin-RevId: 332546916

* Add type checking to unit tests to verify things are working.

PiperOrigin-RevId: 332551203

* test_descriptor: Remove Test Teardown.

Test teardown was deprecated in favor of PhaseGroup teardowns. Fully removing
them.

PiperOrigin-RevId: 333112589

* Add PhaseNode and implement PhaseSequence.

Phase nodes are now the basic building block of the OpenHTF execution engine.

Phase sequence is a phase node that constains a sequence of phase nodes.  Phase
groups now use phase sequences to contain its setup, main, and teardown phases.

PiperOrigin-RevId: 334461205

* Implement Phase Branches

Phase branches run phases conditionally based on triggered diagnosis results.

PiperOrigin-RevId: 335460872

* PhaseExecutor: Raise on invalid phase result.

PiperOrigin-RevId: 335469722

* Implement Subtests.

Subtests are a collection of phases that can indepenently fail and skip the rest
of the phases while still working with PhaseGroup teardowns.

PiperOrigin-RevId: 335472411

* util/test: Allow customizing the test_start_function.

The OpenHTF TestCase can now customize the test start function by setting the
`test_start_function` attribute.

This change will now force unit tests to call super().setUp() in all cases.

PiperOrigin-RevId: 335509542

* Implement Phase Checkpoints

Phase checkpoints are nodes that check if a diagnosis result has been triggered
or a simple set of phases has failed.  In those cases, they will either be
resolved as FAIL_SUBTEST or STOP.

PiperOrigin-RevId: 335513243

* Refactor TestApi and expose diagnoses store.

The TestApi object should just be a proxy to the functions on the PhaseState
and TestState instances.

Add the diagnoses store to the API for simpler access during phases.

PiperOrigin-RevId: 336213159

* Subtest skip phases when they fail.

Subtests should skip the later phases with special handling for other nodes as
documented in event_sequence.md.

PiperOrigin-RevId: 336386553

* Fix typo in TestApi.diagnoses_store

PiperOrigin-RevId: 336408351

* Internal change

PiperOrigin-RevId: 336691169

* Add DimensionPivot.

DimensionPivot is a validator that runs a subvalidator on each value
independently for a dimensioned measurement.  If any value fails, the
measurement is a failure.

PiperOrigin-RevId: 336730422

* Drop PY2 support from OpenHTF.

PiperOrigin-RevId: 337139629
  • Loading branch information
@arsharma1
arsharma1 authored Oct 14, 2020
1 parent 310b64d commit 94bebbc
Show file tree
Hide file tree
Showing 102 changed files with 7,697 additions and 3,741 deletions.
2 changes: 0 additions & 2 deletions .travis.yml
View file
Original file line number Diff line number Diff line change
@@ -1,8 +1,6 @@
language: python
matrix:
include:
- python: 2.7
env: TOXENV=py27
- python: 3.6
env: TOXENV=py36
addons:
Expand Down
22 changes: 22 additions & 0 deletions CHANGELOG
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
Changes for 2.0.

* Dropped Python 2.x support.
* Added type annotations.
* Replaced mutablerecords with attrs.
* PhaseOptions:
* The openhtf.TestPhase alias for PhaseOptions has been deprecated for a
long time. Removing it.
* PhaseDescriptor:
* with_known_plugs and with_known_args are being rolled into with_plugs and
with_args, respectively. They will no longer raise exceptions when
the names are not found.
* If the options name field is a callable, the name property of
PhaseDescriptors will only return the name of the function rather than
the callable. This ensures that the name property is always Text.
* Test:
* The test teardown has been removed in favor of using a PhaseGroup.
* Unit testing:
* Unit tests using openhtf.util.test.TestCase can customize the test start
function when yielding openhtf.Test instances by setting the
`test_start_function` attribute. This can be set to None to remove the
function.
96 changes: 50 additions & 46 deletions bin/units_from_xls.py
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,8 +11,6 @@
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.


"""Read in a .xls file and generate a units module for OpenHTF.
UNECE, the United Nations Economic Commission for Europe, publishes a set of
Expand All @@ -39,23 +37,19 @@
spaces into underscores, and converting to uppercase.
"""


import argparse
import os
import shutil
import re
import shutil
import sys
from tempfile import mkstemp
import tempfile

import six
import xlrd


# Column names for the columns we care about. This list must be populated in
# the expected order: [<name label>, <code label>, <suffix label>].
COLUMN_NAMES = ['Name',
'Common\nCode',
'Symbol']
COLUMN_NAMES = ['Name', 'Common\nCode', 'Symbol']

PRE = '''# coding: utf-8
# THIS FILE IS AUTOMATICALLY GENERATED. DO NOT EDIT.
Expand Down Expand Up @@ -100,7 +94,14 @@
import collections
UnitDescriptor = collections.namedtuple('UnitDescriptor', 'name code suffix')
class UnitDescriptor(
collections.namedtuple('UnitDescriptor', [
'name',
'code',
'suffix',
])):
pass
ALL_UNITS = []
Expand All @@ -120,8 +121,10 @@
# pylint: enable=line-too-long
class UnitLookup(object):
"""Facilitates user-friendly access to units."""
def __init__(self, lookup):
self._lookup = lookup
Expand All @@ -144,41 +147,42 @@ def __call__(self, name_or_suffix):
'''

SHEET_NAME = 'Annex II & Annex III'
UNIT_KEY_REPLACEMENTS = {' ': '_',
',' : '_',
'.': '_',
'-': '_',
'/': '_PER_',
'%': 'PERCENT',
'[': '',
']': '',
'(': '',
')': '',
"'": '',
'8': 'EIGHT',
'15': 'FIFTEEN',
'30': 'THIRTY',
'\\': '_',
six.unichr(160): '_',
six.unichr(176): 'DEG_',
six.unichr(186): 'DEG_',
six.unichr(8211): '_',
}
UNIT_KEY_REPLACEMENTS = {
' ': '_',
',': '_',
'.': '_',
'-': '_',
'/': '_PER_',
'%': 'PERCENT',
'[': '',
']': '',
'(': '',
')': '',
"'": '',
'8': 'EIGHT',
'15': 'FIFTEEN',
'30': 'THIRTY',
'\\': '_',
six.unichr(160): '_', # NO-BREAK SPACE
six.unichr(176): 'DEG_', # DEGREE SIGN
six.unichr(186): 'DEG_', # MASCULINE ORDINAL INDICATOR
six.unichr(8211): '_', # EN DASH
}


def main():
"""Main entry point for UNECE code .xls parsing."""
parser = argparse.ArgumentParser(
description='Reads in a .xls file and generates a units module for '
'OpenHTF.',
'OpenHTF.',
prog='python units_from_xls.py')
parser.add_argument('xlsfile', type=str,
help='the .xls file to parse')
parser.add_argument('xlsfile', type=str, help='the .xls file to parse')
parser.add_argument(
'--outfile',
type=str,
default=os.path.join(os.path.dirname(__file__), os.path.pardir,
'openhtf','util', 'units.py'),
default=os.path.join(
os.path.dirname(__file__), os.path.pardir, 'openhtf', 'util',
'units.py'),
help='where to put the generated .py file.')
args = parser.parse_args()

Expand All @@ -188,14 +192,12 @@ def main():
sys.exit()

unit_defs = unit_defs_from_sheet(
xlrd.open_workbook(args.xlsfile).sheet_by_name(SHEET_NAME),
COLUMN_NAMES)
xlrd.open_workbook(args.xlsfile).sheet_by_name(SHEET_NAME), COLUMN_NAMES)

_, tmp_path = mkstemp()
_, tmp_path = tempfile.mkstemp()
with open(tmp_path, 'w') as new_file:
new_file.write(PRE)
new_file.writelines(
[line.encode('utf8', 'replace') for line in unit_defs])
new_file.writelines([line.encode('utf8', 'replace') for line in unit_defs])
new_file.write(POST)
new_file.flush()

Expand All @@ -209,14 +211,16 @@ def unit_defs_from_sheet(sheet, column_names):
Args:
sheet: An xldr.sheet object representing a UNECE code worksheet.
column_names: A list/tuple with the expected column names corresponding to
the unit name, code and suffix in that order.
Yields: Lines of Python source code that define OpenHTF Unit objects.
the unit name, code and suffix in that order.
Yields:
Lines of Python source code that define OpenHTF Unit objects.
"""
seen = set()
try:
col_indices = {}
rows = sheet.get_rows()

# Find the indices for the columns we care about.
for idx, cell in enumerate(six.next(rows)):
if cell.value in column_names:
Expand All @@ -234,9 +238,9 @@ def unit_defs_from_sheet(sheet, column_names):

# Split on ' or ' to support the units like '% or pct'
for suffix in suffix.split(' or '):
yield "%s = UnitDescriptor('%s', '%s', '''%s''')\n" % (
key, name, code, suffix)
yield "ALL_UNITS.append(%s)\n" % key
yield "%s = UnitDescriptor('%s', '%s', '''%s''')\n" % (key, name, code,
suffix)
yield 'ALL_UNITS.append(%s)\n' % key

except xlrd.XLRDError:
sys.stdout.write('Unable to process the .xls file.')
Expand Down
94 changes: 82 additions & 12 deletions docs/event_sequence.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,13 +6,58 @@ further:
1. `test_start`'s plugs are instantiated
1. `test_start` is run in a new thread
1. All plugs for the test are instantiated
1. Each phase is run
1. The teardown phase is run
1. Each phase node is run
1. If the node is a subtest, each node is run until a FAIL_SUBTEST is
returned by a phase.
1. If the node is a branch, each node is run if the condition is met
1. If the node is a sequence, each node is run
1. If the node is a group, each of the groups sequences is run
1. If the node is a phase descriptor, that phase is run
1. All plugs' `tearDown` function is called
1. All plugs are deleted
1. Test outcome is calculated as PASS or FAIL
1. Output callbacks are called

## Phase node execution

```
[PhaseNode]
|
+--[PhaseDescriptor]
|
+--[Checkpoint]
|
\--[PhaseCollection]
|
+--[PhaseSequence]
| |
| +--[PhaseBranch]
| |
| \--[Subtest]
|
\--[PhaseGroup]
```

`PhaseNode`s are the basic building block for OpenHTF's phase execution. They
are a base class that defines a few basic operations that can get recursively
applied. The `PhaseDescriptor` is the primary executable unit that wraps the
phase functions. `PhaseCollection` is a base class for a node that contains
multiple nodes. The primary one of these is the `PhaseSequence`, which is a
tuple of phase nodes; each of those nodes is executed in order with nested
execution if those nodes are other collections. `PhaseBranch`s are phase
sequences that are only run when the Diagnosis Result-based conditions are met.
`Checkpoint` nodes check conditions, like phase failure or a triggered
diagnosis; if that condition is met, they act as a failed phase. `PhaseGroup`s
are phase collections that have three sequences as described below.

### Recursive nesting

Phase collections allow for nested nodes where each nested level is handled with
recursion.

OpenHTF does not check for or handle the situation where a node is nested inside
itself. The current collection types are frozen to prevent this from happening.

## Test error short-circuiting

A phase raising an exception won't kill the test, but will initiate a
Expand All @@ -23,27 +68,52 @@ If a `test_start` phase is terminal, then the executor will skip to Plug
Teardown, where only the plugs initialized for `test_start` have their
`teardown` functions called.

In all cases with terminal phases, the Test outcome is ERROR for output
callbacks.

### PhaseGroups

`PhaseGroup` collections behave like contexts. They are entered if their
`setup` phases are all non-terminal; if this happens, the `teardown` phases are
guarenteed to run. `PhaseGroup` collections can contain additional `PhaseGroup`
instances. If a nested group has a terminal phase, the outer groups will trigger
the same shortcut logic.

For terminal phases in a `PhaseGroup`,
* If the phase was a `PhaseGroup.setup` phase, then we skip the rest of the
`PhaseGroup`.
* If the phase was a `PhaseGroup.main` phase, then we skip to the
`PhaseGroup.teardown` phases of that `PhaseGroup`.
* If the phase was a `PhaseGroup.teardown` phase, the rest of the `teardown`
phases are run, but outer groups will trigger the shortcut logic.

In all cases with terminal phases, the Test outcome is ERROR for output
callbacks.
For terminal phases (or phases that return `FAIL_SUBTEST`) in a `PhaseGroup`,
* If the phase was in the `setup` sequence, then we do not run the rest of
the `PhaseGroup`.
* If the phase was in the `main` sequence, then we do not run the rest of the
`main` sequence and proceed to the `teardown` sequence of that `PhaseGroup`.
* If the phase was in the `teardown` sequence, the rest of the `teardown`
sequence ndoes are run, but outer groups will trigger the shortcut logic.
This also applies to all nested phase nodes.

NOTE: If a phase calls `os.abort()` or an equivalent to the C++
`die()` function, then the process dies and you cannot recover the results from
this, so try to avoid such behavior in any Python or C++ libraries you use.

### Subtests

`Subtest`s are Phase Sequences that allow phases to exit early, but continue on
with other phases. A phase can indicate this by returning
`htf.PhaseResult.FAIL_SUBTEST` or with a checkpoint with that result as its
action. The details of subtests are included in the output test record.

The rest of the phases in a subtest after the failing node will be processed as:

* Phase descriptors are all skipped.
* Branches are not run at all, as though their condition was evaluated as false.
* Groups entered after the failing node are entirely skipped, including their
`teardown` sequences.
* Groups with the failing node in its `main` sequence will skip the rest of the
`main` sequence, but will run the teardown phases.
* Groups with the failing node in its `setup` sequence will skip the rest of the
setup phases and will record skips for the `main` and `teardown` sequences.
* Groups with the failing node in its `teardown` sequence will still run the
rest of the `teardown` sequence.
* Sequences are recursively processed by these same rules.

Phase group teardowns are run properly when nested in a subtest.

## Test abortion short-circuiting

Expand Down
Loading

0 comments on commit 94bebbc

Please sign in to comment.