tc-testing: Allow test cases to be skipped

By adding a check for an optional key/value pair to the test case
data, individual test cases may be skipped to prevent tdc from
aborting a test run due to setup or teardown failure.

If a test case is skipped, it will still appear in the results
output to allow for a consistent number of executed tests in each
run. However, the test will be marked as skipped.

This support for skipping extends to any plugins that may generate
additional results for each executed test.

Signed-off-by: Lucas Bates <lucasb@mojatatu.com>
Signed-off-by: David S. Miller <davem@davemloft.net>
This commit is contained in:
Lucas Bates 2019-02-28 17:38:40 -05:00 committed by David S. Miller
parent 745732c8ec
commit 255c1c7279
4 changed files with 27 additions and 13 deletions

View file

@ -33,6 +33,11 @@ Each test case has required data:
id: A unique alphanumeric value to identify a particular test case
name: Descriptive name that explains the command under test
skip: A completely optional key, if the corresponding value is "yes"
then tdc will not execute the test case in question. However,
this test case will still appear in the results output but
marked as skipped. This key can be placed anywhere inside the
test case at the top level.
category: A list of single-word descriptions covering what the command
under test is testing. Example: filter, actions, u32, gact, etc.
setup: The list of commands required to ensure the command under test