Interpreting the Test Output

The RDM Responder tests can produce a lot of output and some of the test cases are quite complex. This page aims to provide a general overview on interpreting the output from the tests.

For questions about the behavior of the tests please see the Support page.

Test Categories

Tests are grouped according to the sections in the RDM Categories/Parameter ID Defines table in the E1.20 document. There are some extra categories for specific behavior like error conditions and sub device handling.

Test States

Upon completion of the test suite, there are four possible result states for each test:

Passed
The responder replied with the expected result.
Failed
The responder failed to reply, or replied with an un-expected result.
Not Run
This test wasn’t run because the responder doesn’t support the required functionality or a previous test failed.
Broken
An internal error occurred, this indicates a programming error or an error with the test rig.

Log Messages

Some behavior isn’t explicitly forbidden by the E1.20 Standard but may cause problems for controllers. When detected such behavior is logging with Warning and Advisory messages.
Warnings
Warnings indicate behavior that doesn’t match the standard, but is unlikely to cause usability issues. Warnings are printed in the summary section of the test output.
Advisory Messages
Advisory messages indicate issues that are not covered by the standard but are likely to cause problems i.e a sensor temperature out side of the stated scale range.

Test Output

Let’s go over the output from the tests using some examples. If you’re running the tests from the command line you can see the verbose output by adding -d as an argument. If you’re using the Web UI the verbose output will be visible in the bottom third of the screen.

First a successful test:

GetDeviceModelDescription: GET the device model description.
 GET: uid: 7a70:00000001, pid: DEVICE_MODEL_DESCRIPTION (0x0080), sub device: 0, args: []
 Response: RDMResponse: ACK, PID = 0x0080, data = {u'description': 'Model 1'}
GetDeviceModelDescription: Passed

The first line contains the name of the test, as well as a one line description of what the test does. The second line is the command we sent to the responder, in this case it was a GET DEVICE_MODEL_DESCRIPTION sent to 7a70:00000001 with no parameter data.

The third line is the response we received from the responder. In this case we received an ACK, with the response data “Model 1”. This is a valid response so the test is declared to have passed (line 4).

Now let’s consider a test that fails:

AllSubDevicesDeviceInfo: Send a Get Device Info to ALL_SUB_DEVICES.
 GET: uid: 7a70:00000001, pid: DEVICE_INFO (0x0060), sub device: 65535, args: []
Request failed: Response Timeout
 Failed: expected one of:
  CC: Get, PID 0x0060, NACK Sub device out of range

In this test, we send a GET DEVICE_INFO to all the sub devices on the responder. In this case the responder doesn’t respond with any data, so we declare a response timeout. The test then displays what it was expecting, in this case the correct response was a NR_SUB_DEVICE_OUT_OF_RANGE response code.

Let’s consider another test:

GetSensorDefinitionWithNoData: Get the sensor definition with no data.
 GET: uid: 7a70:00000001, pid: SENSOR_DEFINITION (0x0200), sub device: 0, data: ''
 Response: RDMResponse: ACK, PID = 0x0200, data = {u'normal_min': -32768, u'name': 'Device Lifetime Hours', u'range_min': -32768, u'range_max': 32767, u'normal_max': 32767, u'supports_recording': 0, u'prefix': 0, u'sensor_number': 4, u'type': 32, u'unit': 0}
 Failed: expected one of:
  CC: Get, PID 0x0200, NACK Format Error
GetSensorDefinitionWithNoData: Failed

Here we send a GET SENSOR_DEFINITION but without any parameter data. Recall that the GET SENSOR_DEFINITION command takes one byte of data which represents the sensor index to return the definition for. In this case the responder replies with an ACK, even though it doesn’t have enough information to determine which sensor we asked for! This fails the test since the correct result would be a NR_FORMAT_ERROR.