There are several approaches and areas to test and validate a System Under Test (SUT) is robustly adhering to protocol standards and ready for long term use. Here we describe these areas and provide examples of Valid8.com products and services designed to test these areas:


Simulation Testing
The test-tool provides access at a certain protocol layer and behaves in a controlled manner to run scenarios against the SUT. Examples of simulation include Conformance, Feature, and Robustness Testing. The tests are fully editable and can be modified as desired to meet particular test plan requirements.

Example: SIP Simulation


Emulation Testing
The test-tool acts as a reference device for a particular end-point or set of end-points. For example, an H.248 Access Gateway, or a SIP Proxy is an end-point. The state-machines and architecture are usually not editable but instead run a pre-configured standard call-flow according to the specification.

Example: H.248 Call Server Emulator


Conformance Testing
This involves creating one or more test scenarios from each paragraph in a protocol specification, and then testing against the SUT to measure how it reacts, displaying a PASS/FAIL verdict for each. There are several sub-categories of test within the conformance umbrella:

Valid : Ensure the SUT responds as it should according to the specification. For example, on receipt of an SIP INVITE, the SUT should respond with a 1xx provisional response or a 2xx final response.

Invalid : Ensure the SUT responds as it should under erroneous conditions. An example would be sending a badly constructed SIP INVITE to a SIP Server and verifies the SUT handles it correctly without crashing.

Inopportune : Ensure the SUT is able to handle unexpected messages, i.e. syntactically correct but not received at the usual time. For example sending a CANCEL for a call leg that does not exist.

Timeout : Ensure the SUT handles timeout events and retransmissions. For example if the tester omits to send a response to an INVITE, the SUT should retransmit the original INVITE after T1 seconds.

Example: H.323 Conformance Test Suite


Feature Testing
The test-tool simulates a protocol end-point to test a particular feature for correct behavior. For example a SIP Call-Transfer scenario may need to be tested for a particular SUT type such as a n IP PBX.

Example: SIP PBX Feature Tester


Load Testing
Here the test-tool creates an environment where traffic (e.g. signaling, voice, data) is generated to verify correct operation in the SUT. For example, a load tester could emulate thousands of SIP User Agents making calls against a SIP Server SUT.

Example: SCTP Traffic Generator


Robustness Testing or Fuzzing
The goal in fuzzing is to find bugs and vulnerabilities, and break (crash) the SUT by sending it messages with illegal out-of-bounds or unexpected parameters according to the specification. The SUT is monitored for abnormal behavior such as a crash or out-of-resources, etc. Examples include flooding the SUT with streams of too large or too small messages.

Example: SCTP Test Suite


Regression Testing
Tests that previously uncovered an issue are rerun to ensure they do not get reintroduced after changes are made to the SUT such as software changes.

Example: SIP IMS Test Suite

About Valid8.com
Founded in September 2002 and located in Woburn Massachusetts, Valid8.com was created to develop customized solutions for the needs of the local and global market, focusing primarily on VoIP. Valid8.com is a software development company specializing in ready-to-run solutions for telecommunications signaling and test requirements. Building on optimized standard modules from established technology partners, Valid8.com adds application specific functionality with our specialist design & development teams. With project management, design and development capabilities for software and hardware, Valid8.com is an experienced partner for demanding projects.
http://www.valid8.com.