Glossary
Testing Roles
Testing roles are jobs assigned to a user or a group of users to test the different aspects of an application or a system according to the required specifications.
Test Engineer
A test engineer is a professional who determines how to create a process that would test a particular product to assure that the product meets applicable specifications. A Test engineer produces the test cases.
Tester
A tester is responsible for the core activities of the test effort, which involves conducting the necessary tests and logging the outcomes of testing.
Test Plan
A test plan documents the strategy that will be used to verify and ensure that a product or system meets its design specifications and other requirements. A test plan is typically prepared by test engineers. Technically, the test plan is a tree structure that consists of concrete test cases and folders. Folders are container items to group related test cases or subfolders. In Codebeamer, test plans are built and maintained in the test cases tracker. Both folders and test cases are represented by tracker items in that tracker.
Test Case
A test case is a detailed procedure that fully tests a feature or an aspect of a feature. Test cases consist of detailed descriptions, pre-actions, test steps, and post-actions. Pre-actions should be executed before the actual test steps to initialize the environment, to populate test data and for other types of preparation. Post-actions should be executed after the steps, to close open resources and to clean up the environment. Post-actions should be executed regardless of the actual result of the test case, unless documented otherwise. A test case is a tracker item in the test cases tracker.
Test Step
A test step is one step in a test case procedure.
 |
The default values for the Action and Expected result fields in test case configuration do not have any effect during test step creations and test runs.
|
Test Set
A test set is a logical group of related test cases. It is associated with a list of releases that it is allowed to be executed with, and a list of test configurations that are allowed to be used while executing it. Test Sets are maintained in the test sets tracker.
Test Configuration
A test configuration is one possible configuration of the product under testing. As tests can have different results depending on the configuration used, the configurations used for the tests must be recorded for effective testing. Test configurations are stored in the test configurations tracker.
Test Run
A test run is an actual execution of a test set using one specific release and one specific test configuration. Test runs are initiated by test engineers and are executed by testers. Test runs are maintained in the test runs tracker.
Test Results
The system starts showing results for the test run as soon as a result is available for at least one test case in the run. Depending on the status of the test run, the results shown by the system could be partial or final results.
Passed
Results that have “passed” mean that all executed test cases and all checked aspects worked as expected.
Partly Passed
A parameterized test is considered to have partly passed if some of its parameters are skipped, but all of the executed parameterized test cases have passed.
A test run is considered to have partly passed if a test is parameterized and the tester skips some of the parameters, but all of the executed parameterized tests have passed.
For more information, see: Test Parameterisation and Skipping Some Test Parameters and Partly Passed Result.
Failed
A test run is considered failed, if at least one of the tested aspects or a test case did not work as expected.
Blocked
Tests with a blocked result are not considered as closed. After removing the cause of the block, the tests should be continued from the point of the block.
Not Applicable
When certain test steps or test runs are not relevant, test step and test run results can be set to Not Applicable. Test steps with Not Applicable results can be ignored.
The following logic used for the Not Applicable result:
• When at least one test step result has a value other than Not Applicable, the result of the test run shall be calculated as if the test step with the Not Applicable result did not exist. The test step with a Not Applicable result is ignored.
• When all of the test step results in a test run have a value of Not Applicable, the result of the test run shall be Not Applicable.
• If a test run has only Passed and Not Applicable evaluations, the final result should be Partly Passed (only in case of steps).
Test Coverage
Test coverage is a metric used to express which requirements of your product are verified by test cases, and by how many of those test cases. The more test cases exist for a requirement, the more it is covered. Codebeamer allows you to easily map test cases to requirements.