Automated Quality Assurance of Non-Functional Requirements for Testability

A Software Requirements Specification (SRS) document contains all the requirements to describe a software system to be developed. These requirements are typically separated into Functional Requirements (FRs), which describe the features of the system under development and Non-Functional Requirements (NFRs), which include quality attributes and design constraints, among others. NFRs can have a significant impact on the time of a system?s development process and its total cost, as they frequently describe cross-cutting concerns. NFRs that are not testable are typically ignored in system development, as there is no way to verify them. Thus, NFRs must be checked for testability. However, for natural language requirements, this so far had to be done manually, which is time-consuming and therefore costly.
In order to improve software development support, we propose a semantic quality assurance method that automatically detects non-testable NFRs in natural language specifications. Our work contains four significant contributions towards this goal: (1) building a generic ontology which represents the main concepts in requirements statements and their relations; (2) Based on this generic ontology, two corpora are developed: The first one is a new gold standard corpus containing annotations for different NFR types. The second one is for requirements thematic roles and testability; (3) A Support Vector Machine (SVM) classifier to automatically categorize requirements sentences into the different ontology classes is introduced; (4) Finally, a rule-based text mining system is used to analyze requirement thematic roles and to flag non-testable NFRs. Based on the SRS corpus, our results demonstrate that the proposed approach is feasible and effective, with an F-measure of 80% for non-testability detection.