Assessing The Quality Factors Found in In-Line Documentation Written in Natural Language: The JavadocMiner

An important software engineering artifact used by developers and maintainers to assist in software comprehension and maintenance is source code documentation. It provides the insight needed by software engineers when performing a task, and therefore ensuring the quality of documentation is extremely important. In-line documentation is at the forefront of explaining a programmer's original intentions for a given implementation. Since this documentation is written in informal natural language, ensuring its quality needs to be performed manually. In this works, we present an effective and automated approach for assessing the quality of in-line documentation using a set of heuristics, targeting both the quality of language and consistency between source code and its comments. Our evaluation is made up of three parts: We first apply the JavadocMiner tool to the different modules of two open source applications (ArgoUML and Eclipse) in order to automatically assess their intrinsic comment quality. In the second part of our evaluation, we correlate the results returned by the analysis with bug defects reported for the individual modules in order to examine connections between natural language documentation and source code quality. And finally, we compare the comment quality results generated using our JavadocMiner with the quality assessments performed manually by undergraduate and graduate computer science students.