Assessing the quality factors found in in-line documentation written in natural language: The JavadocMiner

An important software engineering artefact used by developers and maintainers to assist in software comprehension and maintenance is source code documentation. It provides the insight needed by software engineers when performing a task, and therefore ensuring the quality of this documentation is extremely important. In-line documentation is at the forefront of explaining a programmer’s original intentions for a given implementation. Since this documentation is written in natural language, ensuring its quality so far needed to be performed manually. In this paper, we present an effective and automated approach for assessing the quality of in-line documentation using a set of heuristics, targeting both the quality of language and consistency between source code and its comments. Our evaluation is made up of two parts: We first apply the JavadocMiner tool to the different modules of two open source applications (ArgoUML and Eclipse) in order to automatically assess their intrinsic comment quality. In the second part of our evaluation, we correlate the results returned by the analysis with bug defects reported for the individual modules in order to examine connections between natural language documentation and source code quality.