Code coverage as defined by Wikipedia refers to a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs. Thus, serving as a metric to track the percentage of code lines having a corresponding test to validate its functionality. While code coverage itself is not a self sufficient metric and may not always indicate a healthy or well tested product, it is still a quick way to get a sense of the quality of a product.
For every new feature release, we want to check the code coverage numbers for each of the micro service and ensure it meets the minimum code coverage criteria set by the team. And this meant going through multiple code coverage reports by navigating to multiple Jenkins jobs and checking if each of these meet the criteria. We also wanted the flexibility to access the latest code coverage numbers across products at any given time.
We built a central dashboard that can fetch the code coverage numbers for each of the micro services and display it at product levels. This gave us a single view which made monitoring the quality for each of the services easy.
We wanted the dashboard to be color coded to indicate how close a service is in meeting its code coverage criteria. Instead of reinventing, we took advantage of an existing open source dashboard framework named Dashing. It supports many different types of widgets which gave us the flexibility in the visualization of our data as well as in ensuring we always see the latest numbers by scheduling frequency dashboard data updates.
Once we had the dashboard framework finalized, we started working on getting the data that matters to us on it. In this case, it was the code coverage numbers from Jenkins via published HTML reports. As our services are developed using different programming languages, we are using different code coverage tools. We built parsers to extract the code coverage numbers from each of these different formatted code coverage reports.
We currently support parsers for the following code coverage reports.
Scoverage - Scala code coverage tool with statement based coverage details
We now have this code coverage dashboard enabled for multiple products across Ooyala. It serves as the central place to view the quality status for each of the products and their corresponding micro services. It is actively being used as a pre deploy check for many of our services.