If you don't find an answer, please click here to post your question.
Engineering Blog

Code Coverage Dashboard

by Community Manager on ‎04-13-2017 08:13 AM (2,814 Views)

Written by Anupama Shetty 4/13/17

 

Code coverage as defined by Wikipedia refers to a measure used to describe the degree to which the source code of a program is executed when a particular test suite runs. Thus, serving as a metric to track the percentage of code lines having a corresponding test to validate its functionality. While code coverage itself is not a self sufficient metric and may not always indicate a healthy or well tested product, it is still a quick way to get a sense of the quality of a product.

 

At Ooyala, many of our products such as Analytics and Discovery are built using many micro services. As such, it is important for us to ensure each of these services are well tested for every new product release. We use code coverage metric as one of the basic metric to ensure these services are well tested by setting a minimum of 70% code coverage at unit test level. As each of these services were built for a specific need and requirement, they were developed using different programming languages such as Scala, Golang, Javascript or Ruby. Most of the programming languages used today, offer a code coverage tool that can be easily enabled for your code repository. We use Jenkins as our continuous integration environment and have jobs setup to trigger unit test runs for every new code change. These jobs will generate and publish the code coverage report showing the latest code coverage numbers for every Jenkins test run.

 

Problem

 

For every new feature release, we want to check the code coverage numbers for each of the micro service and ensure it meets the minimum code coverage criteria set by the team. And this meant going through multiple code coverage reports by navigating to multiple Jenkins jobs and checking if each of these meet the criteria. We also wanted the flexibility to access the latest code coverage numbers across products at any given time.

 

 

Solution

 

We built a central dashboard that can fetch the code coverage numbers for each of the micro services and display it at product levels. This gave us a single view which made monitoring the quality for each of the services easy.

 

 

Dashboard

 

We wanted the dashboard to be color coded to indicate how close a service is in meeting its code coverage criteria. Instead of reinventing, we took advantage of an existing open source dashboard framework named Dashing. It supports many different types of widgets which gave us the flexibility in the visualization of our data as well as in ensuring we always see the latest numbers by scheduling frequency dashboard data updates.

 

blog_image.png

 

 

Parsers

 

Once we had the dashboard framework finalized, we started working on getting the data that matters to us on it. In this case, it was the code coverage numbers from Jenkins via published HTML reports. As our services are developed using different programming languages, we are using different code coverage tools. We built parsers to extract the code coverage numbers from each of these different formatted code coverage reports.

We currently support parsers for the following code coverage reports.

  1. Scoverage - Scala code coverage tool with statement based coverage details

  2. Jacoco - Scala code coverage tool

  3. Scct - Scala code coverage tool

  4. Gocov - Golang code coverage tool

  5. Istanbul - Javascript code coverage tool that fetches coverage from jasmine based specs.

  6. Simplecov - Ruby code coverage Tool

 

Summary

 

We now have this code coverage dashboard enabled for multiple products across Ooyala. It serves as the central place to view the quality status for each of the products and their corresponding micro services. It is actively being used as a pre deploy check for many of our services.