Software Quality and Test Coverage

Test coverage (code coverage) is one of the most important metrics in software quality. It measures what percentage of your codebase is executed by automated tests. However, high test coverage alone doesn’t guarantee high quality — what matters is what and how you measure.

In a Nutshell

Test coverage quantifies the portion of code executed by tests. The main metrics are line coverage, branch coverage, and function coverage. A coverage value of 80% means: 80% of the lines were executed at least once during testing. But this tells you nothing about whether the tests contain the right assertions.

Quick Technical Overview

Code coverage is a measure of how much of a program’s source code is executed by tests. It’s expressed as a percentage and can be measured at different levels: lines, statements, branches, functions, and execution paths. Coverage tools instrument the code during execution and track which code regions are traversed. The metric helps identify untested areas, but it’s not a guarantee of correctness — a test can execute a line without verifying the expected behavior.

Who needs test coverage and why?

Test coverage matters for:

• Developers: to see which code areas lack tests
• Teams: to establish a minimum quality threshold
• QA Engineers: to identify risk-prone areas
• Project Managers: to assess the maturity of the testing strategy
• Compliance: in regulated industries (medical, automotive, finance), coverage thresholds are often mandatory

Coverage Metrics Explained

Line Coverage

Measures how many lines of source code are executed by tests. The simplest and most widely used metric.

function classify(score) {
  if (score >= 90) return 'A';    // Line 1
  if (score >= 80) return 'B';    // Line 2
  if (score >= 70) return 'C';    // Line 3
  return 'F';                      // Line 4
}

A test with classify(95) covers only line 1 → 25% line coverage. Tests with classify(95) and classify(65) cover lines 1 and 4 → 50%.

Branch Coverage

Measures how many branches (if/else, switch, ternary) are covered by tests. Each branch has two paths: true and false.

function classify(score) {
  if (score >= 90) return 'A';    // Branch 1: true/false
  if (score >= 80) return 'B';    // Branch 2: true/false
  if (score >= 70) return 'C';    // Branch 3: true/false
  return 'F';
}

Six branches total (3× true, 3× false). A test with classify(95) covers branch 1-true but not branch 1-false → 1/6 = 17% branch coverage.

Function Coverage

Measures how many functions in the code are called at least once. Especially important for modules with many small functions.

Path Coverage

Measures how many possible execution paths through the code are covered. The strictest metric because it considers all combinations of branches. With n independent branches, there are 2^n paths — path coverage is often unrealistic for complex code.

Statement Coverage

Similar to line coverage, but refers to individual statements rather than lines. A single line can contain multiple statements.

Real-World Example: Jest with Istanbul Coverage

Setup

// package.json
{
  "scripts": {
    "test": "jest",
    "test:coverage": "jest --coverage"
  },
  "jest": {
    "collectCoverageFrom": [
      "src/**/*.js",
      "!src/**/*.spec.js",
      "!src/index.js"
    ],
    "coverageThreshold": {
      "global": {
        "branches": 80,
        "functions": 80,
        "lines": 80,
        "statements": 80
      }
    }
  }
}

Example Module

// src/utils/discount.js
export function calculateDiscount(price, customerType) {
  if (typeof price !== 'number' || price < 0) {
    throw new Error('Price must be a non-negative number');
  }

  switch (customerType) {
    case 'premium':
      return price * 0.8;   // 20% discount
    case 'vip':
      return price * 0.7;   // 30% discount
    case 'standard':
      return price;          // No discount
    default:
      throw new Error(`Unknown customer type: ${customerType}`);
  }
}

Tests with Complete Branch Coverage

// src/utils/discount.spec.js
import { calculateDiscount } from './discount.js';

describe('calculateDiscount', () => {
  test('premium customer gets 20% discount', () => {
    expect(calculateDiscount(100, 'premium')).toBe(80);
  });

  test('vip customer gets 30% discount', () => {
    expect(calculateDiscount(100, 'vip')).toBe(70);
  });

  test('standard customer gets no discount', () => {
    expect(calculateDiscount(100, 'standard')).toBe(100);
  });

  test('throws on negative price', () => {
    expect(() => calculateDiscount(-1, 'premium'))
      .toThrow('Price must be a non-negative number');
  });

  test('throws on non-number price', () => {
    expect(() => calculateDiscount('100', 'premium'))
      .toThrow('Price must be a non-negative number');
  });

  test('throws on unknown customer type', () => {
    expect(() => calculateDiscount(100, 'unknown'))
      .toThrow('Unknown customer type: unknown');
  });
});

Coverage Report

----------|---------|----------|---------|---------|-------------------
File      | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s
----------|---------|----------|---------|---------|-------------------
All files |     100 |      100 |     100 |     100 |
 discount |     100 |      100 |     100 |     100 |
----------|---------|----------|---------|---------|-------------------

Coverage Goals: What Values Make Sense?

Project Type	Recommended Coverage	Rationale
Library / SDK	90-100%	High stability, many consumers
Web Application	70-80%	Pragmatic, focus on business logic
Prototype / MVP	40-60%	Fast iteration, tests for core logic
Regulated Software	100% Branch	Mandatory (IEC 62304, DO-178C)
Legacy Code	50%+ incrementally	Increase during refactoring

Rule of thumb: 80% line coverage with strong assertions is more valuable than 100% coverage without assertions.

Common Pitfalls

1. 100% Coverage Without Assertions

// Bad: Line is executed but nothing is verified
test('calculateDiscount runs', () => {
  calculateDiscount(100, 'premium'); // No assertion!
});

Coverage: 100%, quality: 0%.

2. Coverage Gaming

Developers write tests only to hit coverage thresholds, without meaningful assertions. Solution: use mutation testing.

3. Overusing Ignore Pragmas

/* istanbul ignore next */
export function complexFunction() { ... }

Every ignore hides untested code. Use it only for platform-specific code or consciously excluded error paths.

4. Relying solely on line coverage

Line coverage is the weakest metric. Branch coverage uncovers untested edge cases that line coverage misses entirely.

Mutation Testing: The next level

Mutation testing systematically modifies your code—for example, changing > to >= or + to -—then checks whether your tests catch the mutants. If a mutant survives, your test is incomplete.

# Stryker Mutation Testing for JavaScript
npx stryker run

Example mutants:

• if (score >= 90) → if (score > 90) — reveals tests that don’t verify the exact boundary at 90
• return price * 0.8 → return price * 0.9 — exposes tests lacking precise assertions
• return 'A' → return 'B' — catches tests that never verify return values

Tools at a glance

Tool	Language	Metrics	Highlights
Istanbul/nyc	JavaScript	Line, Branch, Func	Built into Jest
Coverage.py	Python	Line, Branch	Standard with pytest
JaCoCo	Java	Line, Branch, Method	Standard in Maven/Gradle
gcov	C/C++	Line, Branch	Integrated with GCC
Stryker	JS/TS	Mutation Score	Mutation testing
PIT	Java	Mutation Score	Mutation testing

Exam highlights

• Distinguishing line, branch, path, and function coverage
• Coverage is necessary but not sufficient for quality
• Coverage thresholds in CI/CD pipelines as quality gates
• Mutation testing to complement coverage measurement
• Coverage tools: Istanbul, JaCoCo, Coverage.py, gcov
• 100% coverage without assertions is worthless
• ISO 25010: testability as a quality attribute

FAQ

Next in the Software Quality learning path

The next article in the Software Quality learning path covers Clean Code and SOLID Principles—the foundations for readable, maintainable, and extensible code.

References

1. https://istanbul.js.org/
2. https://stryker-mutator.io/
3. https://jestjs.io/docs/configuration#coveragethreshold-object
4. https://iso25000.com/index.php/en/iso-25000-standards/iso-25010.html

Recommended books on software quality

If you’d like to dive deeper into test coverage, software quality, and testing practices, we recommend the following books: