Performance Testing

Performance Testing is an umbrella term for tests that measure the speed, stability, and scalability of an application. It’s one of the most important non-functional testing disciplines and ensures that software remains performant and stable under realistic load.

In a Nutshell

• Performance Testing measures speed, stability, and scalability.
• Types: Load, Stress, Spike, Endurance, Volume, Scalability.
• Metrics: Response time, Throughput, Error rate, Resource utilization.
• Tools: k6, JMeter, Gatling, Locust.
• Performance Testing is part of Performance Engineering.

What is Performance Testing and why does it matter?

Performance Testing examines how an application behaves under different loads. It answers questions like: How fast does the system respond? How many requests per second can it handle? How stable is it over time? What resources does it consume?

Common goals of Performance Testing include:

• Ensuring user experience: Slow load times drive users away and reduce conversion rates.
• Identifying bottlenecks: Databases, APIs, and algorithms can become chokepoints.
• Validating SLAs: Agreed-upon response times and availability must be met.
• Capacity planning: How many users can the system support? When does it need to scale?
• Checking stability: Does the system run without resource leaks over extended periods?

Types of Performance Tests

Load Testing

Load Testing examines behavior under expected, realistic load. It shows whether the system performs well during normal operation. For example: 1,000 concurrent users with response times under 500ms.

Stress Testing

Stress Testing pushes beyond the expected load. It finds the system’s limits and reveals how it behaves when overloaded. For example: increase load until the system breaks down.

Spike Testing

Spike Testing creates sudden, extreme spikes in load. It simulates, for instance, a sudden rush of traffic when a sale begins. It demonstrates how well the system handles abrupt load changes.

Endurance Testing (Soak Testing)

Endurance Testing runs the application under steady load for an extended period. It’s designed to expose resource leaks—growing memory consumption, connection leaks, or accumulating log files.

Volume Testing

Volume Testing exercises the system with large data sets. It verifies that the database and application remain performant even with millions of records.

Scalability Testing

Scalability Testing examines how well the system scales when additional resources are added. It shows whether horizontal or vertical scaling improves performance.

Key Metrics

Response Time

Response time is the duration from request to complete response. It’s often reported as an average, median, or percentile. The 95th percentile is particularly valuable because it reveals how slow requests are for most users.

Throughput

Throughput indicates how many requests or transactions the system can process within a given time. It’s expressed in requests per second or transactions per minute.

Error Rate

Error rate is the proportion of requests that fail. A sudden spike points to a bottleneck.

Resource Utilization

Resource utilization encompasses CPU usage, memory, network bandwidth, and disk I/O. It helps identify the root cause of performance problems.

Concurrent Users

Concurrent Users are the number of active users sending requests simultaneously. They form the basis of load profiles.

Practical Example

The following example shows a typical Performance Testing plan for a web application. It combines different testing types and demonstrates how metrics and goals work together.

Goals:
- Response time under 200ms
- 95th percentile under 500ms
- Error rate under 0.1 percent
- Throughput 1,000 requests per second

Tests:
- Load Test under normal conditions
- Stress Test to breaking point
- Endurance Test over 24 hours

Why these tests?

• Load Test: Verifies the system meets goals under expected load of 1,000 requests per second. This represents normal operation.
• Stress Test: Finds the system’s limits. What happens at 2,000 or 5,000 requests per second? When does the error rate increase?
• Endurance Test: Does the system remain stable after 24 hours under load? Are there memory or connection leaks?

Metrics are collected continuously during the test and compared against targets. Deviations are analyzed and used to drive optimizations.

Essential Tools

k6

k6 is a modern open-source Performance Testing tool developed by Grafana Labs. Tests are written in JavaScript and can be integrated into CI/CD pipelines.

Apache JMeter

JMeter is one of the most widely-used Performance Testing tools. It provides a graphical interface and supports many protocols including HTTP, SOAP, JDBC, and JMS.

Gatling

Gatling is a powerful tool for writing tests in Scala, Kotlin, or Java. It excels through the high performance of its load generator.

Locust

Locust is a Python-based tool with straightforward script authoring. It’s well-suited for teams already using Python.

Advantages and Disadvantages

Advantages	Disadvantages
Better user experience through optimized response times	Complex test environment setup
Early detection of bottlenecks	Production-like environment required
SLA validation and capacity planning	Effort needed for test data and scenarios
Objective performance measurement	Tools and infrastructure have associated costs
Reduced production outages	Result interpretation requires experience
Foundation for scaling decisions	Tests must be run regularly

Best Practices

• Define goals: Establish clear SLAs and metrics before testing begins.
• Production-like environment: Test environments should mirror production.
• Realistic test data: Use anonymized production data.
• Monitoring: Capture server and application metrics throughout the test.
• Run regularly: Repeat Performance Tests whenever relevant changes occur.
• Integrate with CI/CD: Execute Performance Tests automatically during releases.
• Analyze bottlenecks: Don’t just measure—investigate root causes.

Key exam topics

• Performance Testing: The umbrella term for tests that measure speed, stability, and scalability.
• Load Testing: Testing under expected, realistic load.
• Stress Testing: Testing beyond expected load limits.
• Spike Testing: Simulating sudden, extreme load spikes.
• Endurance Testing: Extended runs to detect resource leaks.
• Volume Testing: Testing with large data sets.
• Scalability Testing: Testing how the system scales with additional resources.
• Important Metrics: Response time, throughput, error rate, resource utilization, concurrent users.
• Percentiles: The 95th percentile is more meaningful than the average.
• Key Tools: k6, JMeter, Gatling, Locust.
• SLA: Service Level Agreement with agreed performance targets.
• Performance Engineering: A holistic approach to performance that extends beyond testing alone.
• Latency: The delay between a request and the first response.

Primary sources

1. https://www.gatling.io
2. https://k6.io
3. https://en.wikipedia.org/wiki/Performance_testing

Continuing Your Software Testing Learning Path

The next article in our Software Testing Learning Path covers Security Testing — how security testing uncovers vulnerabilities.