Continuous Integration – CI Process

In recent times, during application development there are frequent code check-in, build releases, and integration happening every day. Needless to say that it becomes necessary to perform unit tests and functional tests to identify issues and debug them quickly for every new build instead of identifying them in the actual production environment. There are tools in the market like Jenkins, TeamCity, and Bamboo which help developers continuously integrate new or changed code into a shared repository and also verify it with an automated build.

The typical CI process includes the following steps:

 

 

The CI process runs all the tests continuously, enabling teams to detect errors quickly on every pipeline build. The CI tools also facilitate this in an automated fashion helping run tests every day, every hour or on every developer check-in.

Importance of Performance Testing in the CI pipeline

It is widely known what all happens when an application has poor performance in the production. Application performance has direct impact on retaining the users and making revenue and we can’t compromise on either. Therefore it is very important to identify performance bottlenecks in the cycle and address them at the earliest. Most of the development projects push the load and performance testing until the end of the development cycle. There is little or no time left for the developers to make changes. This leads to push back on release dates and delay in getting the new application features out of the door. This might also cause the teams to revisit the application architecture for any changes required which might even take weeks to implement.

Including load and performance testing in the CI pipeline will help catch the performance issues at early stages in the cycle, thereby saving cost. Developers can instantly know whether the new feature in the latest build disrupted the application to meet the Service level agreements. It is mandatory to fix the problem then and there before it becomes exponentially more expensive.

1.Adding performance tests on CI pipeline

Unlike unit tests, we can’t afford to run the load/performance on every developer check-in. We will end up spending a lot of money on resources (load servers, maintenance costs etc.) which will be standing idle most of the time. There are two efficient ways to add performance test on the CI pipeline:

a)Profiling Joba Running a small subset of performance/load test on every developer check-in which sits inside the CI process.

b) It should run parallel with the CI process at a certain stage, because we can’t afford the time consumed by the job even though it’s considerably minimal.

 

2.Nightly performance job

a) Running a complete set of load/performance tests using the CI tools

b) Nightly job runs once every day during the specified time, which stays out of the CI process but stays inside the CD pipeline

 

3.Setting Performance testing Thresholds

The last important step is to set performance testing thresholds. Performance thresholds are the maximum acceptable values for the metrics identified for the project, usually specified in the terms of response time, throughput (transaction per second), and resource utilization levels. Performance thresholds typically equate to requirements, hence it is necessary to baseline the threshold at a very early stage. This means that we can configure what is acceptable for our test. If it doesn’t pass the thresholds that we have set, the build will fail.

Latest posts by Murali Murugan (see all)