INTRODUCTION

Incorporating non-functional test (or indeed any type of testing) early in the software development lifecycle is a great idea, as the earlier in the development cycle defects and issues are found and remediated, the lower the cost of that remediation is, as issues would not have had less time to bed and become an integral part of the application.

WHAT IS NON-FUNCTIONAL TESTING ?

When we describe non-functional testing, we are referring to such aspects as performance, reliability, scalability, security, rather than functional testing, which verifies that the software functionally does what is expected.

How would we go about incorporating non-functional testing early in the software development cycle ?

DEFINE NON-FUNCTIONAL REQUIREMENTS EARLY

At the start of the project, we should define non-functional requirements as well as functional requirements. In order to be able to test, we are going to need some sort of indication / definition of the sort of system behaviour which would be acceptable in terms of aspects such as response times, memory utilisation, latency, acceptable error rates and so on.

It is however not common that non-functional requirements are initially defined at the start of the project, and such requirements tend to be retrofitted at a later date depending on emerging and changing customer requirements into non-functional aspects.

INVOLVE / EDUCATE STAKEHOLDERS ABOUT NON-FUNCTIONAL TESTING

By initially involving all stakeholders, including performance engineers, security experts and business analysts in the requirement gathering process the requirements are known from the beginning and the system can be designed with clear non-functional objectives which will not need to be reworked at a later date.

From the outset it is important all non-functional requirements are well understood by all stakeholders, especially everyone in the development and testing process, and the impact of failing these requirements is communicated and realised.

Educate wider project staff about the impact of non-functional issues can have overall success of a project.

INCORPORATE AUTOMATED NON-FUNCTIONAL TESTING INTO CI/CD CYCLES

Incorporate non-functional testing tasks straight into the overall development process.

Performance testing is often executed as part of the continuous integration and continuous delivery (CI/CD) pipeline, which can give swift feedback on the performance of any recent code changes. When choosing the performance testing tool, you should check the compatibility of the tool when used with the CI/CD tool you are using (e.g. Jenkins, TeamCity, etc...)

Implement a monitoring solution in the CI/CD pipeline (using tools such as DataDog, New Relic, AppDynamics, etc...) to monitor the performance of the deployed application in real-time and send alerts when issues or performance drops below the required standard. Can your performance testing tool integrate with a monitoring / test reporting tool ?

Security testing tools, such as SAST or Static Application Security Testing tools can find potential security defects and vulnerabilities simply by scanning code without needing to deploy the application. As such, they can also be run as part of CI/CD cycles.

Performance testing results from the CI/CD process must be interpreted and presented in a format that can be understood by all stakeholders in the team.

CONCLUSION

By including non-functional testing early in the software development cycle, teams can accelerate non-functional testing and the production of quality releases and identify and remediate non-functional issues early before they manifest into well integrated and therefore hard to remove issues.

‍