INTRODUCTION

Non-Functional Testing (NFT) is a highly specialised area of expertise and to be able to clearly identify requirements for a project requires specialised skills.

It is recognised that non-functional testing (known also as NFT) is a highly specialised area of expertise, and to be able to clearly identify requirements for a project and build a strategy to test them requires specialised skills. Sometimes not all the costs and complexities associated with conducting NFT are recognised. In order to help project managers clearly identify their project’s NFT requirements and to have clear perspective on the non-functional testing groups deliverables, a standardised test strategy that centralises NFT expertise is often a good approach.

In this article, we delve into the specialised skills required, the often underestimated costs, and a strategic approach to streamline the process.

THE FOUR PHASES ON NFT STRATEGY

Non-Functional Testing goes beyond the conventional realms of functional testing, requiring a unique set of skills to identify project requirements accurately. The complexity lies not only in the testing itself but in the ability to decipher and articulate the non-functional aspects of a project.

The strategy consists of four separate phases of activity:

1). REQUIREMENTS GATHERING

• Senior non-functional experts conduct thorough meetings with project stakeholders, technical architects, DBAs, and business analysts.

• Information is gathered from project documentation and design plans.

• A proof-of-concept (PoC) is executed using automated NFT tools to validate gathered information.

2). RESOURCING

• Allocate the necessary resources based on the requirements gathered in the first phase.

3). TEST STRATEGY DEVELOPMENT

• Craft a comprehensive NFT strategy inclusive of deliverables, timescales, resourcing needs, and test environment requirements.

4). TEST DEVELOPMENT, EXECUTION AND REPORTING STAGES

• Execute the NFT strategy, monitor results, and report findings.

REQUIREMENTS GATHERING

In the Requirements Gathering phase, senior non-functional experts gather information from project documentation as well as a series of meetings, which could include the following:

• A high-level meeting with the project manager to qualify their requirements.

• A series of meetings with the project groups technical architects.

• A series of meetings with DBA’s and business analysts.

• Review of the design documentation and system testing test plans.

A streamlined approach for gathering the information whilst maintaining the integrity of the requirements gathering is as follows :

1). REQUIREMENT QUESTIONNAIRE

For each project to be non-functionally tested, the project manager is required to complete a Requirements Questionnaire. This will clarify, qualify and quantify any requirements for non-functional testing. The NFT experts review the completed Requirements Questionnaire and where applicable, confirm that a requirement for non-functional testing exists.

2). TECHNICAL QUESTIONNAIRE

A second Technical Questionnaire is passed to the projects technical architect for completion. This questionnaire will be used to gather and consolidate the technical requirements for non-functional testing.

3). PROOF OF CONCEPT

This is followed by a proof-of-concept exercise (also known as a PoC) using an appropriate automated non-functional performance testing tool. From the information gathered, from both the meetings and the proof-of-concept, a NFT strategy would be produced and presented to the project group. This NFT strategy ideally includes clearly defined deliverables, timescales, resourcing needs and test environment requirements. This is a robust method for engaging with a non-functional test requirement but is demanding on both time and resources.

4). TEST STRATEGY DEVELOPMENT 

A test strategy is then formulated based on the information gathered.

5). REVIEW AND REFINEMENT

The NFT strategy is delivered to the project manager for review and refinement as required. The NFT Strategy will provide the project manager and non-functional testers with definitive objectives and deliverables.

COMPONENTS OF AN EFFECTIVE NFT STRATEGY

A well-crafted NFT strategy should encompass:

• A volumetric model against the test scenarios is developed and the non-functional results are measured. In essence, to run an effective non-functional performance test, we need to know what type of traffic we need to simulate to properly performance test the application.

• The requirements for the non-functional test lab to produce test scripts, inject the load and monitor servers under test.

• A definition of the in-scope activities.

• A definition of the out-of-scope activities.

• Details of the scripts and load scenarios.

• Details of required user access information for monitoring sub-system components such as web, application and database servers.

• Details of the data required for each virtual user to work on the application under test, such as user ids and passwords.

• Details on any supporting application data that is required to be built prior to or during any testing activities.

• Details of responsibilities for each group involved in the non-functional testing effort.

• Details of resource, timelines, costs and deliverables.

• A schedule of test execution including testing and tuning cycles.

CONCLUSION

In the intricate world of Non-Functional Testing, expertise is key. A standardized approach ensures clarity, efficiency, and a holistic understanding of project requirements. By navigating through the phases and streamlining the process, project managers and non-functional testers can embark on a journey that leads to reliable results and optimal system performance.