Performance Methodology

The phases of the performance testing lifecycle are:

Requirements Analysis

The MindTree performance test team will work along with Client to define the performance testing requirements in detail. This will include:

• Documenting the performance requirements and definition of benchmark tests
• Definition of load tests, stress tests and soak tests
• Prepare an inventory of business scenarios comprising the workloads to be tested
• Prepare an Inventory of system transactions within the business scenarios to be executed and response times measured
• Prepare a description of analysis and reporting requirements
• These requirements will then be used as a basis for performance testing after review and signoff from the client

Test Planning

The main objective of this phase is to organize, manage and schedule the load scenario creation, volume test data creation and the scenario scripting effort. The plan is based on the performance testing strategy. The plan identifies the tasks, the transactions/business scenarios to be included in the load scenarios, the types of tests to be carried out and the contingency plans for the identified risks, the schedule, roles and responsibilities and the deliverables from the MindTree performance testing team. The infrastructure required to carry out the tests is identified.

Tool Deployment and Configuration

The performance-testing tool identified for this engagement is deployed in the performance test environment and configured.

Scenario Design and scripting

Scripting the performance test scenarios and preparation of the test database for test execution are the major activities in this phase. This phase starts after the system-testing phase is completed and the application is stable. Scripts are created for the business scenarios/transactions that are identified to be a part of the various workload definitions. Script development is done incrementally as follows:

• Each test script is prepared and tested in isolation to debug it
• The scripts undergo peer reviews to ensure conformance to standards and conventions
• Scripts are integrated into the development version of the workload (performance test scenario) and the workload is executed to test that the new script is compatible
• As the workload grows, the developing test framework is continually refined, debugged, and made more reliable

Test database setup

MindTree team typically uses a production like database for the actual performance test execution. A sanitized version of the production database can be used if the production database is confidential and cannot be used as is. To simulate the database size after a certain period of usage (6 months, 1 year for e.g.), additional test data may have to be created. MindTree will create such data if required based on the information provided by the development and db teams. The test database thus created is baseline for test execution. A backup of the test database is taken to enable restoration of the database between tests/test execution cycles as appropriate.

Dry Run

When the last script is integrated into the workload scenario, the test is executed as a ‘dry run' to ensure it is completely repeatable and reliable, and ready for the formal tests. The dry run can sometimes expose performance problems and provide an early indication of potential bottlenecks when the tests are scaled up.

Test Execution

The MindTree test team executes the identified workload scenarios in this phase. The execution of formal performance tests requires co-ordination between various teams (support, DB, etc.). The captured results/data is archived securely before the start of the next workload scenario. The environment support team is on call for the duration of performance test execution to resolve any issues that may occur during the course of testing. This could involve activities like a) restoring test databases b) uploading test data into the databases c) restarting applications/systems that may fail during the testing activity d) restarting web server and app servers and database servers if required e) resolve network problems f) starting the performance monitoring agents etc.

Result Analysis and Reporting

MindTree team analyses the data captured by the performance testing tool in this phase. In addition to the transaction response time statistics, throughput, transaction rates, database, network, and resource usage data are also captured and analyzed. For each workload scenario, the performance data will be analyzed and the data along with the reports will be delivered to client/development team. Performance tests will be re-executed once the bottlenecks are fixed.

Sign Off

Client will sign off on all the deliverables of performance testing (workload scenario descriptions, test scripts, performance test result data, performance test reports, test summary reports, etc.) on completion of the performance testing phase.

Closure

The closure phase is the final phase of the project. The best practices and learnings from the project will be summarized and shared with the peer group and customer; all the artifacts of the project will be archived and the project formally closed.