The Solution: Maximize Testing Productivity

To manage the challenges inherent to testing environments and testing automation products, organizations must think about testing strategically. Organizations simply can no longer afford to engineer or reengineer automation frameworks for every new application.

Interestingly, most testing-only service providers have started to address these issues before their clients. Such providers are re-shaping their business models to go beyond easy-to-deploy tools and in addition are discussing business processes that can address the failures of rigid testing approaches. Nonetheless, beyond creating comprehensive “how-to” documents and application maps, many providers have not been able to resolve the key issues with automated testing environments.

Despite the complexity of all of these factors, the solution to this industry-wide dilemma resides in strategies that will maximize testing productivity and create more effective testing teams. Such solutions ultimately will produce the development of a single framework that permits growth, flexibility and continuous improvement with each application, as well as organizations that think beyond technology. With these goals in mind, organizations can achieve maximum productivity if they examine and implement the following four key aspects of testing:

1. Structural Discipline at a Process Level
To help maximize productivity, organizations must create structural frameworks that are driven by process comprehensiveness rather than technologies or platforms. This strategy should include:

• Quick turnaround for products with frequent version releases
• Relevance of test scenarios during transposition of products across platforms
• Ability to move programming languages without compromising testing rigor
• Reusing components and viewing them as services rather than programs or code
• Accommodation of user-defined granularity of execution controls

It also is important to be able to import modules within an application in production, either online or during scheduled run-times, without compromising data or interface integrity. This can be achieved through adopting process discipline frameworks such as CMMi, which can be useful in creating the necessary ethos.

From test strategy creation to measuring effectiveness, controlling outcomes in production environments is crucial for minimal performance loss. While this may not be an approach organizations can move towards easily, working with providers who have developed such capabilities imparts the much-needed flexibility to quickly transform an organization? technology landscape. This approach also enables organizations to spend less time and effort on small pilot projects.

2. Platform Independence
It is not easy for technology organizations to think of strategies that go beyond platform or programming language restrictions. Organizations that have been able to develop execution engines that automate distribution, execution and analysis of test cases have been extremely successful. This platform-independent thinking fosters greater flexibility and testing environments that can be easily deployed and monitored without resorting to the sampling method of testing.

Benefits of platform-independent thinking include:

• Increased dexterity of applications
• Faster releases of new versions
• Quicker testing of integration components
• The least number of bugs and patches in production

Consequently, organizations can shrink maintenance costs while obtaining much-needed flexibility to enhance or upgrade applications more quickly in the future.

3. Business-Impacting Success Measures

Traditionally testing ?manual or automated ?typically has been measured using success criteria as defined by test scripts. While automated testing tools, such as GUI monitoring tools that interact and aid with visibility to progress test jobs, have gone slightly beyond such measures, success measurements have remained within the domain of the technology teams.

It is critical to measure the impact of testing automation processes through a lens of rigorous business metrics. Organizations should measure key indicators such Earned Value Analysis, Post-Release Defect Rates, Root Cause Analysis of Failures, Check for Application Readiness, Defect Seeding & Coverage Analysis, User Impact Analysis and others. Such an approach will ensure that testing automation goes beyond technology and promotes business dexterity in an increasingly complex marketplace.

4. Modification of Standard Global Delivery Models for Testing

While sourcing testing providers, organizations should not deploy the standard sourcing model used for development and maintenance projects. Rather, key evaluation criteria for choosing testing providers should rest on the ability to understand the specific business and deploy testing strategies that are consistent with market demand and user perspectives for the specific product or system.

Evaluating the efficacy of delivery capabilities from both a technical and management standpoint is important. Organizations should confirm that testing scenarios are not diluting effectiveness and should explore sourcing partnerships where testing providers collaborate across a range of products and systems. In this context, many global providers supply sufficient flexibilities to ensure replication is avoided while sharing capabilities are encouraged. For this to occur customer organizations will need to treat their testing providers as key business partners to maximize learning.

Furthermore, testing customers and providers should create a separate governance structure that focuses on creating scenarios across a suite of products that: 1) can be replicated and 2) cannot be replicated. For replicable tests, overall efficiencies can increase through leveraging offshore teams. For non-replicable tests, multiple providers can deliver in a collaborative manner through onshore-offshore teams. Ultimately, this maximizes productivity while minimizing inefficiencies and defects in a post-production environment.