Testing is not just an accessory service for IT departments to verify the quality of development delivery. Testing is an important capacity that every IT department must build in order to improve overall organizational productivity and provide better support to the business.

For example, we have a Wall Street-based finance client that has an in-house IT team to develop and support their business and IT applications. In recent years, they found their IT testing capability was relatively weak; the testing efficiency and effectiveness were much lower than the industry benchmarks. It wasn’t because their testing team didn’t work hard.

Despite much overtime and other efforts, it still required a long elapsed time to complete a full round of regression testing which increased their time to market.

In addition, they couldn’t avoid bugs escaping to production systems. Of course, there were complaints from business teams. They felt their IT department was in the way of many initiatives.

Something had to change

The client knew something had to change, and approached us to help them build an in-house Test Center of Excellence (CoE), which included the creation of a test framework, selection of proper test tools, re-organizing test processes, building reporting and a governance model.Using a Test Management Tool which is then integrated with Test Complete, a Test Driver Script is used to create the individual test functions. This separation allows for greater flexibility.

After an initial analysis of GUI-Server application, an order management system (OMS), and their back-end pre-trade risk system which sets and monitors firm limits for different asset types, we discovered one of the biggest problems was the time it took to run regression tests. It tied the project schedule down and even impacted the quality.

The longer it took, the higher the possibility of having additional code changes happening on the package delivery to QA. This in turn could cause other issues (e.g. some bug fixes, some small features that the business pushed the developers to add, and some code change done in good faith).

Time for test automation

This told them and us that it was the right time to implement test automation. From the technical side, TestComplete for GUI application automation testing and FitNesse for back-end automation testing was introduced.

Custom Bleum libraries are used to create the FitNesse wiki pages automatically.A TestComplete framework was built and integrated with client’s test case management tool. A framework was also built for the FitNesse and some core libraries (Java) were developed to make auto test case development more efficient (Automation Drivers and Templates). The following diagram shows the high-level design chart for FitNesse framework we built.

Several other resources were also integrated into the client’s IT department to improve the overall testing and operation productivity. This included JIRA, TestRail, Jenkins, GitHub and simulators.

Now the data can be transferred through different systems with minimal manual effort. And test managers now have access to measurable results such as test case review coverage, automation coverage, test script development productivity, auto execution productivity and technical debt amount.

A 90 percent reduction in duration

After the automation testing frameworks were built, the test case automation was implemented which continued to improve the test coverage. The regression test duration for OMS was reduced by around 90 percent compared to the time before with automation framework was implemented. In the back-end system, the new duration was reduced by 99 percent of the previous elapsed time.

As a result, other benefits such as reductions to the overall feature time to market and better use of QA resources were discovered. This also reduced the cost for the development team because the earlier we can report the bugs, the lower the cost to fix them.
Organizations that understand the need for testing and see it as important part of their IT department rather than a nice add-on will see more benefits and reduce the overall cost to fix bugs and other issues.

Over to you

Does your company have robust testing capabilities? Do you think this is an area your organization has under control or an area that could use some work? We would love to hear your perspective in the comments below.