Project Metrics

To gage the readiness of a software testing project for release, metrics collected on the project level must answer the following questions :

• Is the project on schedule and within cost?
• Are the customer and vendor confident in the test results?
• Does the quality of the software meet its release requirements?

Time & Cost

Schedule and Cost Variance (SV and CV)

Completing projects on time and within budget is often a key concern when outsourcing. Monitoring SV and CV will help a customer track the vendor's progress by measuring project performance against the mutually established schedule and budget.  If a deviation occurs, project managers can use this metric to help identify the root cause of issues inhibiting progress.

Confidence in Test Results

Measuring the Creation and Execution of Requirement Based Tests

In order to have confidence in a testing team’s results, test cases should be developed and executed to cover 100% of the requirements established by the customer.  Any less than 100% coverage may lead to defects after the software is released. The graphic above gives a view of how to track the testing process using the following questions:

• What percentage of requirements have test cases developed?
• What percentage of requirements are verified?

Setting a Defect Detection Goal

The number of defects in a software program? previous version can be used to predict the expected number of defects in future releases. The chart above shows the defect detection target based on the average number of defects in the past releases. This can be used to plan how many defects the test team should find as the project progresses. It also is an effective measurement to determine when enough testing has been completed. If the project manager notices a significant decrease in number of defects found compared to historical data, there are two possible causes: the quality of the development has improved and/or the test team has not found all of the program? defects.

Quality Meets Release Requirements

Measuring the Requirement Pass Rate

Pass rate analysis gives project managers an executive summary of all of the testing that has been completed. This metric will assist project managers in determining if a project is ready for release and help them prioritize ongoing development efforts. If a 100% pass rate means all requirements are validated, the example above shows that a project manager’s first priority would be to improve the performance of module-3, since it is used by most staff on a daily basis and is only 70% complete.