Performance Test Handicap Analysis

The Challenge & History

About a decade ago I was asked by a company to oversee a set of performance tests at a second company, one which the first company was in the process of valuing for acquisition purposes. The outcome of the performance tests would help to establish the value of the company for the acquisition. This seemed to be a straightforward proposition: Arrive, oversee a testing effort to ensure that best practices were followed and then report on my observations.

What I found when I arrived was anything but straightforward. The second company was clearly attempting to "cook the test" to show much better results than would occur in production and thus leading to a higher valuation for the company. All sorts of bad practices were in play, from process issues to significant breaks between the production and the test environments in both hardware and software. My challenge was to find an objective method to convey the risk and quality associated with the test results to my client

I began to look for an established process which I could borrow from. I required a method with particular characteristics:

  • Objective:Any two people could arrive at the same conclusion
  • Easy to follow for business and technical personnel
  • Allowed one effort|test to be compared to another in the form of benchmark of quality
  • Incorporated process elements as well as technical elements
  • The Solution to the Challenge