The term "integrity testing," as it is used here is a euphemism for stress testing. It is used in preference to the term "stress testing" because it seems to better convey the reason for using this type of testing.

If you are trying to field a system with a client-server based architecture, and the server just doesn't want to behave, you can benefit from integrity testing.

It is a well known, but frequently under-appreciated fact that computers and the software that runs on them are essentially non-deterministic systems. Inputs from keyboards, system devices and external peripherals typically arrive at random intervals (your software could scan your keyboard periodically, but it has no control over when a user will press a key). Some of these systems respond to asynchronous interrupts, which may be stacked many layers deep. With modern operating systems, such as Unix, Solaris, Linux, Windows/95/98/NT/2000/XP, VMS, etc., the programming environment not only supports multiple processes running simultaneously, but multiple threads within processes. This is done in an attempt to use CPU cycles in a more efficient manner in order to provide ever-increasing levels of performance.

This leads to significant problems. As parallelism is increased, complexity skyrockets, and crashes occur as a result of unanticipated conditions occurring at random. Exacerbating this situation are the likelihood of hidden flaws in the operating system and the programming libraries provided by third parties. When one of these flaws pops up, it is difficult to diagnose and even more difficult to correct. It isn't acceptable to throw up one's hands at this point and say, "Oh well, it just doesn't work."

This is where integrity testing comes in. This is an approach to creating software reliability in complex systems which Mr. Barchi pioneered at Lockheed Martin Corporation. It involves the following procedures:

• The software to be evaluated is examined to determine if it is a good candidate for integrity testing (many simpler applications aren't). The software architecture is examined for potential weaknesses and a test approach is designed to exploit these in an effort to drive out hidden problems. Typically, the types of problems that the integrity test focuses on are:
  
  
  • Boundary limit problems
    
  • Race conditions
    
  • Memory leaks
    
  • Deadlocks
    

    Boundary limit problems occur when system resources are temporarily exhausted due to overloading: memory queues fill up, system heap is used up, communications traffic uses 100% of available bandwidth, etc.
    
    Race conditions occur when two or more processes compete for the same resource, set a flag, etc., and implicit assumptions that have been made about the possible order of events aren't necessarily true.
    
    Memory leaks occur when software doesn't properly free up dynamically allocated memory once it is done with it. There are several excellent tools available, such as Rational's "Purify" which are designed to detect memory leaks. However, they aren't always practical to use due to the amount of overhead they impose.
    
    Deadlocks occur when two or more processes need something from each other to continue processing but none of them can proceed.

• AJB works with the software developers to create an integrity test application which focuses on key functionality of the product and stresses it to the point where it fails (Mr. Barchi has been doing this for years and has never encountered a complex software product that he couldn't break)
  
  
• AJB works with the software developers to analyze the problems which are uncovered and recommend solutions where appropriate (not all problems need to be fixed).
  
  
• As the behavior of the product when it is under stress becomes better understood, AJB works with the software developers to expand the test to explore newly identified potential weaknesses.
  
  
• As the product reaches the desired level of stability, the client may then get further value from the integrity test application by using it for performance testing and regression testing.
  

The experience gained from integrity testing helps software developers to better understand the characteristics of complex systems and to architect more reliable systems in the future.