White Box Testing

White Box Testing

White box testing assumes that the tester can take a look at the code for the application block and create test cases that look for any potential failure scenarios. During white box testing, you analyze the code of the application block and prepare test cases for testing the functionality to ensure that the class is behaving in accordance with the specifications and testing for robustness.

Input

The following input is required for white box testing:

• Requirements
• Functional specifications
• High-level design documents
• Detailed design documents
• Application block source code

White Box Testing Steps

The white box testing process for an application block is shown in Figure 6.2.

Figure 6.2. White box testing process

White box testing involves the following steps:

1. Create test plans. Identify all white box test scenarios and prioritize them.
2. Profile the application block. This step involves studying the code at run time to understand the resource utilization, time spent by various methods and operations, areas in code that are not accessed, and so on.
3. Test the internal subroutines. This step ensures that the subroutines or the nonpublic interfaces can handle all types of data appropriately.
4. Test loops and conditional statements. This step focuses on testing the loops and conditional statements for accuracy and efficiency for different data inputs.
5. Perform security testing. White box security testing helps you understand possible security loopholes by looking at the way the code handles security.

The next sections describe each of these steps.

Step 1: Create Test Plans

The test plans for white box testing can be created only after a reasonably stable build of the application block is available. The creation of test plans involves extensive code review and input from design review and black box testing. The test plans for white box testing include the following:

• Profiling, including code coverage, resource utilization, and resource leaks
• Testing internal subroutines for integrity and consistency in data processing
• Loop testing; test simple, concatenated, nested, and unstructured loops
• Conditional statements, such as simple expressions, compound expressions, and expressions that evaluate to Boolean.

1. For more information about creating test cases, see Chapter 3, "Testing Process for Application Blocks."

Step 2: Profile the Application Block

Profiling allows you to monitor the behavior of a particular code path at run time when the code is being executed. Profiling includes the following tests:

• Code coverage. Code coverage testing ensures that every line of code is executed at least once during testing. You must develop test cases in a way that ensures the entire execution tree is tested at least once. To ensure that each statement is executed once, test cases should be based on the control structure in the code and the sequence diagrams from the design documents. The control structures in the code consist of various conditions as follows:
  • Various conditional statements that branch into different code paths. For example, a Boolean variable that evaluates to "false" or "true" can execute different code paths. There can be other compound conditions with multiple conditions, Boolean operators, and bit-wise comparisons.
  • Various types of loops, such as simple loops, concatenated loops, and nested loops.
  There are various tools available for code coverage testing, but you still need to execute the test cases. The tools identify the code that has been executed during the testing. In this way, you can identify the redundant code that never gets executed. This code may be left over from a previous version of the functionality or may signify a partially implemented functionality or dead code that never gets called.
  Tables 6.3 and 6.4 list sample test cases for testing the code coverage of ConfigurationManager class of the CMAB.
  Table 6.3: The CMAB Test Case Document for Testing the Code Coverage for InitAllProvider Method and All Invoked Methods

Scenario 1.3	Test the code coverage for the method InitAllProviders()in ConfigurationManager class.
Priority	High
Execution details	Create a sample application for reading configuration data from a data store through the CMAB. Run the application under the following conditions: With a default section present Without a default section Trace the code coverage using an automated tool. Report any code not being called in InitAllProviders().
Tools required	Custom test harness integrating the application block for reading configuration data.
Expected results	The entire code for InitAllProviders()method and all the invoked methods should be covered under the preceding conditions.

Table 6.4: The CMAB Test Case Document for Testing the Code Coverage for Read Method and All Invoked Methods

Scenario 1.4	Test the code coverage for the method Read (sectionName) in the ConfigurationManager class.
Priority	High
Execution details	Create a sample application for reading configuration data from SQL database through the CMAB. Run the application under the following conditions: Give a null section name or a section name of zero length to the Read method. Read a section whose name is not mentioned in the App.config or Web.config files. Read a configuration section that has cache enabled. Read a configuration section that has cache disabled. Read a configuration section successfully with the cache disabled, and then disconnect the database and read the section again. Read a configuration section with the section having no configuration data in the database. Read the configuration section that does not have provider information mentioned in the App.config or Web.config files. Trace the code coverage. Report any code left not being covered in the Read (sectionName) method.
Tools required	Custom test harness integrating the application block for reading of configuration data.
Expected results	The entire code for the Read (sectionName) method and the invoked methods should be covered under the preceding conditions.

• Memory allocation pattern. You can profile the memory allocation pattern of the application block by using code profiling tools. You need to check for the following in the allocation pattern:
  • The percentage of allocations in Gen 0, Gen 1, and Gen 2. If the percentage of objects in Gen 2 is high, the resource cleanup in the application block is not efficient and there are memory leaks. This probably means the objects are held up longer than required (this may be expected in some scenarios). Profiling the application blocks gives you an idea of the type of objects that are being promoted to Gen 2 of the heap. You can then focus on analyzing the culprit code snippet and rectify the problem.
    An efficient allocation pattern should have most of the allocations in Gen 0 and Gen 1 over a period of time.
    There might be certain objects, such as a pinned pool of reusable buffers used for I/O work, that are promoted to Gen 2 when the application starts. The faster this pool of buffers gets promoted to Gen 2, the better.
  • The fragmentation of the heap. The heap fragmentation happens most often in scenarios where the objects are pinned and cannot be moved. The memory cannot be efficiently compacted around these objects. The longer these objects are pinned, the greater the chances of heap fragmentation. As mentioned earlier, there might be a pool of buffers that needs to be used for I/O calls. If these objects are initialized when the application starts, they quickly move the Gen 2, where the overhead of heap allocation is largely removed.
  • "Side effect" allocations. Large number of side effect allocations take place because of some calls in a loop or recursive functions, such as the calls to string-related functions String.ToLower()or concatenation using the + operator happening in a loop. This causes the original string to be discarded and a new string to be allocated for each such operation. These operations in a loop may cause significant increase in memory consumption.
  You can also analyze memory leaks by using debugging tools, such as WinDbg from the Windows Resource Kit. Using these tools, you can analyze the heap allocations for the process.
  
  
• Cost of serialization. There may be certain scenarios when the application block needs to serialize and transmit data across processes or computers. Serializing data involves memory overhead that can be quite significant, depending on the amount of data and the type of serializer or formatter used for serialization. You need to instrument your code to take the snapshots of memory utilized by the garbage collector before and after serialization.
  
  
• Contention and deadlock issues. Contention and deadlock issues mostly surface under high load conditions. The input from load testing (during black box testing) give you information about the potential execution paths where contention and deadlocks issues are suspected. For example, in the case of the CMAB, you may suspect a deadlock if you see the requests timing out when trying to update a particular information in the persistent medium.
  You need to analyze these issues with invasive profiling techniques, such as using WindDbg tool, in the production environment on a live process or by analyzing the stack dumps of the process.
  
  
• Time taken for executing a code path. For scenarios where performance is critical, you can profile the time they take. Timing a code path may require custom instrumentation of the appropriate code. There are also various tools available that help you measure the time it takes for a particular scenario to execute by automatically creating instrumented assemblies of the application block. The profiling for time taken may be for complete execution of a usage scenario, an internal function, or even a particular loop within a function.
  
  
• Profiling for excessive resource utilization. The input from a performance test may show excessive resource utilization, such as CPU, memory, disk I/O, or network I/O, for a particular usage scenario. But you may need to profile the code to track the piece of code that is blocking resources disproportionately. This might be an expected behavior for a particular scenario in some circumstances. For example, an empty while loop may pump up the processor utilization significantly and is something you should track and rectify; whereas, a computational logic that involves complex calculations may genuinely warrant high processor utilization.

Step 3: Test the Internal Subroutines

Thoroughly test all internal subroutines for every type of input. The subroutines that are internally called by the public API to process the input may be working as expected for the expected input types. However, after a thorough code review, you may notice that there are some expressions that may fail for certain types of input. This warrants the testing of internal methods and subroutines by developing NUnit tests for internal functions after a thorough code review. Following are some examples of potential pitfalls:

• The code analysis reveals that the function may fail for a certain input value. For example, a function expecting numeric input may fail for an input value of 0.
• In the case of the CMAB, the function reads information from the cache. The function returns the information appropriately if the cache is not empty. However, if during the process of reading, the cache is flushed or refreshed, the function may fail.
• The function may be reading values in a buffer before returning them to the client. Certain input values might result in a buffer overflow and loss of data.
• The subroutine does not handle an exception where the remote call to a database is not successful. For example, in the CMAB, if the function is trying the update the SQL Server information but the SQL Server database is not available, it does not log the application in the appropriate event sink.

Step 4: Test Loops and Conditional Statements

The application block may contain various types of loops, such as simple, nested, concatenated, and unstructured loops. Although unstructured loops require redesigning, the other types of loops require extensive testing for various inputs. Loops are critical to the application block performance because they magnify seemingly trivial problems by iterating through the loop multiple times.

Some of the common errors could cause the loop to execute infinite times. This could result in excessive CPU or memory utilization resulting in the application failing. Therefore, all loops in the application block should be tested for the following conditions:

• Provide input that results in executing the loop zero times. This can be achieved where the input to the lower bound value of the loop is less than the upper bound value.
• Provide input that results in executing the loop one time. This can be achieved where the lower bound value and upper bound value are the same.
• Provide input that results in executing the loop a specified number of times within a specific range.
• Provide input that the loop might iterate n, n-1, and n+1 times. The out-of-bound loops (n-1 and n+1) are very difficult to detect with a simple code review; therefore, there is a need to execute special test cases that can simulate such cases.

When testing nested loops, you can start by testing the innermost loop, with all other loops set to iterate a minimum number of times. After the innermost loop is tested, you can set it to iterate a minimum number of times, and then test the outermost loop as if it was a simple loop.

Also, all of the conditional statements should be completely tested. The process of conditional testing ensures that the controlling expressions have been exercised during testing by presenting the evaluating expression with a set of input values. The input values ensure that all possible outcomes of the expressions are tested for expected output. The conditional statements can be a relational expression, a simple condition, a compound condition, or a Boolean expression.

Step 5: Perform Security Testing

White box security testing focuses on identifying test scenarios and testing based on knowledge of implementation details. During code reviews, you can identify areas in code that validate data, handle data, access resources, or perform privileged operations. Test cases can be developed to test all such areas. Following are some examples:

• Validation techniques can be tested by passing negative value, null value, and so on, to make sure the proper error message displays.
• If the application block handles sensitive data and uses cryptography, then based on knowledge from code reviews, test cases can be developed to validate the encryption technique or cryptography methods.