Testing Process

Testing Process

The process for testing application blocks is outlined below. The steps listed should be performed regardless of the testing methodology that is used for the blocks.

Input

The following input is required to test an application block:

• Functional specifications
• Requirements
• Performance objectives
• Deployment scenarios

Steps

Figure 3.1 shows the steps in the testing process for application blocks.

Figure 3.1. The testing process for application blocks

1. Create test plans. Create test plan documentation with a prioritized list of test cases and execution details.
2. Review the design. Review the design to ensure that it addresses all the deployment scenarios and the requirements. Also ensure adherence to guidelines for performance, security, globalization, maintainability, and so on.
3. Review the implementation. Review the code to ensure that various best practices are followed.
4. Perform black box testing. Test the code from an end-user perspective by executing all the end-user scenarios.
5. Perform white box testing. Analyze the code for failure scenarios and simulate them by passing invalid data using custom test harnesses.

Step 1: Create Test Plans

The test plans document the test cases you will use to test the application block. The test cases cover all aspects of testing, including design review, code review, profiling, deployment testing, and load testing. The test plans help to ensure that you test all of the features and usage scenarios of the application block.

The test plan documentation consists of two documents:

• Detailed test plan (DTP) document. The detailed test plan document lists test cases in a prioritized order (High, Medium, and Low). The test case descriptions in this document briefly summarize the usage scenarios and features to be tested. For each test case, you assign a priority level based on the case's importance and its overall impact on the goal of meeting the desired objectives and requirements.
• Detailed test case (DTC) document. The detailed test case document maps to the detailed test plan document. This document describes the steps that the user must perform to execute each test case that is listed in the DTP document. It also lists the data that is required for the test and describes the expected results of the test.

For examples of DTP and DTC documents, see the "Appendix" section later in this chapter. You can use these examples as templates when you create the test plans for your application block.

You must update the DTP and DTC documents throughout the development life cycle. For example, you must update these documents if functional specifications or requirements change, or if you have additional input. You must also update them if you add test cases or if you modify the priority of the existing test cases later in the development cycle to account for additional usage scenarios or functional testing.

Step 2: Review the Design

In this step, you execute the test cases from the DTP that are related to reviewing the design of the application block. The design review is very important from the testing point of view, because you can avoid most of the potential pitfalls (for example, those related to performance or security) in the early stages of a project by thoroughly reviewing the design of the application block. In the later stages, considerable rework may be required in the code to make any design changes.

The design review helps to ensure the following:

• The design meets all of the functional specifications and requirements for the application block.
• The design makes appropriate tradeoffs given the deployment scenarios for the application block. Examples of deployment targets that may have special requirements are mobile devices, the DMZ (also known as demilitarized zone or screened subnet) of a Web farm, and tablet PCs.
• The design ensures that the application block meets all performance objectives.
• The design addresses all the security threats possible in various deployment scenarios for the application block.
• The design adheres to best practices and principles related to coupling and cohesion, concurrency, communication, class design, exception management, resource management, caching, and so on, so that developers can easily extend and customize the application block.
• The design adheres to best practices for globalization and localization.

The design review also helps in identifying the scenarios that must be tested for one or more of the following:

• Possible security attacks
• Performance optimizations
• Profiling to ensure that there are no memory leaks

Note   You must update the DTP and DTC to document these possible scenarios.

More Information

For more information about how to conduct a design review for the application block, see Chapter 4, "Design Review for Application Blocks."

Step 3: Review the Implementation

In this step, you execute the test cases from the DTP that are related to reviewing the implementation (code) of the application block. This is an important step in unit testing the application block. By conducting an extensive code review, you can detect errors early in the life cycle. If you do not detect errors until the white box testing stage, correcting them may require significant and costly changes to the system, which may lead to a cascading effect of code changes between various modules.

The implementation of the application block is reviewed for the following:

• The code adheres to the design that was laid out in Step 1.
• Naming guidelines for code elements (class names, variables, method names, and assemblies, and so on) are followed.
• The code has comments at appropriate places to help customers understand the application block so they can easily customize it.
• The code follows best practices related to the following:
  • Performance and scalability
  • Writing secure code
  • Ensuring that relevant design patterns are being followed and looking for any possible anti-patterns in the code
  • Exception management
  • Resource management (memory allocation, connection pooling, object pooling, and so on)
  • Globalization and localization
• The block does not contain redundant or commented code that is never called.

The code review also helps in identifying the possible scenarios that must tested for one or more of the following:

• Boundary level conditions
• Special inputs
• Possible security attacks
• Performance optimizations
• Profiling to ensure that there are no memory leaks
• Thread safety and deadlock issues

Note   You must update the DTP and DTC to document these possible scenarios.

More Information

For more information about implementation review of the application blocks, see Chapter 5, "Code Review for Application Blocks."

Step 4: Perform Black Box Testing

In this step, you execute the test cases in the DTP that are related to black box testing of the application blocks. Black box testing is testing without knowledge of the internal workings of the system that is being tested. This type of testing simulates end-user experience.

Black box testing helps to ensure that the application block:

• Meets all of the objectives listed in the requirements document.
• Covers all of the functionalities that are specified in the functional specifications.
• Can gracefully handle all of the expected and exceptional usage scenarios. The error messages displayed are meaningful and help the user in diagnosing the problem.

You may need to develop one or more of the following to test the functionality of the application blocks:

• Test suites, such as NUnit, to test the API of the application block for various types of input.
• Dummy Windows forms or Web Forms applications that integrate the application blocks and are deployed in simulated target deployments.
• Automated scripts that test the APIs of the application blocks for various types of input.

The black box testing assumes that the tester has no knowledge of code and is intended to simulate the end-user experience. You can use sample applications to integrate and test the application block for black box testing. This approach tests all possible combinations of end-user actions. The test planning for black box testing can begin as soon as the requirements and the functional specifications are available. Black box testing includes the following:

• Testing all of the external interfaces for all possible usage scenarios. This means testing all of the external interfaces (such as public classes rather than the private or internal classes) that end users can integrate with their applications. Testing the external interfaces involves the following:
  • Ensuring that the interfaces meet the functional specifications and address all of the requirements. This type of testing ensures that the application block provides interfaces for meeting all of the requirements as stated in the functional specifications. This type of testing requires that you develop test suites. You must test for all of the ways in which clients of the application block can call the APIs. The usage scenarios include both the expected process flows and random input.
  • Testing for various types of input. This type of testing ensures that the interfaces return the output as expected and are robust enough to handle invalid data and exceptional conditions gracefully. The input data can be randomly generated either within a specified range expected by the application block, outside the specified range, or at the boundary of the range (known as boundary testing). Testing with data outside the specified range ensures that the application block is robust and can handle invalid data, and that the error messages generated are meaningful for the end user. Boundary testing ensures that the highest and lowest permitted values for input produce expected output.
• Performance testing. You execute performance-related test cases from the DTP in a simulated environment that is close to the real-world deployment. Performance testing verifies that the application block can perform under expected normal and peak load conditions, and that it can scale sufficiently to handle increased capacity. There are two main aspects of performance testing: load testing and stress testing. Each aspect has different end goals. You must plan and execute test cases for both aspects as follows:
  • Load testing. Use load testing to verify the application block behavior under normal and peak load conditions. Load testing enables you to verify that the application block can meet the performance objectives and does not overshoot the allocated budget for resource utilization such as memory, processor, and network I/O. It also enables you to measure response times and throughput rates for the application that is using the application block.
    Load testing also helps you to identify the overhead (if any, in terms of resource cost such as processor, memory, disk i/o or network i/o) of using the application block to achieve a desired functionality by testing applications with and without the application block to achieve the same result.
  • Stress testing. Use stress testing to evaluate the application block's behavior when it is pushed beyond peak load conditions. The goal of stress testing is to identify errors that occur only under high load conditions—for example, synchronization issues, race conditions, and memory leaks.
  The analysis from performance tests can serve as input for the implementation review and white box testing. If you detect a problem, you may need to review the code of the module you suspect to isolate the cause. (For example, if you have a problem of increased response wait time, a code review may reveal that a coarse-grained lock is causing the problem). During the white box testing stage, you can profile the suspected code path and optimize it.
• Security testing. Security testing from the black box perspective is done by simulating the target deployment environment and performing tests that simulate the actions an attacker can take in a real-world production environment.

You identify vulnerabilities in the application block (and hence expose threats) by attempting to simulate all possible types of input. The goal is to expose (at run time) any threats that may exist in the code because of poor design or poor coding practices. The scope of security testing for an application block is different from that for a real-world application and depends on the type of functionality that the application block provides. You can execute tests to:

• Check input validation techniques, if the application block has functionality that validates input data. You can provide various types of input with the objective of breaking the validation technique.
• Break the encryption and access sensitive data, if the application block handles sensitive data and uses encryption.
• Check for buffer overflows.
• Check for cross-site scripting errors.
• Validate authorization and authentication mechanisms, if the application block provides this functionality.
• Globalization testing. The design and implementation have already been reviewed for adherence to best practices for globalization. For black box testing, you execute the globalization testing–related test cases from the DTP to ensure that the application block can handle international support and is ready to be used from various locales around the world.
  The goal of globalization testing is to detect problems in application design that could interfere with globalization. Globalization testing ensures that the code can handle all international support and that it supports any of the culture/locale settings without breaking functionality that would cause either data loss or display problems.
  To perform globalization testing, you must install multiple language groups and ensure that the culture/locale is not your local culture/locale. Executing test cases in both Japanese and German environments, for example, can cover most globalization issues.