White Box Testing

White Box Testing

White box testing assumes that the tester can take a look at the code for the application block and create test cases that look for any potential failure scenarios. During white box testing, you analyze the code of the application block and prepare test cases for testing the functionality to ensure that the class is behaving in accordance with the specifications and testing for robustness.

Input

The following input is required for white box testing:

• Requirements
• Functional specifications
• High-level design documents
• Detailed design documents
• Application block source code

White Box Testing Steps

The white box testing process for an application block is shown in Figure 6.2.

Figure 6.2. White box testing process

White box testing involves the following steps:

1. Create test plans. Identify all white box test scenarios and prioritize them.
2. Profile the application block. This step involves studying the code at run time to understand the resource utilization, time spent by various methods and operations, areas in code that are not accessed, and so on.
3. Test the internal subroutines. This step ensures that the subroutines or the nonpublic interfaces can handle all types of data appropriately.
4. Test loops and conditional statements. This step focuses on testing the loops and conditional statements for accuracy and efficiency for different data inputs.
5. Perform security testing. White box security testing helps you understand possible security loopholes by looking at the way the code handles security.

The next sections describe each of these steps.

Step 1: Create Test Plans

The test plans for white box testing can be created only after a reasonably stable build of the application block is available. The creation of test plans involves extensive code review and input from design review and black box testing. The test plans for white box testing include the following:

• Profiling, including code coverage, resource utilization, and resource leaks
• Testing internal subroutines for integrity and consistency in data processing
• Loop testing; test simple, concatenated, nested, and unstructured loops
• Conditional statements, such as simple expressions, compound expressions, and expressions that evaluate to Boolean.

1. For more information about creating test cases, see Chapter 3, "Testing Process for Application Blocks."

Step 2: Profile the Application Block

Profiling allows you to monitor the behavior of a particular code path at run time when the code is being executed. Profiling includes the following tests:

• Code coverage. Code coverage testing ensures that every line of code is executed at least once during testing. You must develop test cases in a way that ensures the entire execution tree is tested at least once. To ensure that each statement is executed once, test cases should be based on the control structure in the code and the sequence diagrams from the design documents. The control structures in the code consist of various conditions as follows:
  • Various conditional statements that branch into different code paths. For example, a Boolean variable that evaluates to "false" or "true" can execute different code paths. There can be other compound conditions with multiple conditions, Boolean operators, and bit-wise comparisons.
  • Various types of loops, such as simple loops, concatenated loops, and nested loops.
  There are various tools available for code coverage testing, but you still need to execute the test cases. The tools identify the code that has been executed during the testing. In this way, you can identify the redundant code that never gets executed. This code may be left over from a previous version of the functionality or may signify a partially implemented functionality or dead code that never gets called.
  Tables 6.3 and 6.4 list sample test cases for testing the code coverage of ConfigurationManager class of the CMAB.
  Table 6.3: The CMAB Test Case Document for Testing the Code Coverage for InitAllProvider Method and All Invoked Methods

Scenario 1.3	Test the code coverage for the method InitAllProviders()in ConfigurationManager class.
Priority	High
Execution details	Create a sample application for reading configuration data from a data store through the CMAB. Run the application under the following conditions: With a default section present Without a default section Trace the code coverage using an automated tool. Report any code not being called in InitAllProviders().
Tools required	Custom test harness integrating the application block for reading configuration data.
Expected results	The entire code for InitAllProviders()method and all the invoked methods should be covered under the preceding conditions.

Table 6.4: The CMAB Test Case Document for Testing the Code Coverage for Read Method and All Invoked Methods

Scenario 1.4	Test the code coverage for the method Read (sectionName) in the ConfigurationManager class.
Priority	High
Execution details	Create a sample application for reading configuration data from SQL database through the CMAB. Run the application under the following conditions: Give a null section name or a section name of zero length to the Read method. Read a section whose name is not mentioned in the App.config or Web.config files. Read a configuration section that has cache enabled. Read a configuration section that has cache disabled. Read a configuration section successfully with the cache disabled, and then disconnect the database and read the section again. Read a configuration section with the section having no configuration data in the database. Read the configuration section that does not have provider information mentioned in the App.config or Web.config files. Trace the code coverage. Report any code left not being covered in the Read (sectionName) method.
Tools required	Custom test harness integrating the application block for reading of configuration data.
Expected results	The entire code for the Read (sectionName) method and the invoked methods should be covered under the preceding conditions.

• Memory allocation pattern. You can profile the memory allocation pattern of the application block by using code profiling tools. You need to check for the following in the allocation pattern:
  • The percentage of allocations in Gen 0, Gen 1, and Gen 2. If the percentage of objects in Gen 2 is high, the resource cleanup in the application block is not efficient and there are memory leaks. This probably means the objects are held up longer than required (this may be expected in some scenarios). Profiling the application blocks gives you an idea of the type of objects that are being promoted to Gen 2 of the heap. You can then focus on analyzing the culprit code snippet and rectify the problem.
    An efficient allocation pattern should have most of the allocations in Gen 0 and Gen 1 over a period of time.
    There might be certain objects, such as a pinned pool of reusable buffers used for I/O work, that are promoted to Gen 2 when the application starts. The faster this pool of buffers gets promoted to Gen 2, the better.
  • The fragmentation of the heap. The heap fragmentation happens most often in scenarios where the objects are pinned and cannot be moved. The memory cannot be efficiently compacted around these objects. The longer these objects are pinned, the greater the chances of heap fragmentation. As mentioned earlier, there might be a pool of buffers that needs to be used for I/O calls. If these objects are initialized when the application starts, they quickly move the Gen 2, where the overhead of heap allocation is largely removed.
  • "Side effect" allocations. Large number of side effect allocations take place because of some calls in a loop or recursive functions, such as the calls to string-related functions String.ToLower()or concatenation using the + operator happening in a loop. This causes the original string to be discarded and a new string to be allocated for each such operation. These operations in a loop may cause significant increase in memory consumption.
  You can also analyze memory leaks by using debugging tools, such as WinDbg from the Windows Resource Kit. Using these tools, you can analyze the heap allocations for the process.
  
  
• Cost of serialization. There may be certain scenarios when the application block needs to serialize and transmit data across processes or computers. Serializing data involves memory overhead that can be quite significant, depending on the amount of data and the type of serializer or formatter used for serialization. You need to instrument your code to take the snapshots of memory utilized by the garbage collector before and after serialization.
  
  
• Contention and deadlock issues. Contention and deadlock issues mostly surface under high load conditions. The input from load testing (during black box testing) give you information about the potential execution paths where contention and deadlocks issues are suspected. For example, in the case of the CMAB, you may suspect a deadlock if you see the requests timing out when trying to update a particular information in the persistent medium.
  You need to analyze these issues with invasive profiling techniques, such as using WindDbg tool, in the production environment on a live process or by analyzing the stack dumps of the process.
  
  
• Time taken for executing a code path. For scenarios where performance is critical, you can profile the time they take. Timing a code path may require custom instrumentation of the appropriate code. There are also various tools available that help you measure the time it takes for a particular scenario to execute by automatically creating instrumented assemblies of the application block. The profiling for time taken may be for complete execution of a usage scenario, an internal function, or even a particular loop within a function.
  
  
• Profiling for excessive resource utilization. The input from a performance test may show excessive resource utilization, such as CPU, memory, disk I/O, or network I/O, for a particular usage scenario. But you may need to profile the code to track the piece of code that is blocking resources disproportionately. This might be an expected behavior for a particular scenario in some circumstances. For example, an empty while loop may pump up the processor utilization significantly and is something you should track and rectify; whereas, a computational logic that involves complex calculations may genuinely warrant high processor utilization.

Step 3: Test the Internal Subroutines

Thoroughly test all internal subroutines for every type of input. The subroutines that are internally called by the public API to process the input may be working as expected for the expected input types. However, after a thorough code review, you may notice that there are some expressions that may fail for certain types of input. This warrants the testing of internal methods and subroutines by developing NUnit tests for internal functions after a thorough code review. Following are some examples of potential pitfalls:

• The code analysis reveals that the function may fail for a certain input value. For example, a function expecting numeric input may fail for an input value of 0.
• In the case of the CMAB, the function reads information from the cache. The function returns the information appropriately if the cache is not empty. However, if during the process of reading, the cache is flushed or refreshed, the function may fail.
• The function may be reading values in a buffer before returning them to the client. Certain input values might result in a buffer overflow and loss of data.
• The subroutine does not handle an exception where the remote call to a database is not successful. For example, in the CMAB, if the function is trying the update the SQL Server information but the SQL Server database is not available, it does not log the application in the appropriate event sink.

Step 4: Test Loops and Conditional Statements

The application block may contain various types of loops, such as simple, nested, concatenated, and unstructured loops. Although unstructured loops require redesigning, the other types of loops require extensive testing for various inputs. Loops are critical to the application block performance because they magnify seemingly trivial problems by iterating through the loop multiple times.

Some of the common errors could cause the loop to execute infinite times. This could result in excessive CPU or memory utilization resulting in the application failing. Therefore, all loops in the application block should be tested for the following conditions:

• Provide input that results in executing the loop zero times. This can be achieved where the input to the lower bound value of the loop is less than the upper bound value.
• Provide input that results in executing the loop one time. This can be achieved where the lower bound value and upper bound value are the same.
• Provide input that results in executing the loop a specified number of times within a specific range.
• Provide input that the loop might iterate n, n-1, and n+1 times. The out-of-bound loops (n-1 and n+1) are very difficult to detect with a simple code review; therefore, there is a need to execute special test cases that can simulate such cases.

When testing nested loops, you can start by testing the innermost loop, with all other loops set to iterate a minimum number of times. After the innermost loop is tested, you can set it to iterate a minimum number of times, and then test the outermost loop as if it was a simple loop.

Also, all of the conditional statements should be completely tested. The process of conditional testing ensures that the controlling expressions have been exercised during testing by presenting the evaluating expression with a set of input values. The input values ensure that all possible outcomes of the expressions are tested for expected output. The conditional statements can be a relational expression, a simple condition, a compound condition, or a Boolean expression.

Step 5: Perform Security Testing

White box security testing focuses on identifying test scenarios and testing based on knowledge of implementation details. During code reviews, you can identify areas in code that validate data, handle data, access resources, or perform privileged operations. Test cases can be developed to test all such areas. Following are some examples:

• Validation techniques can be tested by passing negative value, null value, and so on, to make sure the proper error message displays.
• If the application block handles sensitive data and uses cryptography, then based on knowledge from code reviews, test cases can be developed to validate the encryption technique or cryptography methods.

Read More

Black Box Testing

Overview

After you complete the design and code review of the application block, you need to test the application block to make sure it meets the functional requirements and successfully implements the functionality for the usage scenarios it was designed and implemented for.

The testing effort can be divided into two categories that complement each other:

• Black box testing. This approach tests all possible combinations of end-user actions. Black box testing assumes no knowledge of code and is intended to simulate the end-user experience. You can use sample applications to integrate and test the application block for black box testing. You can begin planning for black box testing immediately after the requirements and the functional specifications are available.
• White box testing. (This is also known as glass box, clear box, and open box testing.) In white box testing, you create test cases by looking at the code to detect any potential failure scenarios. You determine the suitable input data for testing various APIs and the special code paths that need to be tested by analyzing the source code for the application block. Therefore, the test plans need to be updated before starting white box testing and only after a stable build of the code is available.
  A failure of a white box test may result in a change that requires all black box testing to be repeated and white box testing paths to be reviewed and possibly changed.

The goals of testing can be summarized as follows:

• Verify that the application block is able to meet all requirements in accordance with the functional specifications document.
• Make sure that the application block has consistent and expected output for all usage scenarios for both valid and invalid inputs. For example, make sure the error messages are meaningful and help the user in diagnosing the actual problem.

You may need to develop one or more of the following to test the functionality of the application blocks:

• Test harnesses, such as NUnit test cases, to test the API of the application block for various inputs
• Prototype Windows Forms and Web Forms applications that integrate the application blocks and are deployed in simulated target deployments
• Automated scripts that test the API of the application blocks for various inputs

This chapter examines the process of black box testing and white box testing. It includes code examples and sample test cases to demonstrate the approach for black box testing and white box testing application blocks. For the purpose of the examples illustrated in this chapter, it is assumed that functionality testing is being done for the Management Application Block (CMAB). The CMAB has already been through design and code review. The requirements for the CMAB are the following:

• It provides the functionality to read and store configuration information transparently in a persistent storage medium. The storage mediums are SQL Server, the registry, and an XML file.
• It provides a configurable option to store the information in encrypted form and plain text using XML notation.
• It can be used with desktop applications and Web applications that are deployed in a Web farm.
• It caches configuration information in memory to reduce cross-process communication, such as reading from any persistent medium. This reduces the response time of the request for any configuration information. The expiration and scavenging mechanism for the data that is cached in memory is similar to the cron algorithm in UNIX.
• It can store and return data from various locales and cultures without any loss of data integrity.

Black Box Testing

Black box testing assumes the code to be a black box that responds to input stimuli. The testing focuses on the output to various types of stimuli in the targeted deployment environments. It focuses on validation tests, boundary conditions, destructive testing, reproducibility tests, performance tests, globalization, and security-related testing.

Risk analysis should be done to estimate the amount and the level of testing that needs to be done. Risk analysis gives the necessary criteria about when to stop the testing process. Risk analysis prioritizes the test cases. It takes into account the impact of the errors and the probability of occurrence of the errors. By concentrating on the test cases that can lead to high impact and high probability errors, the testing effort can be reduced and the application block can be ensured to be good enough to be used by various applications.

Preferably, black box testing should be conducted in a test environment close to the target environment. There can be one or more deployment scenarios for the application block that is being tested. The requirements and the behavior of the application block can vary with the deployment scenario; therefore, testing the application block in a simulated environment that closely resembles the deployment environment ensures that it is tested to satisfy all requirements of the targeted real-life conditions. There will be no surprises in the production environment. The test cases being executed ensure robustness of the application block for the targeted deployment scenarios.

For example, the CMAB can be deployed on the desktop with Windows Forms applications or in a Web farm when integrated with Web applications. The CMAB requirements, such as performance objectives, vary from the desktop environment to the Web environment. The test cases and the test environment have to vary according to the target environments. Other application blocks may have more restricted and specialized target environments. An example of an application block that requires a specialized test environment is an application block that is deployed on mobile devices and is used for synchronizing data with a central server.

As mentioned earlier, you will need to develop custom test harnesses for functionality testing purpose.

Input

The following input is required for black box testing:

• Requirements
• Functional specifications
• High-level design documents
• Application block source code

The black box testing process for an application block is shown in Figure 6.1.

Figure 6.1 . Black box testing process

Black Box Testing Steps

Black box testing involves testing external interfaces to ensure that the code meets functional and nonfunctional requirements. The various steps involved in black box testing are the following:

1. Create test plans. Create prioritized test plans for black box testing.
2. Test the external interfaces. Test the external interfaces for various type of inputs using automated test suites, such as NUnit suites and custom prototype applications.
3. Perform load testing. Load test the application block to analyze the behavior at various load levels. This ensures that it meets all performance objectives that are stated as requirements.
4. Perform stress testing. Stress test the application block to analyze various bottlenecks and to identify any issues visible only under extreme load conditions, such as race conditions and contentions.
5. Perform security testing. Test for possible threats in deployment scenarios. Deploy the application block in a simulated target environment and try to hack the application by exploiting any possible weakness of the application block.
6. Perform globalization testing. Execute test cases to ensure that the application block can be integrated with applications targeted toward locales other than the default locale used for development.

The next sections describe each of these steps.

Step 1: Create Test Plans

The first step in the process of black box testing is to create prioritized test plans. You can prepare the test cases for black box testing even before you implement the application block. The test cases are based on the requirements and the functional specification documents.

The requirements and functional specification documents help you extract various usage scenarios and the expected output in each scenario.

The detailed test plan document includes test cases for the following:

• Testing the external interfaces with various types of input
• Load testing and stress testing
• Security testing
• Globalization testing

For more information about creating test cases, see Chapter 3, "Testing Process for Application Blocks."

Step 2: Test the External Interfaces

You need to test the external interfaces of the application block using the following strategies:

• Ensure that the application block exposes interfaces that address all functional specifications and requirements. To perform this validation testing, do the following:
  1. Prepare a checklist of all requirements and features that are expected from the application block.
  2. Create test harnesses, such as NUnit, and small "hello world"' applications to use all exposed APIs of the test application block.
  3. Run the test harnesses.
  Using NUnit, you can validate that the intended feature is working if the input is given on the expected lines.
  The sample applications can indicate whether the application block can be integrated and deployed in the target environment. The sample applications are used to test for the possible user actions for the usage scenarios; these include both the expected process flows and the random inputs. For example, a Web application deployed in a Web farm that integrates the CMAB can be used to test reading and writing information from a persistent database, such as the registry, SQL, or an XML file. You need to test the functionality by using various configuration options in the configuration file.
• Testing for various types of inputs. After ensuring that the application block exposes the interfaces that address all of the functional specifications, you need to test the robustness of these interfaces. You need to test for the following input types:
  • Randomly generated input within a specified range
  • Boundary cases for the specified range of input
  • The number zero testing if the input is numeric
  • The null input
  • Invalid input or input that is out of the expected range

This testing ensures that the application block provides expected output for data within the specified range and gracefully handles all invalid data. Meaningful error messages should be displayed for invalid input. Boundary testing ensures that the highest and lowest permitted inputs produce expected output.

You can use NUnit for this type of input testing. Separate sets of NUnit tests can be generated for each range of input types. Executing these NUnit tests on each new build of the application block ensures that the API is able to successfully process the given input.

Read More

Testing Process

Testing Process

The process for testing application blocks is outlined below. The steps listed should be performed regardless of the testing methodology that is used for the blocks.

Input

The following input is required to test an application block:

• Functional specifications
• Requirements
• Performance objectives
• Deployment scenarios

Steps

Figure 3.1 shows the steps in the testing process for application blocks.

Figure 3.1. The testing process for application blocks

1. Create test plans. Create test plan documentation with a prioritized list of test cases and execution details.
2. Review the design. Review the design to ensure that it addresses all the deployment scenarios and the requirements. Also ensure adherence to guidelines for performance, security, globalization, maintainability, and so on.
3. Review the implementation. Review the code to ensure that various best practices are followed.
4. Perform black box testing. Test the code from an end-user perspective by executing all the end-user scenarios.
5. Perform white box testing. Analyze the code for failure scenarios and simulate them by passing invalid data using custom test harnesses.

Step 1: Create Test Plans

The test plans document the test cases you will use to test the application block. The test cases cover all aspects of testing, including design review, code review, profiling, deployment testing, and load testing. The test plans help to ensure that you test all of the features and usage scenarios of the application block.

The test plan documentation consists of two documents:

• Detailed test plan (DTP) document. The detailed test plan document lists test cases in a prioritized order (High, Medium, and Low). The test case descriptions in this document briefly summarize the usage scenarios and features to be tested. For each test case, you assign a priority level based on the case's importance and its overall impact on the goal of meeting the desired objectives and requirements.
• Detailed test case (DTC) document. The detailed test case document maps to the detailed test plan document. This document describes the steps that the user must perform to execute each test case that is listed in the DTP document. It also lists the data that is required for the test and describes the expected results of the test.

For examples of DTP and DTC documents, see the "Appendix" section later in this chapter. You can use these examples as templates when you create the test plans for your application block.

You must update the DTP and DTC documents throughout the development life cycle. For example, you must update these documents if functional specifications or requirements change, or if you have additional input. You must also update them if you add test cases or if you modify the priority of the existing test cases later in the development cycle to account for additional usage scenarios or functional testing.

Step 2: Review the Design

In this step, you execute the test cases from the DTP that are related to reviewing the design of the application block. The design review is very important from the testing point of view, because you can avoid most of the potential pitfalls (for example, those related to performance or security) in the early stages of a project by thoroughly reviewing the design of the application block. In the later stages, considerable rework may be required in the code to make any design changes.

The design review helps to ensure the following:

• The design meets all of the functional specifications and requirements for the application block.
• The design makes appropriate tradeoffs given the deployment scenarios for the application block. Examples of deployment targets that may have special requirements are mobile devices, the DMZ (also known as demilitarized zone or screened subnet) of a Web farm, and tablet PCs.
• The design ensures that the application block meets all performance objectives.
• The design addresses all the security threats possible in various deployment scenarios for the application block.
• The design adheres to best practices and principles related to coupling and cohesion, concurrency, communication, class design, exception management, resource management, caching, and so on, so that developers can easily extend and customize the application block.
• The design adheres to best practices for globalization and localization.

The design review also helps in identifying the scenarios that must be tested for one or more of the following:

• Possible security attacks
• Performance optimizations
• Profiling to ensure that there are no memory leaks

Note   You must update the DTP and DTC to document these possible scenarios.

More Information

For more information about how to conduct a design review for the application block, see Chapter 4, "Design Review for Application Blocks."

Step 3: Review the Implementation

In this step, you execute the test cases from the DTP that are related to reviewing the implementation (code) of the application block. This is an important step in unit testing the application block. By conducting an extensive code review, you can detect errors early in the life cycle. If you do not detect errors until the white box testing stage, correcting them may require significant and costly changes to the system, which may lead to a cascading effect of code changes between various modules.

The implementation of the application block is reviewed for the following:

• The code adheres to the design that was laid out in Step 1.
• Naming guidelines for code elements (class names, variables, method names, and assemblies, and so on) are followed.
• The code has comments at appropriate places to help customers understand the application block so they can easily customize it.
• The code follows best practices related to the following:
  • Performance and scalability
  • Writing secure code
  • Ensuring that relevant design patterns are being followed and looking for any possible anti-patterns in the code
  • Exception management
  • Resource management (memory allocation, connection pooling, object pooling, and so on)
  • Globalization and localization
• The block does not contain redundant or commented code that is never called.

The code review also helps in identifying the possible scenarios that must tested for one or more of the following:

• Boundary level conditions
• Special inputs
• Possible security attacks
• Performance optimizations
• Profiling to ensure that there are no memory leaks
• Thread safety and deadlock issues

Note   You must update the DTP and DTC to document these possible scenarios.

More Information

For more information about implementation review of the application blocks, see Chapter 5, "Code Review for Application Blocks."

Step 4: Perform Black Box Testing

In this step, you execute the test cases in the DTP that are related to black box testing of the application blocks. Black box testing is testing without knowledge of the internal workings of the system that is being tested. This type of testing simulates end-user experience.

Black box testing helps to ensure that the application block:

• Meets all of the objectives listed in the requirements document.
• Covers all of the functionalities that are specified in the functional specifications.
• Can gracefully handle all of the expected and exceptional usage scenarios. The error messages displayed are meaningful and help the user in diagnosing the problem.

You may need to develop one or more of the following to test the functionality of the application blocks:

• Test suites, such as NUnit, to test the API of the application block for various types of input.
• Dummy Windows forms or Web Forms applications that integrate the application blocks and are deployed in simulated target deployments.
• Automated scripts that test the APIs of the application blocks for various types of input.

The black box testing assumes that the tester has no knowledge of code and is intended to simulate the end-user experience. You can use sample applications to integrate and test the application block for black box testing. This approach tests all possible combinations of end-user actions. The test planning for black box testing can begin as soon as the requirements and the functional specifications are available. Black box testing includes the following:

• Testing all of the external interfaces for all possible usage scenarios. This means testing all of the external interfaces (such as public classes rather than the private or internal classes) that end users can integrate with their applications. Testing the external interfaces involves the following:
  • Ensuring that the interfaces meet the functional specifications and address all of the requirements. This type of testing ensures that the application block provides interfaces for meeting all of the requirements as stated in the functional specifications. This type of testing requires that you develop test suites. You must test for all of the ways in which clients of the application block can call the APIs. The usage scenarios include both the expected process flows and random input.
  • Testing for various types of input. This type of testing ensures that the interfaces return the output as expected and are robust enough to handle invalid data and exceptional conditions gracefully. The input data can be randomly generated either within a specified range expected by the application block, outside the specified range, or at the boundary of the range (known as boundary testing). Testing with data outside the specified range ensures that the application block is robust and can handle invalid data, and that the error messages generated are meaningful for the end user. Boundary testing ensures that the highest and lowest permitted values for input produce expected output.
• Performance testing. You execute performance-related test cases from the DTP in a simulated environment that is close to the real-world deployment. Performance testing verifies that the application block can perform under expected normal and peak load conditions, and that it can scale sufficiently to handle increased capacity. There are two main aspects of performance testing: load testing and stress testing. Each aspect has different end goals. You must plan and execute test cases for both aspects as follows:
  • Load testing. Use load testing to verify the application block behavior under normal and peak load conditions. Load testing enables you to verify that the application block can meet the performance objectives and does not overshoot the allocated budget for resource utilization such as memory, processor, and network I/O. It also enables you to measure response times and throughput rates for the application that is using the application block.
    Load testing also helps you to identify the overhead (if any, in terms of resource cost such as processor, memory, disk i/o or network i/o) of using the application block to achieve a desired functionality by testing applications with and without the application block to achieve the same result.
  • Stress testing. Use stress testing to evaluate the application block's behavior when it is pushed beyond peak load conditions. The goal of stress testing is to identify errors that occur only under high load conditions—for example, synchronization issues, race conditions, and memory leaks.
  The analysis from performance tests can serve as input for the implementation review and white box testing. If you detect a problem, you may need to review the code of the module you suspect to isolate the cause. (For example, if you have a problem of increased response wait time, a code review may reveal that a coarse-grained lock is causing the problem). During the white box testing stage, you can profile the suspected code path and optimize it.
• Security testing. Security testing from the black box perspective is done by simulating the target deployment environment and performing tests that simulate the actions an attacker can take in a real-world production environment.

You identify vulnerabilities in the application block (and hence expose threats) by attempting to simulate all possible types of input. The goal is to expose (at run time) any threats that may exist in the code because of poor design or poor coding practices. The scope of security testing for an application block is different from that for a real-world application and depends on the type of functionality that the application block provides. You can execute tests to:

• Check input validation techniques, if the application block has functionality that validates input data. You can provide various types of input with the objective of breaking the validation technique.
• Break the encryption and access sensitive data, if the application block handles sensitive data and uses encryption.
• Check for buffer overflows.
• Check for cross-site scripting errors.
• Validate authorization and authentication mechanisms, if the application block provides this functionality.
• Globalization testing. The design and implementation have already been reviewed for adherence to best practices for globalization. For black box testing, you execute the globalization testing–related test cases from the DTP to ensure that the application block can handle international support and is ready to be used from various locales around the world.
  The goal of globalization testing is to detect problems in application design that could interfere with globalization. Globalization testing ensures that the code can handle all international support and that it supports any of the culture/locale settings without breaking functionality that would cause either data loss or display problems.
  To perform globalization testing, you must install multiple language groups and ensure that the culture/locale is not your local culture/locale. Executing test cases in both Japanese and German environments, for example, can cover most globalization issues.

Read More

Software Development Methodologies and Testing

Software Development Methodologies and Testing

Various generic software development life cycle methodologies are available for executing software development projects. Although each methodology is designed for a specific purpose and has its own advantages and disadvantages, most methodologies divide the life cycle into phases and share tasks across these phases. This section briefly summarizes common methodologies used for software development and describes their relationship to testing.

Waterfall Model

The waterfall model is one of the earliest structured models for software development. It consists of the following sequential phases through which the development life cycle progresses:

• System feasibility. In this phase, you consider the various aspects of the targeted business process, find out which aspects are worth incorporating into a system, and evaluate various approaches to building the required software.
• Requirement analysis. In this phase, you capture software requirements in such a way that they can be translated into actual use cases for the system. The requirements can derive from use cases, performance goals, target deployment, and so on.
• System design. In this phase, you identify the interacting components that make up the system. You define the exposed interfaces, the communication between the interfaces, key algorithms used, and the sequence of interaction. An architecture and design review is conducted at the end of this phase to ensure that the design conforms to the previously defined requirements.
• Coding and unit testing. In this phase, you write code for the modules that make up the system. You also review the code and individually test the functionality of each module.
• Integration and system testing. In this phase, you integrate all of the modules in the system and test them as a single system for all of the use cases, making sure that the modules meet the requirements.
• Deployment and maintenance. In this phase, you deploy the software system in the production environment. You then correct any errors that are identified in this phase, and add or modify functionality based on the updated requirements.

The waterfall model has the following advantages:

• It allows you to compartmentalize the life cycle into various phases, which allows you to plan the resources and effort required through the development process.
• It enforces testing in every stage in the form of reviews and unit testing. You conduct design reviews, code reviews, unit testing, and integration testing during the stages of the life cycle.
• It allows you to set expectations for deliverables after each phase.

The waterfall model has the following disadvantages:

• You do not see a working version of the software until late in the life cycle. For this reason, you can fail to detect problems until the system testing phase. Problems may be more costly to fix in this phase than they would have been earlier in the life cycle.
• When an application is in the system testing phase, it is difficult to change something that was not carefully considered in the system design phase. The emphasis on early planning tends to delay or restrict the amount of change that the testing effort can instigate, which is not the case when a working model is tested for immediate feedback.
• For a phase to begin, the preceding phase must be complete; for example, the system design phase cannot begin until the requirement analysis phase is complete and the requirements are frozen. As a result, the waterfall model is not able to accommodate uncertainties that may persist after a phase is completed. These uncertainties may lead to delays and extended project schedules.

Incremental or Iterative Development

The incremental, or iterative, development model breaks the project into small parts. Each part is subjected to multiple iterations of the waterfall model. At the end of each iteration, a new module is completed or an existing one is improved on, the module is integrated into the structure, and the structure is then tested as a whole.

For example, using the iterative development model, a project can be divided into 12 one- to four-week iterations. The system is tested at the end of each iteration, and the test feedback is immediately incorporated at the end of each test cycle. The time required for successive iterations can be reduced based on the experience gained from past iterations. The system grows by adding new functions during the development portion of each iteration. Each cycle tackles a relatively small set of requirements; therefore, testing evolves as the system evolves. In contrast, in a classic waterfall life cycle, each phase (requirement analysis, system design, and so on) occurs once in the development cycle for the entire set of system requirements.

The main advantage of the iterative development model is that corrective actions can be taken at the end of each iteration. The corrective actions can be changes to the specification because of incorrect interpretation of the requirements, changes to the requirements themselves, and other design or code-related changes based on the system testing conducted at the end of each cycle.

The main disadvantages of the iterative development model are as follows:

• The communication overhead for the project team is significant, because each iteration involves giving feedback about deliverables, effort, timelines, and so on.
• It is difficult to freeze requirements, and they may continue to change in later iterations because of increasing customer demands. As a result, more iterations may be added to the project, leading to project delays and cost overruns.
• The project requires a very efficient change control mechanism to manage changes made to the system during each iteration.

Prototyping Model

The prototyping model assumes that you do not have clear requirements at the beginning of the project. Often, customers have a vague idea of the requirements in the form of objectives that they want the system to address. With the prototyping model, you build a simplified version of the system and seek feedback from the parties who have a stake in the project. The next iteration incorporates the feedback and improves on the requirements specification. The prototypes that are built during the iterations can be any of the following:

• A simple user interface without any actual data processing logic
• A few subsystems with functionality that is partially or completely implemented
• Existing components that demonstrate the functionality that will be incorporated into the system

The prototyping model consists of the following steps.

1. Capture requirements. This step involves collecting the requirements over a period of time as they become available.
2. Design the system. After capturing the requirements, a new design is made or an existing one is modified to address the new requirements.
3. Create or modify the prototype. A prototype is created or an existing prototype is modified based on the design from the previous step.
4. Assess based on feedback. The prototype is sent to the stakeholders for review. Based on their feedback, an impact analysis is conducted for the requirements, the design, and the prototype. The role of testing at this step is to ensure that customer feedback is incorporated in the next version of the prototype.
5. Refine the prototype. The prototype is refined based on the impact analysis conducted in the previous step.
6. Implement the system. After the requirements are understood, the system is rewritten either from scratch or by reusing the prototypes. The testing effort consists of the following:
   • Ensuring that the system meets the refined requirements
   • Code review
   • Unit testing
   • System testing

The main advantage of the prototyping model is that it allows you to start with requirements that are not clearly defined.

The main disadvantage of the prototyping model is that it can lead to poorly designed systems. The prototypes are usually built without regard to how they might be used later, so attempts to reuse them may result in inefficient systems. This model emphasizes refining the requirements based on customer feedback, rather than ensuring a better product through quick change based on test feedback.

Agile Methodology

Most software development life cycle methodologies are either iterative or follow a sequential model (as the waterfall model does). As software development becomes more complex, these models cannot efficiently adapt to the continuous and numerous changes that occur. Agile methodology was developed to respond to changes quickly and smoothly. Although the iterative methodologies tend to remove the disadvantage of sequential models, they still are based on the traditional waterfall approach. Agile methodology is a collection of values, principles, and practices that incorporates iterative development, test, and feedback into a new style of development. For an overview of agile methodology, see the Agile Modeling site at http://www.agilemodeling.com/.

The key differences between agile and traditional methodologies are as follows:

• Development is incremental rather than sequential. Software is developed in incremental, rapid cycles. This results in small, incremental releases, with each release building on previous functionality. Each release is thoroughly tested, which ensures that all issues are addressed in the next iteration.
• People and interactions are emphasized, rather than processes and tools. Customers, developers, and testers constantly interact with each other. This interaction ensures that the tester is aware of the requirements for the features being developed during a particular iteration and can easily identify any discrepancy between the system and the requirements.
• Working software is the priority rather than detailed documentation. Agile methodologies rely on face-to-face communication and collaboration, with people working in pairs. Because of the extensive communication with customers and among team members, the project does not need a comprehensive requirements document.
• Customer collaboration is used, rather than contract negotiation. All agile projects include customers as a part of the team. When developers have questions about a requirement, they immediately get clarification from customers.
• Responding to change is emphasized, rather than extensive planning. Extreme Programming does not preclude planning your project. However, it suggests changing the plan to accommodate any changes in assumptions for the plan, rather than stubbornly trying to follow the original plan.

Agile methodology has various derivate approaches, such as Extreme Programming, Dynamic Systems Development Method (DSDM), and SCRUM. Extreme Programming is one of the most widely used approaches.

Extreme Programming

In Extreme Programming, rather than designing whole of the system at the start of the project, the preliminary design work is reduced to solving the simple tasks that have already been identified.

The developers communicate directly with customers and other developers to understand the initial requirements. They start with a very simple task and then get feedback by testing their software as soon as it is developed. The system is delivered to the customers as soon as possible, and the requirements are refined or added based on customer feedback. In this way, requirements evolve over a period of time, and developers are able to respond quickly to changes.

The real design effort occurs when the developers write the code to fulfill the specific engineering task. The engineering task is a part of a greater user story(which is similar to a use case). The user story concerns itself with how the overall system solves a particular problem. It represents a part of the functionality of the overall system. A group of user stories is capable of describing the system as a whole. The developers refactor the previous code iteration to establish the design needed to implement the functionality.

During the Extreme Programming development life cycle, developers usually work in pairs. One developer writes the code for a particular feature, and the second developer reviews the code to ensure that it uses simple solutions and adheres to best design principles and coding practices.

Discussion of the core practices of Extreme Programming is beyond the scope of this chapter. For more information, see the links referred to in "More Information" later in this section.

Test-driven development, which is one of the core practices in Extreme Programming, is discussed in greater detail later in this chapter.

When to Use Extreme Programming

Extreme Programming is useful in the following situations:

• When the customer does not have a clear understanding of the details of the new system. The developers interact continuously with the customer, delivering small pieces of the application to the customer for feedback, and taking corrective action as necessary.
• When the technology used to develop the system is new compared to other technologies. Frequent test cycles in Extreme Programming mitigate the risk of incompatibility with other existing systems.
• When you can afford to create automated unit and functional tests. In some situations, you may need to change the system design so that each module can be tested in isolation using automated unit tests.
• When the team size is not very large (usually 2 to 12 people). Extreme Programming is successful in part because it requires close team interaction and working in pairs. A large team would have difficulty in communicating efficiently at a fast pace. However, large teams have used Extreme Programming successfully.

Read More

What is Software Testing Methodologies?

In the field of Software testing we use different types ofSoftware Testing Methodologies. In today’s Software Testing Class, we will see what all Software Testing Methodologies & software testing technique is used in the day today software testing life cycle.

In the Software Development Process different software development approaches are used. A software development process also known as a Software Development Life Cycle (SDLC).

Each process model has its own advantages & disadvantages, the choosing of model is based on the requirement & need of your project to do the success in process of software development.
Here are the few commonly used Software Development Life Cycle used in actual testing:

• Waterfall model
• V model
• Agile model
• Spiral model
• RUP
• RAD

1) Waterfall Model

The Waterfall Model approach is most basic life cycle model, which is the first Process Model begins and used broadly in Software Development to make sure project success. This model is developed by Winston Royce in the early 1970. Waterfall model is software development process sequential process, as flowing progressively downwards like waterfall via multiple phases & these simple sequence of phases has its own defines activities and goals. The Waterfall model is divided into following separate phases:

• Requirement Gathering & Analysis phase,
• Software Design,
• Implementation and Testing
• Maintenance

All these phases are the sequential phases & next step is totally depends on the previous step, so that second phase is begins as only if the defined set of objectives and goals are completed first.

The stages of “The Waterfall Model” are:

Requirement Gathering & Analysis:
This is the starting phase of the SDLC in which all possible system requirements are captured & analyzed. Software requirements specification includes the complete information about how actual end users are expecting from the system. This document covers all the necessary requirements for the development of project. Finally after completion of requirement gathering & analysis (validation of requirement against the user needs), a Requirement Specification document is created which give out as a input to the next phase of SDLC. In this model once we moved to the next phase then it won’t possible to add or update the requirements.

System & Software Design: Prior to start actual coding, it is mandatory to be aware of what all features we are going to implement & how it would look like? The requirement specifications document created in the first phase is used as the input to this phase & based on this the system design specifications would be prepared. In which all hardware & system requirements would be specified. The system design specifications document prepared in this phase is used as input for the Implementation & Testing phase model.

Implementation & Unit Testing: Upon getting the system design specifications document the actual coding would be started. Before starting the actual coding requirements are divided into the models/units. In the actual coding the initially develop the small programs called units. After implementing all units the integration would be started in the next phase. The developed unit’s model functionality is tested separately in this phase to check whether the unit models are meets the specified requirements & this individual model testing is called as Unit Testing.

Integration & System Testing: In the previous model the system requirements are divided into models & each model is developed & tested separately. In this phase all units are integrated & done the system testing to check whether the all models/units are integrated properly or not & the system as whole doing as mention as per the system requirement document, so we call this phase as the Integration & System Testing. Upon complete testing of software is done then actual Software is successfully send to customer.

Operations & Maintenance: This phase is the never ending phase of the Waterfall Model. The problems are comes in picture after Implementation & Unit Testing The issues found after the implementation phase i.e. not found in the period of the development life cycle. Some of the issues are not catch after testing cycle done so those are implemented in the maintenance phase, so this phase is called as Operations & Maintenance phase.

Advantages and Disadvantages of Waterfall Model:

Advantages of Waterfall Model:

• This is linear simple model to implement & easy to maintain.
• In the SDLC initial phase spent time on reviewing requirements and design which saves the time later.
• Required resources are minimum in this model as compare to other.
• After every phase of the model a document in created which help & simpler to understand & design the system.
• Upon completion of coding, is done to check for implemented code or correctness of system.
•  For each stage deadlines can be set which will help to develop the system on decided time frame.

Disadvantages of Waterfall Model: 

• The biggest disadvantages of such system is it won’t allow to go back. If the problems in the design phase which creates complication in the implementation phase.
• This model is rigid model. Not flexible & make changes is not possible in the development of system.
• Unable to start the next phase before completing the previous phase so time consuming.
• In the deployment phase all requirements are not covered or all requirements are not cover so adding such requirements may create unsuitable system.
• To make it customer happy such new requirement need to be implemented in new version of system which leads to added cost to system development.

Read More

almirha in indore,mhow,capinets,tijori,all type of steelamlirha,2 or 3 door steel amlirha,almari

almirha in indore,mhow,capinets,tijori,all type of steelamlirha,2 or 3 door steel amlirha,almari,artijori.co.in

Read More

zubaer khan wikipedia

zubaer  khan wikipediahttp://en.wikipedia.org/wiki/User:Zubaer_khan

Read More