7 Testing and Debugging CERTIFICATION OBJECTIVES 7.01 Creating a Unit Test Plan 7.02 Implementing Tracing 7.03 Instrumenting and Debugging a Windows Service, a Serviced Component, a .NET Remoting Object, and an XML Web Service 7.04 Using Interactive Debugging 7.05 Logging Test Results ✓ Two-Minute Drill Q&A Self Test CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:38 AM Color profile: Generic CMYK printer profile Composite Default screen CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 I n this chapter, you will learn about the testing process and specifically how unit testing fits into the overall testing of the application. You will also look at what tracing is and how you can make use of it, as well as how you implement your remote components and retrieve that information. Debugging and especially how you debug remote components interactively will also be handled here. Another interesting area you will look at is how you can use SOAP extensions to debug the components. CERTIFICATION OBJECTIVE 7.01 Creating a Test Plan on the Evolutionary Model This section will deal with software testing and the unit test in particular. Unit testing takes the smallest part of software practical and validates its function. The unit test must be planned and documented according to what is called the unit test plan. The software that usually is produced to perform the unit test is called the test harness—software that has stubs and functions that can call on and respond to any functionality in the unit being tested. Software testing is a high-priority activity in the production of software applications. Among other things, the testing process ■ Measures the quality of the software The assumption is that there will always be flaws in the software that are waiting to be discovered. By testing and debugging prior to release, you can ensure that your software is regarded as high-quality. ■ Validates that the application behaves as the user expects it to The application is tested to ensure that it behaves as described in the documentation, and as the users expect the application to work. ■ Reduces the cost of development Testing will reduce those costly last-minute errors that traditionally have delayed products and added to the cost. The unit test methodology results in individual software components being tested and validated before they are assembled into increasingly more complex units. ■ Reduces the cost of ownership by minimizing the cost of maintenance Maintenance cost is determined on how much testing must be performed to validate the code after the maintenance. Because the testing plans are already written the unit and integration tests are faster and more accurate. 2 Chapter 7: Testing and Debugging P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen Creating a Test Plan on the Evolutionary Model 3 CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 ■ Replaces the traditional development and test planning process The traditional process follows the waterfall model, where one task flows into the next in an ordered manner as in the following list: ■ Analyze requirements ■ Create designs and specifications ■ Create the code ■ Test ■ Release When you reach the testing phase in this model, the application is already finished, and when problems are identified, there is a major issue: in order to make repairs, you make changes, but those changes will modify the application, so you need to start testing again. This cycle of testing and modifications on the finished application can be never-ending. The development cost also increases enormously—there are estimates that the cost can increase by 100 times when testing is delayed until the application is finished. Unit testing becomes almost automatic because the methodology leads us to test as you go along, building test harnesses to validate your implementation. The waterfall model matched the development model for most procedural environments, but when you moved into object-oriented development, the development model changed to the evolutionary model, where testing is an ongoing process that is integrated into the development process. The steps in the evolutionary model are to 1. Analyze requirements: ■ Discover classes ■ Develop classes ■ Test the classes 2. Repeat until the system is complete The repetitious evolution of the application by building increasingly more complex parts of the application on already tested units makes it possible to arrive at the end of the development project with a fully tested application. The additional payback is that at the end when the complete application is assembled from all the units, there will not be any devastating errors that require rewriting parts of the application. P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen 4 Chapter 7: Testing and Debugging CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 Expect questions that make you select between the waterfall model and the evolution model. You can rest assured that the evolution model will be the right answer. The evolution model uses the iterative steps from object-oriented analysis and design (OOAD) to continuously analyze the application and build fully tested parts of that application before moving on to the next cycle, which starts with analysis again. The unit test becomes a central task in the development of the application. As you move forward, you will have more and more fully tested units that can be assembled into larger units that are then tested, and so on until the application is finished. Testing should be planned in a fashion similar to the main development effort. Planning for testing should be requirements based, drawing the test design from the requirements section of the software specification. Testing generally identifies defects (bugs) that allow, create, or cause unexpected behaviors in terms of the requirements. The requirements for the software must be written in such a way that they can be directly translated into test documentation. For example, requirements should be ■ Binding The customer demanded them. ■ Testable If you can’t test the requirement, there is no way of proving compliance. ■ Clear They must be unambiguous and interpretable in only one way. Planning then proceeds by designing test cases that will validate each of the requirements. The test plan will outline the entire process with the individual test cases included. The development of a solid test plan is built on the systematic analysis of the application to make sure that everything is tested without repetitions. The plan ensures that testing procedures are known and do not depend on accidental or random testing. One closely related task to testing is optimization. The optimization task is the process that removes bottlenecks (overuse of resources) in the software and hardware to produce the best combination that minimizes resource use. After performing optimization, you must perform the tests for each unit again to ensure that the optimization exercise did not break the application. Testing The testing exercise consists of three related tasks that together ensure the quality of the developed software. They are P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen Creating a Test Plan on the Evolutionary Model 5 CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 ■ The unit test Testing the smallest possible software unit ■ The integration test Testing combinations of software units ■ The regression test Retesting after modifications to the software units The exam focuses on the unit test, but to fully understand the testing methodology, you will cover the other two as well. Unit Testing The goal of unit testing is to take the smallest possible testable software that is part of the application, remove it from the remainder of the software, and test to ensure it behaves as expected. Unit testing is a natural fit with the OOAD methodology used for modern software design because it flows naturally from the unit-by-unit iterative development that you are used to from the OOP world. The testing process usually requires that you write a test harness that is used to communicate with the software unit. The terminology used to describe this scenario is that you will write a driver to simulate the caller and possibly a stub to simulate a called unit. The test harness becomes part of the applications code base and will be used for validation through the lifetime of the application. The cost in time that is involved in writing the test harness makes it tempting to try to test larger units rather than identifying and testing the smallest possible unit. The problem with testing larger units is that it increases the difficulty in identifying where a problem is located. If you have two software units and you decide to test them together as one unit, you may face the following list of questions, which gives you an idea of the complexity of finding a bug: ■ Is the error caused by a defect in the first unit? ■ Is the error caused by a defect in the second unit? ■ Is the error caused by a defect in the test harness? ■ Is the error caused by a defect in both the first and second units? ■ Is the error caused by a defect in the interface between the two units? The complexity introduced makes it very hard to find the source of the defect—this is one compelling reason for implementing unit testing. Always select the smallest possible unit for the test. P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen 6 Chapter 7: Testing and Debugging CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 EXERCISE 7-3 Unit Testing In this exercise, you will investigate writing a test harness. For this example, you have chosen a class as the unit you will test. The exercise is based on a console application to minimize the complexities. 1. Open a Visual Studio .NET Command Prompt (Start | Programs | Microsoft Visual Studio .NET | Visual Studio .NET Tools | Visual Studio .NET Command Prompt). 2. Change to the C:\VB directory, or create it if you do not have that directory. 3. Make a new directory named Testing (md Testing). 4. Change to the Testing directory (cd Testing). 5. Create a new Visual Basic .NET class file using your favorite editor. Name the file Class.vb. 6. Enter the following source code in the Class.vb source file: Public Class Temperature Private m_temperature As Integer Public Property Temp As Integer Get Return m_temperature End Get Set(value As Integer) m_temperature = value – 273 ' Kelvin End Set End Property Public Function Celsius() As Integer Return m_temperature End Function Public Function Fahrenheit() As Integer Return m_temperature End Function End Class 7. Next, write the test harness to test the Temperature class. The logic behind the test is that you will test the Celsius() and Fahrenheit() functions to ensure that the class returns the correct value. The test plan is to use a value of 30 Celsius and verify that it is correctly returned. Create a new source file in the Testing directory; name the file Harness.vb. P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen Creating a Test Plan on the Evolutionary Model 7 CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 8. Enter the following Visual Basic .NET code in the Harness.vb file: Public Class Tester Public Shared Sub Main() Dim t As Temperature = New Temperature() Static Thirty As Integer = 30 t.Temp = Thirty ' assign the property 30 Celsius ' Now test the behavior when the temperature is set to 30 C Select Case t.Celsius() Case Thirty System.Console.Out.WriteLine( _ "The value is correct: {0} (Expected {1}C)", _ t.Celsius, Thirty) Case Else System.Console.Out.WriteLine( _ "The value is incorrect: {0} (Expected {1}C)", _ t.Celsius, Thirty) End Select Select Case t.Fahrenheit() Case ((Thirty* 9 / 5)+32) System.Console.Out.WriteLine( _ "The value is correct: {0} (Expected {1}F)", _ t.Fahrenheit, ((Thirty* 9 / 5)+32)) Case Else System.Console.Out.WriteLine( _ "The value is incorrect: {0} (Expected {1}F)", _ t.Fahrenheit, ((Thirty* 9 / 5)+32)) End Select End Sub End Class 9. Save the source files and compile them using this command line: C:\VB\Testing>vbc Harness.vb Class.vb 10. Run the harness program; the following is the output: C:\VB\Testing>harness The value is incorrect: -243 (Expected 30C) The value is incorrect: -243 (Expected 86F) As you can see, there is a problem, and if you look at the Temperature class, you will find that the temperature is stored using the absolute Kelvin scale, but there is no conversion in the Celsius() or Fahrenheit() functions. The next step provides the solution to the Celsius() function problem. P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:39 AM Color profile: Generic CMYK printer profile Composite Default screen 8 Chapter 7: Testing and Debugging CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 11. Correct the Celsius() function to correctly convert from Kelvin to Celsius. Here is the correct conversion: Public Function Celsius() As Integer Return m_temperature + 273 ' convert from Kelvin to Celsius End Function 12. Save and compile the harness program. 13. Execute the harness program. This should be the result: C:\VB\Testing>harness The value is correct: 30 (Expected 30C) The value is incorrect: -243 (Expected 86F) 14. Correct the problem with the Fahrenheit() function using the following code segment: Public Function Fahrenheit() As Integer Return (((m_temperature + 273) * 9 ) / 5 ) + 32 End Function 15. Save and compile the harness program. 16. Execute the harness program. C:\VB\Testing>harness The value is correct: 30 (Expected 30C) The value is correct: 86 (Expected 86F) Having tested the Temperature class, you can move on to other classes in the application. Integration Testing When the units of the application are completed and tested, they will be assembled (integrated) into larger units. These new integrated units need to be tested as well, in what is called integration testing. The most basic integration test takes two tested units and tests the interface between those two units. This process continues until all units have been integrated. By working with only two units at a time, the integration testing becomes manageable, and locating a defect will be much easier and faster. There are three approaches to integration testing: ■ The top-down strategy In this strategy, you start at the top of the application and integrate the units to ensure that high-level logic and dataflow are tested early in the development cycle. The top-down strategy minimizes the need for drivers (callers), but the need for stubs complicates management of the test. The low-level units are tested late in the development cycle. The top-down strategy P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:40 AM Color profile: Generic CMYK printer profile Composite Default screen Creating a Test Plan on the Evolutionary Model 9 CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 has very poor support for early (proof of concept), limited-functionality releases of the application. ■ The bottom-up strategy This strategy starts with the integration of the lowest-level units first, with the result that the utility level of the application is tested early in the process. This strategy minimizes the need for stubs while demanding drivers; the high-level logic and data flow are not tested until the later stages of the development cycle. The support for early releases is poor. ■ The umbrella strategy In this strategy, you test along the lines of functional control and data-flow paths. The logic is to integrate low-level functions first—as in the bottom-up strategy—after which the output from each function is integrated in the top-down manner. This strategy minimizes the need for stubs and drivers but also makes management of the testing effort more complicated. The umbrella strategy leads to the possible early release of limited-functionality proof-of-concept versions. The integration testing is completed when the application is ready to be shipped to the customer. The most common model for testing is a combination of the umbrella model and one of the other two models. That way, the early test versions can be brought together with the users to validate the design as well as the formalized top-down or bottom-up model. One word of advice, though: ensure that management of the testing is tightly controlled. Regression Testing Regression testing refers to the retesting of units after the unit has been modified. You perform the regression test by rerunning the original tests that were designed for that unit. The testing will determine if the modification broke the unit or not. Regression testing should not be a long process; rather, it should be a very quick go–no go test. Here are some strategies that can be used during the testing: ■ Look out for side effects of fixes that have been incorporated. ■ Write one regression test for each bug that was fixed. ■ Test fixed bugs directly after the bug is fixed to ensure that the fix has no side effects. ■ If multiple tests are similar, throw away the least efficient ones. ■ Tests that always pass the program should be archived for historical reasons. ■ Test functionality, not design. P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:40 AM Color profile: Generic CMYK printer profile Composite Default screen 10 Chapter 7: Testing and Debugging CertPrs8 / MCAD/MCSD XML Web Services and Server Components Development with Visual Basic .NET / Lind / 222653-6 / Chapter 7 ■ Vary the data for the test, look for data corruption. Try to overflow buffers, test for logical behavior. ■ Track the memory use of the program. The groups of tests that need to be maintained in order to be able to perform regression testing are most effectively stored in a library that constitutes a battery of standard tests that are used whenever the program has to be tested. One of the tricky aspects of the test library is to decide what to keep. Don’t spend too much time analyzing the merits of a particular test case; if you can’t make up your mind, add it to the library. The content of the library should be analyzed every so often to remove duplicated test cases as well as invalid tests. Software testing is as much a science as is software design, and the preceding information is included as an overview of the topic. Remember that the exam will assume that you know how to write a test harness for a unit. Providing Test Data to Components and Applications The selection of test data for use with the test cases is as important as the design of the test cases themselves. The data that is used must be selected from the problem domain of the development project. The data is usually identified during the analysis phase of the project and should be verified with the domain experts of the project to ensure the data is valid. When developing components that quite possibly will be used in an international environment, it is imperative that you provide for data that is valid for the different locales of the users of the component. To that end, work with the domain experts to identify the different locales that will be supported in the design and create data definitions for each locale. The most important aspect of the localized component is that the tests must provide data in the format of the user locale and test that the component performs the correct conversions. For example, if the locale of the user presents dates in the YYYY.MM.DD format (2002.09.07), the test data must include that format to ensure that the conversions work. CERTIFICATION OBJECTIVE 7.02 Logging Test Results The performance of tests is one part of testing—but there is an equally important part that involves the logging or filing of the test data. The logging can be as low-tech as P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:40 AM Color profile: Generic CMYK printer profile Composite Default screen [...]... XML 30 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging SELF-TEST ANSWERS Using the SOAP Extensions for Debugging 1 þ A and D The namespaces for the SOAP extensions are System.Web.Services Protocol.SoapExtension and System.Web.Services.Protocol SoapExtensionAttribute ý B and C Answers B and C are incorrect because there... use the command-line parameters in the Main() method to set the trace switches when the application starts up, or in the case of applications that do not start from the command line, you use the web.config file to set the switch values CERTIFICATION SUMMARY In this chapter, you have explored testing and debugging issues From the point of the exam, unit testing is the most important form of testing Remember... will be easy to design and the iterative nature of OOAD will lend itself to providing a functional test plan Color profile: Generic CMYK printer profile CertPrs8 Composite Default screen / MCAD/MCSD XML 12 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging CERTIFICATION OBJECTIVE 7.03 Instrumenting and Debugging a Service... between multiple units The evolution methodology maximizes test reuse and minimizes cost for testing in that the software units are tested and errors are found early in the development cycle when the costs of fixes are relatively low Debugging during development is achieved by using the Debug class and the methods available that will write debugging information to a file or the event log for further analysis... CertPrs8 Composite Default screen / MCAD/MCSD XML 14 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging The TraceLevel enumeration is used to set the Level property of the TraceSwitch Using this technique, you can use the Main() method and command parameters to indicate the detail of tracing that will take place, as is... CertPrs8 Composite Default screen / MCAD/MCSD XML 22 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging End Get End Property End Class The purpose of the ExtensionType property is to ensure the extension can be found 6 Now comes the fun stuff Define a class and name it SExt It inherits from SoapExtension, as in the following code:... Save and test the XML Web Service to ensure the conversions are correct The help page looks like this: P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:41 AM Color profile: Generic CMYK printer profile CertPrs8 Composite Default screen / MCAD/MCSD XML 16 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging. .. Event Log, and locate the events for CtoF() and FtoC() respectively, as shown in the next two illustrations P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:41 AM Color profile: Generic CMYK printer profile CertPrs8 / screen Composite DefaultMCAD/MCSD XML Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Instrumenting and Debugging a... SOAP information as it travels between the client and the server P:\010Comp\CertPrs8\653-6\ch07.vp Wednesday, October 30, 2002 9:50:42 AM Color profile: Generic CMYK printer profile CertPrs8 Composite Default screen / MCAD/MCSD XML 26 Chapter 7: ✓ Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging TWO-MINUTE DRILL Creating a Unit Test... XML 20 Chapter 7: Web Services and Server Components Development with Visual Basic NET / Lind / 222653-6 / Chapter 7 Testing and Debugging functionality, called the SOAP extensions, allows us to work with the data outside of the SOAP environment In order to take advantage of the SOAP extensions, you need to derive one class from System.Web.Services.Protocol.SoapExtension and one class from System.Web.Services.Protocol.SoapExtensionAttribute . 7 Testing and Debugging CERTIFICATION OBJECTIVES 7.01 Creating a Unit Test Plan 7.02 Implementing Tracing 7.03 Instrumenting and Debugging a. The plan ensures that testing procedures are known and do not depend on accidental or random testing. One closely related task to testing is optimization.