Hung Q. Nguyen Bob Johnson Michael Hackett Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems Second Edition 201006 FM.qxd 6/5/03 11:14 AM Page i 201006 FM.qxd 6/5/03 11:14 AM Page iv Hung Q. Nguyen Bob Johnson Michael Hackett Testing Applications on the Web: Test Planning for Mobile and Internet-Based Systems Second Edition 201006 FM.qxd 6/5/03 11:14 AM Page i Executive Publisher: Robert Ipsen Executive Editor: Carol Long Development Editor: Scott Amerman Editorial Manager: Kathryn A. Malm Production Editor: Felicia Robinson Text Design & Composition: Wiley Composition Services Copyright © 2003 by Hung Q. Nguyen, Bob Johnson, and Michael Hackett. All rights reserved. Published by Wiley Publishing, Inc., Indianapolis, Indiana Published simultaneously in Canada No part of this publication may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning, or otherwise, except as permitted under Section 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, Inc., 222 Rose- wood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 646-8700. Requests to the Pub- lisher for permission should be addressed to the Legal Department, Wiley Publishing, Inc., 10475 Crosspoint Blvd., Indianapolis, IN 46256, (317) 572-3447, fax (317) 572-4447, E-mail: permcoordinator@wiley.com. Limit of Liability/Disclaimer of Warranty: While the publisher and author have used their best efforts in preparing this book, they make no representations or warranties with respect to the accuracy or completeness of the contents of this book and specifically disclaim any implied warranties of merchantability or fitness for a particular purpose. No warranty may be created or extended by sales representatives or written sales materials. The advice and strategies contained herein may not be suitable for your situation. You should consult with a professional where appropriate. Neither the publisher nor author shall be liable for any loss of profit or any other commercial damages, including but not limited to special, inci- dental, consequential, or other damages. For general information on our other products and services please contact our Customer Care Department within the United States at (800) 762-2974, outside the United States at (317) 572-3993 or fax (317) 572-4002. Trademarks: Wiley, the Wiley Publishing logo and related trade dress are trademarks or registered trademarks of Wiley Publishing, Inc., in the United States and other countries, and may not be used without written permission. All other trademarks are the property of their respective owners. Wiley Publishing, Inc., is not associated with any product or ven- dor mentioned in this book. Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not be available in electronic books. Library of Congress Cataloging-in-Publication Data: ISBN: 0-471-20100-6 Printed in the United States of America 10 9 8 7 6 5 4 3 2 1 201006 FM.qxd 6/5/03 11:14 AM Page ii To Heather, Wendy, Denny, Leilani, Jesse and Anne, whose love and friendship give me the endless source of energy and happiness. Hung Q. Nguyen To Victoria, for all the advice, help, support, and love she has given me. Bob Johnson To Ron, from whom I have stolen much time to make this book happen. Thank you for your love and support. Michael Hackett 201006 FM.qxd 6/5/03 11:14 AM Page iii 201006 FM.qxd 6/5/03 11:14 AM Page iv Preface xxi Foreword xxiii Acknowledgments xxv About the Authors xxvii Part One Introduction 1 Chapter 1 Welcome to Web Testing 3 Why Read This Chapter? 3 Introduction 4 The Evolution of Software Testing 4 The Gray-Box Testing Approach 7 Real-World Software Testing 9 Themes of This Book 10 What’s New in the Second Edition 12 New Contents and Significant Updates 12 What Remains from the First Edition 13 Chapter 2 Web Testing versus Traditional Testing 15 Why Read This Chapter? 15 Introduction 16 The Application Model 16 Hardware and Software Differences 20 The Differences between Web and Traditional Client-Server Systems 22 Client-Side Applications 22 Event Handling 23 Application Instance and Windows Handling 26 UI Controls 28 Contents v 201006 FM.qxd 6/5/03 11:14 AM Page v Web Systems 28 Hardware Mix 30 Software Mix 30 Server-Based Applications 31 Distributed Server Configurations 32 The Network 33 Bug Inheritance 33 Back-End Data Accessing 34 Thin-Client versus Thick-Client Processing 35 Interoperability Issues 36 Testing Considerations 37 Bibliography 38 Part Two Methodology and Technology 39 Chapter 3 Software Testing Basics 41 Why Read This Chapter? 41 Introduction 42 Basic Planning and Documentation 42 Common Terminology and Concepts 43 Test Conditions 43 Static Operating Environments 43 Dynamic Operating Environments 44 Test Types 46 Acceptance Testing 46 Feature-Level Testing 50 Phases of Development 58 Test-Case Development 60 Equivalence Class Partitioning and Boundary Condition Analysis 60 State Transition 63 Use Cases 66 Example Test Cases from Use Cases 68 Test Cases Built from Use Cases 71 Templates for Use-Case Diagram, Text, and Test Case 75 Condition Combination 75 The Combinatorial Method 78 Bibliography 80 Chapter 4 Networking Basics 81 Why Read This Chapter? 81 Introduction 82 The Basics 82 The Networks 82 The Internet 83 Local Area Networks (LANs) 84 Wide Area Networks (WANs) 85 Connecting Networks 86 Connectivity Services 86 vi Contents 201006 FM.qxd 6/5/03 11:14 AM Page vi Direct Connection 86 Other Network Connectivity Devices 88 TCP/IP Protocols 89 The TCP/IP Architecture 90 Testing Scenarios 93 Connection Type Testing 94 Connectivity Device Testing 97 Other Useful Information 99 IP Addresses and DNS 99 IP Address 100 Network Classes 100 Domain Name System (DNS) 101 Subnet 103 Subnet Masks 105 Custom Subnets 106 A Testing Example 106 Host Name and IP Resolution Tests 106 Testing Considerations 108 Bibliography 110 Chapter 5 Web Application Components 111 Why Read This Chapter? 111 Introduction 112 Overview 112 Distributed Application Architecture 113 Traditional Client-Server Systems 113 Thin- versus Thick-Client Systems 113 Web-Based Client-Server Systems 114 Software Components 116 Operating Systems 117 Application Service Components 117 Third-Party Components 119 Integrated Application Components 119 Dynamic Link Library (DLL) 119 Potential DLL-Related Errors 122 Scripts 123 Web Application Component Architecture 123 Server-Side Components 123 Core Application Service Components 124 Markup Language Pages 125 XML with SOAP 125 Web-to-Database Connectivity 125 Other Application Service Components 128 Client-Side Components 130 Web Browsers 130 Add-on/Plug-in Components 131 Testing Discussion 133 Test-Case Design Analysis 134 Test Partitioning 138 Contents vii 201006 FM.qxd 6/5/03 11:14 AM Page vii Testing Considerations 141 DLL Testing Issues 142 Script Testing Issues 143 Characteristics of a Script 143 Use of Scripts in Web Applications 144 Testing Scripts in Web Applications 145 Coding-Related Problems 145 Script Configuration Testing 147 Bibliography 147 Chapter 6 Mobile Web Application Platform 149 Why Read This Chapter? 149 Introduction 150 What Is a Mobile Web Application? 150 Various Types of Mobile Web Client 151 Palm-Sized PDA Devices 151 Data Synchronizing 152 Web Connectivity 152 Various Types of Palm-Sized PDA Devices 153 Handheld PCs 154 WAP-Based Phones 155 i-Mode Devices 157 Smart Phones or Mobile Phone/PDA Combos 157 Mobile Web Application Platform Test Planning Issues 159 Microbrowsers 159 Web Clipping Application: How Does It Work? 161 Handheld Device Hardware Restrictions 163 Software-Related Issues 164 Wireless Network Issues 166 Wireless Network Standards 166 Wireless Modem 170 Wireless LAN and Bluetooth 170 Other Software Development Platforms and Support Infrastructures 171 The Device Technology Converging Game: Who Is the Winner? 172 Bibliography and Additional Resources 172 Bibliography 172 Additional Resources 173 Chapter 7 Test Planning Fundamentals 177 Why Read This Chapter? 177 Introduction 178 Test Plans 178 Test-Plan Documentation 180 Test-Plan Templates 182 Test-Plan Section Definitions 182 viii Contents 201006 FM.qxd 6/5/03 11:14 AM Page viii [...]... security testing, and what are my testing responsibilities?” “What do I need to consider in testing mobile Web applications? ” With a combination of general testing methodologies and the information contained in this book, you will have the foundation required to achieve these testing goals—maximizing productivity and minimizing quality risks in a Web application environment Testing Applications on the Web. .. appreciate the complexity of the Web testing problem, the first edition of Testing Applications on the Web became the first book on gray-box testing In so-called black-box testing, we treat the software under test as a black box We specify the inputs, we look at the outputs, but we can’t see inside the box to see how it works The black-box tester operates at the customer’s level, basing tests on knowledge... and anyone who has the responsibility of planning and managing Web- application test projects Testing Applications on the Web begins with an introduction to the clientserver and Web system architectures It offers an in-depth exploration of Web application technologies such as network protocols, component-based architectures, and multiple server types from the testing perspective It then covers testing. .. Compliance Tests Configuration and Compatibility Tests Connectivity Tests Devices with Peripheral Network Connections Latency Transmission Errors Transitions from Coverage to No-Coverage Areas Transitions between Data and Voice Data or Message Race Condition Performance Tests Security Tests Testing Web Applications Using an Emulation Environment Testing Web Applications Using the Physical Environment 516... FM.qxd 6/5/03 11:14 AM Page xxi Preface Testing Applications on the Web introduces the essential technologies, testing concepts, and techniques that are associated with browser-based applications It offers advice pertaining to the testing of business-to-business applications, business-to-end-user applications, Web portals, and other Internet-based applications The primary audience is software testers,... xiii Contents Netscape NetHelp ActiveX Controls Help Elements Approaching Help Testing Two-Tiered Testing Stand-alone Testing Interaction between the Application and the Help System Types of Help Errors Testing Considerations Bibliography Chapter 16 Installation Tests Why Read This Chapter? Introduction The Roles of Installation/Uninstallation Programs Installer Uninstaller Common Features and Options... systems The second edition continues the gray-box analysis by deepening the discussions in the first edition It also adds several new chapters to address business-critical testing issues from server-side, performance- and applicationlevel security testing to the latest mobile Web application testing A final strength of the book is the power of the real-world example Hung Quoc Nguyen is the president of the. .. Setup Options Installation Sources and Destinations Server Distribution Configurations Server-Side Installation Example Media Types Branching Options Common Server-Side-Specific Installation Issues Installer/Uninstaller Testing Utilities Comparison-Based Testing Tools InControl4 and InControl5 Norton Utilities’ Registry Tracker and File Compare Testing Considerations Bibliography and Additional Resources... Test Data Setting Up the Test Bed Setting Up the Test Suite Parameters Performance Test Run Example Analysis Phase Other Testing Considerations Bibliography Chapter 20 Testing Mobile Web Applications Why Read This Chapter? Introduction Testing Mobile versus Desktop Web Applications Various Types of Tests Add -on Installation Tests Data Synchronization-Related Tests UI Implementation and Limited Usability... Introduction ◆ The Evolution of Software Testing ◆ The Gray-Box Testing Approach ◆ Real-World Software Testing ◆ Themes of This Book ◆ What’s New in the Second Edition Introduction This chapter offers a historical perspective on the changing objectives of software testing It touches on the gray-box testing approach and suggests the importance of having a balance of product design, both from the designer’s . 520 Other Testing Considerations 523 Bibliography 525 Chapter 20 Testing Mobile Web Applications 527 Why Read This Chapter? 527 Introduction 528 Testing Mobile versus Desktop Web Applications. 542 Transitions between Data and Voice 542 Data or Message Race Condition 542 Performance Tests 543 Security Tests 544 Testing Web Applications Using an Emulation Environment 544 Testing Web Applications. Page vii Testing Considerations 141 DLL Testing Issues 142 Script Testing Issues 143 Characteristics of a Script 143 Use of Scripts in Web Applications 144 Testing Scripts in Web Applications 145 Coding-Related