TEAMFLY Team-Fly ® Uttama Reddy John Wiley & Sons, Inc. NEW YORK • CHICHESTER • WEINHEIM • BRISBANE • SINGAPORE • TORONTO Wiley Computer Publishing W. H. Inmon Building the Data Warehouse Third Edition Uttama Reddy Uttama Reddy Building the Data Warehouse Third Edition Uttama Reddy Uttama Reddy John Wiley & Sons, Inc. NEW YORK • CHICHESTER • WEINHEIM • BRISBANE • SINGAPORE • TORONTO Wiley Computer Publishing W. H. Inmon Building the Data Warehouse Third Edition Uttama Reddy Publisher: Robert Ipsen Editor: Robert Elliott Developmental Editor: Emilie Herman Managing Editor: John Atkins Text Design & Composition: MacAllister Publishing Services, LLC Designations used by companies to distinguish their products are often claimed as trademarks. In all instances where John Wiley & Sons, Inc., is aware of a claim, the product names appear in initial cap- ital or ALL CAPITAL LETTERS. Readers, however, should contact the appropriate companies for more com- plete information regarding trademarks and registration. This book is printed on acid-free paper. Copyright © 2002 by W.H. Inmon. All rights reserved. Published by John Wiley & Sons, Inc. Published simultaneously in Canada. No part of this publication may be reproduced, stored in a retrieval system or transmitted in any form or by any means, electronic, mechanical, photocopying, recording, scanning or otherwise, except as permitted under Sections 107 or 108 of the 1976 United States Copyright Act, without either the prior written permission of the Publisher, or authorization through payment of the appropriate per-copy fee to the Copyright Clearance Center, 222 Rosewood Drive, Danvers, MA 01923, (978) 750-8400, fax (978) 750-4744. Requests to the Publisher for permission should be addressed to the Permissions Depart- ment, John Wiley & Sons, Inc., 605 Third Avenue, New York, NY 10158-0012, (212) 850-6011, fax (212) 850-6008, E-Mail: PERMREQ @ WILEY.COM. This publication is designed to provide accurate and authoritative information in regard to the subject matter covered. It is sold with the understanding that the publisher is not engaged in professional ser- vices. If professional advice or other expert assistance is required, the services of a competent pro- fessional person should be sought. Library of Congress Cataloging-in-Publication Data: ISBN: 0-471-08130-2 Printed in the United States of America. 10 9 8 7 6 5 4 3 2 1 Uttama Reddy To Jeanne Friedman—a friend for all times Uttama Reddy Uttama Reddy CONTENTS Preface for the Second Edition xiii Preface for the Third Edition xiv Acknowledgments xix About the Author xx Chapter 1 Evolution of Decision Support Systems 1 The Evolution 2 The Advent of DASD 4 PC/4GL Technology 4 Enter the Extract Program 5 The Spider Web 6 Problems with the Naturally Evolving Architecture 6 Lack of Data Credibility 6 Problems with Productivity 9 From Data to Information 12 A Change in Approach 15 The Architected Environment 16 Data Integration in the Architected Environment 19 Who Is the User? 19 The Development Life Cycle 21 Patterns of Hardware Utilization 22 Setting the Stage for Reengineering 23 Monitoring the Data Warehouse Environment 25 Summary 28 Chapter 2 The Data Warehouse Environment 31 The Structure of the Data Warehouse 35 Subject Orientation 36 Day 1-Day n Phenomenon 41 Granularity 43 The Benefits of Granularity 45 An Example of Granularity 46 Dual Levels of Granularity 49 vii Uttama Reddy [...]... External/Unstructured Data and the Data Warehouse External/Unstructured Data in the Data Warehouse Chapter 8 263 295 Chapter 10 The Data Warehouse and the Web 297 Supporting the Ebusiness Environment 307 Moving Data from the Web to the Data Warehouse 307 Moving Data from the Data Warehouse to the Web 308 Web Support 309 Summary 310 Uttama Reddy XII C O N T E N TS Chapter 11 ERP and the Data Warehouse 311 ERP Applications... Normalization/Denormalization 10 2 11 0 Snapshots in the Data Warehouse Meta Data Managing Reference Tables in a Data Warehouse 11 3 11 3 Cyclicity of Data -The Wrinkle of Time 11 5 Complexity of Transformation and Integration 11 8 Triggering the Data Warehouse Record 12 2 12 2 12 3 12 3 Events Components of the Snapshot Some Examples Profile Records 12 4 Managing Volume 12 6 Creating Multiple Profile Records 12 7 Uttama Reddy... Going from the Data Warehouse to the Operational Environment 12 8 Direct Access of Data Warehouse Data 12 9 Indirect Access of Data Warehouse Data 13 0 13 0 13 2 13 3 An Airline Commission Calculation System A Retail Personalization System Credit Scoring Indirect Use of Data Warehouse Data Star Joins 13 7 Supporting the ODS 14 3 Summary 14 5 Granularity in the Data Warehouse 14 7 Raw Estimates 14 8 Input to the Planning... in the Data Warehouse 76 Summary Chapter 3 59 77 AM FL Y Structuring Data in the Data Warehouse The Data Warehouse and Design 81 Beginning with Operational Data 82 Data/ Process Models and the Architected Environment 87 The Data Warehouse and Data Models 89 92 94 98 The Data Warehouse Data Model The Midlevel Data Model The Physical Data Model TE viii The Data Model and Iterative Development 10 2 Normalization/Denormalization... Technology 18 1 Multidimensional DBMS and the Data Warehouse 18 2 Data Warehousing across Multiple Storage Media 18 8 Meta Data in the Data Warehouse Environment 18 9 Context and Content 19 2 19 3 Three Types of Contextual Information Capturing and Managing Contextual Information Looking at the Past 19 4 19 5 Refreshing the Data Warehouse Testing 19 8 Summary 19 8 The Distributed Data Warehouse 2 01 Types of... Outside the Data Warehouse 312 Building the Data Warehouse inside the ERP Environment 314 Feeding the Data Warehouse through ERP and Non-ERP Systems 314 The ERP-Oriented Corporate Data Warehouse 318 Summary 320 Chapter 12 Data Warehouse Design Review Checklist When to Do Design Review Who Should Be in the Design Review? What Should the Agenda Be? The Results Administering the Review A Typical Data Warehouse. .. Placement 17 1 Parallel Storage/Management of Data 17 1 17 1 Meta Data Management Language Interface 17 3 Efficient Loading of Data 17 3 Efficient Index Utilization 17 5 Compaction of Data 17 5 Compound Keys 17 6 Variable-Length Data 17 6 Lock Management 17 6 Uttama Reddy x C O N T E N TS Index-Only Processing 17 8 Fast Restore 17 8 Other Technological Features 17 8 DBMS Types and the Data Warehouse 17 9 Changing... Reddy Preface for the Third Edition This book is the first in a series of books relating to data warehouse The next book in the series is Using the Data Warehouse (Wiley, 19 94) Using the Data Warehouse addresses the issues that arise once you have built the data warehouse In addition, Using the Data Warehouse introduces the concept of a larger architecture and the notion of an operational data store (ODS)... serve as a guideline for the designer and the developer xiii Uttama Reddy PREFACE FOR TH E TH I RD EDITION When the first edition of Building the Data Warehouse was printed, the database theorists scoffed at the notion of the data warehouse One theoretician stated that data warehousing set back the information technology industry 20 years Another stated that the founder of data warehousing should not... Process 14 9 Data in Overflow? Chapter 4 13 6 14 9 15 1 Overflow Storage What the Levels of Granularity Will Be Some Feedback Loop Techniques 15 6 Levels of Granularity-Banking Environment 15 8 Summary Chapter 5 15 5 16 5 The Data Warehouse and Technology 16 7 Managing Large Amounts of Data 16 7 Managing Multiple Media 16 9 Index/Monitor Data 16 9 Interfaces to Many Technologies 17 0 Programmer/Designer Control of Data . 310 CONTENTS xi Uttama Reddy Chapter 11 ERP and the Data Warehouse 311 ERP Applications Outside the Data Warehouse 312 Building the Data Warehouse inside the ERP Environment 314 Feeding the Data. Data Model 98 The Data Model and Iterative Development 10 2 Normalization/Denormalization 10 2 Snapshots in the Data Warehouse 11 0 Meta Data 11 3 Managing Reference Tables in a Data Warehouse 11 3 Cyclicity. 295 Chapter 10 The Data Warehouse and the Web 297 Supporting the Ebusiness Environment 307 Moving Data from the Web to the Data Warehouse 307 Moving Data from the Data Warehouse to the Web 308 Web