Eating the IT Elephant: Moving from Greenfield Development to Brownfield by Richard Hopkins; Kevin Jenkins Publisher: IBM Press Pub Date: May 06, 2008 Print ISBN-10: 0-13-713012-0 Print ISBN-13: 978-0-13-713012-2 eText ISBN-10: 0-13-714946-8 eText ISBN-13: 978-0-13-714946-9 Pages: 256 Table of Contents | Index Overview A Practical, Start-to-Finish Approach to Managing, Evolving, and Transforming Legacy IT Systems For every IT executive, manager, architect, program leader, project leader, and lead analyst "Richard and Kevin introduce us to a reality that's often neglected in our industry: the problem of evolving legacy systems, a domain they call 'Brownfield development.' The authors identify the root of the problem as that of complexity, and offer an approach that focuses on the fundamentals of abstraction and efficient communication to nibble at this problem of transformation bit by bit As the old saying goes, the way you eat the elephant is one bite at a time Richard and Kevin bring us to the table with knife and fork and other tools, and show us a way to devour this elephant in the room." Grady Booch, IBM Fellow, co-creator of UML "Most organizations in the 21st century have an existing, complex systems landscape It is time that the IT industry face up to the reality of the situation and the need for new development methods and tools that address it This book describes a new approach to the development of future systems: a structured approach that recognizes the challenges of 'Brownfield' development, is based on engineering principles, and is supported by appropriate tooling." Chris Winter, CEng CITP FBCS FIET, IBM Fellow, Member of the IBM Academy of Technology Most conventional approaches to IT development assume that you're building entirely new systems Today, "Greenfield" development is a rarity Nearly every project exists in the context of existing, complex system landscapes often poorly documented and poorly understood Now, two of IBM's most experienced senior architects offer a new approach that is fully optimized for the unique realities of "Brownfield" development Richard Hopkins and Kevin Jenkins explain why accumulated business and IT complexity is the root cause of large-scale project failure and show how to overcome that complexity "one bite of the elephant at a time." You'll learn how to manage every phase of the Brownfield project, leveraging breakthrough collaboration, communication, and visualization tools including Web 2.0, semantic software engineering, model-driven development and architecture, and even virtual worlds This book will help you reengineer new flexibility and agility into your IT environment…integrate more effectively with partners… prepare for emerging business challenges… improve system reuse and value…reduce project failure rates…meet any business or IT challenge that requires the evolution or transformation of legacy systems · System complexity: understand it, and harness it Go beyond the comforting illusion of your high-level architecture diagrams · How conventional development techniques actually make things worse Why traditional decomposition and abstraction don't work-and what to do instead · Reliably reengineer your IT in line with your business priorities New ways to understand, communicate, visualize, collaborate, and solve complex IT problems · Cut the elephant down to size, one step at a time Master all four phases of a Brownfield project: survey, engineer, accept, and deploy Eating the IT Elephant: Moving from Greenfield Development to Brownfield by Richard Hopkins; Kevin Jenkins Publisher: IBM Press Pub Date: May 06, 2008 Print ISBN-10: 0-13-713012-0 Print ISBN-13: 978-0-13-713012-2 eText ISBN-10: 0-13-714946-8 eText ISBN-13: 978-0-13-714946-9 Pages: 256 Table of Contents | Index Copyright Foreword by Grady Booch Foreword by Chris Winter Preface Acknowledgments About the Authors Part I: Introducing Brownfield Chapter 1 Eating Elephants Is Difficult Today's Delivery Methods Why Do Big Projects Fail? Environmental Complexity Brownfield Sites Must Be Surveyed Endnotes Chapter 2 The Confusion of Tongues Introducing Brownfield Key Communication Problems Overcoming Communication Complexity Endnotes Chapter 3 Big-Mouthed Superhero Required Elephant-Eating Strategies Consuming the Environment Architecting the Elephant Eater The Elephant Eater in Action The Brownfield Beliefs Endnotes Chapter 4 The Trunk Road to the Brain Alternative Wallpapers Invading Hilbert Space Architecture Is the Solution Bridging the Business/IT Gap Endnotes Chapter 5 The Mythical Metaman When You Eliminate the Impossible The Long Tail of Business Services Business Attractors for Attractive Businesses The Death of Brownfield Endnotes Part II: The Elephant Eater Chapter 6 Abstraction Works Only in a Perfect World Considerations for an Elephant Eater Systems Integration and Engineering Techniques Abstraction Is the Heart of Architecture Do We Need a Grand Unified Tool? The Connoisseur's Guide to Eating Elephants Endnotes Chapter 7 Evolution of the Elephant Eater The Sources of Brownfield Isn't This CASE? Isn't This MDA? Endnotes Chapter 8 Brownfield Development Agile Waterfalls The Brownfield Development Approach Endnotes Chapter 9 Inside the Elephant Eater Looking Inside the Elephant Eater Step 1: Parse View and Identify Patterns Step 2: Merge Views Step 3: Create Transforms Step 4: Generate Artifacts Steps 5.1: Test Artifacts and 5.1a: Identify Generation Faults Step 5.1b: Add and Update Information A Portrait of an Elephant Eater Endnotes Chapter 10 Elephant Eater at Work Making the Move to Brownfield Taking the First Step A Better Way to Build Interfaces A Better Way to Build an Enterprise Service Bus The End of Middleware? Evolving a Deployable Enterprise Architecture Index Copyright The authors and publisher have taken care in the preparation of this book, but make no expressed or implied warranty of any kind and assume no responsibility for errors or omissions No liability is assumed for incidental or consequential damages in connection with or arising out of the use of the information or programs contained herein © Copyright 2008 by International Business Machines Corporation All rights reserved Note to U.S Government Users: Documentation related to restricted right Use, duplication, or disclosure is subject to restrictions set forth in GSA ADP Schedule Contract with IBM Corporation IBM Press Program Managers: Tara Woodman, Ellice Uffer Cover design: IBM Corporation Associate Publisher: Greg Wiegand Marketing Manager: Kourtnaye Sturgeon Publicist: Heather Fox Acquisitions Editor: Katherine Bull Development Editors: Kevin Ferguson, Ginny Bess Managing Editor: Gina Kanouse Designer: Alan Clements Senior Project Editor: Lori Lyons Copy Editor: Krista Hansing Indexer: Lisa Stumpf Compositor: Nonie Ratcliff Proofreader: Anne Goebel Manufacturing Buyer: Dan Uhrig Published by Pearson plc Publishing as IBM Press IBM Press offers excellent discounts on this book when ordered in quantity for bulk purchases or special sales, which may include electronic versions and/or custom covers and content particular to your business, training goals, marketing focus, and branding interests For more information, please contact: U S Corporate and Government Sales 1-800-382-3419 corpsales@pearsontechgroup.com For sales outside the U S., please contact: International Sales international@pearsoned.com The following terms are trademarks or registered trademarks of International Business Machines Corporation in the United States, other countries, or both: IBM, the IBM logo, IBM Press, AD/Cycle, DB2, developerWorks, Rational, System 360, Tivoli and WebSphere Java and all Java-based trademarks are trademarks of Sun Microsystems, Inc in the United States, other countries, or both Microsoft, Windows, Windows NT, and the Windows logo are trademarks of Microsoft Corporation in the United States, other countries, or both Other company, product, or service names may be trademarks or service marks of others Library of Congress Cataloging-in-Publication Data Hopkins, Richard Eating the IT elephant : moving from greenfield development to brownfiel Hopkins and Kevin Jenkins p cm Includes index ISBN 0-13-7130120 (pbk : alk paper) Information technology Business enterprises—Planning I Jenkins, Kevin II Title T58.5.H69 2008 004.068—dc22 2008004231 All rights reserved This publication is protected by copyright, and permission must be obtained from the publisher prior to any prohibited reproduction, storage in a retrieval system, or transmission in any form or by any means, electronic, mechanical, photocopying, recording, or likewise For information regarding permissions, write to: Pearson Education, Inc Rights and Contracts Department 501 Boylston Street, Suite 900 Boston, MA 02116 Fax (617) 671 3447 ISBN-13: 978-013713012 Text printed in the United States on recycled paper at R.R Donnelley in Crawfordsville, Indiana First printing May 2008 Dedication To Mam, Dad, Lyndsay, and the three best kids in the world: Kirsty, Julia, and Nicholas They would have preferred a book on real elephants, but at least they approve of the cover —R.H To Anna, for supporting me through the stressful times of delivering large projects —K.J Foreword by Grady Booch A simple back-of-the-envelope calculation suggests that, worldwide, we produce about 33 billion lines of new or modified code every year Cumulatively, this means that since the 1940s and '50s (when higher order programming languages began to gain some traction), we've produced somewhere around one trillion source lines of code On the one hand, this volume of output suggests that ours is an incredibly vibrant and innovative industry On the other hand, it's a humbling thought, for through those trillion lines of code, all handcrafted by individual human labor, we've changed the world Truth be told, some nontrivial percentage of the 33 billion lines yearly is dead on arrival or so transitory that it's thrown away quickly Much of that code, however, has a longer half-life, and even some of that code lives after 10, 20, or even 30 or more years For many developers, the code they write today becomes tomorrow's legacy that their children or their children's children may stare at some day, trying to use it, adapt it, evolve it, asking the question, "What the heck was this developer thinking?" Greenfield development, quite honestly, is great fun, simply because you get to start with a clean slate and, thus, are not burdened by anything from the past For the most part, we teach Greenfield development in our schools; furthermore, start-up companies look so much more nimble than their older counterparts because they don't have the millstone of legacy around their necks Woe be unto the student who enters the real world (it's not like being at the university, unless you move from the college womb to the start-up womb immediately), and woe be unto the start-up company that begins to mature into sustainable development and soon realizes that you can't just start over creating Elephant Eaters empowering business change interfaces Mythical Man Month, The Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Naur, Peter nonfunctional requirements Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Occam's Razor OMG (Open Management Group) On Demand ontologies Open Management Group (OMG) organization, risk areas of project failure outputs of Brownfield development approach OWL (Web Ontology Language) 2nd Inventory 2nd Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Palmisano, Sam parable of the blind men and the elephant parochialism Views parsing Views Pattern Driven Engineering, MDA (Model Driven Architecture) patterns identifying Peopleware phases of Brownfield development approach physically separated teams PIM (Platform Independent Model) planning, risk areas of project failure Platform Independent Model (PIM) Platform Specific Model (PSM) plumbing, comparison PowerPoint precision architectures, merging views presentations private languages process flows project reporting, risk areas of project failure PSM (Platform Specific Model) Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] quality assurance checkpoints Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] RAD (Rapid Application Development) RDF (Resource Description Framework) 2nd RDF graphs RDF/XML extract regulatory compliance, IT spending reporting on projects, risk areas of project failure representing triples requirements conflicting requirements functional requirements nonfunctional requirements risk areas of project failure Resource Description Framework (RDF) reuse, Brownfield Beliefs reversing MDA (Model Driven Architecture) ripple effect 2nd abstraction effects of environmental complexity risk areas of project failure change management globalization induced complexity organization and planning project reporting requirements RSA (Rational Software Architect) rules Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Sarbanes-Oxley Second Life semantic technologies semantic web 2nd 3rd semantics Elephant Eaters, consuming the environment overcoming inconsistency and ambiguity Service Oriented Architecture (SOA) services customers dynamic services singing pigs site surveys 2nd 3rd 4th Brownfield sites skills, developing for Elephant Eaters SOA (Service Oriented Architecture) software archaeology business options structures software engineering, abstraction sources of Brownfield 2nd splitting systems integration Standish Group static testing steady state, IT spending Stock, Gregory stove-pipe systems strange attractors strategies for elephant-eating structure of Inventory structures, software archaeology subphases of Brownfield development approach Survey Phase, Brownfield development approach syntax overcoming inconsistency and ambiguity System/360 (IBM) 2nd systems integration bottom-up approach splitting or abstraction top-down approach Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] TADDM (Tivoli Application Dependency Discovery Manager) 2nd taxonomy teams, physically separating Test Artifacts testing Artifacts Brownfield, early testing static testing transforms, artifacts text messages The Open Group Architecture Framework (TOGAF) three-dimensional displays time dimensions, merging Views time slices Tivoli Application Dependency Discovery Manager (TADDM) 2nd TOGAF (The Open Group Architecture Framework) tool unification tools, developing top-down approach, systems integration transformations Transforms creating Artifacts Elephant Eater architecture merging Views testing transparency, lack of triples Inventory representing truth, Brownfield Beliefs Turner, Richard Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] UML (Unified Modeling Language) 2nd UML diagrams unified tools updating information use cases 2nd Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Venn diagrams Views 2nd communication ambiguity inconsistency parochialism Elephant Eater architecture merging 2nd extracting information identifying missing or incorrect information precision architectures time dimensions 2nd transforms one-size-fits-all approach parsing software archaeology splitting virtual worlds (v-worlds) 2nd Second Life visualizations VITA (Views, Inventory, Transforms, and Artifacts) 2nd Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] W3C (World Wide Web Consortium) waterfall development waterfall methods agile approach to problems versus agile methods Web 2.0 Web Ontology Language [See OWL (Web Ontology Language).] WebSphere Business Modeler 2nd World Wide Web Consortium (W3C) 2nd Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] XML Index [A] [B] [C] [D] [E] [F] [G] [H] [I] [J] [L] [M] [N] [O] [P] [Q] [R] [S] [T] [U] [V] [W] [X] [Z] Zachman Framework ... engineer, accept, and deploy Eating the IT Elephant: Moving from Greenfield Development to Brownfield by Richard Hopkins; Kevin Jenkins Publisher: IBM Press Pub Date: May 06, 2008 Print ISBN- 10: 0-13-713012-0 Print ISBN- 13: 978-0-13-713012-2... For many developers, the code they write today becomes tomorrow's legacy that their children or their children's children may stare at some day, trying to use it, adapt it, evolve it, asking the question, "What the heck was this developer... systems landscapes It is time that the IT industry face up to the reality of the situation and the need for new development methods and tools that address these issues and take our industry into the twenty-first century