Books for professionals By professionals ® US $59.99 Shelve in: Databases / Oracle User level: Intermediate–Advanced THE APRESS ROADMAP Pro Oracle Database 11g Administration Pro Oracle SQL Beginning Oracle Database 11g Administration Beginning Oracle SQL Pro Oracle Database 11g RAC on Linux Expert Oracle Practices www.apress.com SOURCE CODE ONLINE Companion eBook Steve Shaw, Author of Pro Oracle Database 10g RAC on Linux Martin Bach Pro Oracle Database 11g RAC on Linux Dear Reader, You are holding in your hands the result of our experience and best-thinking when it comes to installing, configuring, and tuning Oracle Database 11g RAC on Linux. Real Application Clusters, or RAC as it is commonly called, is Oracle’s industry- leading architecture for building scalable and fault-tolerant databases. RAC pro- vides redundancy through multiple servers, allowing you to scale up and down simply by adding or subtracting servers. Using practical examples and illustrations we take you through all the stages of building the infrastructure for the latest 11g Release 2 clustered environments from selecting the right hardware components to installing and configuring Oracle Enterprise Linux. We detail how to install and configure Oracle VM—Oracle’s own virtualization solution—to enable anyone to begin working with Oracle RAC straight away. We show the spectrum of configurations from single server to a fully virtualized RAC implementation. Building upon the Linux foundation, you will see how to successfully imple- ment Oracle’s Grid infrastructure and RAC database software even when upgrad- ing from an Oracle Database 10g release. You will also learn how to manage and monitor your new clustered installation through workload management and per- formance monitoring, and parallel execution. We make no assumptions on your experience with Oracle 11g RAC Release 2, or with Linux. Our goal in this book is to provide a complete reference to all of the information you will need, beginning with the essential grounding of concepts and architecture. We have comprehensively researched, tested, and detailed every step of the process so this book can be your guide to taking the next step in the evolu- tion of grid and cloud computing with Oracle 11g Release 2 RAC on Linux. Steve Shaw & Martin Bach The eXperT’s Voice ® in oracle Pro Oracle Database 11g RAC on Linux Installation, Administration, Performance Steve Shaw and Martin Bach Create robust and scalable database systems using Oracle’s clustering and grid technologies Covers 11g Release 2 Shaw Bach Oracle Database 11g RAC on Linux Companion eBook Available Pro www.it-ebooks.info www.it-ebooks.info Pro Oracle Database 11g RAC on Linux          Steve Shaw Martin Bach www.it-ebooks.info Pro Oracle Database 11g RAC on Linux Copyright © 2010 by Steve Shaw and Martin Bach All rights reserved. No part of this work may be reproduced or transmitted in any form or by any means, electronic or mechanical, including photocopying, recording, or by any information storage or retrieval system, without the prior written permission of the copyright owner and the publisher. ISBN-13 (pbk): 978-1-4302-2958-2 ISBN-13 (electronic): 978-1-4302-2959-9 Printed and bound in the United States of America 9 8 7 6 5 4 3 2 1 Trademarked names, logos, and images may appear in this book. Rather than use a trademark symbol with every occurrence of a trademarked name, logo, or image we use the names, logos, and images only in an editorial fashion and to the benefit of the trademark owner, with no intention of infringement of the trademark. The use in this publication of trade names, trademarks, service marks, and similar terms, even if they are not identified as such, is not to be taken as an expression of opinion as to whether or not they are subject to proprietary rights. President and Publisher: Paul Manning Lead Editor: Jonathan Gennick Technical Reviewer: Bernhard Cock Buning and Sandesh Rao Editorial Board: Clay Andres, Steve Anglin, Mark Beckner, Ewan Buckingham, Gary Cornell, Jonathan Gennick, Jonathan Hassell, Michelle Lowman, Matthew Moodie, Duncan Parkes, Jeffrey Pepper, Frank Pohlmann, Douglas Pundick, Ben Renow-Clarke, Dominic Shakeshaft, Matt Wade, Tom Welsh Coordinating Editor: Anita Castro Copy Editor: Patrick Meader and Mary Ann Fugate Compositor: Bytheway Publishing Services Indexer: BIM Indexing & Proofreading Services Artist: April Milne Cover Designer: Anna Ishchenko Distributed to the book trade worldwide by Springer Science+Business Media, LLC., 233 Spring Street, 6th Floor, New York, NY 10013. Phone 1-800-SPRINGER, fax (201) 348-4505, e-mail orders-ny@springer- sbm.com, or visit www.springeronline.com. For information on translations, please e-mail rights@apress.com, or visit www.apress.com. Apress and friends of ED books may be purchased in bulk for academic, corporate, or promotional use. eBook versions and licenses are also available for most titles. For more information, reference our Special Bulk Sales–eBook Licensing web page at www.apress.com/info/bulksales. The information in this book is distributed on an “as is” basis, without warranty. Although every precaution has been taken in the preparation of this work, neither the author(s) nor Apress shall have any liability to any person or entity with respect to any loss or damage caused or alleged to be caused directly or indirectly by the information contained in this work. www.it-ebooks.info  CONTENTS AT A GLANCE iii Contents at a Glance  About the Author xxi  About the Technical Reviewer xxii  Acknowledgments xxiii  Chapter 1: Introduction 1  Chapter 2: RAC Concepts 27  Chapter 3: RAC Architecture 63  Chapter 4: Hardware 97  Chapter 5: Virtualization 165  Chapter 6: Linux Installation and Configuration 231  Chapter 7: Grid Infrastructure Installation 323  Chapter 8: Clusterware 379  Chapter 9: Automatic Storage Management 455  Chapter 10: RDBMS Installation and Configuration 505  Chapter 11: Workload Management 559  Chapter 12: Oracle Performance Monitoring 607  Chapter 13: Linux Performance Monitoring 653  Chapter 14: Parallel Execution 687  Chapter 15: Upgrading to Oracle 11g Release 2 717  Index 771 www.it-ebooks.info  CONTENTS iv Contents  About the Authors xxi  About the Technical Reviewers xxii  Acknowledgments xxiii  Chapter 1: Introduction 1 Introducing Oracle Real Application Clusters 1 Examining the RAC Architecture 3 Deploying RAC 4 Maintaining High Availability 5 Defining Scalability 6 Scaling Vertically vs. Horizontally 7 Increasing Manageability 8 Assessing the Cost of Ownership 10 Clustering with Oracle on Linux 13 Running Linux on Oracle 16 Understanding the Role of Unix 16 Liberating Software 17 Developing Linux 18 Expanding the Concept of Free with Open Source 19 Combining Oracle, Open Source, and Linux 20 Drilling Down on Unbreakable Linux 21 Creating and Growing Red Hat Enterprise Linux 22 Extending Red Hat with Oracle Enterprise Linux 23 Drilling Down on SuSE Linux Enterprise Server 24 www.it-ebooks.info  CONTENTS v Taking Linux to Asia 25 Summary 25  Chapter 2: RAC Concepts 27 Clustering Concepts 27 Configuring Active/active Clusters 27 Implementing Active/passive Clusters 28 Configuring a Shared-All Architecture 28 Configuring a Shared-Nothing Architecture 29 Exploring the Main RAC Concepts 29 Working with Cluster Nodes 29 Leveraging the Interconnect 30 Clusterware/Grid Infrastructure 31 Leveraging Automatic Storage Management 39 Installing Real Application Clusters 44 Using the Global Resource Directory (GRD) 49 Transferring Data Between Instances with Cache Fusion 51 Achieving Read Consistency 52 Synchronizing System Change Numbers 52 Exploring the New Features of 11g Release 2 52 Leveraging Grid Plug and Play 53 Modeling Resources with Server Pools 55 Ensuring POSIX Compliance with ACFS 56 Using Oracle Restart Instead of RAC 57 Simplifying Clusterd Database Access with SCAN Listener 59 Summary 60  Chapter 3: RAC Architecture 63 Availability Considerations 63 Deciding the Number of Nodes 65 www.it-ebooks.info  CONTENTS vi Online Maintenance and Patching 67 Instance Recovery in RAC 72 Failover Considerations 74 Transparent Application Failover 75 Fast Connection Failover and Fast Application Notification 76 Scalability Considerations 77 Scalability Enhancers 78 Scalability Inhibitors 79 Standby Databases 81 Introduction to Oracle Standby Databases 82 Types of Standby Database 83 Active Data Guard 85 Extended Distance Clusters 90 Oracle Streams 91 Streams Processing 92 Oracle Streams Prerequisites 93 Cluster Topologies 94 Summary 95  Chapter 4: Hardware 97 Oracle Availability 98 Server Processor Architecture 99 x86 Processor Fundamentals 99 Multicore Processors and Hyper-Threading 103 CPU Cache 106 CPU Power Management 109 Virtualization 111 Memory 112 Virtual Memory 112 www.it-ebooks.info  CONTENTS vii Physical Memory 113 NUMA 116 Memory Reliability 125 Additional Platform Features 125 Onboard RAID Storage 126 Machine Check Architectures 126 Remote Server Management and IPMI 127 Network Interconnect Technologies 127 Server I/O 128 Private Interconnect 131 Storage Technologies 136 RAC I/O Characteristics 137 Hard Disk and Solid State Disk Drive Performance 143 RAID 147 Storage Protocols for Linux 153 Summary 164  Chapter 5: Virtualization 165 Virtualization Definition and Benefits 165 Oracle VM 168 Oracle VM Server Architecture 168 Oracle VM Design 174 Oracle VM Server Installation 178 Oracle VM Manager Installation 183 Oracle VM CLI Installation and Configuration 186 Configuring Oracle VM 187 Network Configuration 187 Server Pool Configuration 192 Installing and Configuring Guests 208 www.it-ebooks.info  CONTENTS viii Importing a Template 209 Creating a Guest from a Template 210 Accessing a Guest 212 Configuring a Guest for RAC 214 Managing Domains 216 Oracle VM Agent 216 Oracle VM Manager 218 Oracle VM Manager CLI 220 The xm Command-Line Interface 222 Summary 230  Chapter 6: Linux Installation and Configuration 231 Selecting the Right Linux Software 231 Reviewing the Hardware Requirements 232 Drilling Down on Networking Requirements 233 Configuring a GNS or a Manual IP 233 Configuring DNS and DHCP 236 Downloading the Linux Software 243 Preparing for a Network Install 243 Installing Oracle Enterprise Linux 5 247 Starting the Installation 247 Installation Media Check 247 Anaconda Installation 247 Install or Upgrade 248 Disk Partitioning 248 Configuring the Boot Loader and Network 259 Selecting a Time Zone 260 Configuring the Root Password 261 Reviewing the Package Installation Defaults 261 www.it-ebooks.info [...]... hardware and Oracle RAC for data processing The RAC option has been available since Oracle 9i Release 1 in the summer of 2001 Prior to that, the clustered Oracle database option was known as the Oracle Parallel Server option RAC offers fundamental improvements over Oracle Parallel Server—and the introduction of Cache Fusion has helped improve application scalability and inter-instance communication, as well... technology stack (see Chapter 3 for a much more in-depth discussion of the RAC architecture) Figure 1-1 The Oracle Real Application Clusters (RAC) software stack As you can see in Figure 1-1, Oracle RAC is based around the following software components: • Oracle RAC runs on top of an operating system • Oracle RAC builds on the Oracle software stack • Oracle recommends installing Grid Infrastructure—the clustering... the evolution of Oracle clustering products, culminating with the product we know now Introducing Oracle Real Application Clusters Oracle Real Application Clusters (RAC) is an option that sits on top of the Oracle database Using the shared disk architecture, the database runs across a set of computing nodes offers increased availability, allows applications to scale horizontally, and improves manageability... Installing Additional CDs 265 Configuring Oracle Enterprise Linux 5 265 Configuring a Server with the Oracle Validated RPM 266 Verifying the Oracle Validated RPM Actions 270 Post Oracle Validated RPM Configuration 282 Completing the Linux Configuration for RAC 292 Configuring Shared Storage 298 Discovering and Configuring... (virtualization) products are increasingly being used To counter that trend, RAC One Node builds on the RAC technology stack: Oracle Clusterware, Oracle Automatic Storage Management, and the Oracle database Oracle RAC One Node will be discussed in more detail in Chapter 3 2 www.it-ebooks.info CHAPTER 1  INTRODUCTION Examining the RAC Architecture Figure 1-1 provides an overview of the RAC technology... recovery solutions for mission critical 24x7 systems Martin is a proud member of the Oracle Certified Master community, having successfully passed the exam for Database 10g Release 2 Additionally, he has been nominated as an Oracle Ace, based on his significant contribution and activity in the Oracle technical community With this accreditation, Oracle Corporation recognized his proficiency in Oracle technology... installed and configured From Oracle 10g Release 1 to Oracle 11g Release 1, Oracle s software components could be installed on certified cluster file systems such as Oracle s own OCFS2, a so-called shared Oracle home Beginning with Oracle 11g Release 2, only the RDBMS software binaries can be installed as a shared home, Grid Infrastructure, the cluster foundation, can no longer be installed on a shared... Beginning with Oracle 11, they can also be upgraded once their calls are completed This is especially useful in conjunction with connection pooling and web applications where one session can no longer be directly associated with an individual, as it was in the days of dedicated server connections and Oracle Forms applications In the connection pooling scenario, the application simply grabs a connection out... schemas in their own database or share one database with other projects The more static an application’s data, the more suited that app is for consolidation A different approach to server consolidation is to have multiple databases run on the same cluster, instead of employing one database with multiple schemas Tom Kyte’s web site (http://asktom .oracle. com) includes an ongoing discussion where participants... partitioning powerful hardware Oracle RAC offers a very interesting avenue for Oracle database server consolidation One of the arguments used in favor of consolidation is the fact that it is more expensive (not only from a license point of view) to support a large number of small servers, each with its own storage and network connectivity requirements, than a large cluster with one or only a few databases . Administration Pro Oracle SQL Beginning Oracle Database 11g Administration Beginning Oracle SQL Pro Oracle Database 11g RAC on Linux Expert Oracle Practices www.apress.com SOURCE CODE ONLINE Companion. Practices www.apress.com SOURCE CODE ONLINE Companion eBook Steve Shaw, Author of Pro Oracle Database 10g RAC on Linux Martin Bach Pro Oracle Database 11g RAC on Linux Dear