Effective MySQL Backup and Recovery About the Author Ronald Bradford has worked in the relational database field for over 20 years With his professional background and a decade of working knowledge in database architecture, performance tuning, and management of large enterprise systems using Ingres and Oracle, Ronald has for the past 13 years worked primarily with MySQL, the world’s most popular open source database He has worked both at Oracle Corporation (1996–1999) as an Oracle Consultant and MySQL, Inc (2006–2008) as a senior MySQL Consultant His contributions to the MySQL community include being recognized as the all-time top individual MySQL blog contributor at Planet MySQL (2010), and international recognitions include being named an Oracle ACE Director (2010) and MySQL Community Member of the Year (2009) Ronald combines his extensive consulting expertise with a passion to share the knowledge and benefits of using MySQL Starting in 2006, his many public speaking engagements have included over 60 presentations in 20 countries in 2010–2011 About the Technical Editors Hans Forbrich has been working with computers since the early 1970s, in particular with entity-relationship and relational databases starting in 1979 using an engine on IBM mainframes called GERM (General Entity Relationship Model) Since that time, Hans has been a DBA, an operations architect for a number of organizations, and an Oracle University instructor but always heavily involved in high availability and recoverability As a fellow ACE Director, Hans is pleased and honored to have been a technical reviewer for this book Backup and recovery is an important and often overlooked area of MySQL, and Ronald’s expertise and experience in this area shines through Chris Schneider has been a MySQL community member, user, and evangelist for the past ten years Throughout his career he has designed, implemented, and maintained small to large scale MySQL installations while training and mentoring teams of DBAs This has included building architecture from the ground up and improving on those that are currently in place while emphasizing scalability, performance, and ease of use Since 2009, Chris has been an expert speaker at many U.S conferences including the MySQL Conference and Expo, ODTUG KScope, and Oracle Open World Lenz Grimmer first encountered MySQL in 1995, when he had his first job as a systems administrator in a small Internet startup company, which already used what was later called the “LAMP stack” to provide web hosting services for customers He then worked as a distribution developer at SUSE Linux from 1998–2002, before he joined MySQL AB as a Release Engineer in charge of producing the official MySQL builds for all platforms After having been with the MySQL team for nine years, he recently returned to Linux, as a member of the Oracle Linux product management team at Oracle Lenz is the maintainer of the mylvmbackup script and has given numerous talks on the topics of MySQL backup and recovery In his spare time, Lenz enjoys spending time with his family or tinkering with remote controlled quadrocopters, powered by the Arduino platform Effective MySQL Backup and Recovery Ronald Bradford New York Chicago San Francisco Lisbon London Madrid Mexico City Milan New Delhi San Juan Seoul Singapore Sydney Toronto Effective MySQL: Backup and Recovery Copyright © 2012 by The McGraw-Hill Companies, Inc All rights reserved Except as permitted under the United States Copyright Act of 1976, no part of this publication may be reproduced or distributed in any form or by any means, or stored in a database or retrieval system, without the prior written permission of the publisher Oracle is a registered trademark of Oracle Corporation and/or its affiliates All other trademarks are the property of their respective owners, and McGraw-Hill makes no claim of ownership by the mention of products that contain these marks Screen displays of copyrighted Oracle software programs have been reproduced herein with the permission of Oracle Corporation and/or its affiliates ISBN: 978-0-07-178858-8 MHID: 0-07-178858-1 The material in this eBook also appears in the print version of this title: ISBN 978-0-07178857-1, MHID 0-07-178857-3 McGraw-Hill eBooks are available at special quantity discounts to use as premiums and sales promotions, or for use in corporate training programs To contact a representative please e-mail us at bulksales@mcgraw-hill.com All trademarks are trademarks of their respective owners Rather than put a trademark symbol after every occurrence of a trademarked name, we use names in an editorial fashion only, and to the benefit of the trademark owner, with no intention of infringement of the trademark Where such designations appear in this book, they have been printed with initial caps Sponsoring Editor Paul Carlstroem Editorial Supervisor Patty Mon Project Manager Sapna Rastogi, Cenveo Publisher Services Acquisitions Coordinator Ryan Willard Technical Editors Hans Forbrich Chris Schneider Lenz Grimmer Copy Editor Lisa McCoy Proofreader Paul Tyler Indexer Karin Arrigoni Production Supervisor Jean Bodeaux Composition Cenveo Publisher Services Illustration Cenveo Publisher Services Art Director, Cover Jeff Weeks Cover Designer Pattie Lee Information has been obtained by Publisher from sources believed to be reliable However, because of the possibility of human or mechanical error by our sources, Publisher, or others, Publisher does not guarantee to the accuracy, adequacy, or completeness of any information included in this work and is not responsible for any errors or omissions or the results obtained from the use of such information Oracle Corporation does not make any representations or warranties as to the accuracy, adequacy, or completeness of any information contained in this Work, and is not responsible for any errors or omissions TERMS OF USE This is a copyrighted work and The McGraw-Hill Companies, Inc (“McGraw-Hill”) and its licensors reserve all rights in and to the work Use of this work is subject to these terms Except as permitted under the Copyright Act of 1976 and the right to store and retrieve one copy of the work, you may not decompile, disassemble, reverse engineer, reproduce, modify, create derivative works based upon, transmit, distribute, disseminate, sell, publish or sublicense the work or any part of it without McGraw-Hill’s prior consent You may use the work for your own noncommercial and personal use; any other use of the work is strictly prohibited Your right to use the work may be terminated if you fail to comply with these terms THE WORK IS PROVIDED “AS IS.” McGRAW-HILL AND ITS LICENSORS MAKE NO GUARANTEES OR WARRANTIES AS TO THE ACCURACY, ADEQUACY OR COMPLETENESS OF OR RESULTS TO BE OBTAINED FROM USING THE WORK, INCLUDING ANY INFORMATION THAT CAN BE ACCESSED THROUGH THE WORK VIA HYPERLINK OR OTHERWISE, AND EXPRESSLY DISCLAIM ANY WARRANTY, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO IMPLIED WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE McGraw-Hill and its licensors not warrant or guarantee that the functions contained in the work will meet your requirements or that its operation will be uninterrupted or error free Neither McGraw-Hill nor its licensors shall be liable to you or anyone else for any inaccuracy, error or omission, regardless of cause, in the work or for any damages resulting therefrom McGraw-Hill has no responsibility for the content of any information accessed through the work Under no circumstances shall McGrawHill and/or its licensors be liable for any indirect, incidental, special, punitive, consequential or similar damages that result from the use of or inability to use the work, even if any of them has been advised of the possibility of such damages This limitation of liability shall apply to any claim or cause whatsoever whether such claim or cause arises in contract, tort or otherwise For MySQL culture, past, present, and future To many in the MySQL and growing Oracle community: you are more than colleagues; you are, and always will remain, great friends CONTENTS Acknowledgments Introduction The Five Minute DBA Approaching a MySQL Backup Determining Your Database Size Choosing a Locking Strategy Execution Time Combining Information Performing a MySQL Backup Running mysqldump Securing Your Backup Benefits with mysqldump More Information Other Options Conclusion Understanding Backup Options Terminology Choosing a Backup Strategy Database Availability Storage Engines Locking Strategies MySQL Topology Static Backup Options Filesystem Backup SQL Dump Table Extract Filesystem Snapshot InnoDB Hot Backup Options Not Discussed nc (netcat) netcat (nc) Network Attached Storage devices network bandwidth network devices nice command —no-history-logging option —no-schemas option —no-timestamp option O objects completeness consistency schema table OLTP systems —only-innodb option —only-innodb-with-frm option —only-known-file-types option Open Query Open Stack cloud software openssl utility Openstack Keystone Identity Service —opt option optimization See also compression architectural considerations backup security and example backup environment incremental backups overview parallel processing partial backups remote backups streaming backups Oracle Corporation Oracle Secure Backup (OSB) OSB (Oracle Secure Backup) P PalominoDB parallel copying —parallel option parallel processing partial backups per table dump files Percona Percona disaster Percona toolkit Percona XtraDB storage engine performance See also optimization binary logging and compression and permissions considerations remote backups pigz utility piped commands PITR See point in time recovery Planet MySQL point in time recovery (PITR) AWS RDS binary logs considerations described HP Public Cloud replication stream requirements standalone recovery power disruption power loss —prepare option processing, parallel production environment pt-table-checksum utility Puppet tool PURGE BINARY LOGS command pvdisplay command Pythian Group Python scripts Q query cache R RAID system RAID-1 configurations —raw option RBR (row-based replication) RDS (Remote Database Service) RDS service rds-describe-db-snapshots command read lock —read-from-remote-server option read_only option recovery See also restores AWS RDS options for backup/recovery quiz binary logs See binary logs business requirements for case study cloud impact on considerations data access points and defining requirements defining responsibilities determining type of disaster See disaster scenarios filesystem copy InnoDB crash recovery InnoDB storage engine MEB MyISAM crash recovery MySQL configuration and MySQL data physical hardware and planning for worst situation point in time See point in time recovery software failures SQL dump static strategic plans for strategy system variables table definition file terminology testing and verification of XtraBackup recovery point objective (RPO) recovery time object (RTO) “red bus” policies Red Dwarf project redo logs redundancy reference manuals references relay log relay log directories relay_log variable relay_log_index_system variable relay-log-info-file option relay-log-info-file system variable remote backups Remote Database Service See RDS remote hosts remote servers —remote-host option replicate_do_db variable replicate_ignore_db variable replication architecture backup considerations characteristics of considerations design considerations disabling filesystem snapshots and high availability and inconsistency limitations MEB and mysqldump options new features options/variables prerequisite checks read only replication slaves row-based scalable architecture options security and semi-synchronous statement-based system variables triggers workflow XtraBackup and replication lag replication servers replication stream restores See also recovery AWS RDS file snapshot filesystem copy Google Cloud SQL incremental backups and MEB recovery on running instances SQL dump recovery XtraBackup —routines option row-based replication (RBR) —rows option RPO (recovery point objective) rsync command RTO (recovery time object) rzip utility S SA (system administrator) —safe-slave-backup option —safe-updates option SAN (Storage Area Network) SAN failures SBR (statement-based replication) SBT (System Backup to Tape) interface ScaleDB schema definition schema objects schema sync utility schemas comparison completeness consistency Schooner SQL storage engine security applications backups cloud deployments considerations data encryption firewalls lack of MEB backups optimization and permissions See permissions replication and security patch upgrades semi-synchronous replication server_id variable servers downtime causes master See MySQL masters remote replication single slave See MySQL slaves Service Level Agreement (SLA) SHOW BINARY LOGS command SHOW command SHOW MASTER LOGS command SHOW PROCESSLIST command SHOW PROCESSLIST command SHOW SLAVE STATUS command —show-long option Sidekick/Microsoft data loss —single-transaction option skip_networking option —skip-quote-names option skip-slave-start option SkySQL SLA (Service Level Agreement) slave host error log slave relay log slave_exec_mode variable —slave-info option slaves See MySQL slaves slave-skip-error option slave_skip_errors variable snapshot files snapshot volumes snapshots AWS RDS filesystem HP Public Cloud LVM rds-describe-db-snapshots command replication and socket file software failures software updates software upgrades Solid State Drives (SSD) SQL CLI tool SQL dump recovery SQL dumps See mysqldump SQL server modes SQL statements sql_mode option SSD (Solid State Drives) SSH connections SSL support START TRANSACTION command —start-lsn option statement-based replication (SBR) static backups described options for static recovery status options stock analysis system STOP SLAVE SQL_THREAD command —stop-never option Storage Area Network (SAN) storage engines See also specific storage engines considerations default locking strategies third party verifying —stream option streaming backups —stream=tar option sync_binlog variable synchronization files replication sync_relay_log_info option system administrator (SA) system alerts System Backup to Tape (SBT) interface system variables data management recovery replication T table data table definition recovery table extracts table objects tables locks MyISAM temporary —tables option tablespaces tar files TCP connections TD Bank disaster technical resource responsibilities temporary tables terminology test environment testing considerations described importance of InnoDB crash recovery purpose of recovery process throttling time command Tokutek TokuDB storage engine transaction logs transactions isolating key steps single trigger operation U UDP connections Unix commands UNLOCK TABLES command unsafe statements updates See software updates V variables See system variables W wait_timeout WAN (wide area network) wide area network (WAN) —with-timestamp option World Backup Day X XBM (XtraBackup Manager) Xeround XtraBackup binary log statements compression examples incremental backups parallel copying partial backups recovery options remote hosts remote hosts and replication and restore process xtrabackup command XtraBackup Manager (XBM) xz utility Z zip utility Zmanda Recovery Manager for MySQL ... Incremental Backups MySQL Enterprise Backup (MEB) XtraBackup Partial Backups mysqldump MySQL Enterprise Backup (MEB) XtraBackup MySQL Backup Security Encryption with ezNcrypt Architectural Considerations... Chapter 1 introduced a number of important terms that are essential to fully understand the principles for backup and recovery with MySQL Term Description static backup This is a backup of data at a given point in time Generally a MySQL backup would be performed daily, for example 2:00... Oracle DBA for understanding and learning to master backup and recovery for MySQL I am indebted to Chris Schneider, who in addition to being a technical editor of this book and co-author for the next book in this Effective MySQL series, was able to assist