The Red Gate Guide SQL Server Team-based Development Phil Factor, Grant Fritchey, Alex Kuznetsov, and Mladen Prajdic ´ ISBN: 978-1-906434-48-9 The Red Gate Guide to SQL Server Team-based Development By Phil Factor, Grant Fritchey, Alex Kuznetsov, and Mladen Prajdić First published by Simple Talk Publishing 2010 Copyright Phil Factor, Grant Fritchey, Alex Kuznetsov, and Mladen Prajdić 2010 ISBN 978-1-906434-48-9 The right of Phil Factor, Grant Fritchey, Alex Kuznetsov and Mladen Prajdić to be identified as the authors of this work has been asserted by them in accordance with the Copyright, Designs and Patents Act 1988 All rights reserved No part of this publication may be reproduced, stored or introduced into a retrieval system, or transmitted, in any form, or by any means (electronic, mechanical, photocopying, recording or otherwise) without the prior written consent of the publisher Any person who does any unauthorized act in relation to this publication may be liable to criminal prosecution and civil claims for damages This book is sold subject to the condition that it shall not, by way of trade or otherwise, be lent, re-sold, hired out, or otherwise circulated without the publisher's prior consent in any form other than that in which it is published and without a similar condition including this condition being imposed on the subsequent publisher Editor: Tony Davis Technical Reviewer: Peter Larsson Additional Material: Roger Hart and Allen White Cover Image:Paul Vlaar Copy Edit: Gower Associates Typeset & Designed: Matthew Tye & Gower Associates Table of Contents Introduction xiii Chapter 1: Writing Readable SQL 16 Why Adopt a Standard? 16 Object Naming Conventions 18 Tibbling .18 Pluralizing 19 Abbreviating (or abrvtng) .19 [Escaping] 20 Restricting 22 A guide to sensible object names 23 Code Layout 26 Line-breaks 26 Indenting 27 Formatting lists 27 Punctuation 28 Capitalization 29 Getting off the fence… 30 Summary 34 Chapter 2: Documenting your Database 36 Why Bother to Document Databases? .36 Where the Documentation Should Be Held 37 What Should Be In the Documentation? 39 How Should the Documentation Be Published? 39 What Standards Exist? 40 XMLDOCS 40 YAML and JSON 44 How Headers are Stored in the Database 46 Extended properties 47 Get Information Out of Headers 51 Publishing the Documentation 55 Summary 58 Chapter 3: Change Management and Source Control 59 The Challenges of Team-based Development 60 Environments 61 Development environments 63 Testing, staging and production environments 65 Source Control 69 Source control features 70 Source control systems 72 Database objects in source control .75 Getting your database objects into source control 77 Managing data in source control 87 Summary 93 Chapter 4: Managing Deployments 94 Deployment Schemes 94 Visual Studio 2010 Premium tools 96 Red Gate SQL Source Control 105 Automating Builds for Continuous Integration 114 What is continuous integration? 115 Example: deploying to test 116 Creating test data 118 Automation with MSBuild, NAnt, and PowerShell 118 Automation with CruiseControl 123 Summary 125 Chapter 5: Testing Databases 126 Why Test a Database? 127 Essential Types of Database Testing 127 Black-box and white-box testing 128 Unit testing 130 Integration and acceptance testing .131 Stress testing databases 132 Error testing 133 Essentials for Successful Database Testing 133 The right attitude 133 A test lab 135 Source control 136 Database schema change management 137 Semi- or fully-automated deployment 138 A testing tool 139 A data generation tool 139 How to Test Databases 141 Reverting the database state 141 Simplifying unit tests 145 Testing existing databases 146 Unit Testing Examples: Testing Data and Schema Validity 148 Testing the database interface 148 Testing the database schema 151 Testing tables, views, and UDFs 156 Testing stored procedures 160 Testing authentication and authorization 163 Summary .166 Chapter 6: Reusing T-SQL Code 167 The Dangers of Copy-and-Paste .168 How Reusing Code Improves its Robustness 173 Wrapping SELECTs in Views 177 Reusing Parameterized Queries: Stored Procedures versus Inline UDFs 178 Scalar UDFs and Performance 183 Multi-Statement Table-Valued UDFs .188 Reusing Business Logic: Stored Procedure, Trigger, Constraint or Index? .188 Use constraints where possible 189 Turn to triggers when constraints are not practical 191 Unique filtered indexes (SQL Server 2008 only) 196 Summary .196 Chapter 7: Maintaining a Code Library 198 Coding for Reuse 199 Code comments 199 Parameter naming 201 Unit tests 203 Storing Script Libraries 204 Source control 205 A single file or individual files? 205 Tools for Creating and Managing Code Libraries 206 SQL Server Management Studio 207 Text editors 213 Wikis 215 SQL Prompt 219 Summary .224 Chapter 8: Exploring your Database Schema 225 Building a Snippet Library 226 Interrogating Information Schema and Catalog Views 227 Searching Structural Metadata in Schema-scoped Objects within a Database 229 Tables with no primary keys 230 Tables with no referential constraints 231 Tables with no indexes 232 A one-stop view of your table structures 233 How many of each object… 236 Too many indexes… 237 Seeking out troublesome triggers .238 What objects have been recently modified? 240 Querying the documentation in extended properties 242 Object permissions and owners 243 Searching All Your Databases 245 Investigating Foreign Key Relationships 246 Interrogating Object Dependencies 251 Finding the closest relations 252 Finding the dependency chain 253 Summary .258 Chapter 9: Searching DDL and Build Scripts 259 Searching Within the DDL 260 Why isn't it in SSMS? 260 So how you it? .261 Using SSMS to Explore Table Metadata 274 SSMS shortcut keys 278 Useful shortcut queries 279 Useful shortcut stored procedures 284 Generating Build Scripts 285 Summary .292 Chapter 10: Automating CRUD 293 First, Document Your Code 294 Automatically Generating Stored Procedure Calls 297 Automating the Simple Update Statement 301 Generating Code Templates for Table-Valued Functions 306 Automatically Generating Simple INSERT Statements 307 Summary .308 Chapter 11: SQL Refactoring 309 Why Refactor SQL? 309 Requirements for Successful SQL Refactoring .311 A set-based mindset .311 Consistent naming conventions 315 Thorough testing 316 A database abstraction layer 316 Where to Start? 317 SQL Refactoring in Action: Tackling Common Anti-Patterns 320 Using functions on columns in the WHERE clause 320 The "SELECT *" anti-pattern 323 Huge, many-parameter stored procedures .328 The "one subquery per condition" anti-pattern .330 The "cursor is the only way" anti-pattern 333 Using data types that are too large .339 The "data in code" anti-pattern 342 Summary .346 Chapter 11: SQL Refactoring /* The solution is to store this data logic in a table not in our code like this */ DROP TABLE Users GO CREATE TABLE Users ( UserId INT IDENTITY(1, 1) PRIMARY KEY , UserName VARCHAR(50) , UserPassword NVARCHAR(10) , IsActive BIT , IsDeleted BIT , IsSleeping BIT ) INSERT INTO Users ( UserName , UserPassword , IsActive , IsDeleted , IsSleeping ) SELECT 'User 1' , 'MyPwd1' , , , UNION ALL SELECT 'User 2' , 'BlaBla' , , , UNION ALL SELECT 'User 3' , 'aaaaaa' , , , UNION ALL SELECT 'User 4' , 'bbbbbb' , , , 346 Chapter 11: SQL Refactoring GO SELECT FROM WHERE UserId , UserName , UserPassword , IsActive , IsDeleted , IsSleeping Users IsActive = AND IsSleeping = Listing 11-19: Refactoring the DDL to avoid the "Tell, don't ask" anti-pattern By moving each status code to its own column, we don't have to keep the data in code; it's stored in the schema like it should be It also means that we can mix and match the different statuses, as required There are times when you might still use the "data in code" method, but it should be used only with caution For example, we might consider using it if the UserStatus values really were static and used through multiple applications I still don't like it, though The hard part is changing your thinking processes so that you know when it's still needed and when it isn't Summary In this chapter, we've looked at why refactoring is important, in the context of the broader picture of database development Refactoring allows us to improve the performance, maintainability, and security of database code, without affecting any of the systems that access the database code In order to this, we need a set of unit tests that cover our database functionality, a database abstraction layer that isolates the underlying structure from the outside 347 Chapter 11: SQL Refactoring world, and good and consistent naming conventions We also need to adopt a set-based approach to SQL programming My basic approach was to apply concepts and analogies from object-oriented development to the world of database refactoring As it turns out, pretty much the same principles apply in both development branches We just have to learn how to recognize them By examining and refactoring a few common anti-patterns, we've taken the first steps into how refactoring should be done, and what improvements we can expect 348 SQL Server and NET Tools from Red Gate Software Pricing and information about Red Gate tools are correct at the time of going to print For the latest information and pricing on all Red Gate's tools, visit www.red-gate.com SQL Compare® Pro $595 Compare and synchronize SQL Server database schemas  Eliminate mistakes migrating database changes from dev, to test, to production  Speed up the deployment of new database schema updates  Find and fix errors caused by differences between databases  Compare and synchronize within SSMS "Just purchased SQL Compare With the productivity I'll get out of this tool, it's like buying time." Robert Sondles Blueberry Island Media Ltd SQL Data Compare Pro $595 Compares and synchronizes SQL Server  Save time by automatically comparing and synchronizing your data  Copy lookup data from development databases to staging or production  Quickly fix problems by restoring damaged or missing data to a single row  Compare and synchronize data within SSMS "We use SQL Data Compare daily and it has become an indispensable part of delivering our service to our customers It has also streamlined our daily update process and cut back literally a good solid hour per day." George Pantela GPAnalysis.com Visit www.red-gate.com for a 14-day, free trial SQL Prompt Pro $295 Write, edit, and explore SQL effortlessly  Write SQL smoothly, with code-completion and SQL snippets  Reformat SQL to a preferred style  Keep databases tidy by finding invalid objects automatically  Save time and effort with script summaries, smart object renaming, and more "SQL Prompt is hands-down one of the coolest applications I've used Makes querying/developing so much easier and faster." Jorge Segarra University Community Hospital SQL Source Control Connect your existing source control system to SQL Server  Bring all the benefits of source control to your database  Source control database schemas and data within SSMS, not with offline scripts  Connect your databases to TFS, SVN, SourceGear Vault, Vault Pro, Mercurial, Perforce, Git, Bazaar, and any source control system with a capable command line  Work with shared development databases, or individual copies  Track changes to follow who changed what, when, and why  Keep teams in sync with easy access to the latest database version  View database development history for easy retrieval of specific versions Visit www.red-gate.com for a free trial from $395 Deployment Manager from Automated deployment for your applications and databases Deploys your whole application – ASP.NET sites, dependent assemblies, and databases – in one process  Makes deployment repeatable with a minimum of custom scripting  Shows you which version of your software is running on each dev, test, staging, and production environment, from a central dashboard  Works with local, remote, and Cloud-based servers  Uses public/private key encryption to keep deployments over the Internet secure  "This tool ranks up there with NuGet and Team City as the tools that have made the biggest difference to web application development productivity these last months – it truly deserves to be a roaring success!" Mark Roberts NET Web Developer, Red Gate Software Visit www.red-gate.com for a 28-day, free trial $295 SQL Backup Pro Compress, verify, and encrypt SQL Server backups  Compress SQL Server database backups by up to 95% for faster, smaller backups  Protect your data with up to 256-bit AES encryption  Strengthen your backups with network resilience to enable a fault-tolerant transfer of backups across flaky networks   Control your backup activities through an intuitive interface, with powerful job management and an interactive timeline Get integrated backup verification – schedule regular restores and include a database integrity check (DBCC CHECKDB) "SQL Backup Pro cut the backup time for our most mission-critical database by 92%, and provided us with 95% compression Builtin network resilience has also reduced our failure rate to zero I'm absolutely amazed at how well it performs." Kiara Rodemaker Manager, IT Accounting Services, Czarnowski Visit www.red-gate.com for a 14-day, free trial $795 SQL Monitor from SQL Server performance monitoring and alerting      Intuitive overviews at global, cluster, machine, SQL Server, and database levels for up-to-the-minute performance data Use SQL Monitor's web UI to keep an eye on server performance in real time on desktop machines and mobile devices Intelligent SQL Server alerts via email and an alert inbox in the UI, so you know about problems first Comprehensive historical data, so you can go back in time to identify the source of a problem View the top 10 expensive queries for an instance or database based on CPU usage, duration, and reads and writes  PagerDuty integration for phone and SMS alerting  Fast, simple installation and administration  Add your own T-SQL scripts with the custom metrics feature to expand SQL Monitor's range "Being web based, SQL Monitor is readily available to you, wherever you may be on your network You can check on your servers from almost any location, via most mobile devices that support a web browser." Jonathan Allen Senior DBA, Careers South West Ltd Visit www.red-gate.com for a 14-day, free trial $795 SQL DBA Bundle $1,395 Five essential tools for database administration Backup & Recovery Protect your organization's data by creating highly compressed, fully verified and encrypted backups, and ensure reliable restores Performance Monitoring & Tuning Monitor your servers in real time and obtain the performance data and alerts that are important to your business Storage & Capacity Planning Proactively monitor data growth and make the most of your storage space with backup compression and the ability to virtually restore Troubleshooting Get an alert within seconds of a problem arising, gain insight into what happened, and diagnose the issue, fast Security Protect your data from unauthorized access with strong backup encryption The SQL DBA Bundle contains: SQL Backup Pro SQL Monitor SQL Virtual Restore SQL HyperBac SQL Multi Script The tools in the bundle can be bought separately with a combined value of $3,375, or purchased together for $1,395, saving 60% on the individual tool prices Visit www.red-gate.com for a 14-day, free trial SQL Toolbelt $1,995 The essential SQL Server tools for database professionals You can buy our acclaimed SQL Server tools individually or bundled Our most popular deal is the SQL Toolbelt: sixteen of our SQL Server tools in a single installer, with a combined value of $5,930 but an actual price of $1,995, a saving of 66% Fully compatible with SQL Server 2000, 2005, and 2008 SQL Toolbelt contains:  SQL Compare Pro  SQL Dependency Tracker  SQL Data Compare Pro  SQL Packager  SQL Source Control  SQL Multi Script Unlimited  SQL Backup Pro  SQL Search  SQL Monitor  SQL Comparison SDK  SQL Prompt Pro  SQL Object Level Recovery Native  SQL Data Generator  SQL Connect  SQL Doc "The SQL Toolbelt provides tools that database developers, as well as DBAs, should not live without."  SQL Test William Van Orden Senior Database Developer, Lockheed Martin Visit www.red-gate.com for a 14-day, free trial ANTS Memory Profiler $495 Find memory leaks and optimize memory usage of your NET applications  Zero in on the causes of memory leaks, fast  Visualize the relationship between your objects and identify references which should no longer be held in memory  Optimize your application's memory usage "Freaking sweet! We have a known memory leak that took me about four hours to find using our current tool, so I fired up ANTS Memory Profiler and went at it like I didn't know the leak existed Not only did I come to the conclusion much faster, but I found another one!" Aaron Smith IT Manager, R.C Systems Inc ANTS Performance Profiler from $395 Identify performance bottlenecks within minutes  Drill down to slow lines of code with line-level code timings  Analyse both your NET code and SQL queries in a single profiling session  Optimize NET application performance "ANTS Performance Profiler took us straight to the specific areas of our code which were the cause of our performance issues." Terry Phillips Sr Developer, Harley-Davidson Dealer Systems "I have integrated ANTS Profiler into my entire development process and it has truly helped us write better performing applications from the beginning by ensuring that we always know exactly what is going on." Mitchell Sellers MVP Visit www.red-gate.com for a 14-day, free trial "EVERY DEVELOPER NEEDS THIS TOOL!" Daniel Larson Software Architect, NewsGator Technologies NET Reflector $95 Decompile, debug, and understand any NET code  See inside assemblies, libraries, and frameworks so you can understand how they work, even if you don't have the source  Decompile, search, and analyze any NET assembly in C#, Visual Basic, and IL  Step straight into decompiled assemblies while debugging in Visual Studio, so you can debug 3rd-party code just like your own "One of the most useful, practical debugging tools that I have ever worked with in NET! It provides complete browsing and debugging features for NET assemblies, and has clean integration with Visual Studio." Tom Baker Consultant Software Engineer, EMC Corporation SmartAssembly $795 Prepare your application for the world  Obfuscation: Obfuscate your NET code to secure it against reverse engineering Multiple layers of protection defend your software against decompilation and cracking  Automated Error Reporting: Get quick and automatic reports on exceptions your end-users encounter, and identify unforeseen bugs within hours or days of shipping Receive detailed reports containing a stack trace and values of the local variables, making debugging easier "Knowing the frequency of problems (especially immediately after a release) is extremely helpful in prioritizing and triaging bugs that are reported internally Additionally, by having the context of where those errors occurred, including debugging information, really gives you that leap forward to start troubleshooting and diagnosing the issue." Ed Blankenship Technical Lead and MVP Visit www.red-gate.com for a 14-day, free trial SQL Server Execution Plans (2nd Edition) Grant Fritchey Every Database Administrator, developer, report writer, and anyone else who writes T-SQL to access SQL Server data, must understand how to read and interpret execution plans This book leads you right from the basics of capturing plans, through to how to interpret them in their various forms, graphical or XML, and then how to use the information you find there to diagnose the most common causes of poor query performance, and so optimize your SQL queries, and improve your indexing strategy ISBN: 978-1-906434-93-9 Published: October 2012 SQL Server Concurrency: Locking, Blocking and Row Versioning Kalen Delaney Your application can have impeachable indexes and queries, but they won't help you if you can't get to your data because another application has it locked That's why every DBA and developer must understand SQL Server concurrency and how to troubleshoot excessive blocking or deadlocking ISBN: 978-1-906434-91-5 Published: September 2012 The Red Gate Guides SQL Server Backup and Restore Shawn McGehee A DBA's tasks from day to day are rarely constant; with one exception: the need to ensure each and every day that any database in their charge can be restored and recovered, in the event of error or disaster In this book, you'll discover how to perform each of these backup and restore operations using SQL Server Management Studio (SSMS), basic T-SQL scripts and Red Gate's SQL Backup tool ISBN: 978-1-906434-86-1 Published: May 2012 SQL Server Team-based Development Phil Factor, Grant Fritchey, Alex Kuznetsov, and Mladen Prajdić This book shows how to use a mixture of home-grown scripts, native SQL Server tools, and tools from the Red Gate SQL Toolbelt, to successfully develop database applications in a team environment, and make database development as similar as possible to "normal" development ISBN: 978-1-906434-59-5 Published: November 2010 ... reliably to any number of systems This book shows how to use of mixture of home-grown scripts, native SQL Server tools, and tools from the Red Gate SQL toolbelt (such as SQL Compare, SQL Source.. .The Red Gate Guide to SQL Server Team-based Development By Phil Factor, Grant Fritchey, Alex Kuznetsov, and Mladen Prajdić First published by Simple Talk Publishing 2010 Copyright Phil Factor,... eye, but there is no equivalent tool to refactor the naming of all your SQL objects to conform to a given standard (though SQL Refactor will help you with renaming tables) Naming has to be done