Microsoft SQL Server 2012 Internals Kalen Delaney Bob Beauchemin Conor Cunningham Jonathan Kehayias Benjamin Nevarez Paul S Randal Published with the authorization of Microsoft Corporation by: O’Reilly Media, Inc 1005 Gravenstein Highway North Sebastopol, California 95472 Copyright © 2013 by Kalen Delaney All rights reserved No part of the contents of this book may be reproduced or transmitted in any form or by any means without the written permission of the publisher ISBN: 978-0-7356-5856-1 LSI Printed and bound in the United States of America Microsoft Press books are available through booksellers and distributors worldwide If you need support related to this book, email Microsoft Press Book Support at mspinput@microsoft.com Please tell us what you think of this book at http://www.microsoft.com/learning/booksurvey Microsoft and the trademarks listed at http://www.microsoft.com/about/legal/en/us/IntellectualProperty/ Trademarks/EN-US.aspx are trademarks of the Microsoft group of companies All other marks are property of their respective owners The example companies, organizations, products, domain names, email addresses, logos, people, places, and events depicted herein are fictitious No association with any real company, organization, product, domain name, email address, logo, person, place, or event is intended or should be inferred This book expresses the author’s views and opinions The information contained in this book is provided without any express, statutory, or implied warranties Neither the authors, O’Reilly Media, Inc., Microsoft Corporation, nor its resellers, or distributors will be held liable for any damages caused or alleged to be caused either directly or indirectly by this book Acquisitions and Developmental Editor: Michael Bolinger Production Editor: Kara Ebrahim Editorial Production: Box 12 Communications Technical Reviewers: Benjamin Nevarez and Jonathan Kehayias Copyeditor: Box 12 Communications Indexer: Box 12 Communications Cover Design: Twist Creative • Seattle Cover Composition: Karen Montgomery Illustrator: Rebecca Demarest Contents at a glance Introduction xix Chapter SQL Server 2012 architecture and configuration Chapter The SQLOS 35 Chapter Databases and database files 99 Chapter Special databases 139 Chapter Logging and recovery 171 Chapter Table storage 203 Chapter Indexes: internals and management 297 Chapter Special storage 381 Chapter Special indexes 457 Chapter 10 Query execution 513 Chapter 11 The Query Optimizer 611 Chapter 12 Plan caching and recompilation 703 Chapter 13 Transactions and concurrency 765 Chapter 14 DBCC internals 837 Index 903 Contents Introduction xix Chapter SQL Server 2012 architecture and configuration SQL Server editions SQL Server installation and tools SQL Server metadata Compatibility views Catalog views Dynamic Management Objects Other metadata Components of the SQL Server engine 10 Protocols 11 Query processor 12 The storage engine 14 SQL Server 2012 configuration 17 Using SQL Server Configuration Manager 18 Managing services 19 SQL Server system configuration 21 Operating system configuration 21 Trace flags 23 SQL Server configuration settings 24 Conclusion 33 What you think of this book? We want to hear from you! Microsoft is interested in hearing your feedback so we can continually improve our books and learning resources for you To participate in a brief online survey, please visit: microsoft.com/learning/booksurvey v Chapter The SQLOS 35 NUMA architecture 36 The scheduler 37 Understanding SQL Server schedulers 38 Binding schedulers to CPUs 41 Observing scheduler internals 42 Understanding the Dedicated Administrator Connection (DAC) 45 Memory 47 The buffer pool and the data cache 47 Column store object pool 48 Access to in-memory data pages 48 Page management in the data cache 48 The free buffer list and the lazywriter 49 Checkpoints 50 Memory management in other caches 52 The Memory Broker 54 Memory sizing 54 Buffer pool sizing 55 SQL Server Resource Governor 61 Resource Governor overview 61 Resource Governor controls 70 Resource Governor metadata 71 Extended Events 73 Extended Events architecture 73 Event execution life cycle 73 Core concepts 75 Extended Events DDL and querying 83 Extended Events UI 86 Conclusion 97 Chapter Databases and database files 99 Working with sample databases 100 AdventureWorks 100 pubs 101 vi Contents Northwind 101 Understanding database files 101 Creating a database 104 Using CREATE DATABASE: an example 106 Expanding or shrinking a database 106 Automatic file expansion 106 Manual file expansion 107 Fast file initialization 107 Automatic shrinkage 108 Manual shrinkage 108 Using database filegroups 109 The default filegroup 110 A FILEGROUP CREATION example 112 Filestream filegroups 113 Altering a database 114 ALTER DATABASE examples 115 Databases under the hood 116 Space allocation 116 Setting database options 119 State options 122 Cursor options 125 Auto options 125 SQL options 126 Database recovery options 128 Other database options 129 Understanding database security 129 Database access 130 Database security 132 Databases vs schemas 133 Principals and schemas 133 Default schemas 134 Moving or copying a database 134 Detaching and reattaching a database 135 Backing up and restoring a database 136 Contents vii Understanding compatibility levels 137 Conclusion 138 Chapter Special databases 139 System databases 139 Understanding the master database 139 Understanding the model database 140 Introducing the tempdb database 140 Understanding the resource database 140 Understanding the msdb database 141 Moving system databases 142 Moving the master database 143 The tempdb database 144 Objects in tempdb 144 Optimizations in tempdb 146 Best practices 147 tempdb contention 148 tempdb space monitoring 153 Database snapshots 155 Creating a database snapshot 156 Understanding space used by database snapshots 159 Managing your snapshots 161 Partially contained databases 162 Configuring a contained database 162 Creating contained users 163 Understanding database collation changes 166 Detecting uncontained features 168 Conclusion 169 Chapter Logging and recovery 171 Transaction log internals 171 Phases of recovery 174 Page LSNs and recovery 175 Log reading 176 viii Contents Disk I/O,  28–30 memory options,  26–27 query processing options,  31–33 scheduling options,  27–28 SET TRANSACTION ISOLATION LEVEL command,  813 Set Working Set Size option,  26 SGAM (Shared Global Allocation Map) pages,  117, 856 SGAM (Shared Global Allocation Map) page, tempdb database,  148 Shared Global Allocation Map (SGAM) pages,  117, 856 Shared Global Allocation Map (SGAM) page, tempdb database,  148 shared locks,  775–776 Shared Memory,  11 Shared Table lock running DBCC CHECKDB in tempdb database,  144 shell queries,  712 short data cluster,  428 short data region (CD format),  428–429 SHOWPLAN_ALL output,  571 showplan options graphical plans,  517 text plans,  518 XML plans,  518 shrinking databases,  106–109 automatic shrinkage,  108 manual shrinkage,  108–109 log files,  109 shrinking databases tempdb,  147 shrinking files and databases,  837–841 AUTO_SHRINK option,  841–842 data file shrinking,  838–839 DBCC SHRINKFILE command,  840 log file shrinking,  840 shutdowns orderly shutdowns,  51 sibling facts,  847 side-tables,  458 simple parameterization,  713, 747 SIMPLE parameterization model,  698 SIMPLE recovery model (databases),  193 Simplification phase (Query Optimizer),  625 simultaneous transactions,  772 single-byte character strings,  214 SORT_IN_TEMPDB option single-byte character types Windows collations,  219–220 single-column indexes, seekable predicates,  528 SingleFragmentDocidFilter iterator,  504 SingleFragmentSeekFilter iterator,  504 SINGLE_USER mode,  122 size column (sys.database_files),  103 size management plan cache,  740–741 SIZE property,  105 sizing buffer pools,  55–57 memory,  54–55 pools Resource Governor,  65–66 skipping recompile steps,  728–729 sliding window benefits, partitions,  450–453 slot ID,  847 smalldatetime data type,  210 SMP (symmetric multiprocessing) NUMA,  36 snapshot-based isolation levels, row versioning,  813–830 DDL statements,  819–822 RCSI (read committed snapshot isolation),  813–814, 821–822 snapshot isolation,  814–815 snapshot isolation scope,  815–816 summary,  821–822 transaction metadata,  827–830 update conflicts,  817–818 version store,  823–827 viewing database state,  816–817 Snapshot isolation (transactions),  772 row versioning,  814–815 scope, row versioning,  815–816 snapshots (databases),  155–161 alternatives to,  844–845 consistency checks,  843 creating,  156–159 management,  161 space monitoring,  159–161 Snapshot Transactions counter,  827 SNI (Server Name Indication),  37 SNI (SQL Server Network Interface) protocol layer,  11 snowflake schemas,  659 snowflake schemas, data warehouses,  600 SORT_IN_TEMPDB option,  154 Index  941 SORT_IN_TEMPDB option (CREATE INDEX command) SORT_IN_TEMPDB option (CREATE INDEX command),  363 SORT_IN_TEMPDB (XML index option),  460 sort merge joins,  540–542 Sort operator,  675 sort order SQL Server collations, character data types (tables),  225 Windows collations, character data types (tables),  220–222 sort units tempdb database,  145 SOS_RESERVEDMEMBLOCKLIST waits,  749 SOS Scheduler,  38 space allocation altering databases,  116–119 space monitoring database snapshots,  159–161 tempdb database,  153–155 sparse column checks, processing columns,  869 sparse columns (storage format),  411–423 COLUMN_SET construct,  414–417 management,  411–414 altering tables,  413–414 creating tables,  412–413 metadata,  419 physical storage,  416–419 storage savings,  420–423 sparse column updates, Query Optimizer,  681 sparse files,  156 sparse vector,  417–418 spatial data types,  231 GEOGRAPHY,  471 GEOMETRY,  471 spatial-index consistency checks,  885 spatial indexes,  471–491 columns,  458, 476 composition of,  475–477 diagnostic stored procedures,  484–491 ensuring use,  478–479 nearest neighbor queries,  481–484 purpose of,  472–476 query plans,  479–480 spatial queries,  477 SpatialIntersectFilterOverGridIndex optimization,  477 SpatialJoinToApply optimization,  477 spatial queries,  477 SPATIAL_WINDOW_MAX_CELLS query hint,  479 SPATIAL_WINDOW_MAX_CELLS value,  491 942   Index sp_cacheobjects view,  745 sp_configure Recovery Interval option,  51 sp_configure stored procedure,  184 sp_control_plan_guide procedure,  758 sp_create_plan_guide_from_handle procedure,  763–764 sp_create_plan_guide procedure,  755 special data types (tables),  230–233 binary data types,  230 bit data types,  230 LOB data types,  230 rowversion data types,  231 spatial data types,  231 sql_variant data types,  231 table data types,  231 uniqueidentifier data types,  231 xml data types,  231 special indexes full-text indexes,  492–505 configuring,  500–501 creating,  498–499 diagnostic information,  500–501 extended event information,  503–505 internal tables,  494–497 maintenance,  499–500 metadata views,  497 query plans,  502–503 status metadata,  500–501 use in a query,  501–502 semantic indexes,  505–510 spatial columns,  458 spatial indexes,  471–491 composition of,  475–477 diagnostic stored procedures,  484–491 ensuring use,  478–479 nearest neighbor queries,  481–484 purpose of,  472–476 query plans,  479–480 spatial queries,  477 versus ordinary indexes,  457–458 XML,  458–471 creating and maintaining,  459–463 query plans,  465–467, 470–471 schema-validated columns,  469–470 secondary XML indexes,  468–469 XQuery,  463–465 sp_estimate_data_compression_savings stored procedure,  442 sp_executesql procedure,  717–718, 748 sp_filestream_force_garbage_collection procedure,  404 sp_fulltext_keymappings system stored procedure,  496 sp_fulltext_pendingchanges system procedure,  495 sp_fulltext_semantic_register_language_statistics_db system stored procedure,  505 sp_fulltext_semantic_unregister_language_statistics_ db system stored procedure,  505 sp_get_query_template procedure,  757, 761 sp_helpdb procedure,  sp_help_spatial_geography_histogram system stored procedure,  491 sp_help_spatial_geography_index diagnostic procedure,  484 sp_help_spatial_geography_index_xml system stored procedure,  484 sp_help_spatial_geometry_histogram system stored procedure,  491 sp_help_spatial_geometry_index diagnostic procedure,  484 sp_help_spatial_geometry_index_xml system stored procedure,  484 SPIDs (session IDs) compared to tasks,  39 spilling hash aggregates,  552–553 hash joins,  543 spinlocks,  775, 810 SPINLOOP value (task_state),  44 Split operator,  675–676 Split/Sort/Collapse optimization (Query Optimizer),  674–676 splitting pages, indexes,  342–346 intermediate index pages,  344 leaf-level pages,  344–347 root pages,  343 sp_migrate_user_to_contained stored procedure,  164 spool operators (Query Optimizer),  623 spools lazy,  535 sp_sequence_get_range stored procedure,  242 SQL-92 standard,  7, 205 SQL Azure,  SQLCMD tool,  sql_handle parameter,  731 sql_handle value,  735 SQL_Latin1_General_CP1_CI_AS collation,  218, 227 SQL Server Analysis Services SQLMail fiber mode,  40 SQL Manager Cache (SQLMGR),  735 SQL methods XML data types,  463–465 SQLMGR (SQL Manager Cache),  735 SQL options,  126–127 SQLOS (SQl Server Operating System) memory DMVs (Dynamic Management Objects),  57–59 SQLOS (SQL Server Operating System).  See SQLOS memor column store object pool,  48 memory,  47–61 buffer pool,  47 buffer pool sizing,  55–57 checkpoints,  50–52 column store object pool,  48 data cache,  47, 47–50 free buffer list,  49–50 lazywriter,  49–50 management in other caches,  52–53 Memory Broker,  54 NUMA (Non-Uniform Memory Access),  59–60 page management in data cache,  48–49 read-ahead,  60 sizing,  54–55 NUMA (Non-Uniform Memory Access) architecture,  36–37 SQLOS (SQL Server Operating Sytem) memory access to pages in data cache,  48 SQL plan guides,  756–757 SQL Plans cache store,  732 SQL PowerShell,  SQL Server,  editions,  1–2 SQL Server 2012,  configuring,  17–20 default network configuration,  18–19 network protocols,  18 system configuration,  21–33 editions memory configurations,  56 installing,  SQL Server Agent service msdb database,  141 SQL Server Analysis Services schemas,  133 Index  943 SQL Server Books Online SQL Server Books Online editions,  SQL Server Browser service,  20 SQL Server collations, character data types (tables),  224–230 defined collations during setup,  227–228 Installation Wizard,  228 performance considerations,  229 sort orders,  225 tertiary collations,  225–227 traps,  229–230 SQL Server Configuration Manager configuring network protocols,  18 configuring SQL Server 2012,  18–19 implementing default network configuration,  18–19 managing services,  19 moving the master database,  143 SQL Server Browser service,  20 SQL Server Database Engine,  components,  10–17 protocols,  11–12 query processor,  12–14 storage engine,  14–17 SQL Server error log, DBCC CKECKDB output,  888 SQL Server Express,  SQL Server Express with Advanced Services,  SQL Server Express with Tools,  sqlserver.latch_suspend_end event,  151 SQL Server Management Studio,  2, 184 Object Explorer pane,  server property pages,  25 SQL Server Network Interface (SNI) protocol layer,  11 SQL Server Operating System.  See SQLOS SQL Server Profiler,  176 SQL Server Resolution Protocol (SSRP),  20 SQL Server Resource Governor.  See Resource Governor SQL Server schedulers,  37–47 dynamic affinity,  41 fibers,  40 NUMA (Non-Uniform Memory Access),  40–41 tasks,  39–40 threads,  40 workers,  39 SQLServer:Transactions performance object,  826–827 SQL statements DAC (Dedicated Administrator Connection),  46 944   Index sqlvariant columns,  382 sql_variant data, tables,  273–276 sql_variant data types,  231 SQLXML fiber mode,  40 SRID, spatial index columns,  458 SSRP (SQL Server Resolution Protocol),  20 stack_bytes_used (sys.dm_os_threads),  44 stale statistics, optimality-based recompiles,  725–727 Standard edition,  standards SQL-92,  standby, restoring with,  200–201 star join queries,  600 Star Join queries,  660 star schema, data warehouses,  600 star schemas,  659 started_by_sqlserver (sys.dm_os_threads),  44 START FULL POPULATION option (ALTER FULLTEXT INDEX statement),  499 START INCREMENTAL POPULATION s option (ALTER FULLTEXT INDEX statement),  499 Starting value property, SEQUENCE objects,  240 start parallelism exchange,  585 START UPDATE POPULATION option (ALTER FULLTEXT INDEX statement),  499 State column (sys.database_files),  102 state_desc column (sys.database_files),  103 statements,  67 ALTER CONTAINMENT option,  163 ALTER DATABASE,  114–119 FILEGROUP keyword,  110 ALTER EVENT SESSION,  83 ALTER INDEX manually controlling unit of locking,  804 ALTER TABLE escalating table locks,  805 CREATE DATABASE CONTAINMENT option,  163 FILEGROUP keyword,  110 CREATE EVENT SESSION,  83 CREATE INDEX monitoring space usage in tempdb,  154 CREATE SCHEMA,  133 CREATE VIEW WITH SCHEMABINDING option,  335 DBCC CHECKIDENT(tablename),  240 DBCC OPTIMIZER_WHATIF,  602 DDL (Data Definition Language) ALTER FULLTEXT INDEX,  499 CREATE DATABASE FOR ATTACH,  505 CREATE FULLTEXT CATALOG,  492–493 CREATE PRIMARY XML INDEX,  459 CREATE SPATIAL INDEX,  472 snapshot isolation,  819–822 DELETE,  598–599 GRANT CREATE TABLE TO,  133 INSERT,  598–599 RESTORE DATABASE PAGE clause,  199 SET DEADLOCK_PRIORITY,  809 UPDATE,  598–599 WRITE clause,  190 UPDATE STATISTICS,  606 UPDATETEXT,  190 WRITETEXT,  190 element, displaying query plans,  524 state options,  122–125 states, virtual log files,  178 STATISTICAL_SEMANTICS keyword,  493, 505 statistics optimality-based recompiles stale,  725–727 updated,  725 Query Optimizer,  630–638 density/frequency information,  634–636 design,  631–634 filtered,  636–637 string,  637–638 wait,  748–749 STATISTICS_NORECOMPUTE option, creating indexes,  314 STATISTICS PROFILE output,  635 STATMAN aggregate function,  633 Status column (VLFs),  180 STDistance() method,  472–473 STIntersects operation,  488 STIntersects() spatial operator,  480 element, displaying query plans,  524 STOPLIST specification full-text indexes,  493 stop parallelism exchange,  585 stopping event sessions,  86 storage columnstore indexes,  371–375 sys.column_store_segments view data data compression,  423–443 FILESTREAM data,  394–405 FileTable data,  404–410 LOB (large object) data,  381–393 metadata,  245–246 partitioning,  444–455 sparse columns,  411–423 tables,  203–296 storage engine,  14–17 access methods,  14–15 transaction services,  15–17 stored procedures,  719–720, 747 sp_sequence_get_range,  242 stream aggregations,  548–552 String Statistics feature, Query Optimizer,  637–638 structure B-tree indexes,  308–312, 315–332 clustered indexes,  316–322 clustering key dependency,  308–311 constraints,  312 index row formats,  315–316 nonclustered B-tree indexes,  311–312 nonclustered indexes,  322–332 stub, Optimize for Ad Hoc Workloads option,  709–711 subqueries,  566–580 CASE expressions,  578–580 correlated,  566 correlated scalar subqueries,  570–577 multirow,  566 noncorrelated,  566 noncorrelated scalar subqueries,  567–569 removing correlations,  576–577 scalar,  566 subtype resources,  786–787 supplementary characters (SC) tokens,  218 suspect_pages table,  198–199 SUSPECT state,  123 SUSPENDED value (task_state),  44 switching recovery models,  193–194 SWITCH operations,  658 SWITCH option, partitions,  450 SWITCH PARTITION operator,  670 symmetric multiprocessing.  See SMP syscacheobjects,  syscacheobjects object,  744–746 sys.column_store_segments catalog view,  376–380 sys.column_store_segments view,  454 Index  945 sys.database_files sys.database_files columns,  102–103 sys.database_recovery_status catalog view,  185 sys.databases,  4, sysdatabases,  sys.databases catalog view,  119–129 sys.databases view,  8, 120–129 columns,  120 state options,  122–125 sys.data_spaces columns,  446 sys.destination_data_spaces columns,  447 sys.dm_db_database_page_allocations DMV locating physical pages, tables,  261 sys.dm_db_file_space_usage DMV monitoring space usage in tempdb,  153 sys.dm_db_fts_index_physical_stats DMV,  506 sys.dm_db_session_space_usage DMV monitoring space usage in tempdb,  153 sys.dm_db_task_space_usage DMV monitoring space usage in tempdb,  153 sys.dm_db_uncontained_entities DMV detecting uncontained features in databases,  168 sys.dm_exec_cached_plan_dependent_objects function,  738 sys.dm_exec_cached_plans,  6, 72 sys.dm_exec_cached_plans view,  738–739 sys.dm_exec_connections,  sys.dm_exec_procedure_stats view,  740–743 sys.dm_exec_query_memory_grants,  72 sys.dm_exec_query_resource_semaphores,  72 sys.dm_exec_query_stats function,  739–740 sys.dm_exec_requests,  6, 72 sys.dm_exec_requests catalog view,  889 sys.dm_exec_requests view,  739–740 sys.dm_exec_sessions,  6, 72 sys.dm_exec_sql_text function,  736–738 sys.dm_exec_text_query_plan function,  737 sys.dm_fts_index_keywords_by_document function,  497 sys.dm_fts_index_keywords_by_property function,  497 sys.dm_fts_index_keywords function,  497 sys.dm_fts_index_population DMV,  500, 506 sys.dm_fts_memory_buffers DMV,  500 sys.dm_fts_memory_pools DMV,  500 sys.dm_fts_outstanding_batches DMV,  500 sys.dm_fts_population_ranges DMV,  500 sys.dm_fts_semantic_similarity_population DMV,  506 sys.dm_io_virtual_file_stats function,  159 sys.dm_os_memory_brokers,  72 946   Index sys.dm_os_memory_cache_clock_hands,  53 columns,  59 sys.dm_os_memory_cache_counters columns,  58 sys.dm_os_memory_cache_hash_tables columns,  58 sys.dm_os_memory_clerks,  54 columns,  57–58 sys.dm_os_performance_counters view,  188 sys.dm_os_process_memory,  56 columns,  56 sys.dm_os_schedulers columns,  43 sys.dm_os_sys_info columns,  56 sys.dm_os_tasks,  44–45 sys.dm_os_threads,  44 sys.dm_os_waiting_tasks,  45 sys.dm_os_wait_stats view wait statistics,  748–749 sys.dm_os_workers,  44 sys.dm_resource_governor_configuration,  72 sys.dm_resource_governor_resource_pools,  72 sys.dm_resource_governor_workload_groups,  72 sys.dm_tran_current_transaction view,  827–828 sys.dm_tran_locks view viewing locks,  786–789 sys.dm_xe_packages,  75 sys.dm_xe_sessions columns,  96–97 sys,dn_db_database_page_allocations,  824 sys.fn_PhysLocFormatter function locating physical pages, tables,  261–262 sys.fn_validate_plan_guide function,  763 sys.fulltext_index_catalog_usages metadata table,  497 sys.fulltext_semantic_languages metadata view,  505 sys.fulltext_semantic_language_statistics_database metadata view,  505 sys.indexes,  sysindexes,  sys.indexes Catalog view, tables,  244–245 sys.language_model_mapping_table internal table,  505 syslockinfo tables,  800–802 sysmembers,  sysmessages,  sys.objects,  sysobjects,  sys.partition_functions columns,  447 tables sys.partition_range_values columns,  447 sys.partition_schemes columns,  447 sys.partitions view,  245 sys.plan_guides view,  759 sysprocesses,  sys.resource_governor_configuration,  71 sys.resource_governor_resource_pools,  72 sys.resource_governor_workload_groups,  72 sys schema,  205 sys.sysallocunits system catalog,  855 sys.syshobtcolumns table,  855 sys.syshobts table,  855 sys.sysrowsets system catalog,  855 sys.sysrscols system catalog,  855 sys.system_internals_partition_columns view,  264 sys.tables,  SysTab value (objtype columns),  706 system base tables,  system catalog consistency checks,  855–856 system configuration configuring,  21–33 operating system,  21–23 settings,  24–33 Disk I/O,  28–30 memory options,  26–27 query processing options,  31–33 scheduling options,  27–28 system databases master,  139–140 model,  140 moving,  142–144 msdb,  141–142 partially contained,  162–169 collation changes,  166–168 configuring,  162–163 creating,  163–166 detecting uncontained features,  168–169 resource,  140–141 snapshots,  155–161 creating,  156–159 management,  161 space monitoring,  159–161 tempdb,  140, 144–156 best practices,  147–148 contention,  148–152 objects,  144–145 optimizations,  146–147 space monitoring,  153–155 system functions,  8–9 system_internals_allocation_units view,  446 system metadata objects catalog views,  4–5 compatibility views,  3–4 Dynamic Management Objects,  6–7 system stored procedures,  system tables Compatibility Views,  3–4 sysusers,  T table data types,  231 table-level data modification, indexes,  357 Table metadata objects,  861 tables,  203–296 altering,  279–286 adding columns,  281 changing data types,  280–281 constraints,  281–282 dropping columns,  283 internals,  283–286 sparse columns (storage format),  413–414 constraints,  276–279 failures in transactions,  278–279 names and catalog view information,  277–278 creating,  203–204 data types,  208–233 delimited identifiers,  206–207 FILESTREAM data,  397–398 naming,  204–205 naming conventions,  207–208 NULL values,  233–235 reserved keywords,  205–206 sparse columns (storage format),  412–413 user-defined data types,  235–236 data types,  208–233 character,  213–230 date and time,  210–213 numeric,  208–210 special data types,  230–233 user-defined,  235–236 dimension,  600 fact,  600 filegroups,  111 hash,  48 heap modification internals,  286–295 allocation structures,  286–288 deleting rows,  288–292 inserting rows,  288 updating rows,  292–295 Index  947 table scans IDENTITY property,  237–240 internal storage,  243–276 catalog view queries,  246–248 data and time data,  270–273 data pages,  248–257 data rows,  257–259 data storage metadata,  245–246 fixed-length rows,  262–265 locating a physical page,  259–262 NULL values,  267–270 sql_variant data,  273–276 sys.indexes Catalog view,  244–245 variable-length columns,  267–270 variable-length rows,  265–267 naming,  204–208 conventions,  207–208 delimited identifiers,  206–207 reserved keywords,  205 NULL issues,  233–235 partitioning,  444–455 columnstore indexes,  452–454 functions and schemes,  444–446 metadata,  446–449 sliding window benefits,  450–453 per-table logical consistency checks,  860–866 data and index page processing,  864–865 metadata consistency checks,  861–862 page audits,  863–865 sequence objects,  240–243 system Compatibility Views,  3–4 system base,  table scans read-ahead,  60 table-valued functions tempdb database,  145 Table-Valued Function [XML Reader] iterator,  471 Table-Valued Function [XML Reader with XPath Filter] iterator,  470 table variables tempdb database,  145 TABLOCK hint,  833 TABLOCK option (DBCC CHECKDB),  892 TABLOCKX hint,  833 tabular data stream (TDS) endpoints,  12 packets,  11–12 TagB row header,  315 Tag Index (TI) semantic index,  506 Target Memory,  56 948   Index Target Recovery Time database configuration option,  51 TARGET_RECOVERY_TIME option,  29 targets Extended Events,  80–82 viewing data,  90–95 tasks,  39–40 Blocked Process Threshold option,  31 compared to SPIDs (session IDs),  39 managing,  21–22 task_state (sys.dm_os_tasks),  44 TCP/IP,  11 TDS (tabular data stream) endpoints,  12 packets,  11–12 tempdb database,  140, 144–156 best practices,  147–148 contention,  148–152 DDL contention,  153 DML contention,  148–152 objects,  144–145 optimizations,  146–147 space monitoring,  153–155 temporary tables,  166 Temp Tables Creation Rate performance counter,  153 Temp Tables for Destruction performance counter,  153 termination options,  124–125 tertiary collations SQL Server collations, character data types (tables),  225–227 TERTIARY_WEIGHTS function,  225 tessellation levels, cells,  472 TextAdd UDX operator,  471 TEXT_DATA pages,  388 text_in_row_limit value,  389 TEXT_MIXED pages,  388 text pages, processing columns,  870–871 text plans, queries,  518 threads compared to fibers,  40 parallel query execution plans,  32 serial execution plans,  32 three-part name specification, tables,  204 ticks,  744 time data internal storage, tables,  270–273 time data types,  210–213 TI (Tag Index) semantic index,  506 UNION queries toggling on/off trace flags,  23 tokens, collations,  218 tools analyzing indexes,  302–307 dm_db_database_page_allocations function,  306–307 dm_db_index_physical_stats DMV,  302–305 DBCC MEMORYSTATUS,  691 SET STATISTICS TIME ON,  691 SQLCMD,  ToolTip merge join operator,  520 rewind and rebind information,  572 TopNByRank iterator,  504 TOP N Sort iterator,  482 Top operations,  671 TORN_PAGE_DETECTION option,  128 total_virtual_address_space_kb column (sys.dm_os_ process_memory),  56 traceflag 1106,  149 trace flag 2549,  853 trace flag 2562,  853 trace flags,  23–24 1211,  805 1224,  806 2371,  727 toggling on/off,  23 tracking type, changing in full-text indexes,  493 transaction_id column,  827 transaction_is_snapshot column,  827 transaction logs,  171–177 changes in log size,  178–188 autoshrinking,  187–188 maintaining recoverable logs,  185–187 viewing log file size,  188 virtual log files,  178–185 LSN (Log Sequence Number),  158 recovery log cache,  177 page LSNs,  175–176 phases,  174–175 readable records,  176–177 truncating,  176 transactions,  766–774 ACID properties,  15, 767–768 dependencies,  768–769 isolation levels,  770–775 Read Committed isolation,  771 Read Uncommitted isolation,  770 Repeatable Read isolation,  771–772 Serializable isolation,  773–774 Snapshot isolation,  772 manipulating FILESTREAM data,  401–402 metadata snapshot-based isolation levels,  827–830 Recovery Interval option,  29 transaction sequence number (XSN),  811 transaction_sequence_num column,  827 transaction services,  15–17 Transact-SQL (T-SQL) catalog views,  tree format, Query Optimizer,  612–613 trie trees,  637–638 Trigger value (objtype columns),  705 trivial plans (Query Optimizer),  616, 625–626 troubleshooting plan cache issues,  748–752 TRUNCATEONLY option, shrinking files and databses,  839 truncating databases,  185 transaction logs,  176 T-SQL BULK INSERT command,  238 T-SQL query constructs,  458 T-SQL Reference,  298 T-SQL syntax,  156 T-SQL (Transact-SQL) catalog views,  TYPE COLUMN specification full-text indexes,  493 type_desc column (sys.database_files),  102 types Extended Events,  79 U UDTs (user-defined data types),  631 UDT (user-defined data type),  852 UDX iterator,  471 UMS (User Mode Scheduler),  38 uncontained features (partially contained databases), detecting,  168–169 undo phase of recovery,  173–174 Unicode character strings,  213 Uniformity assumption, Query Optimizer,  637 UNION ALL queries,  555–559 UNION operations, Query Optimizer,  693–694 UNION queries,  555–559 indexes,  562–564 Index  949 UNIQUE constraint UNIQUE constraint,  276 UNIQUEIDENTIFIER data type,  507 uniqueidentifier data types,  231 UNIQUE KEY constraints,  312 universal unique identifier (UUID) data type,  231 unrestricted-length LOB data, storage,  386–393 appending data into columns,  393 data rows,  389–392 max-length data,  392–393 Unused state (VLFs),  179 UPDATE command,  671 Update Conflict Ratio counter,  827 update conflicts snapshot-based isolation levels, row versioning,  817–818 updated statistics, optimality-based recompiles,  725 update locks,  776 updates FILESTREAM data,  400 Query Optimizer,  670–686 Halloween Protection feature,  674–675 locking,  685–686 MERGE operation,  676–678 non-updating updates,  681–682 partitioned table updates,  682–685 partition-level lock escalation,  686 sparse column updates,  681 Split/Sort/Collapse optimization,  674–676 wide update plans,  679–681 UPDATE statement,  598–599 WRITE clause,  190 UPDATE STATISTICS command,  725 UPDATE STATISTICS statement,  606 UPDATETEXT statement,  190 updating rows heap data modifications,  292–295 modifying indexes,  354–358 UPDLOCK hint,  819, 833 USE PLAN hint,  754 USE PLAN hint, Query Optimizer,  699–700 User Connections option,  26 user data lock types for,  775–786 lock duration,  785 lock granularity,  778–784 lock modes,  775–778 lock ownership,  785–786 user-defined data types (tables),  235–236 user-defined data types (UDTs),  631 950   Index user-defined data type (UDT),  852 user-defined scalar functions, forcing recompilation,  720–721 User Interfaces Extended Events,  86–97 aggregating data,  93–95 Data Storage page,  89 Events page,  87–88 General page,  86–87 grouping data,  93–95 viewing target data,  90–95 User Mode Scheduler (UMS),  38 user objects tempdb database,  145 user stores,  53 UsrTab value (objtype columns),  706 UUID (universal unique identifier) data type,  231 V validation plan guides,  763–764 value-based encoding,  372 value method,  464 VALUE secondary XML index,  463, 468 varbinary data,  382 varchar(MAX) data type,  389 vardecimal (data compression),  423 vardecimal storage format property,  209 variable-length columns, tables,  214, 267–270 variable-length rows, tables,  265–267 variables @BlobEater,  850 VAS (virtual address space),  55–57 Version Cleanup Rate counter,  826 Version Generation Rate counter,  826 versioning tempdb database,  146 versioning code,  16 versioning info,  430 versioning operations,  15 versions Windows collations, character data types (tables),  219 version store snapshot-based isolation levels, row versioning,  823–827 tempdb database,  146 Version Store Size counter,  826 very large databases (VLDBs), partial backups,  198 word breaker view sys.databases,  120–129 viewa sys.resource_governor_workload_groups,  72 view expansion preoptimization activity,  625 viewing collation options, character data types (tables),  218 database state, row versioning,  816–817 locks,  786–789 request_Columns,  788–789 resource_Columns,  786–787 log file size,  188 query plans,  520–526 resource database contents,  141 target data (Extended Events UI),  90–95 views sys.databases columns,  120 state options,  122–125 sys.databases catalog,  119 sys.dm_exec_cached_plans,  72 sys.dm_exec_query_memory_grants,  72 sys.dm_exec_query_resource_semaphores,  72 sys.dm_exec_requests,  72 sys.dm_exec_sessions,  72 sys.dm_os_memory_brokers,  72 sys.dm_os_schedulers,  43 sys.dm_os_tasks,  44–45 sys.dm_os_threads,  44 sys.dm_os_waiting_tasks,  45 sys.dm_os_workers,  44 sys.dm_resource_governor_configuration,  72 sys.dm_resource_governor_resource_pools,  72 sys.dm_resource_governor_workload_groups,  72 sys.dm_tran_current_transaction,  827–828 sys.dm_tran_locks viewing locks,  786–789 sys.plan_guides,  759 sys.resource_governor_configuration,  71 sys.resource_governor_resource_pools,  72 View Server State permission,  42 View value (objtype columns),  706 virtual_address_space_available_kb column (sys dm_os_process_memory),  57 virtual_address_space_committed_kb column (sys dm_os_process_memory),  57 virtual_address_space_reserved_kb column (sys dm_os_process_memory),  56 virtual address space (VAS),  55–57 virtual log files.  See VLFs virtual_memory_committed_kb column (sys.dm_os_ memory_clerks),  58 virtual_memory_kb column (sys.dm_os_sys_info),  56 virtual_memory_reserved_kb column (sys.dm_os_ memory_clerks),  58 VLDBs (very large databases), partial backups,  198 VLFs (virtual log files),  178–185 automatic truncation,  184–185 observing,  179–181 states,  178 using multiple files,  181–183 volumes NTFS, creating snapshots,  159 W wait_duration_ms (sys.dm_os_waiting_tasks),  45 wait statistics,  748–749 wait_type (sys.dm_os_waiting_tasks),  45 warehouses, data,  599–603 Web edition,  WHERE clause full-text index queries,  501 wide update plans, Query Optimizer,  679–681 width sensitivity (WS) tokens,  218 windowing functions, Query Optimizer,  658–660 windows New Database,  104 Windows collations, character data types (tables),  218–224 binary collations,  223–224 character ranges,  222–223 naming,  218–219 single-byte character types,  219–220 sort order,  220–222 versions,  219 Window Spool operator,  658 Windows principal contained users,  163 Windows scheduler,  37, 38 Windows Server 2008 R2 Server Core SP1 installing SQL Server 2012,  WITH (NOEXPAND) hint,  650 WITH RECOMPILE option,  720 WITH RECOVERY option, RESTORE command,  196 WITH SCHEMABINDING option (CREATE VIEW statement),  335 WITH VALUES clause (ALTER TABLE command),  281 word breaker,  498 Index  951 workers workers,  39 affinity,  39 work files, tempdb database,  145 workload groups,  66–68 Default group,  66 Internal groups,  66 properties,  67–68 work_queue_count (sys.dm_os_schedulers),  43 work tables, tempdb database,  145 write-ahead logging,  15, 172 WRITE clause,  190 WRITETEXT statement,  190 WS (width sensitivity) tokens,  218 X XACT_ABORT SET option,  279 XLOCK hint,  834 xmaxlen columns,  263 xml data types,  231 XML data types, built-in SQL methods,  463–465 XML-Index consistency checks,  884–885 XML indexes,  458–471, 881 creating and maintaining,  459–463 primary XML indexes,  460–463 secondary XML indexes,  463 query plans,  465–467, 470–471 schema-validated columns,  469–470 secondary XML indexes,  468–469 XQuery,  463–465 XML query plans,  518 XQuery, XML indexes,  463–465 XSN (transaction sequence number),  811 952   Index About the authors K ALE N DE L ANE Y (primary author) has been working with Microsoft SQL Server for over 26 years, and she provides advanced SQL Server training to clients around the world She has been a SQL Server MVP (Most Valuable Professional) since 1992 and has been writing about SQL Server almost as long Kalen has spoken at dozens of technical conferences, including almost every PASS Community Summit held in the United States since the organization’s founding in 1999 Kalen is a contributing editor and columnist for SQL Server Magazine and the author or co-author of many Microsoft Press books on SQL Server, including Inside Microsoft SQL Server 7, Inside Microsoft SQL Server 2000, Inside Microsoft SQL Server 2005: The Storage Engine, Inside Microsoft SQL Server 2005: Query Tuning and Optimization and SQL Server 2008 Internals Kalen blogs at www.sqlblog.com, and her personal website can be found at www.SQLServerInternals.com BOB BE AUCHE MIN (author) is a database-centric instructor, course author, writer, conference speaker, application practitioner and architect, and a Developer Skills Partner for SQLskills Bob has been a Microsoft MVP since 2002 He's written and taught courses on SQL Server and data access worldwide since the mid-1990s, and currently writes and teaches SQLskills' developer and DBA-centric immersion course offerings He is lead author of the books A Developer's Guide to SQL Server 2005 and A First Look at SQL Server 2005 For Developers, and sole author of Essential ADO.NET He's written numerous Microsoft whitepapers, as well as articles on SQL Server and other databases, database security, ADO.NET, and OLE DB for a number of magazines CONOR CUNNINGHAM (author) is principal architect of the SQL Server Core Engine Team, with over 15 years experience building database engines for Microsoft He specializes in query processing and query optimization, and he designed and/or implemented a number of the query processing features available in SQL Server Conor holds a number of patents in the field of query optimization, and he has written numerous academic papers on query processing Conor blogs at “Conor vs SQL” at http://blogs.msdn.com/b/ conor_cunningham_msft JONATHAN KE HAYIA S (author and technical reviewer) is a Principal Consultant and Trainer for SQLskills, one of the most well-known and respected SQL Server training and consulting companies in the world Jonathan is a SQL Server MVP and was the youngest person ever to obtain the Microsoft Certified Masters for SQL Server 2008 Jonathan is a performance tuning expert, for both SQL Server and hardware, and has architected complex systems as a developer, business analyst, and DBA Jonathan also has extensive development (T-SQL, C#, and ASP.Net), hardware and virtualization design expertise, Windows expertise, Active Directory experience, and IIS administration experience Jonathan frequently blogs about SQL Server at http://www.SQLskills.com/blogs/Jonathan, and can be reached by email at Jonathan@SQLskills.com, or on Twitter as @SQLPoolBoy He regularly presents at PASS Summit, SQLBits, SQL Intersections, SQL Saturday events, and local user groups and has remained a top answerer of questions on the MSDN SQL Server Database Engine forum since 2007 BE NJAMIN NE VARE Z (author and technical reviewer) is a database professional based in Los Angeles, California, and author of Inside the SQL Server Query Optimizer, published by Red Gate books He has 20 years of experience with relational databases, and has been working with SQL Server since version 6.5 Benjamin holds a Master’s degree in computer science and has been a speaker at many technology conferences, including the PASS Summit and SQL Server Connections His blog is at http://benjaminnevarez.com and can be reached by email at admin@benjaminnevarez.com and on twitter at @BenjaminNevarez PAUL S R ANDAL (author) is the CEO of SQLskills.com, the world-renowned training and consulting company that he runs with his wife Kimberly L Tripp He is also a SQL Server MVP and a Microsoft Regional Director Paul worked at Microsoft for almost nine years, after spending five years at DEC working on the OpenVMS file system He wrote various DBCC commands for SQL Server 2000 and then rewrote all of DBCC CHECKDB and repair for SQL Server 2005 before moving into management in the SQL team During SQL Server 2008 development, he was responsible for the entire Storage Engine Paul regularly consults and teaches at locations around the world, including the SQLskills Immersion Events on internals, administration, high-availability, disaster recovery, and performance tuning He also wrote and taught the SQL Server Microsoft Certified Master certification for Microsoft He is a top-rated presenter at conferences such as the PASS Summit, and owns and manages the SQLintersections conferences Paul's popular blog is at www.SQLskills.com/blogs/paul and he can be reached at Paul@SQLskills.com and on Twitter as @paulrandal 954   Microsoft SQL Server 2012 Internals Now that you’ve read the book Tell us what you think! Was it useful? Did it teach you what you wanted to learn? Was there room for improvement? Let us know at http://aka.ms/tellpress Your feedback goes directly to the staff at Microsoft Press, and we read every one of your responses Thanks in advance! ... the Inside SQL Server series, which included Inside SQL Server 6.5, Inside SQL Server 7, Inside SQL Server 2000, and Inside SQL Server 2005 (in four volumes) and the SQL Server 2008 Internals. .. http://twitter.com/MicrosoftPress Introduction  xxv CHAPTER SQL Server 2012 architecture and configuration Kalen Delaney M icrosoft SQL Server is Microsoft s premier database management system, and SQL Server. .. the SQL Server Books Online topic, “Features Supported by the Editions of SQL Server 2012. ” A value of for EngineEdition indicates that your SQL Server edition is Express, which includes SQL Server