564 Chapter 15: Query Expressions Three things in the output are remarkable First, notice that after selection is assigned, DelegateInvocations remains at zero At the time of assignment to selection, no iteration over Keywords is performed If Keywords were a property, the property call would run—in other words, the from clause executes at the time of assignment However, neither the projection, the filtering, nor anything after the from clause will execute until the code iterates over the values within selection It is as though at the time of assignment, selection would more appropriately be called “query.” However, once we call Count(), a term such as selection or items that indicates a container or collection is appropriate because we begin to count the items within the collection In other words, the variable selection serves a dual purpose of saving the query information and acting like a container from which the data is retrieved A second important characteristic to notice is that calling Count() twice causes func to again be invoked once on each item selected Since selection behaves both as a query and as a collection, requesting the count requires that the query be executed again by iterating over the IEnumerable collection selection refers to and counting the items—returning the most up-to-date results Similarly, a foreach loop over selection would trigger func to be called again for each item The same is true of all the other extension methods provided via System.Linq.Enumerable Filtering In Listing 15.1, we include a where clause that filters out pure keywords but not contextual keywords The where clause filters the collection vertically so that there are fewer items within the collection The filter criteria are expressed with a predicate—a lambda expression that returns a bool such as word.Contains() (as in Listing 15.1) or File.GetLastWriteTime(file) < DateTime.Now.AddMonths(-1) (as in Listing 15.6, the output of which appears in Output 15.5) Listing 15.6: Anonymous Types within Query Expressions using using using using System; System.Collections.Generic; System.Linq; System.IO; Introducing Query Expressions // static void FindMonthOldFiles( string rootDirectory, string searchPattern) { IEnumerable files = from fileName in Directory.GetFiles( rootDirectory, searchPattern) where File.GetLastWriteTime(fileName) < DateTime.Now.AddMonths(-1) select new FileInfo(fileName); foreach (FileInfo file in files) { // As simplification, current directory is // assumed to be a subdirectory of // rootDirectory string relativePath = file.FullName.Substring( Environment.CurrentDirectory.Length); Console.WriteLine(".{0}({1})", relativePath, file.LastWriteTime); } } // OUTPUT 15.5: \TestData\Bill.cs(8/10/2007 9:33:55 PM) \TestData\Contact.cs(8/19/2007 11:40:30 PM) \TestData\Employee.cs(8/17/2007 1:33:22 AM) \TestData\Person.cs(10/22/2007 10:00:03 PM) Sorting To order the items using a query expression we rely on the orderby clause (see Listing 15.7) Listing 15.7: Sorting Using an Query Expression with an orderby Clause using using using using System; System.Collections.Generic; System.Linq; System.IO; // static void ListByFileSize1( string rootDirectory, string searchPattern) { 565 566 Chapter 15: Query Expressions IEnumerable fileNames = from fileName in Directory.GetFiles( rootDirectory, searchPattern) orderby (new FileInfo(fileName)).Length, fileName ascending select fileName; foreach (string fileName in fileNames) { // As simplification, current directory is // assumed to be a subdirectory of // rootDirectory string relativePath = fileName.Substring( Environment.CurrentDirectory.Length); Console.WriteLine(".{0}", relativePath); } } // Listing 15.7 uses the orderby clause to sort the files returned by Directory.GetFiles() first by file size and then by filename in ascending order Multiple sort criteria are separated by a comma such that first the items are ordered by size, and if the size is the same they are ordered by filename ascending and descending are contextual keywords indicating the sort order If the keyword is omitted, the default is ascending Let In Listing 15.8, we have a query that is very similar to that in Listing 15.7, except that the type argument of IEnumerable is FileInfo One of the problems with the groupby clause in Listing 15.8 is that in order to evaluate the size of the file, an instance of FileInfo needs to be available in both the orderby clause and the select clause Listing 15.8: Projecting a FileInfo Collection and Sorting by File Size using using using using System; System.Collections.Generic; System.Linq; System.IO; // static void ListByFileSize2( string rootDirectory, string searchPattern) { IEnumerable files = Introducing Query Expressions from fileName in Directory.GetFiles( rootDirectory, searchPattern) orderby new FileInfo(fileName).Length, fileName select new FileInfo(fileName); foreach (FileInfo file in files) { // As simplification, current directory is // assumed to be a subdirectory of // rootDirectory string relativePath = file.FullName.Substring( Environment.CurrentDirectory.Length); Console.WriteLine(".{0}({1})", relativePath, file.Length); } } // Unfortunately, although the result is correct, Listing 15.8 ends up instantiating a FileInfo object twice for each item in the source collection FileInfo is instantiated not only in the select clause, but also when the orderby clause is evaluated To avoid unnecessary overhead like this, overhead that could potentially be expensive, the query expression syntax includes a let expression, as demonstrated in Listing 15.9 Listing 15.9: Ordering the Results in a Query Expression using using using using System; System.Collections.Generic; System.Linq; System.IO; // static void ListByFileSize3( string rootDirectory, string searchPattern) { IEnumerable files = from fileName in Directory.GetFiles( rootDirectory, searchPattern) let file = new FileInfo(fileName) orderby file.Length, fileName select file; foreach { // // // (FileInfo file in files) As simplification, current directory is assumed to be a subdirectory of rootDirectory 567 568 Chapter 15: Query Expressions string relativePath = file.FullName.Substring( Environment.CurrentDirectory.Length); Console.WriteLine(".{0}({1})", relativePath, file.Length); } } // The let clause provides a location to place an expression that is used throughout the query expression To place a second let expression, simply add it as an additional clause to the query after the first from clause but before the final select/groupby clause No operator is needed to separate out the expressions Grouping Another common collection scenario is the grouping of items In SQL, this generally involves aggregating the items into a summary header or total— an aggregate value However, C# is more expressive than this In addition to providing aggregate information about each grouping, query expressions allow for the individual items in the group to form a series of subcollections to each item in the overall parent list For example, it is possible to group the contextual keywords separately from the regular keywords and automatically associate the individual words within the keyword type grouping to each other Listing 15.10 and Output 15.6 demonstrate the query expression Listing 15.10: Grouping Together Query Results using System; using System.Collections.Generic; using System.Linq; // private static void GroupKeywords1() { IEnumerable selection = from word in Keywords group word by word.Contains('*'); foreach (IGrouping wordGroup in selection) { Introducing Query Expressions Console.WriteLine(Environment.NewLine + "{0}:", wordGroup.Key ? "Contextual Keywords" : "Keywords"); foreach (string keyword in wordGroup) { Console.Write(" " + (wordGroup.Key ? keyword.Replace("*", null) : keyword)); } } } // OUTPUT 15.6: Keywords: abstract as base bool break byte case catch char checked class const continue decimal default delegate double else enum event explicit extern false finally fixed float for foreach goto if implicit in int interface internal is lock long namespace new null object operator out override params private protected public readonly ref return sbyte sealed short sizeof stackalloc static string struct switch this throw true try typeof uint ulong unchecked unsafe ushort using virtual void volatile while Contextual Keywords: add ascending by descending from get group into join let orderby partial remove select set value where yield There are several things to note in this listing First, each item in the list is of type IGrouping The type parameters of IGrouping are determined by the data type following group and by—that is, TElement is a string because word is a string Type parameter TKey is determined by the data type following by In this case, word.Contains() returns a Boolean, so TKey is a bool A second characteristic of a query expression’s groupby clause is that it enables a nested foreach loop via which the code can iterate over the subcollection mentioned earlier in this section In Listing 15.10, we first iterate over the groupings and print out the type of keyword as a header Nested within the first iteration is a second foreach loop that prints each keyword as an item below the header Third, we can append a select clause to the end of a groupby clause, allowing support for projection (see Listing 15.11 and Output 15.7) More 569 570 Chapter 15: Query Expressions generally, the addition of the select clause is enabled via query continuation—any query body can be appended to any other query body Listing 15.11: Selecting an Anonymous Type Following the groupby Clause using System; using System.Collections.Generic; using System.Linq; // private static void GroupKeywords1() { var selection = from word in Keywords group word by word.Contains('*') into groups select new { IsContextualKeyword = groups.Key, Items = groups }; foreach (var wordGroup in selection) { Console.WriteLine(Environment.NewLine + "{0}:", wordGroup.IsContextualKeyword ? "Contextual Keywords" : "Keywords"); foreach (var keyword in wordGroup.Items) { Console.Write(" " + keyword.Replace("*", null)); } } } // Following the groupby clause is an into clause that allows us to name each item returned by the groupby clause with a range variable (groups in Listing 15.11) The into clause serves as a generator for additional query commands, specifically a select clause in this case The ability to run additional queries on the results of an existing query is supported via query continuation using into and it is not specific to groupby clauses, but rather is a feature of all query expressions The select clause defines an anonymous type, renaming Key to be IsContextualKeyword and naming the subcollection property Items With Introducing Query Expressions OUTPUT 15.7: Keywords: abstract as base bool break byte case catch char checked class const continue decimal default delegate double else enum event explicit extern false finally fixed float for foreach goto if implicit in int interface internal is lock long namespace new null object operator out override params private protected public readonly ref return sbyte sealed short sizeof stackalloc static string struct switch this throw true try typeof uint ulong unchecked unsafe ushort using virtual void volatile while Contextual Keywords: add ascending by descending from get group into join let orderby partial remove select set value where yield this change, the nested foreach uses wordGroup.Items rather than wordGroup directly, as shown in Listing 15.10 Another potential property to add to the anonymous type would be the count of items within the subcollection However, this is available on wordGroup.Items.Count(), so the benefit of adding it to the anonymous type directly is questionable BEGINNER TOPIC Distinct Members Often, it is desirable to only return distinct items from within a collection— all duplicates are combined into a single item Query expressions don’t have explicit syntax for distinct members, but the functionality is available via the query operator Distinct(), as introduced in the preceding chapter Listing 15.12 demonstrates calling it directly from the query expression, and Output 15.8 shows the results Listing 15.12: Obtaining Distinct Members from a Query Expression using System; using System.Collections.Generic; using System.Linq; // public static void ListMemberNames() { IEnumerable enumerableMethodNames = ( from method in typeof(Enumerable).GetMembers( System.Reflection.BindingFlags.Static | 571 572 Chapter 15: Query Expressions System.Reflection.BindingFlags.Public) select method.Name).Distinct(); foreach(string method in enumerableMethodNames) { Console.Write(" {0},", method); } } // OUTPUT 15.8: Enumerable methods are: First, FirstOrDefault, Last, LastOrDefault, Single, SingleOrDefault, ElementAt, ElementAtOrDefault, Repeat, Empty, Any, All, Count, LongCount, Contains, Aggregate, Sum, Min, Max, Average, Where, Select, SelectMany, Take, TakeWhile, Skip, SkipWhile, Join, GroupJoin, OrderBy, OrderByDescending, ThenBy, ThenByDescending, GroupBy, Concat, Distinct, Union, Intersect, Except, Reverse, SequenceEqual, AsEnumerable, ToArray, ToList, ToDictionary, ToLookup, DefaultIfEmpty, OfType, Cast, Range In this example, typeof(Enumerable).GetMembers() returns a list of all the members (methods, properties, and so on) on System.Linq.Enumerable However, many of these members are overloaded, sometimes more than once Rather than displaying the same member multiple times, Distinct() is called from the query expression This eliminates the duplicate names from the list (I cover the details of typeof() and GetMembers() in Chapter 17.) ADVANCED TOPIC Query Expression Compilation Under the covers, a query expression is a series of method calls to the underlying API The CIL itself does not have any concept of query expressions In fact, except for some corner cases with expression trees, there was no change to the underlying CLR in order to support query expressions Rather, query expressions were supported via changes to the C# compiler only This worked because the compiler translates the query expression to method calls For example, the query expression from Listing 15.1 translates to a call to System.Linq.Enumerable’s Where() extension method and becomes Keywords.Where() The criteria identified by the where Query Expressions as Method Invocations clause are just like they were in the Where() (or FindAll()) method described in the preceding chapter ADVANCED TOPIC Implementing Implicit Execution The capability of saving the selection criteria into selection (see Listing 15.1) rather than executing the query at the time of assignment is implemented through delegates The compiler translates the query expression to methods on the from target (i.e., GetFiles()) that take delegates as parameters Delegates are objects that save information about what code to execute when the delegate is called, and since delegates contain only the data regarding what to execute, they can be stored until a later time when they are executed In the case of collections that implement IQueryable (LINQ providers), the lambda expressions are translated into expression trees An expression tree is a hierarchical data structure broken down recursively into subexpressions Each subexpression represents a portion of the lambda expression that is further broken down until each part is the most fundamental unit that can no longer be broken down Frequently, expression trees are then enumerated and reconstructed as the original expression tree is translated into another language, such as SQL Query Expressions as Method Invocations In spite of the power and relative simplicity associated with query expressions, the CLR and IL not require any query expression implementation Rather, the C# compiler translates query expressions into method calls Consider, for example, the query expression from Listing 15.1, a portion of which appears in Listing 15.13 Listing 15.13: Simple Query Expression private static void ShowContextualKeywords1() { IEnumerable selection = from word in Keywords where word.Contains('*') select word; 573 636 Chapter 17: Reflection and Attributes Given the constructor call, the objects returned from PropertyInfo.GetCustomAttributes() will be initialized with the specified constructor arguments, as demonstrated in Listing 17.14 Listing 17.14: Retrieving a Specific Attribute and Checking Its Initialization PropertyInfo property = typeof(CommandLineInfo).GetProperty("Help"); CommandLineSwitchAliasAttribute attribute = (CommandLineSwitchAliasAttribute) property.GetCustomAttributes( typeof(CommandLineSwitchAliasAttribute), false)[0]; if(attribute.Alias == "?") { Console.WriteLine("Help(?)"); }; Furthermore, as Listing 17.15 and Listing 17.16 demonstrate, you can use similar code in a GetSwitches() method on CommandLineAliasAttribute that returns a dictionary collection of all the switches, including those from the property names, and associate each name with the corresponding attribute on the command-line object Listing 17.15: Retrieving Custom Attribute Instances using System; using System.Reflection; using System.Collections.Generic; public class CommandLineSwitchAliasAttribute : Attribute { public CommandLineSwitchAliasAttribute(string alias) { Alias = alias; } public string Alias { get { return _Alias; } set { _Alias = value; } } private string _Alias; public static Dictionary GetSwitches( object commandLine) { PropertyInfo[] properties = null; Attributes Dictionary options = new Dictionary(); properties = commandLine.GetType().GetProperties( BindingFlags.Public | BindingFlags.NonPublic | BindingFlags.Instance); foreach (PropertyInfo property in properties) { options.Add(property.Name.ToLower(), property); foreach (CommandLineSwitchAliasAttribute attribute in property.GetCustomAttributes( typeof(CommandLineSwitchAliasAttribute), false)) { options.Add(attribute.Alias.ToLower(), property); } } return options; } } Listing 17.16: Updating CommandLineHandler.TryParse() to Handle Aliases using System; using System.Reflection; using System.Collections.Generic; public class CommandLineHandler { // public static bool TryParse( string[] args, object commandLine, out string errorMessage) { bool success = false; errorMessage = null; Dictionary options = CommandLineSwitchAliasAttribute.GetSwitches( commandLine); foreach (string arg in args) { PropertyInfo property; string option; if (arg[0] == '/' || arg[0] == '-') { string[] optionParts = arg.Split( new char[] { ':' }, 2); option = optionParts[0].Remove(0, 1).ToLower(); 637 638 Chapter 17: Reflection and Attributes if (options.TryGetValue(option, out property)) { success = SetOption( commandLine, property, optionParts, ref errorMessage); } else { success = false; errorMessage = string.Format( "Option '{0}' is not supported.", option); } } } return success; } private static bool SetOption( object commandLine, PropertyInfo property, string[] optionParts, ref string errorMessage) { bool success; if (property.PropertyType == typeof(bool)) { // Last parameters for handling indexers property.SetValue( commandLine, true, null); success = true; } else { if ((optionParts.Length < 2) || optionParts[1] == "" || optionParts[1] == ":") { // No setting was provided for the switch success = false; errorMessage = string.Format( "You must specify the value for the {0} option.", property.Name); } else if ( property.PropertyType == typeof(string)) { property.SetValue( commandLine, optionParts[1], null); Attributes success = true; } else if (property.PropertyType.IsEnum) { success = TryParseEnumSwitch( commandLine, optionParts, property, ref errorMessage); } else { success = false; errorMessage = string.Format( "Data type '{0}' on {1} is not supported.", property.PropertyType.ToString(), commandLine.GetType().ToString()); } } return success; } } BEGINNER TOPIC Using Hashtable Rather Than Dictionary Listing 17.15 uses the generic collection Dictionary Unfortunately, this is not available prior to C# 2.0, and instead, you have to use System.Collections.Hashtable This is virtually a searchand-replace substitution, except in the call to TryGetValue(), which is not available on Hashtable In that case, you can retrieve the value using the index operator and then check for null, as follows: if((property = (PropertyInfo)options[option])!=null) { // } System.AttributeUsageAttribute Most attributes are intended to decorate only particular constructs For example, it makes no sense to allow CommandLineOptionAttribute to decorate a class or an assembly Those contexts would be meaningless To avoid inappropriate use of an attribute, custom attributes can be decorated with System.AttributeUsageAttribute Listing 17.17 (for CommandLineOptionAttribute) demonstrates how to this 639 640 Chapter 17: Reflection and Attributes Listing 17.17: Restricting the Constructs an Attribute Can Decorate [AttributeUsage(AttributeTargets.Property)] public class CommandLineSwitchAliasAttribute : Attribute { // } If the attribute is used inappropriately, as it is in Listing 17.18, it will cause a compile-time error, a characteristic unique to predefined attributes, as Output 17.5 demonstrates Listing 17.18: AttributeUsageAttribute Restricting Where to Apply an Attribute // ERROR: The attribute usage is restricted to properties [CommandLineSwitchAlias("?")] class CommandLineInfo { } OUTPUT 17.5: Program+CommandLineInfo.cs(24,17): error CS0592: Attribute ’CommandLineSwitchAlias’ is not valid on this declaration type It is valid on ’property, indexer’ declarations only AttributeUsageAttribute’s constructor takes an AttributesTargets flag This enum provides a list of all the possible targets the runtime allows an attribute to decorate For example, if you also allowed CommandLineSwitchAliasAttribute on a field, you would update the AttributeUsageAttribute application as shown in Listing 17.19 Listing 17.19: Limiting an Attribute’s Usage with AttributeUsageAttribute // Restrict the attribute to properties and methods [AttributeUsage( AttributeTargets.Field | AttributeTargets.Property)] public class CommandLineSwitchAliasAttribute : Attribute { // } Attributes Named Parameters In addition to restricting what an attribute can decorate, AttributeUsageAttribute provides a mechanism for allowing duplicates of the same attribute on a single construct The syntax appears in Listing 17.20 Listing 17.20: Using a Named Parameter [AttributeUsage(AttributeTargets.Property, AllowMultiple=true )] public class CommandLineSwitchAliasAttribute : Attribute { // } The syntax is different from the constructor initialization syntax discussed earlier The AllowMultiple parameter is a named parameter, which is a designation unique to attributes Named parameters provide a mechanism for setting specific public properties and fields within the attribute constructor call, even though the constructor includes no corresponding parameters The named attributes are optional designations, but they provide a means of setting additional instance data on the attribute without providing a constructor parameter for the purpose In this case, AttributeUsageAttribute includes a public member called AllowMultiple Therefore, you can set this member using a named parameter assignment when you use the attribute Assigning named parameters must occur as the last portion of a constructor, following any explicitly declared constructor parameters Named parameters allow for assigning attribute data without providing constructors for every conceivable combination of which attribute properties are specified and which are not Since many of an attribute’s properties may be optional, this is a useful construct in many cases BEGINNER TOPIC FlagsAttribute Chapter introduced enums and included an Advanced Topic in regard to FlagsAttribute This is a framework-defined attribute that targets enums 641 Chapter 17: Reflection and Attributes 642 that represent flag type values Here is similar text as a Beginner Topic, starting with the sample code shown in Listing 17.21 Listing 17.21: Using FlagsAttribute // FileAttributes defined in System.IO [Flags] // Decorating an enum with FlagsAttribute public enum FileAttributes { ReadOnly = 1