4281book.fm Page 347 Sunday, February 29, 2004 5:12 PM 347 SUMMARY when you are working with delimited files that you open using the VBA Open statement (rather than the OpenText method of the Workbooks object). The syntax of Split is as follows. Split(Expression, [Delimiter], [Limit], [Compare]) The parameters of Split are explained in the following list. Expression Expression is a required parameter. It is the string that you would like to split apart. Expression should be a string consisting of subparts and delimiters such as one line from a delim- ited text file. Delimiter Delimiter is an optional parameter. Set the Delimiter parameter equal to the character used to delimit the string given by Expression. By default, Split assumes that Expression is space delimited. Limit Limit is an optional parameter that can be used to specify how many subparts to return. Compare Compare is an optional parameter that can be used to specify the type of comparison to use when comparing substrings. Use one of the following: vbUseCompareOption, vbUseBina- ryCompare, vbUseTextCompare, or vbUseDatabaseCompare. Summary Working with text files can be more tedious than working with other data transfer mechanisms such as using a database or XML. Nonetheless, working with text files offers some advantages. For one, text files are universally supported. If you need to get data from one application to another, if nothing else, you can export data to text and then import it in the other application. Another advantage of using text files is that Excel can open delimited and fixed-width files without writing any special VBA code. Yet another advantage of using text files is that operations on text files are fast. You can use text files in your application in two ways. One way is to open the text file in Excel, much like you would an Excel workbook. You can open the file manually or you can open it pro- grammatically using the OpenText method of the Workbooks object. The second way to use a text file can only be achieved programmatically—you open the text file using VBA’s Open statement. This method opens the file without actually displaying it. Your code can work with the text file and then close it when finished without the user knowing that it is even being used. VBA provides lots of native abilities for working with strings (all text file data is essentially a string right?). When you use functions like Left, Right, Mid, Split, and the various Trim functions, you will have no problem slicing and dicing text files. In the next chapter, things get a little more interesting as you learn how to retrieve data from data- bases. Combining Excel with the data retrieval capabilities of a database opens up a new world of pos- sibilities in terms of the sophistication, scale, and functionality you can offer with your custom solution. As you’ll see, VBA is the glue that makes it all work. This page intentionally left blank 4281book.fm Page 349 Monday, March 1, 2004 1:19 AM Chapter 16 Dealing with Databases Your skills as a developer reach a new level when you learn how to develop solutions that harness the power of a database to handle an application’s data storage and management chores. By designing solutions that utilize Excel’s analytical and presentation capabilities and the data management capa- bilities of a database, you can create applications that perform faster, support more users, handle vast amounts of data, and enable sophisticated application functionality. I aim to do three things with this chapter. First, I’ll provide an overview of databases in general so that I can establish some common ground. Second, I’ll show you the multiple ways that you can inter- act with a database from Excel without writing a line of code. Finally, I’ll provide the details you need to programmatically interact with a database. As a special treat, I’ll end the chapter by demonstrating how to work with Microsoft Analysis Services. Analysis Services is an online analytical processing (OLAP) product that enables rapid data analysis. Database Basics The simplest definition of a database is that it is one or more sets of persistent, related data. By that definition, text files and even a list in Excel could qualify as a database. However, generally when the term database is used, it also refers to the software used to create and manage the database. This type of software is referred to as database management systems (DBMS). Most databases created using a DBMS are relational databases. A relational database is a database that allows the database developer to create relationships between tables. For example, if you have a salesperson table and an orders table, you can define a relationship between the tables that would associate a salesperson with the orders she generated. Further, the database developer can specify certain rules that should be enforced by the relationship. For example, what would happen if you tried to add a record to the orders table that wasn’t associated with a salesperson? You can create relationships that would either allow or disallow the record from being added. Benefits of a relational database include the following: Relational integrity You can ensure that the data in a relational database conforms to certain business rules (i.e., all orders must be associated with a salesperson). Data integrity A relational database allows you to define rules and data types specific to a partic- ular field in a table. For example, you could specify that a particular field can only contain integers 4281book.fm Page 350 Monday, March 1, 2004 1:19 AM 350 CHAPTER 16 DEALING WITH DATABASES and that it can’t contain a null value. Further, you can define constraints such that the field must be within a certain range of values. Scalability Most database products are designed to handle a great deal more data than Excel can handle. For example, Excel has 65,536 rows. Have you ever used all of them? If so, what happened? The size of the spreadsheet increased dramatically and your computer’s performance probably slowed to a crawl. Even basic database products such as Microsoft Access can handle hundreds of thousands of rows. Unlike with Excel, a database does not necessarily load the entire contents of the database into memory. Performance Databases offer increased data retrieval speed, faster and more comprehensive sorting capabilities, and increased data manipulation performance. Stability Many database products have extensive logging, backup, and transactional features that help ensure the integrity of a database in the event of software or hardware failure. Collaboration Though Excel offers some multiuser capabilities, data in a database can be accessed by many people at once. By storing data in a central location (rather than distributing it among many spreadsheets), your organization gains the benefit of making the data available to many different kinds of applications in addition to Excel. Once you’ve decided to use a database, you’ll find many database products on the market from which to choose. There are not any hard and fast guidelines for choosing a database product. Many factors may weigh into the final decision including these: ◆ Expected database usage ◆ Number of concurrent users ◆ Amount of data to be stored ◆ Skill set of the technical staff ◆ The cost of the software For learning purposes and departmental databases, Microsoft Access is a common choice. Access is an easy-to-use and widely available (it was probably installed along with Excel) database. Access and Excel are tightly integrated, which means that you have more options for moving data between the two applications. For example, you can copy/paste entire worksheets into Access and Access will create a table out of them. Microsoft SQL Server is more of an industrial strength database. You can use SQL Server for every- thing from a personal database using SQL Server Personal Edition, to enterprise databases using SQL Server Standard or Enterprise Edition. SQL Server is not as easy to use as Access, but it offers much more in terms of database development flexibility, database management features, security, and scalability. Of course, you have tons of other choices from other vendors including IBM, Oracle, and Sybase. You’ll even come across open source products such as MySQL. In theory, you’ll be able to write applications that will work with any of these databases. That said, you may encounter slight func- tional differences between products, so the actual mechanics involved may vary somewhat from prod- uct to product. 4281book.fm Page 351 Monday, March 1, 2004 1:19 AM 351 DEVELOPING YOUR SKILLS Note The examples in this chapter use Microsoft Access. The final section of the chapter demonstrates how to retrieve data from Microsoft Analysis Services, a special kind of database product that ships with Microsoft SQL Server. Data in a database is manipulated using Structured Query Language (SQL). Although your effi- ciency will improve as you learn to craft SQL statements by hand, most database products ship with some sort of visual query tool that allows you to build SQL statements (aka queries) visually. Though you can get by using visual query tools to build queries, I feel it is also important to learn how to write them by hand. One of the reasons it is important to learn the syntax of SQL so that you can write queries manually is that, as a developer, you’ll typically need to write SQL statements that you pass to the database via an intermediate mechanism. If you can’t write queries by hand, you have to build them in your visual query tool and then copy/paste them into the VBE. In the process of copying and pasting, chances are you’ll need to do a little rearranging of the query once it is in the VBE. This whole process is rather inefficient. Note The intermediate mechanism that will be presented in this chapter is known as ActiveX Data Objects (ADO). ADO is a set of objects that you can use programmatically to work with a database. Basically, it is an abstraction layer between your code and the database. One of the benefits of an abstraction layer is that it shields you from the necessity of becoming intimately familiar with the particular details of working with different database products. Instead you learn how to use ADO and let your ADO provider worry about handling the details. I’ll go over this in more detail later in the chapter. Another reason it is beneficial to learn how to write queries manually is that visual query tools don’t always write the best SQL for the task at hand. In fact, sometimes you won’t even be able to design a given query using a visual query tool. Armed with a better understanding of writing your own SQL, you can easily get around these query tool shortcomings. Without this understanding, you’re at the mercy of your tool of choice. Developing Your Skills In order to develop the skills necessary to incorporate database functionality into your solutions, you must invest time learning many new things. Some of the areas that you may need to learn or brush up on include the following: ◆ The general use of one or more database products such as Microsoft Access or Microsoft SQL Server ◆ Database design and development techniques. Unless you are using a database that someone else put together, you’ll need to create and populate the tables in a database. ◆ Structured Query Language (SQL) ◆ ActiveX Data Objects (ADO) Entire books have been written on each of the items mentioned, so you can correctly surmise that it is impossible for me to tell you all that you need to know about each topic in one chapter. Hopefully 4281book.fm Page 352 Monday, March 1, 2004 1:19 AM 352 CHAPTER 16 DEALING WITH DATABASES you’ll find enough material in this chapter to help you solve a current problem or get you started in the right direction. Note Check out the informative, developer-oriented book, the Access 2002 Developer’s Handbook Set, by Paul Lit- win, Ken Getz, and Mike Gunderloy (Sybex, 2001), for in-depth coverage of Microsoft Access. Alternatively, Mastering SQL Server 2000, by Mike Gunderloy and Joseph L. Jorden (Sybex, 2000) provides a wide range of information related to Microsoft SQL Server. If you are totally new to working with databases and this all seems overwhelming, it is important that you just start learning and trust that it will become clear. It gets easier. Synergies exist between all of the items I just mentioned. As you learn more about one of them, you’ll find that it gets easier and easier to learn about the others. Native Excel Database Integration Once you put your data in a database, you need an efficient way to get it out. Without writing a single line of code, you can easily incorporate data from a database into an Excel workbook using Microsoft Query (MS Query). MS Query is included with every edition of Microsoft Office. As you can see in Figure 16.1, MS Query is a visual query tool that looks similar to the query design view in Access. Using MS Query, you can define a query that runs and returns data to Excel. A query is basically a question that is phrased in terms that a database can understand. The data that a query returns is referred to as a result set or a recordset. Figure 16.1 Microsoft Query in action 4281book.fm Page 353 Monday, March 1, 2004 1:19 AM 353 NATIVE EXCEL DATABASE INTEGRATION MS Query is a useful, but for some reason, underused application. This may reflect the general user’s lack of understanding about databases. Alternatively, MS Query may turn some people off because of some of its usage quirks. However, if you give MS Query a chance and invest some time learning how to use it, you’ll find that it allows you to do many useful things. Data retrieved using MS Query is associated with an external data range. As you learned in the last chapter, an external data range is a range of data in Excel that is somehow associated with an external data source. You can set up an external data range so that it refreshes itself at specific times to ensure that it always contains the most up-to-date data. In addition, MS Query allows you to harness the power of parameter queries. A parameter query is a query that is set up to prompt for or accept some criteria when the query executes. This allows you to use the same query to return data associated with a specific data item. For example, rather than creating 12 queries where each returns a specific month’s data, you could create a single query that prompts you to enter the month desired. When you create a parameter query in Microsoft Query, you can instruct Excel to retrieve the parameter from a particular cell. Further, you can set up the external data range associated with the data to refresh the data whenever the cell containing the parameter changes. This is powerful stuff. I’ll demonstrate an example of this later in the chapter. Note Microsoft Query is not installed by default. You may require your Microsoft Office setup CD to install MS Query the first time you try and use it. Excel, Meet My Database. Database, This Is Excel To use MS Query, select Data � Import External Data � New Database Query from the Excel menu. If this is the first time you’re querying a particular data source, you need to set up a new data source by choosing New Data Source as shown in Figure 16.2. This displays the Create New Data Source form shown in Figure 16.3. In step one in Figure 16.3, you provide a name for the database. The name you put here is the name that shows up in the list of databases that you can see in Figure 16.2. I like to use the name of the data- base followed by the server or computer on which the database resides. This practice is handy in the development process because you may have two copies of a given database; one for testing and another “production” or live version. Figure 16.2 Selecting a database 4281book.fm Page 354 Monday, March 1, 2004 1:19 AM 354 CHAPTER 16 DEALING WITH DATABASES Figure 16.3 Creating a new data source Figure 16.4 Connecting to an Access database In step two, you select a driver for the database. The choice of a driver is critical because the driver handles all of the communication between MS Query and the database. Database vendors usually provide drivers that are specific to their product. For connecting to an Access database select the Microsoft Access Driver (*.mdb). In step three, you specify where the database resides and any other connection-related details. This step is database specific. If you selected the Access driver, the dialog box shown in Figure 16.4 appears. For an Access database, most of the time all you need to do is click Select and locate the desired data- base on your filesystem. In the following screen shot, I’ve located the Northwind sample database. 4281book.fm Page 355 Monday, March 1, 2004 1:19 AM 355 NATIVE EXCEL DATABASE INTEGRATION Figure 16.5 Choosing a data source Figure 16.6 The Query Wizard leads you through the process of creating elementary queries. Note The Northwind sample database ships with Microsoft Access. It will be used for all Access examples. In step four, you indicate a default table. This is an optional step. If you are sure that you’ll always use the same table, go ahead and select it here; otherwise leave it blank. When you’re finished creating the new data source, it will appear in the list of databases as shown in Figure 16.5. At this point you’re ready to select the desired database and define your query. You Are an Advanced Player See that check box at the bottom of Figure 16.5? The one that says “Use the Query Wizard to create/ edit queries?” If you just need to return data associated with a single table or view in the database, you could use the Query Wizard. The Query Wizard (Figure 16.6) is a tool meant to make it easier to create simple queries. Although Query Wizard does make it easier, it also limits you to the most elementary types of que- ries. I’d encourage you to avoid the Query Wizard and learn how to use MS Query’s normal design view (Figure 16.1). There are five basic steps to creating a query using MS Query. To illustrate the process, I’ll create a query that summarizes sales in the US by an employee in the Northwind database. [...]... particular Cell object Summary Although Excel has some data management capabilities, they are no where near the capabilities of even the most elementary database product The more data you are working with, the more evident Excel s weaknesses become This isn’t a strike against Excel Excel simply wasn’t designed to be a database By learning how to use Excel in conjunction with a database, you’ll be able to... ADODB.Command With cmd ActiveConnection = conn CommandText = sSQL CommandType = adCmdText Execute lRecordsAffected End With ' Clean up Set cmd = Nothing ExitPoint: ActionQuery = lRecordsAffected Exit Function ErrHandler: Debug.Print "ActionQuery error: " & Err.Description Resume ExitPoint End Function 373 374 CHAPTER 16 DEALING WITH DATABASES I Like Treats At this point, I would like to share a treat with you... (millions of records) Analysis Services and Excel are two complimentary products You can use PivotTables to connect to an analysis server without writing any code For programmatic access to an analysis server consider using either ADO or ADOMD, a flavor of ADO intended for Analysis Services Excel s XML capabilities received a huge boost with the release of Excel 2003 As more and more data is either stored... MAGIC WITH ADO Figure 16.14 Use Data Validation on the parameter cell Figure 16.15 MS Query + Parameter Query + Data Validation = Power Work Magic with ADO I feel it is important to learn how to use MS Query I’ve seen numerous VBA applications that were created to retrieve data from a database into Excel that could have been done using MS Query alone There’s no sense in writing custom code to work with. .. Many, if not most, Excel applications are analytical and numerical in nature With Analysis Services, you can analyze an obscene amount of data using Analysis Services to serve the data and Excel to view the data Excel can natively connect to Analysis Services via a PivotTable Figure 16.19 shows an example of Analysis Services data as viewed through a PivotTable Microsoft also has an Excel add-in called... Hierarchy Contains a set of members, each of which has the same rank within a hierarchy Cellset None Embodies the results of a multidimensional query Cell Cellset Represents the data at the intersection of axis coordinates in a cellset Axis (Axes) Cellset Represents a positional or filter axis of a cellset 377 378 CHAPTER 16 DEALING WITH DATABASES Table 16.5: Key ADOMD Objects (continued) Object* Parent... users will need to install the required version of ADO before using your application ADO version 2.1 was distributed with Office 2000 whereas version 2.5 was distributed with Office XP and Office 2003 Figure 16. 17 You will probably have many versions of ADO available on your PC WORK MAGIC WITH ADO ADO has a very flexible object model For many tasks, you could go about completing them mul­ tiple ways For... Figure 16.18 Output of the RecordsetExample procedure 369 370 CHAPTER 16 DEALING WITH DATABASES Listing 16.3: Looping Through a Recordset Sub LoopThroughRecordset(rst As ADODB.Recordset, rg As Range) Dim nColumnOffset As Integer Dim fld As ADODB.Field ' Use With End With on rst to ' save typing & increase performance ' Downside - harder to read With rst ' Loop until we hit the end of the ' recordset Do... all purposes For other purposes, you can use your VBA skills in conjunction with ADO to retrieve Analysis Services data just the way you (or your users) want it From a developer perspec­ tive, data in Analysis Services is much easier to work with when compared to the same data from a traditional database SQL statements can get pretty complex when working with data that your users want to view along different... ActionQuery by adding a new record to the Catagories table in the Northwind database and then editing an existing record 371 372 CHAPTER 16 DEALING WITH DATABASES Table 16.4: Key Command Object Properties and Methods Property/Method Description ActiveConnection property Returns/sets the connection with which the object is associated CommandText property Returns/sets the text of the command to be issued to the . 4281book.fm Page 3 47 Sunday, February 29, 2004 5:12 PM 3 47 SUMMARY when you are working with delimited files that you open using the VBA Open statement (rather than the OpenText. the text file using VBA s Open statement. This method opens the file without actually displaying it. Your code can work with the text file and then close it when finished without the user knowing. application. ADO version 2.1 was distributed with Office 2000 whereas version 2.5 was distributed with Office XP and Office 2003. Figure 16. 17 You will probably have many versions of ADO