treatments. Table of treatments for the 3 3 design These treatments may be displayed as follows: TABLE 3.37 The 3 3 Design Factor A Factor B Factor C 0 1 2 0 0 000 100 200 0 1 001 101 201 0 2 002 102 202 1 0 010 110 210 1 1 011 111 211 1 2 012 112 212 2 0 020 120 220 2 1 021 121 221 2 2 022 122 222 Pictorial representation of the 3 3 design The design can be represented pictorially by FIGURE 3.24 A 3 3 Design Schematic 5.3.3.9. Three-level full factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri339.htm (3 of 4) [5/1/2006 10:30:44 AM] Two types of 3 k designs Two types of fractions of 3 k designs are employed: Box-Behnken designs whose purpose is to estimate a second-order model for quantitative factors (discussed earlier in section 5.3.3.6.2) ● 3 k-p orthogonal arrays.● 5.3.3.9. Three-level full factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri339.htm (4 of 4) [5/1/2006 10:30:44 AM] 5. Process Improvement 5.3. Choosing an experimental design 5.3.3. How do you select an experimental design? 5.3.3.10.Three-level, mixed-level and fractional factorial designs Mixed level designs have some factors with, say, 2 levels, and some with 3 levels or 4 levels The 2 k and 3 k experiments are special cases of factorial designs. In a factorial design, one obtains data at every combination of the levels. The importance of factorial designs, especially 2-level factorial designs, was stated by Montgomery (1991): It is our belief that the two-level factorial and fractional factorial designs should be the cornerstone of industrial experimentation for product and process development and improvement. He went on to say: There are, however, some situations in which it is necessary to include a factor (or a few factors) that have more than two levels. This section will look at how to add three-level factors starting with two-level designs, obtaining what is called a mixed-level design. We will also look at how to add a four-level factor to a two-level design. The section will conclude with a listing of some useful orthogonal three-level and mixed-level designs (a few of the so-called Taguchi "L" orthogonal array designs), and a brief discussion of their benefits and disadvantages. Generating a Mixed Three-Level and Two-Level Design Montgomery scheme for generating a mixed design Montgomery (1991) suggests how to derive a variable at three levels from a 2 3 design, using a rather ingenious scheme. The objective is to generate a design for one variable, A, at 2 levels and another, X, at three levels. This will be formed by combining the -1 and 1 patterns for the B and C factors to form the levels of the three-level factor X: TABLE 3.38 Generating a Mixed Design Two-Level Three-Level B C X -1 -1 x 1 +1 -1 x 2 -1 +1 x 2 +1 +1 x 3 Similar to the 3 k case, we observe that X has 2 degrees of freedom, which can be broken out into a linear and a quadratic component. To illustrate how the 2 3 design leads to the design with one factor at two levels and one factor at three levels, consider the following table, with particular attention focused on the column labels. 5.3.3.10. Three-level, mixed-level and fractional factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri33a.htm (1 of 5) [5/1/2006 10:30:45 AM] Table illustrating the generation of a design with one factor at 2 levels and another at 3 levels from a 2 3 design A X L X L AX L AX L X Q AX Q TRT MNT Run A B C AB AC BC ABC A X 1 -1 -1 -1 +1 +1 +1 -1 Low Low 2 +1 -1 -1 -1 -1 +1 +1 High Low 3 -1 +1 -1 -1 +1 -1 +1 Low Medium 4 +1 +1 -1 +1 -1 -1 -1 High Medium 5 -1 -1 +1 +1 -1 -1 +1 Low Medium 6 +1 -1 +1 -1 +1 -1 -1 High Medium 7 -1 +1 +1 -1 -1 +1 -1 Low High If quadratic effect negligble, we may include a second two-level factor If we believe that the quadratic effect is negligible, we may include a second two-level factor, D, with D = ABC. In fact, we can convert the design to exclusively a main effect (resolution III) situation consisting of four two-level factors and one three-level factor. This is accomplished by equating the second two-level factor to AB, the third to AC and the fourth to ABC. Column BC cannot be used in this manner because it contains the quadratic effect of the three-level factor X. More than one three-level factor 3-Level factors from 2 4 and 2 5 designs We have seen that in order to create one three-level factor, the starting design can be a 2 3 factorial. Without proof we state that a 2 4 can split off 1, 2 or 3 three-level factors; a 2 5 is able to generate 3 three-level factors and still maintain a full factorial structure. For more on this, see Montgomery (1991). Generating a Two- and Four-Level Mixed Design Constructing a design with one 4-level factor and two 2-level factors We may use the same principles as for the three-level factor example in creating a four-level factor. We will assume that the goal is to construct a design with one four-level and two two-level factors. Initially we wish to estimate all main effects and interactions. It has been shown (see Montgomery, 1991) that this can be accomplished via a 2 4 (16 runs) design, with columns A and B used to create the four level factor X. Table showing design with 4-level, two 2-level factors in 16 runs TABLE 3.39 A Single Four-level Factor and Two Two-level Factors in 16 runs Run (A B) = X C D 1 -1 -1 x 1 -1 -1 2 +1 -1 x 2 -1 -1 3 -1 +1 x 3 -1 -1 4 +1 +1 x 4 -1 -1 5 -1 -1 x 1 +1 -1 6 +1 -1 x 2 +1 -1 7 -1 +1 x 3 +1 -1 8 +1 +1 x 4 +1 -1 5.3.3.10. Three-level, mixed-level and fractional factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri33a.htm (2 of 5) [5/1/2006 10:30:45 AM] 9 -1 -1 x 1 -1 +1 10 +1 -1 x 2 -1 +1 11 -1 +1 x 3 -1 +1 12 +1 +1 x 4 -1 +1 13 -1 -1 x 1 +1 +1 14 +1 -1 x 2 +1 +1 15 -1 +1 x 3 +1 +1 16 +1 +1 x 4 +1 +1 Some Useful (Taguchi) Orthogonal "L" Array Designs L 9 design L 9 - A 3 4-2 Fractional Factorial Design 4 Factors at Three Levels (9 runs) Run X1 X2 X3 X4 1 1 1 1 1 2 1 2 2 2 3 1 3 3 3 4 2 1 2 3 5 2 2 3 1 6 2 3 1 2 7 3 1 3 2 8 3 2 1 3 9 3 3 2 1 L 18 design L 18 - A 2 x 3 7-5 Fractional Factorial (Mixed-Level) Design 1 Factor at Two Levels and Seven Factors at 3 Levels (18 Runs) Run X1 X2 X3 X4 X5 X6 X7 X8 1 1 1 1 1 1 1 1 1 2 1 1 2 2 2 2 2 2 3 1 1 3 3 3 3 3 3 4 1 2 1 1 2 2 3 3 5 1 2 2 2 3 3 1 1 6 1 2 3 3 1 1 2 2 7 1 3 1 2 1 3 2 3 8 1 3 2 3 2 1 3 1 9 1 3 3 1 3 2 1 2 10 2 1 1 3 3 2 2 1 11 2 1 2 1 1 3 3 2 12 2 1 3 2 2 1 1 3 13 2 2 1 2 3 1 3 2 14 2 2 2 3 1 2 1 3 15 2 2 3 1 2 3 2 1 16 2 3 1 3 2 3 1 2 17 2 3 2 1 3 1 2 3 18 2 3 3 2 1 2 3 1 5.3.3.10. Three-level, mixed-level and fractional factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri33a.htm (3 of 5) [5/1/2006 10:30:45 AM] L 27 design L 27 - A 3 13-10 Fractional Factorial Design Thirteen Factors at Three Levels (27 Runs) Run X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 2 2 2 2 2 2 2 2 2 3 1 1 1 1 3 3 3 3 3 3 3 3 3 4 1 2 2 2 1 1 1 2 2 2 3 3 3 5 1 2 2 2 2 2 2 3 3 3 1 1 1 6 1 2 2 2 3 3 3 1 1 1 2 2 2 7 1 3 3 3 1 1 1 3 3 3 2 2 2 8 1 3 3 3 2 2 2 1 1 1 3 3 3 9 1 3 3 3 3 3 3 2 2 2 1 1 1 10 2 1 2 3 1 2 3 1 2 3 1 2 3 11 2 1 2 3 2 3 1 2 3 1 2 3 1 12 2 1 2 3 3 1 2 3 1 2 3 1 2 13 2 2 3 1 1 2 3 2 3 1 3 1 2 14 2 2 3 1 2 3 1 3 1 2 1 2 3 15 2 2 3 1 3 1 2 1 2 3 2 3 1 16 2 3 1 2 1 2 3 3 1 2 2 3 1 17 2 3 1 2 2 3 1 1 2 3 3 1 2 18 2 3 1 2 3 1 2 2 3 1 1 2 3 19 3 1 3 2 1 3 2 1 3 2 1 3 2 20 3 1 3 2 2 1 3 2 1 3 2 1 3 21 3 1 3 2 3 2 1 3 2 1 3 2 1 22 3 2 1 3 1 3 2 2 1 3 3 2 1 23 3 2 1 3 2 1 3 3 2 1 1 3 2 24 3 2 1 3 3 2 1 1 3 2 2 1 3 25 3 3 2 1 1 3 2 3 2 1 2 1 3 26 3 3 2 1 2 1 3 1 3 2 3 2 1 27 3 3 2 1 3 2 1 2 1 3 1 3 2 L 36 design L36 - A Fractional Factorial (Mixed-Level) Design Eleven Factors at Two Levels and Twelve Factors at 3 Levels (36 Runs) Run X1 X2 X3 X4 X5 X6 X7 X8 X9 X10 X11 X12 X13 X14 X15 X16 X17 X18 X19 X20 X21 X22 X23 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 1 2 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 3 1 1 1 1 1 1 1 1 1 1 1 3 3 3 3 3 3 3 3 3 3 3 3 4 1 1 1 1 1 2 2 2 2 2 2 1 1 1 1 2 2 2 2 3 3 3 3 5 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 3 3 3 3 1 1 1 1 6 1 1 1 1 1 2 2 2 2 2 2 3 3 3 3 1 1 1 1 2 2 2 2 7 1 1 2 2 2 1 1 1 2 2 2 1 1 2 3 1 2 3 3 1 2 2 3 8 1 1 2 2 2 1 1 1 2 2 2 2 2 3 1 2 3 1 1 2 3 3 1 9 1 1 2 2 2 1 1 1 2 2 2 3 3 1 2 3 1 2 2 3 1 1 2 10 1 2 1 2 2 1 2 2 1 1 2 1 1 3 2 1 3 2 3 2 1 3 2 11 1 2 1 2 2 1 2 2 1 1 2 2 2 1 3 2 1 3 1 3 2 1 3 12 1 2 1 2 2 1 2 2 1 1 2 3 3 2 1 3 2 1 2 1 3 2 1 5.3.3.10. Three-level, mixed-level and fractional factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri33a.htm (4 of 5) [5/1/2006 10:30:45 AM] 13 1 2 2 1 2 2 1 2 1 2 1 1 2 3 1 3 2 1 3 3 2 1 2 14 1 2 2 1 2 2 1 2 1 2 1 2 3 1 2 1 3 2 1 1 3 2 3 15 1 2 2 1 2 2 1 2 1 2 1 3 1 2 3 2 1 3 2 2 1 3 1 16 1 2 2 2 1 2 2 1 2 1 1 1 2 3 2 1 1 3 2 3 3 2 1 17 1 2 2 2 1 2 2 1 2 1 1 2 3 1 3 2 2 1 3 1 1 3 2 18 1 2 2 2 1 2 2 1 2 1 1 3 1 2 1 3 3 2 1 2 2 1 3 19 2 1 2 2 1 1 2 2 1 2 1 1 2 1 3 3 3 1 2 2 1 2 3 20 2 1 2 2 1 1 2 2 1 2 1 2 3 2 1 1 1 2 3 3 2 3 1 21 2 1 2 2 1 1 2 2 1 2 1 3 1 3 2 2 2 3 1 1 3 1 2 22 2 1 2 1 2 2 2 1 1 1 2 1 2 2 3 3 1 2 1 1 3 3 2 23 2 1 2 1 2 2 2 1 1 1 2 2 3 3 1 1 2 3 2 2 1 1 3 24 2 1 2 1 2 2 2 1 1 1 2 3 1 1 2 2 3 1 3 3 2 2 1 25 2 1 1 2 2 2 1 2 2 1 1 1 3 2 1 2 3 3 1 3 1 2 2 26 2 1 1 2 2 2 1 2 2 1 1 2 1 3 2 3 1 1 2 1 2 3 3 27 2 1 1 2 2 2 1 2 2 1 1 3 2 1 3 1 2 2 3 2 3 1 1 28 2 2 2 1 1 1 1 2 2 1 2 1 3 2 2 2 1 1 3 2 3 1 3 29 2 2 2 1 1 1 1 2 2 1 2 2 1 3 3 3 2 2 1 3 1 2 1 30 2 2 2 1 1 1 1 2 2 1 2 3 2 1 1 1 3 3 2 1 2 3 2 31 2 2 1 2 1 2 1 1 1 2 2 1 3 3 3 2 3 2 2 1 2 1 1 32 2 2 1 2 1 2 1 1 1 2 2 2 1 1 1 3 1 3 3 2 3 2 2 33 2 2 1 2 1 2 1 1 1 2 2 3 2 2 1 2 1 1 3 1 1 3 3 34 2 2 1 1 2 1 2 1 2 2 1 1 3 1 2 3 2 3 1 2 2 3 1 35 2 2 1 1 2 1 2 1 2 2 1 2 1 2 3 1 3 1 2 3 3 1 2 36 2 2 1 1 2 1 2 1 2 2 1 3 2 3 1 2 1 2 3 1 1 2 3 Advantages and Disadvantages of Three-Level and Mixed-Level "L" Designs Advantages and disadvantages of three-level mixed-level designs The good features of these designs are: They are orthogonal arrays. Some analysts believe this simplifies the analysis and interpretation of results while other analysts believe it does not. ● They obtain a lot of information about the main effects in a relatively few number of runs. ● You can test whether non-linear terms are needed in the model, at least as far as the three-level factors are concerned. ● On the other hand, there are several undesirable features of these designs to consider: They provide limited information about interactions. ● They require more runs than a comparable 2 k-p design, and a two-level design will often suffice when the factors are continuous and monotonic (many three-level designs are used when two-level designs would have been adequate). ● 5.3.3.10. Three-level, mixed-level and fractional factorial designs http://www.itl.nist.gov/div898/handbook/pri/section3/pri33a.htm (5 of 5) [5/1/2006 10:30:45 AM] 5. Process Improvement 5.4.Analysis of DOE data Contents of this section Assuming you have a starting model that you want to fit to your experimental data and the experiment was designed correctly for your objective, most DOE software packages will analyze your DOE data. This section will illustrate how to analyze DOE's by first going over the generic basic steps and then showing software examples. The contents of the section are: DOE analysis steps● Plotting DOE data● Modeling DOE data● Testing and revising DOE models● Interpreting DOE results● Confirming DOE results● DOE examples Full factorial example❍ Fractional factorial example❍ Response surface example❍ ● Prerequisite statistical tools and concepts needed for DOE analyses The examples in this section assume the reader is familiar with the concepts of ANOVA tables (see Chapter 3 or Chapter 7)● p-values● Residual analysis● Model Lack of Fit tests● Data transformations for normality and linearity● 5.4. Analysis of DOE data http://www.itl.nist.gov/div898/handbook/pri/section4/pri4.htm [5/1/2006 10:30:45 AM] 5. Process Improvement 5.4. Analysis of DOE data 5.4.1.What are the steps in a DOE analysis? General flowchart for analyzing DOE data Flowchart of DOE Analysis Steps DOE Analysis Steps Analysis steps: graphics, theoretical model, actual model, validate model, use model The following are the basic steps in a DOE analysis. Look at the data. Examine it for outliers, typos and obvious problems. Construct as many graphs as you can to get the big picture. Response distributions (histograms, box plots, etc.)❍ Responses versus time order scatter plot (a check for possible time effects)❍ Responses versus factor levels (first look at magnitude of factor effects)❍ Typical DOE plots (which assume standard models for effects and errors) Main effects mean plots■ Block plots■ Normal or half-normal plots of the effects■ ❍ 1. 5.4.1. What are the steps in a DOE analysis? http://www.itl.nist.gov/div898/handbook/pri/section4/pri41.htm (1 of 2) [5/1/2006 10:30:46 AM] Interaction plots■ Sometimes the right graphs and plots of the data lead to obvious answers for your experimental objective questions and you can skip to step 5. In most cases, however, you will want to continue by fitting and validating a model that can be used to answer your questions. ❍ Create the theoretical model (the experiment should have been designed with this model in mind!). 2. Create a model from the data. Simplify the model, if possible, using stepwise regression methods and/or parameter p-value significance information. 3. Test the model assumptions using residual graphs. If none of the model assumptions were violated, examine the ANOVA. Simplify the model further, if appropriate. If reduction is appropriate, then return to step 3 with a new model. ■ ❍ If model assumptions were violated, try to find a cause. Are necessary terms missing from the model? ■ Will a transformation of the response help? If a transformation is used, return to step 3 with a new model. ■ ❍ 4. Use the results to answer the questions in your experimental objectives finding important factors, finding optimum settings, etc. 5. Flowchart is a guideline, not a hard-and -fast rule Note: The above flowchart and sequence of steps should not be regarded as a "hard-and-fast rule" for analyzing all DOE's. Different analysts may prefer a different sequence of steps and not all types of experiments can be analyzed with one set procedure. There still remains some art in both the design and the analysis of experiments, which can only be learned from experience. In addition, the role of engineering judgment should not be underestimated. 5.4.1. What are the steps in a DOE analysis? http://www.itl.nist.gov/div898/handbook/pri/section4/pri41.htm (2 of 2) [5/1/2006 10:30:46 AM] [...]... (confirmatory runs) 5 Process Improvement 5.4 Analysis of DOE data 5.4.6 How to confirm DOE results (confirmatory runs) Definition of confirmation runs When the analysis of the experiment is complete, one must verify that the predictions are good These are called confirmation runs At least 3 confirmation runs should be planned In an industrial setting, it is very desirable to have a stable process Therefore,... http://www.itl.nist.gov/div898/handbook/pri/section4/pri42.htm (2 of 3) [5/1/2006 10:30:46 AM] 5.4.2 How to "look" at DOE data http://www.itl.nist.gov/div898/handbook/pri/section4/pri42.htm (3 of 3) [5/1/2006 10:30:46 AM] 5.4.3 How to model DOE data 5 Process Improvement 5.4 Analysis of DOE data 5.4.3 How to model DOE data DOE models should be consistent with the goal of the experiment In general, the trial model that will be fit to DOE data should... tests of model fit are used to confirm or adjust models, as needed http://www.itl.nist.gov/div898/handbook/pri/section4/pri43.htm (2 of 2) [5/1/2006 10:30:46 AM] 5.4.4 How to test and revise DOE models 5 Process Improvement 5.4 Analysis of DOE data 5.4.4 How to test and revise DOE models Tools for testing, revising, and selecting models All the tools and procedures for testing, revising and selecting final... and fractional factorial experiments) Youden plots r Other methods r http://www.itl.nist.gov/div898/handbook/pri/section4/pri44.htm (2 of 2) [5/1/2006 10:30:47 AM] 5.4.5 How to interpret DOE results 5 Process Improvement 5.4 Analysis of DOE data 5.4.5 How to interpret DOE results Final model used to make conclusions and decisions Assume that you have a final model that has passed all the relevant tests...5.4.2 How to "look" at DOE data 5 Process Improvement 5.4 Analysis of DOE data 5.4.2 How to "look" at DOE data The importance of looking at the data with a wide array of plots or visual displays cannot be over-stressed Plots for viewing... why it makes sense to plan to experiment sequentially in order to meet the goals http://www.itl.nist.gov/div898/handbook/pri/section4/pri46.htm (2 of 2) [5/1/2006 10:30:47 AM] 5.4.7 Examples of DOE's 5 Process Improvement 5.4 Analysis of DOE data 5.4.7 Examples of DOE's Software packages do the calculations and plot the graphs for a DOE analysis: the analyst has to know what to request and how to interpret... factorial experiment 2 A fractional factorial experiment 3 A response surface experiment http://www.itl.nist.gov/div898/handbook/pri/section4/pri47.htm [5/1/2006 10:30:47 AM] 5.4.7.1 Full factorial example 5 Process Improvement 5.4 Analysis of DOE data 5.4.7 Examples of DOE's 5.4.7.1 Full factorial example Data Source This example uses data from a NIST high performance ceramics experiment This data set was . of 3) [5/1/20 06 10:30: 46 AM] 5.4.2. How to "look" at DOE data http://www.itl.nist.gov/div898/handbook/pri/section4/pri42.htm (3 of 3) [5/1/20 06 10:30: 46 AM] 5. Process Improvement 5.4 3 26 3 3 2 1 2 1 3 1 3 2 3 2 1 27 3 3 2 1 3 2 1 2 1 3 1 3 2 L 36 design L 36 - A Fractional Factorial (Mixed-Level) Design Eleven Factors at Two Levels and Twelve Factors at 3 Levels ( 36 Runs) Run. analysis? http://www.itl.nist.gov/div898/handbook/pri/section4/pri41.htm (2 of 2) [5/1/20 06 10:30: 46 AM] 5. Process Improvement 5.4. Analysis of DOE data 5.4.2.How to "look" at DOE data The importance of