Sample residuals versus fitted values plot showing increasing residuals Sample residuals versus fitted values plot that does not show increasing residuals 5.2.4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (6 of 10) [5/1/2006 10:30:22 AM] Interpretation of the residuals versus fitted values plots A residual distribution such as that in Figure 2.6 showing a trend to higher absolute residuals as the value of the response increases suggests that one should transform the response, perhaps by modeling its logarithm or square root, etc., (contractive transformations). Transforming a response in this fashion often simplifies its relationship with a predictor variable and leads to simpler models. Later sections discuss transformation in more detail. Figure 2.7 plots the residuals after a transformation on the response variable was used to reduce the scatter. Notice the difference in scales on the vertical axes. Independence of Residuals from Factor Settings Sample residuals versus factor setting plot 5.2.4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (7 of 10) [5/1/2006 10:30:22 AM] Sample residuals versus factor setting plot after adding a quadratic term 5.2.4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (8 of 10) [5/1/2006 10:30:22 AM] Interpreation of residuals versus factor setting plots Figure 2.8 shows that the size of the residuals changed as a function of a predictor's settings. A graph like this suggests that the model needs a higher-order term in that predictor or that one should transform the predictor using a logarithm or square root, for example. Figure 2.9 shows the residuals for the same response after adding a quadratic term. Notice the single point widely separated from the other residuals in Figure 2.9. This point is an "outlier." That is, its position is well within the range of values used for this predictor in the investigation, but its result was somewhat lower than the model predicted. A signal that curvature is present is a trace resembling a "frown" or a "smile" in these graphs. Sample residuals versus factor setting plot lacking one or more higher-order terms 5.2.4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (9 of 10) [5/1/2006 10:30:22 AM] Interpretation of plot The example given in Figures 2.8 and 2.9 obviously involves five levels of the predictor. The experiment utilized a response surface design. For the simple factorial design that includes center points, if the response model being considered lacked one or more higher-order terms, the plot of residuals versus factor settings might appear as in Figure 2.10. Graph indicates prescence of curvature While the graph gives a definite signal that curvature is present, identifying the source of that curvature is not possible due to the structure of the design. Graphs generated using the other predictors in that situation would have very similar appearances. Additional discussion of residual analysis Note: Residuals are an important subject discussed repeatedly in this Handbook. For example, graphical residual plots using Dataplot are discussed in Chapter 1 and the general examination of residuals as a part of model building is discussed in Chapter 4. 5.2.4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (10 of 10) [5/1/2006 10:30:22 AM] 5. Process Improvement 5.3.Choosing an experimental design Contents of Section 3 This section describes in detail the process of choosing an experimental design to obtain the results you need. The basic designs an engineer needs to know about are described in detail. Note that this section describes the basic designs used for most engineering and scientific applications Set objectives1. Select process variables and levels2. Select experimental design Completely randomized designs1. Randomized block designs Latin squares1. Graeco-Latin squares2. Hyper-Graeco-Latin squares3. 2. Full factorial designs Two-level full factorial designs1. Full factorial example2. Blocking of full factorial designs3. 3. Fractional factorial designs A 2 3-1 half-fraction design1. How to construct a 2 3-1 design2. Confounding3. Design resolution4. Use of fractional factorial designs5. Screening designs6. Fractional factorial designs summary tables7. 4. Plackett-Burman designs5. Response surface (second-order) designs Central composite designs1. 6. 3. 5.3. Choosing an experimental design http://www.itl.nist.gov/div898/handbook/pri/section3/pri3.htm (1 of 2) [5/1/2006 10:30:22 AM] Box-Behnken designs2. Response surface design comparisons3. Blocking a response surface design4. Adding center points7. Improving fractional design resolution Mirror-image foldover designs1. Alternative foldover designs2. 8. Three-level full factorial designs9. Three-level, mixed level and fractional factorial designs10. 5.3. Choosing an experimental design http://www.itl.nist.gov/div898/handbook/pri/section3/pri3.htm (2 of 2) [5/1/2006 10:30:22 AM] 5. Process Improvement 5.3. Choosing an experimental design 5.3.1.What are the objectives? Planning an experiment begins with carefully considering what the objectives (or goals) are The objectives for an experiment are best determined by a team discussion. All of the objectives should be written down, even the "unspoken" ones. The group should discuss which objectives are the key ones, and which ones are "nice but not really necessary". Prioritization of the objectives helps you decide which direction to go with regard to the selection of the factors, responses and the particular design. Sometimes prioritization will force you to start over from scratch when you realize that the experiment you decided to run does not meet one or more critical objectives. Types of designs Examples of goals were given earlier in Section 5.1.2, in which we described four broad categories of experimental designs, with various objectives for each. These were: Comparative designs to: choose between alternatives, with narrow scope, suitable for an initial comparison (see Section 5.3.3.1) ❍ choose between alternatives, with broad scope, suitable for a confirmatory comparison (see Section 5.3.3.2) ❍ ● Screening designs to identify which factors/effects are important when you have 2 - 4 factors and can perform a full factorial (Section 5.3.3.3) ❍ when you have more than 3 factors and want to begin with as small a design as possible (Section 5.3.3.4 and 5.3.3.5) ❍ when you have some qualitative factors, or you have some quantitative factors that are known to have a non-monotonic effect (Section 3.3.3.10) ❍ Note that some authors prefer to restrict the term screening design to the case where you are trying to extract the most important factors from a large (say > 5) list of initial factors (usually a fractional factorial design). We include the case with a smaller ● 5.3.1. What are the objectives? http://www.itl.nist.gov/div898/handbook/pri/section3/pri31.htm (1 of 2) [5/1/2006 10:30:22 AM] number of factors, usually a full factorial design, since the basic purpose and analysis is similar. Response Surface modeling to achieve one or more of the following objectives: hit a target ❍ maximize or minimize a response❍ reduce variation by locating a region where the process is easier to manage ❍ make a process robust (note: this objective may often be accomplished with screening designs rather than with response surface designs - see Section 5.5.6) ❍ ● Regression modeling to estimate a precise model, quantifying the dependence of response variable(s) on process inputs. ❍ ● Based on objective, where to go next After identifying the objective listed above that corresponds most closely to your specific goal, you can proceed to the next section in which we discuss selecting experimental factors ● and then select the appropriate design named in section 5.3.3 that suits your objective (and follow the related links). ● 5.3.1. What are the objectives? http://www.itl.nist.gov/div898/handbook/pri/section3/pri31.htm (2 of 2) [5/1/2006 10:30:22 AM] 5. Process Improvement 5.3. Choosing an experimental design 5.3.2.How do you select and scale the process variables? Guidelines to assist the engineering judgment process of selecting process variables for a DOE Process variables include both inputs and outputs - i.e., factors and responses. The selection of these variables is best done as a team effort. The team should Include all important factors (based on engineering judgment). ● Be bold, but not foolish, in choosing the low and high factor levels.● Check the factor settings for impractical or impossible combinations - i.e., very low pressure and very high gas flows. ● Include all relevant responses.● Avoid using only responses that combine two or more measurements of the process. For example, if interested in selectivity (the ratio of two etch rates), measure both rates, not just the ratio. ● Be careful when choosing the allowable range for each factor We have to choose the range of the settings for input factors, and it is wise to give this some thought beforehand rather than just try extreme values. In some cases, extreme values will give runs that are not feasible; in other cases, extreme ranges might move one out of a smooth area of the response surface into some jagged region, or close to an asymptote. Two-level designs have just a "high" and a "low" setting for each factor The most popular experimental designs are two-level designs. Why only two levels? There are a number of good reasons why two is the most common choice amongst engineers: one reason is that it is ideal for screening designs, simple and economical; it also gives most of the information required to go to a multilevel response surface experiment if one is needed. 5.3.2. How do you select and scale the process variables? http://www.itl.nist.gov/div898/handbook/pri/section3/pri32.htm (1 of 3) [5/1/2006 10:30:22 AM] [...]... response surface method (RSM) designs RSM designs are used to: r Find improved or optimal process settings http://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm (1 of 3) [5/1/2006 10:30:23 AM] 5.3.3 How do you select an experimental design? Troubleshoot process problems and weak points r Make a product or process more robust against external and non-controllable influences "Robust" means relatively... can be added to check for curvature in a 2-level screening design and backup resources are available to redo runs that have processing mishaps http://www.itl.nist.gov/div898/handbook/pri/section3/pri33.htm (3 of 3) [5/1/2006 10:30:23 AM] 5.3.3.1 Completely randomized designs 5 Process Improvement 5.3 Choosing an experimental design 5.3.3 How do you select an experimental design? 5.3.3.1 Completely randomized... measurements after a diffusion process taking place in a furnace They have four different dosages they want to try and enough experimental wafers from the same lot to run three wafers at each of the dosages Furnace run is a nuisance factor The nuisance factor they are concerned with is "furnace run" since it is known that each furnace run differs from the last and impacts many process parameters http://www.itl.nist.gov/div898/handbook/pri/section3/pri332.htm...5.3.2 How do you select and scale the process variables? Consider adding some center points to your two-level design The term "two-level design" is something of a misnomer, however, as it is recommended to include some center points during... described in more detail in the DOE glossary The Model or Analysis Matrix http://www.itl.nist.gov/div898/handbook/pri/section3/pri32.htm (2 of 3) [5/1/2006 10:30:22 AM] 5.3.2 How do you select and scale the process variables? Design matrices If we add an "I" column and an "X1*X2" column to the matrix of 4 trials for a two-factor experiment described earlier, we obtain what is known as the model or analysis... for levels of X1 are shown in the section on one-way ANOVA in Chapter 7 http://www.itl.nist.gov/div898/handbook/pri/section3/pri331.htm (3 of 3) [5/1/2006 10:30:23 AM] 5.3.3.2 Randomized block designs 5 Process Improvement 5.3 Choosing an experimental design 5.3.3 How do you select an experimental design? 5.3.3.2 Randomized block designs Blocking to "remove" the effect of nuisance factors For randomized... X1 and X2: (-1)(-1) + (+1)(-1) + (-1)(+1) + (+1)(+1) = 0 http://www.itl.nist.gov/div898/handbook/pri/section3/pri32.htm (3 of 3) [5/1/2006 10:30:22 AM] 5.3.3 How do you select an experimental design? 5 Process Improvement 5.3 Choosing an experimental design 5.3.3 How do you select an experimental design? A design is selected based on the experimental objective and the number of factors The choice of... average of all the data - = average of all Y for which X2 = j http://www.itl.nist.gov/div898/handbook/pri/section3/pri332.htm (4 of 4) [5/1/2006 10:30:24 AM] 5.3.3.2.1 Latin square and related designs 5 Process Improvement 5.3 Choosing an experimental design 5.3.3 How do you select an experimental design? 5.3.3.2 Randomized block designs 5.3.3.2.1 Latin square and related designs Latin square (and related)... Latin square designs can be found in Box, Hunter, and Hunter (1978) http://www.itl.nist.gov/div898/handbook/pri/section3/pri3321.htm (6 of 6) [5/1/2006 10:30:24 AM] 5.3.3.2.2 Graeco-Latin square designs 5 Process Improvement 5.3 Choosing an experimental design 5.3.3 How do you select an experimental design? 5.3.3.2 Randomized block designs 5.3.3.2.2 Graeco-Latin square designs These designs handle 3 nuisance . objectives? http://www.itl.nist.gov/div898/handbook/pri/section3/pri31.htm (2 of 2) [5/1 /20 06 10:30 :22 AM] 5. Process Improvement 5.3. Choosing an experimental design 5.3 .2. How do you select and scale the process variables? Guidelines to. discussed in Chapter 4. 5 .2. 4. Are the model residuals well-behaved? http://www.itl.nist.gov/div898/handbook/pri/section2/pri24.htm (10 of 10) [5/1 /20 06 10:30 :22 AM] 5. Process Improvement 5.3.Choosing. experimental design http://www.itl.nist.gov/div898/handbook/pri/section3/pri3.htm (2 of 2) [5/1 /20 06 10:30 :22 AM] 5. Process Improvement 5.3. Choosing an experimental design 5.3.1.What are the objectives? Planning