3.3 Analytic Development of Reliability and Performance in Engineering Design 193 where: a j and x j = regression parameters and covariates, β and μ = the shape and scale parameters. It is often more convenientto define an additionalcovariate,x o = 1, in order to allow the Weibull scale parameter to be included in the vector of regression coefficients, and the proportional hazards model expressed solely by the beta (shape parameter), together with the regression parameters and covariates. The PH failure rate can then be written as λ (t,X)= β (t) β −1 exp  m ∑ j= 0 a j x j  . (3.149) The PH reliability function is thus given by the expression R(t,X )=exp ⎡ ⎣ − t  0 λ (u)du ⎤ ⎦ R(t,X )=exp ⎡ ⎣ − t  0 λ (u,X)du ⎤ ⎦ R(t,X )=exp  −t β ·exp  m ∑ j= 0 a j x j  (3.150) The prob a bility density function (p.d.f.) can be obtained by taking the partial deriva- tive with respect to time of the reliability function given by Eq. (3.150). The PH probability density function is given by the expression f(t,X )= λ (t,X)R(t,X).The total number of unknownsto solvein this model is m+2(i.e. β , μ ,a 1 ,a 2 ,a 3 , ,a m ). The maximum likelihood estimation method can be used to determine these pa- rameters. Solving for the parameters that maximise the maximum likelihood esti- mation will yield the parameters for the PH Weibull model. For β = 1, the equation then becomes the likelihood function for the PH exponentialmodel, which is similar to the original form of the proportional hazards model proposed by Cox (1972). c) Maximum Likelihood Estimation (MLE) Parameter Estimation The idea b ehind maximum likelihood parameter estimation is to determine the pa- rameters that maximise the probability (likelihood) of the sample d ata. From a sta- tistical point of view, the method of maximum likelihood is considered to be more robust (with some exceptions) and yields estimators with good statistical proper- ties. In other words, MLE methods are versatile and apply to most models and to different types of data. In addition, they provide efficient methods for quantifying uncertainty through confidence bounds. Although the methodology for maximum likelihood estimation is simple, the implementation is mathematically complex. By utilising computerised models, however, the mathematical complexity o f MLE is not an obstacle. 194 3 Reliability and Performance in Engineering Design Asymptotic behaviour In many cases, estimation is performed using a set of in- dependent, identically distributed measurements. In such cases, it is of interest to determine the behaviour of a given estimator as the set of measurements increases to infinity, referred to as asymptotic behaviour. Under certain conditions, the MLE exhibits several characteristics that can be interpreted to mean it is ‘asymptotically optimal’. While these asymptotic properties become strictly true only in the limit of infinite sample size, in practice they are often assumed to be approximately true, especially with a large sample size. In particular, inference about the estimated pa- rameters is often based on the asympto tic Gaussian distribution of the MLE. As MLE can generally be applied to failure-related sample data that are available for critical components during the detail design phase of the engineering design process, it is necessary to examine more closely the theory that underlies maximum likelihood estimation for the quantification of complete data. Alternately, when no data are available, the method of qualitative parameter estimation becomes essen- tial, as co nsidered in detail later in Sectio n 3.3.3.3. Background theory If x is a continuous random variable with probability density function: f(x; θ 1 , θ 2 , θ 3 , , θ k ) , where: θ 1 , θ 2 , θ 3 , , θ k are k unknownand constant parameters that needto be estimated through n independent observations, x 1 ,x 2 ,x 3 , ,x n . Then, the likelihood function is given by the following expression L(x 1 ,x 2 ,x 3 , ,x n )= n ∏ i=1 f(x i ; θ 1 , θ 2 , θ 3 , , θ k ) i = 1,2,3, ,n . (3.151) The logarithmic likelihood function is given by Λ = lnL = n ∑ i=1 ln f(xi; θ 1 , θ 2 , θ 3 , , θ k ) . (3.152) The maximum likelihood estimators (MLE) of θ 1 , θ 2 , θ 3 , , θ k are obtained b y maximising Λ . By maximising Λ , which is much easier to work with than L,the maximum likelihood estimators (MLE) of the range θ 1 , θ 2 , θ 3 , , θ k are the simul- taneous solutions of k equations where the partial derivatives of Λ are equal to zero: ∂ ( Λ ) ∂θ j = 0 j = 1,2,3, ,k . Even though it is common practice to plot the MLE solutions using median ranks (points are plotted according to median ranks and the line according to the MLE so- lutions), this method is not completely accurate. As can be seen from the equations above,the MLE method is independentof any kind ofranks or plotting methods. For this reason, the MLE solution appears many times not to track the data on a prob- 3.3 Analytic Development of Reliability and Performance in Engineering Design 195 ability plot. This is p erfectly acceptable, since the two methods are independent of each other. Illustrating the MLE Method Using the Exponential Distribution: To estimate λ , for a sample of n units (all tested to failure), the likelihood function is obtained L( λ |t 1 ,t 2 ,t 3 , ,t n )= n ∏ i=1 f(t i ) = n ∏ i=1 λ e − λ t i = λ n e − λ ∑ t i (3.153) Taking the natural log of both sides Λ = ln(L)=nln( λ ) − λ n ∑ i=1 t i ∂ ( Λ ) ∂λ = n λ − n ∑ i=1 t i = 0 Solving for λ gives: λ = n/ n ∑ i=1 t i . (3.154) Notes on Lambda The value of λ is an estimate because, if another sample from the same popula- tion is obtained and λ re-estimated, then the new value would differ from the one previously calculated. How close is the value of the estimate to the true value? To answer this ques- tion, one must first determine the distribution of the parameter λ . This methodology introduces another term, the confidence level, which allows for the specification of a range for the estimate with a certain confidence level. The treatment of confidence intervals is integral to reliability engineer ing, and to statistics in general. Illustrating the MLE Method Using the Normal Distribution To obtain the MLE estimates for the mean, T , and standard deviation, σ T ,forthe normal distribution, the probability density function of the normal distribution is 196 3 Reliability and Performance in Engineering Design given by F(T)= 1 σ T √ 2 π exp  − 1 2 (T −T) 2 σ T  , (3.155) where: T = mean of the normal distribution, σ T = standard deviation of the normal distribution. If T 1 ,T 2 ,T 3 , ,T n are known times to failure (and with no suspensions), then the likelihood function is given by L(T 1 ,T 2 ,T 3 , ,T n |T, σ T ) : L = n ∏ i=1  1 σ T √ 2 π exp  − 1 2 (T −T) 2 σ T  L = 1  σ T √ 2 π  n exp  − 1 2 n ∑ i=1 (T i −T) 2 σ T  (3.156) Λ = ln(L): ln(L)=− n 2 ln(2 π ) −nln σ T − 1 2 n ∑ i=1 (T i −T) 2 σ T Then, taking the partial derivatives of Λ with respect to each one of the parameters, and setting th ese equal to zero yields: ∂ ( Λ ) ∂ T = 1 σ 2 T − n ∑ i=1 (T i −T)=0 and: ∂ ( Λ ) ∂σ T = n σ T + 1 σ 3 T n ∑ i=1 (T i −T) 2 = 0. Solving these equations simultaneously yields T = 1 n n ∑ i=1 T i (3.157) σ 2 T = 1 n n ∑ i=1 (T i −T) 2 (3.158) These solutions are valid only for data with no suspensions, i.e. all units are tested to failure. In cases in which suspensions are present, the methodology changes and the problem becomes much more complicated. Estimator As indicated, the parameters obtained from maximising the likelihood function are estimators of the true value. It is clear that the sample size determines the accuracy of an estimator. If the sample size equals the whole population,then the 3.3 Analytic Development of Reliability and Performance in Engineering Design 197 estimator is the true value. Estimators have properties such as non-bias and consis- tency (as well as properties of sufficiency and efficiency, which are not considered here). Unbiased estimator An estimator given by the relationship θ = d(x 1 ,x 2 ,x 3 , ,x n ) is considered to be unbiased if and only if the estimator satisfies the condition E( θ )= θ for all θ . In this case, E(x) denotes the expected value of x and is de- fined by the following expression for continuous distributions E(x)= ψ  xf(x)dxx∈ ψ . (3.159) This implies that the true value is not consistently underestimatednor overestimated. Consistent estimator An unbiased estimator that converges more closely to the true value as the sample size increases is called a consistent estimator. The standard deviation of the normal distribution was obtained using MLE. However, this estima- tor of the true standard deviation is a biased one. It can be shown that the consistent estimate of the variance and standard deviation for complete data (for the normal distribution) is given by σ 2 T = 1 n−1 n ∑ i=1 (T i −T) 2 . (3.160) Analysis of censored data So far, parameter estimation has been considered for complete data only. Further expansion on the maximum likelihood parameter esti- mation method needs to include estimating parameters with right censored data. The method is based on the same principles covered previously, but modified to take into account the fact that some of the data are censored. MLE analysis of right censored data The maximum likelihood method is by far the most appropriate analysis method for censored data. When performing maxi- mum likelihood analysis, the likelihood function needs to be expanded to take into account the suspended items. A great advantage of using MLE when dealing with censored data is that each suspension term is included in the likelihood function. Thus, the estimates of the parameters are obtained from consideration of the entire sample population of tested components. Using MLE properties, confidence bounds can be obtained that also account for all the suspension terms. In the case of sus- pensions, and where x is a continuous random variable with p.d.f. and c.d.f. of the following forms f(x; θ 1 , θ 2 , θ 3 , , θ k ) F(x; θ 1 , θ 2 , θ 3 , , θ k ) θ 1 , θ 2 , θ 3 , , θ k are the k unknown parameters that need to be estimated from R failures at (T 1 ,V T 1 ),(T 2 ,V T 2 ),(T 3 ,V T 3 ), ,(T R ,V T R ), and from M suspensions at (S 1 ,V S 1 ),(S 2 ,V S 2 ),(S 3 ,V S 3 ), ,(S M ,V S M ),whereV T R is the Rth stress level corre- sponding to the Rth observed failure, and V S M the Mth stress level corresponding to the Mth observed suspension. 198 3 Reliability and Performance in Engineering Design The likelihood function is then formulated, and the parameters solved by max- imising L((T 1 ,V T 1 ), ,(T R ,V T R ),(S 1 ,V S 1 ), ,(S M ,V S M )| θ 1 , θ 2 , θ 3 , , θ k )= R ∏ i=1 f(T i ,V T i ; θ 1 , θ 2 , θ 3 , , θ k ) M ∏ j= 1  1−F(S j ,V S j ; θ 1 , θ 2 , θ 3 , , θ k )  . (3.161) 3.3.3.2 Expansion of the Exponential Failure Distribution Estimating failure rate As indicated previously in Section 3.2.3.2, the exponen- tial distribution is a very commonly used d istribution in reliability engineering. Due to its simplicity, it has been widely employed in designing for reliability.Theex- ponential distribution describes components with a single parameter, the constant failure rate. The single-parameter exponential probability density function is given by f(T)= λ e − λ T =(1/MTBF)e −T/MTBF . (3.162) This distribution requires the estimation of only one parameter, λ , for its application in designing for reliability, where: λ = constant failure rate, λ > 0, λ = 1/MTBF, MTBF = mean time between failures, or to a failure, MTBF > 0, T = operating time, life or age, in hours, cycles, etc. T ≥ 0. There are several m ethods for estimating λ in the single-parameter exponential fail- ure distribution. In designing for reliability, however, it is important to first under- stand some of its statistical properties. a) Characteristics of the One-Parameter Exponential Distribution The statistical characteristics of the one-parameter exponential distribution are bet- ter understood by examining its parameter, λ , and the effect that this parameter has on the exponential probability density function as well as the reliability function. Effects of λ on the probability density function: • The scale parameter is 1/ λ = m. The only parameter it has is the failure rate, λ . • As λ is decreased in value, the distribution is stretched to the right. • This distribution h as no shape parameter because it has only one shape, i.e. the exponential. • The distribution starts at T = 0wheref(T = 0)= λ and decreases exponentially as T increases (Fig. 3.34), and is convex as T →∞, f(T) → 0. 3.3 Analytic Development of Reliability and Performance in Engineering Design 199 • This probability density function (p.d.f.) can be thought of as a special case of the Weibull probability density function with β = 1. Fig. 3.34 Effects of λ on the probability density function Fig. 3.35 Effects of λ on the reliability function 200 3 Reliability and Performance in Engineering Design Effects of λ on the reliability function : • The failure rate of the function is represented by the parameter λ . • The failure rate of the reliability function is constant (Fig. 3.35). • The one-parameter exponential reliability function starts at the value of 1 at T = 0. • As T → ∞, R(T) → 0. b) Estimating the Parameter of the Exponential Distribution The parameter of the exponential distribution can be estimated graphically by prob- ability plotting or analytically by either least squares or maximum likelihood. Probability plotting The g raphical method of estimating the parameter of the ex- ponential distribution is by p robability plotting, illustrated in the following exam- ple. Estimating the parameter of the exponential distribution with probability plot- ting Assume six identical units have pilot reliability test results at the same ap- plication and operation stress levels. All of these units appear to have failed after operating for the following testing periods, measured in hours: 96, 257, 498, 763, 1,051 and 1,744. Steps for estimating the parameter of the exponential probability density function, using probability plotting, are as follows (Table 3.22). The times to failure are sorted from sma ll to large values, and median rank per- centages calculated. Median rank positions are u sed instead of other ranking meth- ods because median ranks are at a specific confidence level (50%). Exponential probability plots use scalar data arranged in rank order for th e x-axis of the prob- ability plot. The y-axis plot is found from a statistical technique, Benard’s median rank position (Abernethy 1992). Determining the X and Y positions of the plot points The points plotted repre- sent times-to-failure data in reliability analysis. For example, the times to failure in Table 3.22 would be used as the x values or time values. Determining what the appropriate y plot position, or the unreliability values should be is a little more complex. To determine the y plot positions, a value indicating the corresponding Table 3.22 Median rank table for failure test results Time to failure Failure order number Median rank (h) (%) 96 1 10.91 257 2 26.44 498 3 42.14 763 4 57.86 1,051 5 73.56 1,744 6 89.10 3.3 Analytic Development of Reliability and Performance in Engineering Design 201 unreliability for that failure must first be determined. In other words, the cumula- tive percent failed must be obtained for each time to failure. In the example, the cumulative percent failed by 96 h is 17%, by 257 h 34% and so forth. This is a sim- ple method illustrating the concept. The problem with this method is that the 100% point is not defined on most probability plots. Thus, an alternative and more robust approach must be used, such as the method of obtaining the median rank for each failure. Method of median ranks Median ranks are used to obtain an estimate of the un- reliability, U(T j ), f or each failure. It is the value that the true probability of failure, Q(T j ), should have at the jth failure out of a sample of N components,at a 50% con- fidence level. This essentially means that this is a best estimate for the unreliability: half of the time the true value will be greater than the 50% confidence estimate, while the other half of the time the true value will be less than the estimate. The estimate is then based on a solution of the binomial distribution. The rank can be found for any percentage point, P, greater than zero and less than one, by solving the cumulative binomial distribution for Z. This represents the rank, or unreliability estimate, for the jth failure in the following equation for the cumulative binomial distribution P = N ∑ k= j (N k )Z k (1−Z) N−k , (3.163) where: N = the sample size, j = the order number. The median rank is obtained by solving for Z at P = 0.50 in 0.50 = N ∑ k= j (N k )Z k (1−Z) N−k . (3.164) For example, if N = 6 and we have six failures, then the median rank eq uation would be solved six times, once for each failure with j = 1,2,3,4, 5 and 6, for the value of Z. This result can then be used as the unreliability estimate for each failure, or the y plotting position. The solution of Eq. (3.164) for Z requires the use of numerical methods. A quick though less accurate approximation of the median ranks is given by the following expression. This approximation of the median ranks is known as Benard’s approximation (Abernethy 1992): MR = j −0.3 N + 0.4 . (3.165) For the six failures in Table 3.22, the following values are equated (Table 3.23): 202 3 Reliability and Performance in Engineering Design Table 3.23 Median rank table for Bernard’s approximation Failure order number Bernard’s approximation (×10 −2 ) Binomial equation Error margin Failure 1 MR 1 = 0.7/6.4 = 10.94 10.91 +0.275% Failure 2 MR 2 = 1.7/6.4 = 26.56 26.44 +0.454% Failure 3 MR 3 = 2.7/6.4 = 42.19 42.14 +0.120% Failure 4 MR 4 = 3.7/6.4 = 57.81 57.86 −0.086% Failure 5 MR 5 = 4.7/6.4 = 73.44 73.56 −0.163% Kaplan–Meier estimator The Kaplan–Meier estimator is used as an alternative to the median ranks method for calculating the estimates of the unreliability for probability plotting purposes F(t i )=1− i ∏ j= 1 n j −r j n j , (3.166) where: i = 1,2,3 , ,m, m = total number of data points, n = total number of units. and: n i = i−1 ∑ j= 0 S j − i−1 ∑ j= 0 R j , where: i = 1,2,3 , ,m, R j = number of failures in the jth data group, S j = number of surviving units in the jth data group. The exponential probability graph is based on a log-linear scale, as illustrated in Fig. 3.36. The best possible straight line is drawn that goes through the t = 0and R(t)=100% point, and through the plotted points on the x-axis and their corre- sponding rank values on the y-axis. A horizontal line is drawn at the ordinate point Q(t)=63.2% or at the point R(t)=36.8%, until this lin e intersects the fitted straight line. A vertical line is then drawn through this intersection until it crosses the ab- scissa. The value at the abscissa is the estimate of the mean. For this example, MTBF = 833 h, which means that λ = 1/MTBF = 0.0012. This is always at 63.2%, since Q(T)=1 − e −1 = 63.2%. The reliability value for any mission or operational time t can be obtained. For example, the reliability for an operational duration of 1,200h can now be obtained. To obtain the value from the plot, a vertical line is drawn from the abscissa, at t = 1,200h, to the fitted line. A horizontal line from this intersection to the ordinate is drawn and R(t) obtained. This value can also be obtained analytically from the exponential reliability function. In this case, R(t)=98.15% where R(t)=1−U and U = 1.85% at t = 1,200. . and Performance in Engineering Design Asymptotic behaviour In many cases, estimation is performed using a set of in- dependent, identically distributed measurements. In such cases, it is of interest. Analytic Development of Reliability and Performance in Engineering Design 193 where: a j and x j = regression parameters and covariates, β and μ = the shape and scale parameters. It is often more convenientto. several m ethods for estimating λ in the single-parameter exponential fail- ure distribution. In designing for reliability, however, it is important to first under- stand some of its statistical properties. a)