SEQUENTIAL APPROACHES IN GRAPHICAL MODELS AND MULTIVARIATE RESPONSE REGRESSION MODELS JIANG YIWEI (B.Sc., WUHAN UNIVERSITY) A THESIS SUBMITTED FOR THE DEGREE OF DOCTOR OF PHILOSOPHY DEPARTMENT OF STATISTICS AND APPLIED PROBABILITY NATIONAL UNIVERSITY OF SINGAPORE 2015 [...]... aims at expanding the idea of the sequential LASSO approach in linear regression models to the areas of graphical models and multivariate response regression models under the high-dimension-low-samplesize circumstances First, a sequential scaled pairwise selection (SSPS) method is developed for the edge detection in sparse high-dimensional nonparanormal graphical models The extended Bayesian information... the response variables’ precision matrix estimation can be involved The joint estimation of the sparse coe cient matrix and precision matrix are usually formulated in two ways: the penalized likelihood and the penalized conditional regressions In the thesis, we propose some sequential methods for the edge detection and variable selection involved in the graphical models and the multivariate response regression. .. many sequential methods’ stopping rules, which are likely to include many irrelevant features, such as those for the OMP in Cai and Wang (2011) 1.2.3 Sparse Estimation in Multivariate Response Regression Models The regularized coe cient estimation of the univariate response regression models has been extensively studied in various literature Their natural extensions are the multivariate response regression. .. presented in this thesis further explores the sequential approaches to the relevant problems in the graphical models and the multivariate response regression models The specific objectives of the thesis include: • Development of a sequential edge detection method in the nonparanormal graphical models; • Applications of this edge detection technique to the precision matrix estimation; • Extension of the sequential. .. nonzero regression coe cient in (1.1) The identification and estimation of the nonzero !jk ’s are boiled down to variable selection and estimation in these linear regression models Various regularized estimation methods for linear regression 5 Chapter 1 Introduction models have been used in this framework The approach of LASSO was adopted by Meinshausen and B¨hlmann (2006) separately for each of the u p regression. .. approach estimates the precision matrix by optimizing over the support recovered by the SSPS edge detection The second approach adapts some existing methods by performing an SSPS screening in the beginning Chapter 4 is on the joint covariance matrix and precision matrix estimation in multivariate response regression models Two sequential conditional regression (SCR) methods are considered Their selection... i.i.d Np (0, ⌃) Naively, B can be fitted by decomposing (1.5) into a series of marginal linear regression models with 13 Chapter 1 Introduction univariate response, thus the coe cients for di↵erent responses are estimated independently This approach works well if Y1 , , Yp are truly independent Nevertheless, in practice, these responses, rather than being independent, are probably correlated, especially... common linear models, another similar sequential procedure was successfully developed for the interactive linear models by He and Chen (2014) It has been found that they not only excel at the feature selection in the large-p-small-n scenario, but also appear computationally attractive Initial attempt on the sequential edge detection in the Gaussian graphical models has also been realized in Luo and Chen... methods in linear regression models and Gaussian graphical models are combined and exploited to derive these two sequential methods in the conditional regression formulation (SCR) One relies on the alternate updating framework; the other depends on the simultaneous estimation scheme Considerable simulation examples are used to show the SCR methods’ overall advantages in terms of model selection and prediction... a screening tool for the other existing methods Follow-up simulations and a real example are employed to assess the proposed methods’ estimation accuracy in comparison to the others xi Summary Another aspect considered in this thesis is two sequential approaches to the joint estimation of the coe cient matrix and the precision matrix in multivariate response regression models The ideas of sequential . models. The ideas of sequential methods in linear regression models and Gaussian graphical models are combined and exploited to derive these two sequential methods in the conditional regres- sion formulation. some sequential methods for the edge detection and variable selection involved in the graphical models and the multivariate response regression models. In the rest of this chapter, we summarize the evolution. SEQUENTIAL APPROACHES IN GRAPHICAL MODELS AND MULTIVARIATE RESPONSE REGRESSION MODELS JIANG YIWEI (B.Sc., WUHAN UNIVERSITY) A THESIS SUBMITTED FOR