MINISTRY OF EDUCATION AND TRAINING MINISTRY OF NATIONAL DEFENCE MILITARY TECHNICAL ACADEMY CHU THI HUONG SEMANTICS-BASED SELECTION AND CODE BLOAT REDUCTION TECHNIQUES FOR GENETIC PROGRAMMIN DOCTORAL DISSERTATION: MATHEMATICAL FOUNDATION FOR INFORMATICS HA NOI - 2019 MINISTRY OF EDUCATION AND TRAINING MINISTRY OF NATIONAL DEFENCE MILITARY TECHNICAL ACADEMY CHU THI HUONG SEMANTICS-BASED SELECTION AND CODE BLOAT REDUCTION TECHNIQUES FOR GENETIC PROGRAMMIN DOCTORAL DISSERTATION Major: Mathematical Foundations for Informatics Code: 46 01 10 RESEARCH SUPERVISORS: Dr Nguyen Quang Uy Assoc Prof Dr Nguyen Xuan Hoai HA NOI - 2019 ASSURANCE I certify that this dissertation is a research work done by the author under the guidance of the research supervisors The dissertation has used citation information from many di erent references, and the ci-tation information is clearly stated Experimental results presented in the dissertation are completely honest and not published by any other author or work Author Chu Thi Huong ACKNOWLEDGEMENTS The rst person I would like to thank is my supervisor, Dr Nguyen Quang Uy, the lecturer of Faculty of Information Technology, Military Technical Academy, for directly guiding me through the PhD progress Dr Uy’s enthusiasm is the power source to motivate me to carry out this research His guide has inspired much of the research in this dissertation I also wish to thank my co-supervisor, Assoc Prof Dr Nguyen Xuan Hoai at AI Academy He has given and discussed a lot of new issues with me Working with Prof Hoai, I have learnt how to research systematically Particularly, I would like to thank the leaders and lecturers of the Faculty of Information Technology, Military Technical Academy for supporting me with favorable conditions and cheerfully helping me in the study and research process Last, but most important, I also would like to thank my family, my parents for always encouraging me, especially my husband, Nguyen Cong Minh for sharing a lot of happiness and di culty in the life with me, my children, Nguyen Cong Hung and Nguyen Minh Hang for trying to grow up and study by themselves Author Chu Thi Huong CONTENTS Contents i Abbreviations v List of gures vii List of tables ix INTRODUCTION Chapter BACKGROUNDS 1.1 Genetic Programming 1.1.1 GP Algorithm 1.1.2 Representation of Candidate Solutions 1.1.3 Initialising the Population 10 1.1.4 Fitness Evaluation 11 1.1.5 GP Selection 12 1.1.6 Genetic Operators 14 1.1.7 GP parameters 16 1.1.8 GP benchmark problems 18 1.2 Some Variants of GP 18 1.2.1 Linear Genetic Programming 20 1.2.2 Cartesian Genetic Programming 21 1.2.3 Multiple Subpopulations GP 21 1.3 Semantics in GP 23 1.3.1 GP Semantics 23 i 1.3.2 Survey of semantic methods in GP 27 1.3.3 Semantics in selection and control of code bloat 35 1.4 Semantic Backpropagation 37 1.5 Statistical Hypothesis Test 38 1.6 Conclusion 40 Chapter TOURNAMENT SELECTION USING SEMANTICS 41 2.1 Introduction 41 2.2 Tournament Selection Strategies 43 2.2.1 Sampling strategies 44 2.2.2 Selecting strategies 45 2.3 Tournament Selection based on Semantics 48 2.3.1 Statistics Tournament Selection with Random 49 2.3.2 Statistics Tournament Selection with Size 50 2.3.3 Statistics Tournament Selection with Probability 51 2.4 Experimental Settings 53 2.4.1 Symbolic Regression Problems 54 2.4.2 Parameter Settings 54 2.5 Results and Discussions 57 2.5.1 Performance Analysis of Statistics Tournament Selection 57 2.5.2 Combining Semantic Tournament Selection with Semantic Crossover 65 2.5.3 Performance Analysis on The Noisy Data 69 2.6 Conclusion 76 ii SEMANTIC APPROXIMATION FOR REDUCING CODE BLOAT Chapter 78 3.1 Introduction 78 3.2 Controlling GP Code Bloat 81 3.2.1 Constraining Individual Size 81 3.2.2 Adjusting Selection Techniques 81 3.2.3 Designing Genetic Operators 83 3.3 Methods 85 3.3.1 Semantic Approximation 85 3.3.2 Subtree Approximation 87 3.3.3 Desired Approximation 89 3.4 Experimental Settings 90 3.5 Performance Analysis 92 3.5.1 Training Error 92 3.5.2 Generalization Ability 96 3.5.3 Solution Size 98 3.5.4 Computational Time 99 3.6 Bloat, Over tting and Complexity Analysis 102 3.6.1 Bloat Analysis 102 3.6.2 Over tting Analysis 103 3.6.3 Function Complexity Analysis 107 3.7 Comparing with Machine Learning Algorithms 109 3.8 Applying semantic methods for time series forecasting 110 3.8.1 Some other versions 112 3.8.2 Time series prediction model and parameter settings 113 iii [81] Nguyen, Q.U., O’Neill, M., Nguyen, X.H.: Examining semantic diversity and semantic locality of operators in genetic programming Ph.D thesis, University College Dublin (2011) [82] Nguyen, Q.U., Pham, T.A., Nguyen, X.H., McDermott, J.: Subtree semantic ge-ometric crossover for genetic programming Genetic Programming and Evolvable Machines 17(1), 25{53 (2016) [83] Oksanen, K., Hu, T.: Lexicase selection promotes e ective search and behavioural diversity of solutions in linear genetic programming In: 2017 IEEE Congress on Evolutionary Computation (CEC) pp 169{176 IEEE (2017) [84] Oliveira, L.O.V., Casadei, F., Pappa, G.L.: Strategies for improving the distribution of random function outputs in gsgp In: European Conference on Genetic Programming pp 164{177 Springer (2017) [85] Oliveira, L.O.V., Miranda, L.F., Pappa, G.L., Otero, F.E., Takahashi, R.H.: Reducing dimensionality to improve search in semantic genetic programming In: International Conference on Parallel Problem Solving from Nature pp 375{385 Springer (2016) [86] Oliveira, L.O.V., Otero, F.E., Pappa, G.L.: A dispersion operator for geometric semantic genetic programming In: Proceedings of the Genetic and Evolutionary Computation Conference 2016 pp 773{780 ACM (2016) [87] Oltean, M., Grosan, C., Diosan, L., Mihaila, C.: Genetic programming with linear representation: a survey International Journal on Arti cial Intelligence Tools 18(02), 197{238 (2009) [88] O’Neill, M., Vanneschi, L., Gustafson, S.M., Banzhaf, W.: Open issues in genetic programming Genetic Programming and Evolvable Machines 11, 339{363 (2010) [89] Panait, L., Luke, S.: Alternative bloat control methods In: Genetic and Evolutionary Computation Conference pp 630{641 Springer (2004) 140 [90] Pawlak, T.P., Krawiec, K.: Progress properties and tness bounds for geometric semantic search operators Genetic Programming and Evolvable Machines 17(1), 5{23 (2016) [91] Pawlak, T.P., Krawiec, K.: Competent geometric semantic genetic programming for symbolic regression and boolean function synthesis Evolutionary computation 26(2), 177{212 (2018) [92] Pawlak, T.P., Wieloch, B., Krawiec, K.: Review and comparative analysis of geometric semantic crossovers Genetic Programming and Evolvable Machines 16(3), 351{386 (2015) [93] Pawlak, T.P., Wieloch, B., Krawiec, K.: Semantic backpropagation for designing search operators in genetic programming IEEE Transactions on Evolutionary Computation 19(3), 326{340 (2015) [94] Pedregosa, F., Varoquaux, G., Gramfort, A., Michel, V., Thirion, B., Grisel, O., Blondel, M., Prettenhofer, P., Weiss, R., Dubourg, V., Vanderplas, J., Passos, A., Cournapeau, D., Brucher, M., Perrot, M., Duchesnay, E.: Sklearn tutorial [online] (2011), https://scikit-learn.org/stable/ Accessed: 2019-11-24 [95] Poli, R.: A simple but theoretically-motivated method to control bloat in genetic programming Genetic programming pp 43{76 (2003) [96] Poli, R.: Covariant tarpeian method for bloat control in genetic programming Genetic Programming Theory and Practice VIII pp 71{89 (2011) [97] Poli, R., Langdon, W.B., McPhee, N.F., Koza, J.R.: A eld guide to genetic programming Lulu com (2008) [98] Poli, R., McPhee, N.F., Citi, L., Crane, E.: Memory with memory in tree-based genetic programming In: European Conference on Genetic Programming pp 25{36 Springer (2009) 141 [99] Purohit, A., Choudhari, N.S., Tiwari, A.: Code bloat problem in genetic programming International Journal of Scienti c and Research Publications 3(4), 1612 (2013) [100] Rumpf, D.L.: Statistics for dummies Technometrics 46(3) (2004) [101] Saez, J.A., Galar, M., Luengo, J., Herrera, F.: Tackling the problem of classi ca- tion with noisy data using multiple classi er systems: Analysis of the performance and robustness Information Sciences 247, 1{20 (2013) [102] Saez, J.A., Galar, M., Luengo, J., Herrera, F.: Analyzing the presence of noise in multi-class problems: alleviating its in uence with the one-vs-one decomposition Knowledge and Information Systems 38(1), 179{206 (2014) [103] Sara, S., Leonardo, V.: The importance of being at-studying the program length distributions of operator equalisation Genetic Programming Theory and Practice IX pp 211{233 (2011) [104] Silva, S., Costa, E.: Dynamic limits for bloat control in genetic programming and a review of past and current bloat theories Genetic Programming and Evolvable Machines 10(2), 141{179 (2009) [105] Silva, S., Dignum, S.: Extending operator equalisation: Fitness based self adap- tive length distribution for bloat free gp In: European Conference on Genetic Programming pp 159{170 Springer (2009) [106] Silva, S., Dignum, S., Vanneschi, L.: Operator equalisation for bloat free genetic programming and a survey of bloat control methods Genetic Programming and Evolvable Machines 13(2), 197{238 (2012) [107] Silva, S., Vanneschi, L.: Operator equalisation, bloat and over tting: a study on human oral bioavailability prediction In: Proceedings of the 11th Annual conference on Genetic and evolutionary computation pp 1115{1122 ACM (2009) 142 [108] Sokolov, A., Whitley, D.: Unbiased tournament selection In: Proceedings of the 7th annual conference on Genetic and evolutionary computation pp 1131{1138 ACM (2005) [109] Suganuma, M., Shirakawa, S., Nagao, T.: A genetic programming approach to designing convolutional neural network architectures In: Proceedings of the Genetic and Evolutionary Computation Conference pp 497{504 ACM (2017) [110] Szubert, M., Kodali, A., Ganguly, S., Das, K., Bongard, J.C.: Semantic forward propagation for symbolic regression In: International Conference on Parallel Problem Solving from Nature pp 364{374 Springer (2016) [111] Trujillo, L., Emigdio, Z., Juarez-Smith, P.S., Legrand, P., Silva, S., Castelli, M., Vanneschi, L., Schutze, O., Mu~noz, L., et al.: Local search is underused in genetic programming Genetic Programming Theory and Practice XIV pp 119{137 (2018) [112] Trujillo, L., Mu~noz, L., Galvan-Lopez, E., Silva, S.: neat genetic programming: Controlling bloat naturally Information Sciences 333, 21{43 (2016) [113] Trujillo, L., Olague, G., Lutton, E., de Vega, F.F., Dozal, L., Clemente, E.: Speciation in behavioral space for evolutionary robotics Journal of Intelligent and Robotic Systems 64(3-4), 323{351 (2011) [114] Vanneschi, L., Castelli, M., Manzoni, L., Silva, S.: A new implementation of geometric semantic gp and its application to problems in pharmacokinetics In: European Conference on Genetic Programming pp 205{216 Springer (2013) [115] Vanneschi, L., Castelli, M., Silva, S.: Measuring bloat, over tting and functional complexity in genetic programming In: Proceedings of the 12th annual conference on Genetic and evolutionary computation pp 877{884 ACM (2010) [116] Vanneschi, L., Castelli, M., Silva, S.: A survey of semantic methods in genetic programming Genetic Programming and Evolvable Machines 15(2), 195{214 (2014) 143 [117] Vanneschi, L., Galvao, B.: A parallel and distributed semantic genetic programming system In: 2017 IEEE Congress on Evolutionary Computation (CEC) pp 121{128 IEEE (2017) [118] Vyas, R., Bapat, S., Goel, P., Karthikeyan, M., Tambe, S.S., Kulkarni, B.D.: Application of genetic programming gp formalism for building disease predictive models from protein-protein interactions ppi data IEEE/ACM Transactions on Computational Biology and Bioinformatics (TCBB) 15(1), 27{37 (2018) [119] Whigham, P.A., Dick, G.: Implicitly controlling bloat in genetic programming IEEE Transaction on Evolutionary Computation 14(2), 173{190 (2010) [120] White, D.R., McDermott, J., Castelli, M., Manzoni, L., Goldman, B.W., Kronberger, G., Jaskowski, W., O’Reilly, U.M., Luke, S.: Better GP benchmarks: community survey results and proposals Genetic Programming and Evolvable Machines 14(1), 3{29 (2013) [121] Wilson, D.G., Cussat-Blanc, S., Luga, H., Miller, J.F.: Evolving simple programs for playing atari games In: Proceedings of the Genetic and Evolutionary Computation Conference pp 229{236 ACM (2018) [122] Xie, H.: Diversity control in gp with adf for regression tasks In: Australasian Joint Conference on Arti cial Intelligence pp 1253{1257 Springer (2005) [123] Xie, H., Zhang, M.: Impacts of sampling strategies in tournament selection for genetic programming Soft Computing 16(4), 615{633 (2012) [124] Xie, H., Zhang, M.: Parent selection pressure auto-tuning for tournament selec-tion in genetic programming IEEE Transactions on Evolutionary Computation 17(1), 1{19 (2013) [125] Xie, H., Zhang, M., Andreae, P.: Automatic selection pressure control in genetic programming In: Sixth International Conference on Intelligent Systems Design and Applications vol 1, pp 435{440 IEEE (2006) 144 [126] Xie, H., Zhang, M., Andreae, P., Johnson, M.: An analysis of multi-sampled issue and no-replacement tournament selection In: Proceedings of the 10th annual conference on Genetic and evolutionary computation pp 1323{1330 ACM (2008) [127] Xie, H., Zhang, M., Andreae, P., Johnston, M.: Is the not-sampled issue in tournament selection critical? In: 2008 IEEE Congress on Evolutionary Compu-tation pp 3710{3717 IEEE (2008) [128] Yoo, S., Xie, X., Kuo, F.C., Chen, T.Y., Harman, M.: Human competitiveness of genetic programming in spectrum-based fault localisation: theoretical and empirical analysis ACM Transactions on Software Engineering and Methodology (TOSEM) 26(1), (2017) [129] Zegklitz, J., Pos k, P.: Model selection and over tting in genetic programming: Empirical study In: Proceedings of the Companion Publication of the 2015 An-nual Conference on Genetic and Evolutionary Computation pp 1527{1528 ACM (2015) 145 Appendix Remaining results of the statistics tournament selection methods This appendix presents the remaining results of the methods tested in Chapter The table results include: Mean best tness on training noise data with tour size=3 and tour size=7 Average of solutions size on training noise data with tour size=3 and tour size=7 Mean of best tness of GP and three semantics tournament selections with tour size=5 Median of testing error of GP and three semantics tournament se-lections with tour size=5 Average of solution’s size of GP and three semantics tournament selections with tour size=5 Mean of best tness of TS-RDO and four other techniques with tour size=5 Median of ttest of TS-RDO and four other techniques with tour size=5 Average of solutions size of TS-RDO and four other techniques with tour size=5 146 Table A.1: Mean best tness on training noise data with tour-size=3 (the left) and tour-size=7 (the right) Pro GP neatGP TS-S RDO TS-RDO GP neatGP TS-S RDO TS-RDO A Benchmarking Problems F1 F2 F3 F4 F5 F6 F7 F8 F9 F10 F11 F12 2.06 4.78{ 0.22 0.41{ { + 2.43 1.69 4.78 0.57 0.05 { 5.39 13.11 6.63 0.17+ { + 0.10 0.17{ 0.11 0.08 0.14 0.16{ 0.14 0.13 { + 0.21 0.22 0.41 0.76 1.00{ 0.48 0.54{ 66.8 3.41 0.15 { { + 0.53 0.45 + 67.3 3.22 { 67.2 65.9 3.99 5.64{ 9.93 10.9 { 4.61 2.95 6.82 2.72+ 0.21 0.18+ 0.21 0.30{ 7.15 7.52{ { 7.17 6.76 + 0.88 0.93{ 0.89{ 0.87 F14 102.6 109.4{ 104.5{ 94.9+ F15 3.04 3.95{ 3.02 1.86+ F13 4.75 + 0.10 2.85 { 0.58 { 6.33 13.11 { + 0.06 0.21 + { + 0.46 66.0 67.4 3.34 66.5 69.2 5.40 5.64 + 7.96 10.9 + 0.22 0.30 7.03 7.52 { 0.89 0.89 102.4 103.1 + { { { { { { 0.93 109.4 2.52 3.95 18.6 23.8 3.62 4.31 45.4 56.6 24.3 28.5 16.3 16.9 { { 1.26 0.27 0.57 0.27 67.3 { { + 6.98 3.58 + + { 0.89 2.71 0.18 7.17 6.81 { + 0.19 + + 7.06 0.87 0.89 + { { 6.74 2.96 0.21 + { 0.54 { { 0.15 0.61 0.45 { 0.39 { + { 1.00 { + 3.38 1.52 0.10 0.62 + 0.17 { + 0.14 { 0.19 6.98 2.01 { 3.55 0.19 0.12 0.08 { 0.14 0.14 0.16 0.15 + 0.56 0.26 69.2 0.09 { 1.23 0.28 + 0.91 { { + + { + + 102.7 96.2 2.65 { 103.6 + 2.02 1.86 B UCI Problems F16 19.3 23.82 { 20.0 3.97 4.31{ F18 45.8 56.6{ { F17 F19 26.0 28.50 F20 16.6 16.9 F22 F23 5.07 F24 11.6 { 7.14 { 13.6 2.82 34.6 { 31.5 22.1 { 4.49 4.68{ 3.44 4.22{ F21 4.36 45.8 9.49 16.7 4.54 3.75 { { 15.0 2.78 5.07 1.59 { 14.3 5.50 + + + 4.05 { + + + + + + 9.72 3.69 + 35.6 28.3 + { 15.6 + + 4.18 3.45 + 3.03 11.0 5.46 6.79{ 7.04{ 2.33+ 4.77 F26 53.12 53.64{ 53.25 52.63 53.07 F25 147 { { { { { 19.6 4.37 31.7 4.41 3.19 4.22 4.09 7.14 10.1 13.6 4.81 6.79 53.23 { { { 9.78 + 2.57 3.78 45.7 33.9 16.7 { 4.68 4.51 { { 9.37 { { + 22.2 14.8 + + { + { + { + { + 8.81 1.36 15.6 4.57 + 28.6 4.00 3.85 2.80 35.7 15.7 + 4.19 + 3.57 3.68 11.8 { { { 7.48 2.07 5.49 { { 52.85 53.31 53.64 53.52 { Table A.2: Average of solutions size on training noise data with tour-size=3 (the left) and tour-size=7 (the right) Pro GP neatGP TS-S RDO TS-RDO GP neatGP TS-S RDO TS-RDO A Benchmarking Problems F1 F2 F3 F4 273 123+ 184 65+ 260 103+ 250 54+ + F5 85 10 F6 178 48 F7 145 47 F8 F9 235 135+ 165 68+ F10 172 66 F11 149 52 F12 244 64 F13 178 54 F14 323 72 F15 166 64 + + + + + 120 248 + + 97 + 128 190 69 52 45 46 + 312 50 + + 226 { { + 92 153 + 67 171 110 + 173 + + + + 38 160 + { + 240 + + 209 156 + 98 135 + + 35 174 69 141 + 100 179 + 92 + 98 174 + 48 + 142 47 + 366 135+ 220 68+ 70 142 + 60 191 + + 93 185 + + 35 + 192 66 + 159 52 + 297 64 + 161 54 + 361 72 + 191 64 284 109 232 70 220 74 317 87 331 92 + 237 66 + 211 82 + 212 52 + 214 61 + 217 70 + 209 37 + { + 45 + 231 51 + + 98 18 312 174 78 33 + 78 + + 77 25 + 104 183 49 10 75 + 260 103+ 205 54+ + 48 87 104 22 + 100 231 + 38 165 + 16 25 295 123+ 168 65+ 44 + + + + + + + 69 + + 18 + + + + + 16 46 19 + + + + + { 72 132 + 101 + 170 139 12 69 84 158 + 26 142 + 132 + { 57 115 + 73 208 + 84 + 31 + + + + + 18 B UCI Problems F19 186 109+ 124+ 296{ 194 70+ 45+ 198 + { 168 74+ 97 340 + + 213 87+ 13 86 F20 240 92 F21 183 66 F22 194 82 F23 168 52 F24 169 61 F25 174 70 F26 137 37 F16 F17 F18 + + + + + + + + { 91 397 + 200 88 + 84 190 + 53 35 233 + { { 228 + 220 34 70 + 64 + 174 84 204 10 212 + 110 52 + 108 54 72 33 148 + + + + + + + + + + + 117 349 + 243 33 86 + 407 + 100 86 58 + 70 { + 462 { + 16 21 + + + 46 + + + + 171 + 101 39 284 275 { { 260 54 171 + + 242 + 188 61 20 149 + + 73 35 39 + 21 + + + Table A.3: Mean of best tness with tour size=5 The left is original data and the right is noise data Pro GP TS-R TS-S TS-P A Benchmarking Problems { F1 1.59 2.50 F2 0.23 0.35 F3 4.56 F4 0.05 6.20 0.04 F5 0.12 0.13 F6 0.35 F7 0.42 0.58 0.45 F8 5.44 4.98 5.48 F9 2.06 1.73 2.50 F10 7.92 7.47 F11 0.09 0.09 F12 6.96 7.13 F13 0.88 F14 72.8 0.88 74.3 F15 2.30 2.50 { { 2.94 0.58 6.57 0.05 { { { 1.01 0.52 0.28 5.08 { { + { 7.07 0.88 78.5 1.83 0.21 2.56 5.08 5.74 0.37 0.14 0.14 { 0.56 0.61 1.02 { 0.41 0.46 0.49 { 5.01 { + 66.5 { { 7.13 0.88 77.6 0.11 4.15 8.23 8.60 3.33 { 0.59 { 6.70 0.12 0.14 { { 1.21 { 0.56 7.02 7.16 { 0.88 0.89 5.56 6.89 { { + 0.20 2.51 2.87 9.22 8.69 18.3 3.68 20.1 { { { { 0.90 { { { 0.29 4.90 0.10 0.14 { { 66.9 3.96 { { { 7.83 { { + 2.62 2.50 0.81 0.47 + 7.13 { 2.56 { { 103.6 102.5 2.11 TS-P 67.2 0.21 { 103.6 { { 67.1 4.38 0.21 TS-S { 0.10 1.39 + 7.39 5.58 0.07 0.08 { TS-R 0.04 0.13 0.13 { 2.46 GP { 0.21 7.14 0.89 102.7 { { + 2.91 { B UCI Problems F16 8.08 8.78 F17 3.47 F18 10.2 4.00 11.8 F19 25.7 29.8 F20 9.36 9.84 F21 4.26 4.38 F22 0.84 1.14 F23 3.56 4.83 F24 8.39 F25 4.57 10.5 { { { { { { { 31.8 9.77 4.36 1.10 6.04 11.7 { { { { { 6.97 52.06 3.80 { + 45.3 28.3 9.58 { 25.4 29.7 16.4 16.7 4.30 4.40 4.50 3.25 3.69 4.18 5.51 { 10.4 13.2 { 5.00 6.29 53.11 53.35 1.00 4.23 9.74 { { 4.27 46.4 8.9 { { 5.69 F26 51.80 51.94 4.07 10.4 { 5.42 51.88 { 149 { { { { { { { { 19.6 18.8 4.35 44.9 { { 31.6 28.0 { 16.7 4.46 3.78 7.95 16.6 { { { { 3.59 5.18 12.3 { { 53.58 { { 4.48 15.2 7.26 4.07 45.9 { { { { 5.94 53.29 { { Table A.4: Median of testing error with tour size=5 The left is original data and the right is noise data Pro GP TS-R TS-S A Benchmarking Problems + + F1 8.86 6.07 F2 0.96 0.88 F3 31.1 F4 0.051 15.3 0.048 14.1 0.050 F5 0.135 0.135 F6 1.36 F7 + + 4.08 0.87 + + TS-P 6.12 0.96 10.9 0.94 + 32.4 TS-R TS-S 6.10 0.83 + + + 5.17 0.80 + + + TS-P 7.90 0.92 0.147 0.129 0.042 0.134 0.140 0.140 0.139 0.140 1.71 1.91 1.92 2.08 2.23 2.06 2.23 1.67 1.77 1.61 1.77 1.83 1.69 1.81 F8 7.37 7.26 1.59 7.39 6.78 67.1 F9 1.69 + 1.64 5.16 F10 59.7 1.59 48.9 66.9 5.49 + 39.7 61.9 F11 0.07 0.08 0.08 F12 7.44 F13 0.877 7.33 0.874 F14 126.8 F15 4.59 + + + + 16.1 0.143 + 17.4 + GP 16.2 0.143 19.3 0.141 + 66.8 5.21 67.0 61.6 57.1 56.2 0.199 0.199 0.198 + 0.201 7.39 7.33 7.30 + 7.36 0.90 0.90 0.90 + 0.90 127.9 0.871 124.6 7.37 0.876 126.7 122.7 122.6 + 122.7 4.99 3.58 5.03 4.36 5.00 122.5 4.13 23.3 37.3 36.6 36.0 5.03 5.65 + 1.62 25.4 0.06 + 7.33 + + + 5.28 5.03 { B UCI Problems F16 21.3 22.1 25.3 F17 5.12 4.90 F18 9.77 10.78 4.71 9.63 F19 40.7 F20 9.59 38.6 9.83 F21 4.33 4.36 F22 1.90 F23 6.84 2.14 7.54 F24 19.1 F25 + + + + 6.78 39.9 47.6 5.59 47.4 43.1 40.3 9.69 9.32 9.14 4.48 + 36.8 9.46 + 34.5 + + 5.28 44.8 37.7 + + { 4.34 4.31 4.51 9.13 4.56 { 1.82 1.66 5.95 5.90 5.86 8.04 6.53 7.38 7.48 8.48 + 16.5 24.1 + 8.12 9.45 19.5 8.73 + 46.99 46.64 46.51 + 9.01 16.4 8.51 12.8 8.33 F26 48.35 46.95 46.28 150 + 16.8 5.52 47.0 42.2 9.18 4.57 + 5.81 { 8.69 + 22.7 + 8.82 8.31 46.63 + 46.48 Table A.5: Average of solution’s size with tour size=5 The left is original data and the right is noise data Pro GP TS-R TS-S TS-P A Benchmarking Problems 113 F2 302 258+ 169 140+ F3 277 281 99 F4 171 205 70 F5 93 92 44 164 146+ 149 150 56 93 F9 241 199+ 209 141+ F10 180 168 102 F11 157 145 74 F12 90 F13 281 209+ 157 109+ F14 312 275 171 F15 158 147 92 F1 F6 F7 F8 33 + 106 + + 270 273 104 + 184 270 219 + 110 + 149 + 137 + + + + 34 TS-S TS-P 292 245+ 174 148+ + 70 TS-R 250 164 43 + + + + GP 67 + + 39 182 139+ 138 137+ 52 298 189+ 206 126+ 74 198 178+ 156 144 91 229 148 292 212+ 172 141 86 292 338 319 156 159 191 165 79 215 250 234 110 217 168+ 195 175 32 + 140 168 + 149 + + + 89 201 84 274 29 + 34 228 { + + 153 + + + 61 293 116 163 58 60 + 253 159 + + + + + 187 139 + + 167 157 + 248 + 147 343 + 186 + 219 B UCI Problems F16 227 226 180 F17 231 172+ 198 198 41 F18 + 127 F20 257 100+ 240 244 152 F21 226 197 89 F22 207 189 87 F23 186 146+ 186 134+ 33 206 143+ 220 201 26 F19 F24 F25 F26 11 + + + 26 + + 186 182 171 233 + 197 + 201 + 160 + + 116 + 156 + + + 159 218 151 284 + 94 87 11 + + + 301 190+ 207 177+ 91 + 209 176+ 187 131+ 72 201 121+ 202 139+ 20 171 147 57 81 24 24 178 183 150 + + + 215 188 + 177 + + + + + 147 141 + + + 158 143 Table A.6: Mean of best tness of TS-RDO and four other techniques with tour size=5 The left is original data and the right is noise data Pro GP neatGP TS-S RDO TS-RDO GP neatGP TS-S RDO TS-RDO A Benchmarking Problems F1 1.59 4.64 F2 0.23 0.40 F3 4.56 F4 0.05 F5 0.12 0.15 F6 0.35 0.77 F7 0.42 0.50 F8 5.44 F9 2.06 F10 7.92 F11 0.09 { { { + + 0.06 0.16 + + 0.01 0.13 { 0.13 { + { + 1.01 0.01 { 0.52 0.19 { 5.48 + 16.61 0.39 { { + 3.58 2.50 0.20 11.50 5.58 0.95+ { + 0.29 0.07 0.03 { 7.44 F13 0.88 0.92 F15 2.30 { 0.58 { 6.57 12.63 { 0.11 0.05 F12 6.96 F14 72.8 { 2.94 0.16 { { 7.07 6.74 + { 0.88 0.86 { 78.5 + 83.8 53.8 { + 3.53 2.11 1.10 { 4.78 0.21 0.41 + 5.08 13.11 + 0.10 0.17 0.14 0.16 0.61 1.00 0.46 0.54 66.5 69.2 + 4.15 5.64 + 8.23 10.9 0.21 0.30 { 7.02 7.52 + 0.88 0.93 + 103.6 109.4 + 2.51 3.95 18.3 23.8 3.68 4.31 + 45.3 56.6 { 25.4 28.5 + 16.4 16.9 4.40 4.68 3.25 4.22 4.18 7.14 10.4 13.6 5.00 6.79 53.11 53.64 1.06 0.01 0.15 { + 0.01 0.40 0.37 + 0.20 0.32 0.06 7.04 0.87 65.9 1.11 { 1.83 2.29 0.31 { { { { { { { { 3.33 0.59 6.70 0.12 0.14 1.21 { { { { 0.20 { { 0.90 { { + 0.48 + 67.4 3.30 2.94 + + + 6.74 0.87 { + { 96.1 2.86 0.19 { + + 0.25 65.8 0.18 { 1.38 0.10 + 3.14 102.5 2.62 + 3.02 0.31 0.15 0.58 0.28 { { { + { { 7.13 0.08 + 0.14 5.56 6.89 { 0.06 0.16 0.56 67.2 0.14 { { + + { 7.03 { + 0.89 103.1 + 2.02 1.87 B UCI Problems F16 8.08 16.73 F17 3.47 4.18 F18 10.2 F19 25.7 { 9.22 { { 4.07 { 10.4 26.4 { F20 9.36 28.9 31.8 { 13.5 9.77 F21 4.26 4.59 F22 0.84 2.37 F23 3.56 6.23 F24 8.39 11.02 F25 4.57 6.43 F26 51.80 2.01 { { { { 3.13 23.2 6.72 { 4.36 { 2.41 3.89 + + + + + + { + { + { + { + 1.10 0.55 6.04 0.88 11.7 3.53 2.18 3.31 3.29 + 27.9 7.65 4.05 0.71 2.31 + + 9.38 4.62 { 6.97 2.07 { 52.63 52.07 50.88 51.57 152 { { { { { { { { { { { 19.6 + 9.3 { + 4.35 2.64 44.9 34.1+ 31.6 16.7 4.46 3.78 { 22.0 + 14.9 4.01 { 7.95 15.2 { 2.75 { { 7.26 { 53.58 1.38 + + + 4.87 { + + + 9.74 3.71 + 35.7 + 28.5 15.7 4.17 + + 3.53 3.30 11.4 5.29 2.09 52.79 53.24 { { { Table A.7: Median of ttest of TS-RDO and four other techniques with tour size=5 The left is original data and the right is noise data Pro GP neatGP TS-S RDO TS-RDO GP neatGP TS-S RDO TS-RDO A Benchmarking Problems F1 8.86 12.59{ F2 0.96 0.84+ 31.1 32.2 F3 F5 0.05 0.12{ 0.135 0.135 F6 F4 + 4.08 0.87 + 8.23 1.15 { + + 14.1 4.92 0.05 0.02+ + 0.138 1.36 1.74 0.129 1.91 F7 1.67 1.61 1.59 1.22 F8 7.37 7.41 7.39 0.00 F9 1.69 2.41 1.62 0.20 25.4 0.00 0.06 0.00 F10 59.7 F11 F12 41.0 0.07 0.30{ 7.44 7.34+ 7.33 + 0.00 + + + + + + + 10.9 13.1 5.17 0.80 + 0.94 0.84+ 32.4 32.2 4.16 1.00 7.49 1.85 + 0.02 0.138 + { + 16.1 7.15 0.14 0.14 + 6.31 0.15 0.19{ 0.140 0.140 0.139 0.141 0.14 3.07 + 1.77 1.73 1.69 1.61 1.25 1.62 68.5 66.7 + + 0.23 + 0.00 0.08 + + 0.871 0.874 0.870 { F14 126.8 131.3 124.6 124.1 122.6+ + + F15 4.59 5.92{ 3.58 3.24 3.24 67.1 66.9 66.8 5.16 5.68 5.21 61.9 + 56.4 57.1 0.20 0.32{ 7.39 7.41 0.20 + 5.02 50.9 + + 0.20 + { 0.898 122.7 128.8 4.36 6.21{ { 122.5 122.7 4.13 4.14+ + + + + 4.95 46.7 0.20 7.30 7.53 0.898 0.896 0.901 + 0.141 2.06 + F13 0.877 0.874 1.23 6.63 1.00 2.08 2.19 0.00 7.29 + 10.2 + 0.00 1.19 + + + + 7.31 0.896 122.6 4.12 + B UCI Problems F16 21.3 F17 33.7 5.12 4.95 { 25.3 6.86 + { F18 4.71 9.63 F20 9.59 36.8 9.46 F21 4.33 4.52{ 1.90 3.29{ 4.34 4.23 1.82 F23 6.84 8.44{ F24 19.1 17.7 8.04 1.14 6.42 F25 12.8 8.33 9.77 28.4{ F19 40.7 38.3+ F22 9.18 9.01 8.89 F26 48.35 47.26 + 5.66 3.60 + 37.4 11.7 + + + { + + 25.2 { 15.25 46.28 46.35 5.86 + 4.88 + 52.9 + 43.1 40.2 { 11.5 + 4.18 + 1.18 7.77 + + + 45.11 + 153 36.0 12.5 + 5.65 5.45 47.6 32.2 14.1 36.3 + 3.58 4.38 37.3 { + + 11.5 { 5.28 6.56 44.8 38.6+ + + + 5.36 + 36.7 + + 37.7 39.3 9.32 8.72+ 9.14 11.5 { 4.51 4.67{ 4.48 4.41 35.6 5.95 6.19{ 7.38 9.15{ 5.52 5.95 24.1 19.1 9.45 9.42 + 5.86 10.17 + 27.6 16.8 + { { 8.31 12.15 46.64 46.58 46.63 46.73 4.34 { + + 6.02 8.48 10.4 16.0 7.50 46.75 + + { Table A.8: Average of solutions size of TS-RDO and four other techniques with tour size=5 The left is original data and the right is noise data Pro GP neatGP TS-S RDO TS-RDO GP neatGP TS-S RDO TS-RDO A Benchmarking Problems F1 302 124+ 113+ 227+ + F2 169 60 F3 277 112 F4 171 60 F5 93 12 F6 164 45 F7 149 50 F8 241 118 F9 209 62 F10 180 60 F11 157 44 F12 281 67 F13 157 49 F14 312 66 F15 158 58 + + + + + + + + + + + + + 33 + + 99 163 161 + 70 336 44 56 + + + 43 + 93 70 + + 102 74 + + 90 34 171 + + + 92 + + 43 + { 207 + + + 96 34 179 127 164 51 + + + + + + + 48 178 15 18 10 35 50 15 41 22 60 31 + + 70 13 69 + 62 { 36 62 + + + 292 123+ + 174 65 + 273 103 270 54 84 10 182 48 138 47 + + + + 298 135+ + 206 68 + 198 + 156 52 + 292 64 + 172 54 + 338 72 + 191 64 66 + + + + + + + 106 242 + 29 166 64 67 + + + { 104 190 67 336 39 + 37 + + { + { + + 52 234 58 224 74 168 + 60 190 14 79 67 21 + + + + + + + + + 86 188 + 34 146 21 57 24 + + + + + { 79 138 + 101 + 156 154 + 83 143 72 + 91 181 61 145 + 36 15 + + + + + B UCI Problems F17 227 103+ + 231 62 F18 198 F19 257 F20 240 87 F21 226 63 F22 207 83 F23 186 55 F24 186 68 F25 206 63 F26 220 40 F16 71 + 79 + + + + + + + + + 180 321 + 41 232 + 127 362 11 + { 97 188 { + { 152 374 + 89 228 87 33 26 + + + 129 272 265 + 257 26 116 + + + 85 + 172 110 + { 53 + + 59 77 + 29 195 74 + 87 301 92 207 + 66 + 82 + 187 52 + 201 61 + 202 70 170 36 + 154 + 284 209 92 { 54 + 222 250 109+ + 217 70 + + + + + 110 339 + 32 219 + 87 392 11 + { + + 447 + 229 81 + 72 194 24 20 24 + + + 57 259 { { 78 172 + + 190 110 46 + 95 41 248 46 55 + + 260 + + + 96 91 161 + 22 + + + ... punishing the largest individuals, or adjust-ing population size distribution at each generation However, the bloat control methods are often di cult to t the training data leading to a reduction... Fitness that directly re ects the ability of an individual to solve the problem as above is also called raw tness In many situations, raw tness can be standardised (it is called standardised tness)... winner individual Randomly select a individual from the population; A for i to T ourSize Randomly select a individual from the population; B if B is better than A then A B; end end The winner individual