UNIVERSITY OF INFORMATION TECHNOLOGY FACULTY OF SOFTWARE ENGINEERING STUDENT INTERNSHIP REPORT COMPUTER VISION ENGINEER Company : VCCORP Supervisor : PhD Huynh Ngoc Tin Mr Nguyen Thanh Anh Tuyen Student Internship : Le Thi Phuong Ngan Ho Chi Minh, 14 – Jul - 2020 ABSTRACT This report describes the activities carried out during a weeks, final, fulltime internship program at the VCCorp The document contains information about the organization and the responsibilities performed throughout the period between April and June 2020 More than a plain account of tasks, the objective of this text is to reflect upon the experiences collected during the internship from the perspective of a BSc student The first part of the report offers an overview of the organization, followed by an outline of all the duties carried out during this time Following, it proceeds to describe in some detail the most relevant projects carried out and their respective analysis Finally, the report wraps-up with a few closing remarks and conclusions from the experience ACKNOWLEDGEMENT First and foremost, I would like to say thanks to VCCorp for the opportunity given to me as an intern in the Information technology section Special thanks go to my academic supervisor Nguyen Thanh Anh Tuyen, head department Mr Huynh Ngoc Tin of AdTech-HCM team for their personal efforts, practical skills, professional guidance and direction towards successful internship Finally, I would like to extend my heartfelt gratitude to Faculty of Software Engineer for their invaluable support throughout my training Le Thi Phuong Ngan Ho Chi Minh, 14 Jul 2020 FACULTY’S COMMENT CONTENT CHAPTER 1: WHERE I WORKED: VCCORP CHAPTER 2: INTERNSHIP OVERVIEW ……………………………………………………6 Learn the company rules 1.1 Tools 1.2 Technical skills Project implementation Work schedule CHAPTER 3: PROJECT Introduce about project Prepare data 2.1 Create list celebrity name 2.2 Collect data Pipeline project 10 Face Detection 11 Face Alignment 11 Face Embedding 12 Select Threshold 13 Result 13 REFENCES………………… …………………………………………………………………14 CONCLUSION…………………………………………………………………………………15 CHAPTER 1: WHERE I WORKED: VCCORP Founded in 2006, VCCorp Joint Stock Company (VCCorp) is a pioneer in the field of digital content and technology With more than 10 years of establishment and development, VCCorp has built a vast Internet ecosystem with a lot of creative and useful products in many fields (online advertising, e-commerce, online games ., covering over 90% of Internet and mobile users, has contributed greatly to the development of Vietnam Internet in the past decade To have a series of innovative and preeminent technology products today, VCCorp has built a strong workforce with over 1800 employees working and operating professionally in Hanoi, Ho Chi Minh City Chi Minh and other representative offices and branches in 06 most developed cities in Vietnam With the spirit of "Innovation-NonStop - Creative for the community", VCCorp is a common home for young people who love technology and creative passion CHAPTER 2: INTERNSHIP OVERVIEW Learn the company rules Time: day Contents: Introduce about the company, the organization of the company Hear the person in charge of introducing the company, the process of establishment and development (as mentioned above), the process from high to low, the organization of the company In addition, trainees are introduced to how to work in the company such as working time, the rules to comply, how to use email at work Tools and Technical skills 1.1 Tools Time: week Contents: During this time, the supervisor instructed trainees to learn about the tools that will help in the future Some of them are like VPN, Git, 1.2 Technical skills Time: days (1 week) Contents: Keras, Linux, InsightFace, Project implementation After weeks of training and practice, the trainee has mastered the basic knowledge about CNN, euclidean distance,… For the next weeks, the trainer instructed the trainees to apply the knowledge they have learned to implement a celebrity identification project Project details will be discussed in the following section Work schedule Week Job Supervisor -Learn about the company -Learn how to communicate, work via email -Learn the tools that work in a company Learn technical skills Mr Tuyen Nguyen Mr Tuyen Nguyen Rate Supervior’s Comment Create list celebrity name Collect data Face Detector and Face Alignment Get embedding of training data Compare Euclidean distance and select threshold Test with test set and verify data Complete and optimize code Mr Tuyen Nguyen Mr Tuyen Nguyen Mr Tuyen Nguyen Mr Tuyen Nguyen Mr Tuyen Nguyen Mr Tuyen Nguyen CHAPTER 3: PROJECT Introduce about project Face recognition has become a common application in many current technologies Facebook has an auto-tagging feature on the face, the iPhone X uses face recognition to unlock, companies use facial recognition for attendance, and many security systems are also using the technology In this project, I will work with a database of faces of famous people throughout Vietnam The problem is that from a given data set of 100 people, I need to build a model that predicts a new picture that corresponds to someone or from a completely new person Prepare data 2.1 Create list celebrity name Located on the same platform as Wikipedia is the Wikidata repository, a free, multilingual secondary database, specialized in structuring data collection to support other projects on the Wikimedia platform To get the Vietnamese name data on Wikipedia in the simplest way, you just need to go to Wikidata Query, perform query as follows: Result: 2.2 Collect data After getting a list of people named on Wikipedia, I started to search their photos on Google Image Search Fortunately, this part of the script doesn't have to be crawled because I already have an open source called Google Images Download The use of open source is quite simple, just install the instructions at the home page of the package and run the following script: For example, the image obtained by singer Toc Tien as shown below: Pipeline project Face Detection Face Embedding Face Alignment Save embedding 10 Select Threshold Face Detection The face detection algorithm that still uses most source face recognition is mtcnn because its accuracy is relatively fast However, because in the dataset there are many faces in the image are very blurry, bent down, so I chose the algorithm with higher accuracy is Single Stage Headless Face Detector Link https://github.com/deepinsight/mxnet-SSH Pretrain model: https://github.com/deepinsight/insightface/wiki/Model-Zoo Face Alignment To align my face, I used the mtcnn algorithm but not the whole network but a part, mtcnn includes consecutive subnets: Pnet, Rnet, Onet, Lnet I will not talk carefully about mtcnn model here because it is not the purpose of this article, I use subnets behind Onet and Lnet, the purpose of these networks is to detect landmark and refine again However, 11 because of dataset nature, in Onet network I also reduced the threshold from 0.8 (origin) to 0.5 Mtcnn: https://github.com/deepinx/mtcnn-face-detection Also can refer to some very good model align faces such as wingloss, Face Embedding I choose arcface algorithm as model face embedding (LResNet100E-IR, ArcFace @ ms1m-refine-v2) This is a pretrained model that has been trained with several million images so the model has learned a very good (general) ability Retraining the model with this dataset I think will result in no good Link: https://github.com/deepinsight/insightface Pretrained model: https://github.com/deepinsight/insightface/wiki/Model-Zoo Because I don't have the time and the power gpu base so I just use one model face embedding arcface Lresnet100 here and not ensemble model Create embedding file: With the photos in the training set without detecting the face, I will remove and not use these embedding to classify the test set (1 wrong embedding in the train practice can lead to many mis-classified test photos and Reduce results by a lot) In addition, when creating the embedding train file to predict the test set, for each photo in the train set I augment by taking symmetrical image through the mirror (to select threshold, not use this augment) 12 As for the images in the test set without detectable faces, I still create embedding for these images normally by processing images Select Threshold I use the pretrained facenet from this repo https://github.com/nyoki-mtl/keras-facenet Calculate the embedding for each image, compare each embedding for each image in the test set with each group of embeddings belonging to the same person in the train set, find the shortest distance of each photo to each person To insert class 1000 into predictions, I used a simple method of using thresholds First I will draw the probability distribution of variables: The euclidean distance between images if they belong to the same face The euclidean distance between images if they are on different sides The goal is to find a suitable distance so that if the distance from one image to all images is higher than it, it will be unkown I will choose the intersection between sets ie 1.1 Result Number of classes: 98 Accuracy: 95% 13 REFENCES [1] Deep Face Recognition: A Survey - Mei Wang, Weihong Deng School of Information and Communication Engineering, Beijing University of Posts and Telecommunications, Beijing, China [2] MobileFaceNets: Efficient CNNs for Accurate RealTime Face Verification on Mobile Devices 14 CONCLUSION This internship has been very rewarding for me on several aspects First the work I realized allowed me to go very deeply into machine learning/deep learning Thus it let me take great advantage of architecture course I had at the University Also, working on my project is very interesting, first because it is not common to modify the network, and also because to so we need a complete understanding of the hardware It is also interesting to how we can change the behavior of a program by doing that, and we need to be extra careful not to change the logic of the program Another aspect of the internship that fulfilled my expectation, even if it is not completely dependent of the internship itself, is that it allowed me to improve English’s skills Allowing me to improve both my technical and daily life English I would like to thanks especially Mr Huynh Ngoc Tin and Mr Nguyen Thanh Anh Tuyen for there help And of course there quiet impressive technical knowledge I want to thank Mr Nguyen Thanh Anh Tuyen for all his very helpful advises and especially about presenting and writing I believe that if this experience has been as rewarding for me, Mr Huynh and Mr Nguyen are responsible of it for a very important part 15 ... text is to reflect upon the experiences collected during the internship from the perspective of a BSc student The first part of the report offers an overview of the organization, followed by an...ABSTRACT This report describes the activities carried out during a weeks, final, fulltime internship program at the VCCorp The document contains information... professional guidance and direction towards successful internship Finally, I would like to extend my heartfelt gratitude to Faculty of Software Engineer for their invaluable support throughout my